Find a New Purpose, Vision, and Mission for Your Data Warehouse
As user bases and data platforms grow, what should be the role of your data warehouse?
- By Teresa Wingfield
- January 31, 2022
Traditionally, the purpose, vision, and mission of a data warehouse have been driven by what, in most organizations, constitutes a relatively small set of users: the data engineers, data scientists, and business analysts interested in complex analytics. However, as the power of a data platform capable of running not just in the data center but also in the cloud or at the edge becomes more accessible, it will invariably attract a broader base of business users who want to use it to run queries and perform analytics to inform different operational decisions.
To satisfy this ever-expanding user base and their different requirements, organizations need to reconsider the purpose, vision, and mission of a data warehouse. In this new world, what purpose does the data platform serve? What should it deliver? What is its mission (and how will it achieve the vision)? Many aspects of the data warehouse’s purpose and vision will still apply to the data platform, but they will expand to encompass more strategic, tactical, and operational opportunities. The mission, though, must include a focus on data democratization, which requires a far different approach than was required of legacy data warehouse architectures.
New Purpose
Until recently, the data warehouse served as a central repository of historical data to help users analyze different time periods and trends. Data was consolidated from many sources to avoid impacting the performance of operations systems, improve data quality, optimize query performance, and provide a business representation of data that made it easier for users to access information.
A data warehouse’s ability to provide historical analytics will continue to be valuable, but capturing and understanding critical events in real time -- to improve operational decision making and response times -- will continue to grow in importance. True, a complementary operational data store (ODS), with its snapshot of transactional data that is often more current than that in the data warehouse, has provided additional support for operational decisions. However, even an ODS does not provide the real-time access required when decisions must be made in minutes, or perhaps even seconds. Examples include personalized e-commerce, supply chain optimization (scheduling, inventory, equipment use, etc.), credit and loan approvals, investment portfolio decisions, and many more use cases. Data fabrics and data meshes are emerging data architectures that can make data more accessible, available, and discoverable for real-time data ingestion (through built-in data warehouse integration) than a singularly-focused semantic layer can.
New Vision
Data warehouse vision statements commonly speak of making information accessible and easy to use, but traditionally they do so with that rarified audience of data engineers, data scientists, and sophisticated business analysts in mind. The vision associated with a modern data platform must expand the scope of these concepts to embrace data democratization and emphasize universal access to data for anyone in the organization.
This won’t just apply to data in a data warehouse but will also include disparate and diverse data from sources throughout the enterprise. Because data democratization will expand the role of the data platform from strategic and tactical to operational, the vision should consider broader opportunities to generate new revenue and drive operational efficiencies throughout the organization.
New Mission
How do you achieve this new purpose and vision? Your new mission requires specificity. A cloud-native data platform leveraging containers and Kubernetes is the only way you’ll ever be able to support the increasing number of users and their requirements in support of artificial intelligence, machine learning, streaming analytics, and other resource-intensive activities. These decision intelligence workloads can quickly strain legacy data warehouse architectures, so containers are key to enabling the elasticity that an organization needs to meet demand.
You’ll also need Kubernetes orchestration to automate the provisioning, deployment, networking, scaling, availability, and life cycle management of the containers themselves. After you dig into the specifics and determine your strategy to achieve this new mission, purpose, and vision, you will be able to better meet the needs and requirements of your user base as it continues to grow.
About the Author
As the director of product marketing at Actian, Teresa Wingfield focuses on the company’s leading hybrid cloud data solutions. Prior to joining Actian, Teresa managed cloud and security product marketing at industry leaders such as Cisco, VMware, and McAfee. She was also Datameer’s first vice president of marketing where she led all marketing functions for the company’s big data analytics solution built on Hadoop. Before this, Teresa was vice president of research at Giga Information Group, acquired by Forrester, providing strategic advisory services for data warehousing and analytics. Teresa holds graduate degrees in management from MIT’s Sloan School and software engineering from Harvard University.