Why Data Warehouse Modernization Must be Coordinated with Other Modernization Projects
Technology modernization rarely occurs in a vacuum. It usually supports or is driven by business modernization.
- By Philip Russom
- April 17, 2017
One of the hottest trends in data warehousing (DW) is modernization -- where DW professionals upgrade, redesign, and re-implement warehouses to give them future-facing capacity, speed, interoperability, and analytics.
We talk about (and even perform) data warehouse modernization as if it were an isolated project with isolated goals, but the reality is just the opposite. Data warehouse modernization is, in fact, usually one of many attempts at modernization that occur concurrently and have project dependencies. Here are examples of dependent modernizations you must coordinate with your data warehouse modernization.
Business Modernization
In an ideal world, upper management leads the way by deciding how to modernize the business to keep pace and stay relevant with evolving customers, partners, marketplaces, and economies. Business modernization and its goals are, in turn, articulated "down the org chart."
At some point in that process, people in IT and similar groups (such as a data warehouse group) should collaborate with business managers to determine how data, applications, and technology can support the stated business modernization by thinking globally but acting locally. Even if you do not work in an ideal world, some semblance of that process should still be present to guide your alignment of warehouse modernization with business modernization.
Analytics Modernization
Online analytical processing (OLAP) continues to be the most common analytics method, and it's too valuable to replace or abandon. Instead, analytics modernization tends to introduce additional analytics methods that an organization has not deployed before, typically so-called advanced analytics, which are based on technologies for mining, clustering, graph, statistics, and natural language processing (NLP).
Often, new analytics are needed to support business modernization, such as when your organization wants to compete using analytics, improve operational excellence via analytics, and make decisions based on facts and analyses (whether the decisions are strategic, tactical, or operational).
Similarly, data warehouse modernization can be driven by analytics modernization because most warehouses were built for reporting and OLAP and therefore need to be extended or redesigned to accommodate the new data requirements of advanced analytics.
Data Platform Modernization
Technologies come, technologies go, but the data and the warehouse carry on. In TDWI's definition, a data warehouse has three characteristics: it is (1) a data architecture with attendant data models, etc. that (2) are populated with data and (3) organized via metadata, indices, and other semantic mechanisms. By definition, the data warehouse and its underlying server platforms are separate and can be modernized separately.
Warehouse professionals have repeatedly migrated warehouse data and related pieces from SMP to MPP hardware, from 16-bit to 32-bit to 64-bit CPUs, from one vendor brand to another, and from server boxes to racks, grids, and clusters. Whether you realize it or not, these are data platform modernizations, driven by new requirements for scale, speed, price, and future-proofing.
More often than not, modernizing warehouse data (to embrace dimensionality, real time, unstructured data, and detailed sources for analytics) may depend on data platform modernization for appropriate storage, capacity, interfaces, in-place processing, and multistructured data support. This is why modern data warehouses are still logical data architectures at heart, although the data is physically distributed across an increasing number of platform types, including new ones such as those based on columns, clouds, appliances, graph, complex event processing, and Hadoop.
Report Modernization
The style of reports has evolved dramatically since the early 1990s. Back then, reports were only on paper and consisted of one giant table of numbers after the next. Because a single report served dozens of user constituencies, the content of each report was mostly irrelevant to individual report consumers.
Luckily, waves of modernization have greatly improved reports, bringing them online (for greater distribution and ease of use, plus drill-down), giving them a visual presentation (for interpretation at a glance), organizing them around metrics and KPIs (in support of performance management methods), and personalizing them so users go straight to what they need (for productivity and relevance).
The majority of data warehouses continue to be designed by users and deployed mostly in support of reporting and OLAP. As the style of reporting has evolved, warehouse data structures have had no trouble modernizing to keep pace with report change. More dramatic change is seen in users' portfolios of tools for reporting, which still include older enterprise reporting platforms but are now augmented with newer tools for dashboarding, data visualization, and data exploration.
[Editor's note: To learn more about data warehouse modernization and its coordination with other modernizations, attend the TDWI Chicago Leadership Summit in Chicago May 8 and 9, 2017.]
For Further Reading:
About the Author
Philip Russom is director of TDWI Research for data management and oversees many of TDWI’s research-oriented publications, services, and events. He is a well-known figure in data warehousing and business intelligence, having published over 600 research reports, magazine articles, opinion columns, speeches, Webinars, and more. Before joining TDWI in 2005, Russom was an industry analyst covering BI at Forrester Research and Giga Information Group. He also ran his own business as an independent industry analyst and BI consultant and was a contributing editor with leading IT magazines. Before that, Russom worked in technical and marketing positions for various database vendors. You can reach him at [email protected], @prussom on Twitter, and on LinkedIn at linkedin.com/in/philiprussom.