Data Warehouse Directions: Plot a Realistic, Modifiable Course

Michael A. Schiff
Principal Consultant, MAS Strategies

Topic: Experts in Data Warehousing

Which of the following best describes your organization's approach to data warehousing?

A) "Set a course and stick to it" and "Steady as she goes"

or

B) "Recalculating, Recalculating"

Perhaps it should be a combination of both. After all, an often-heard cliché is that a data warehouse is an ongoing journey, not a point destination. Sometimes the most interesting and exciting parts of a journey consist of the side trips we take. Unfortunately, we don't have a GPS-like device to advise us when our data warehouse is no longer meeting the needs of our organization. We should, however, recognize that an organization's data warehousing requirements will vary over time and proactively work with our users to ensure that our data warehousing efforts continue to meet the current and anticipated needs of our organizations.

I believe that one of the most important characteristics of a successful data warehousing program is that it proactively encourages user feedback, both positive and negative. In my experience, many users can't really anticipate their true requirements until after they receive their first taste of the benefits that a data warehouse and business intelligence can provide. This is one of the reasons that prototyping has proven to be so useful. Analysis is an iterative process and most of us will think of additional questions after the results of an initial request are presented.

It is almost impossible to build a data warehouse that will meet everyone's needs when it is first implemented. Over the years, however, I have observed many efforts that tried to do just that; unfortunately, so much time was spent attempting to evaluate and juggle requirements that when the warehouse was ultimately implemented, it was months, if not years, too late.

Building an all-encompassing enterprise data warehouse all at once is akin to trying to boil the ocean; it's simply not very realistic. Another approach many organizations take is to build an enterprise data warehouse in incremental phases instead. Although this is certainly a legitimate technique, I have one caution: make sure you allow for deviation from the original plan. I have observed many occasions when, after the first phase was implemented the users almost immediately requested other features such as new data sources, additional levels of detail, or reports and analyses that were not in the original incremental plan. In some of these situations, the users were advised that, although their requests had merit, they would only be considered after all of the originally planned phases were completed!

Your data warehouse direction should not be limited to a single platform choice. I have been a long-time advocate of an overall data warehouse architecture encompassing a variety of data warehouse platforms. These platforms include data marts, operational data stores, and data warehouse appliances, in addition to enterprise data warehouses. In some situations, it may be more realistic to use special-purpose data warehouse appliances rather than to add new content, or users with compute-intensive requirements, to an existing enterprise data warehouse .

The bottom line is that an organization's data warehousing direction should allow for change based on evolving user requirements and company growth. Survey users on items such as additional data sources (external and internal), report formats, analysis, and the need for historical values. An excellent question to ask is: "What information did you not have last year that, if you did have it, would have made your job easier, reduced costs, or identified new opportunities?"

You should also monitor report usage to determine what analyses and data sources may no longer be needed and should be removed or archived. One of my data axioms -- "Data in a warehouse is like the clothes in a closet; even if we haven't accessed some data in two years, we still tend not to throw it away" (see note below) -- is well worth remembering when evaluating your data warehouse content and direction. It might also be worthwhile to establish a data warehousing steering committee to ensure that your data warehousing efforts follow your organization's potentially changing course. If you do so, be sure to include several different layers of management and staff -- people that use the analyses for both operational and analytical needs.

Note: Axiom from Michael A. Schiff, "Data Management: Observations from a Practitioner." TDWI Experts (September 9, 2009). Based on a collection of data axioms I created and presented at several industry conferences and vendor events.

Michael A. Schiff is a principal consultant for MAS Strategies. He can be reached at [email protected]

Vendor Q & A - Quiterian

Answer provided by Al Saavedra, Vice President of Sales U.S., Quiterian.

Q: What are the key differences between traditional BI and self-service BI?

A: The development cycle for new reports, dashboards and scorecards is typically measured in weeks or months when using traditional BI tools. This creates a backlog of requests by business users. These users offer input about the content of reports only at the beginning or at the end of development cycle. This fragmented conversation between business users and IT also tends to produce less-than- satisfactory results.

With self-service BI, non-technical business users can gain deep insights about the data they need to analyze almost instantly through an easy drag-and-drop, Web based, graphical interface on their own without IT involvement. This process taps directly into the business knowledge and imagination of decision makers and analysts at all levels of the organization.

This direct and instant interaction between business users and data is compatible with decision making in a dynamic and competitive business climate.

Copyright 2011. TDWI. All rights reserved.