TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

LESSON - Strategy and Approach for the Next-Generation Data Warehouse

• April 26, 2005

By Glenn Peipert, EVP, Chief Operating Officer, and Mark Albala, Principal Consultant, Data Warehousing Practice, Conversion Services International, Inc.

The Business Need

For most organizations, data warehouse success remains obscure. Many organizations have data warehouses that no longer align with current business requirements, that provide incomplete information, or that deliver information too late to enable organizations to make accurate and quick decisions. The result is a data warehouse that provides diminishing value at escalating cost.

Competitive pressures, evaporating profits, shrinking budgets, increasing compliance requirements, customer demands, and the need for greater operational efficiencies are forcing change. These business pressures continue to drive the need for a new and improved, next-generation data warehouse that is more strategic to the organization.

The Business Value

The information supply chain—orchestrated through the emergence of a more valuable data warehouse environment—will drive faster, more responsible decisions among customers, suppliers, and vendors through automated means. The next-generation data warehouse will provide real-time access to critical business performance metrics with actionable tasks that enhance and accelerate decision-making processes. This significantly impacts an organization by:

• Optimizing operational efficiency
• Better positioning the organization for competitive advantage
• Containing and reducing cost
• Creating a healthier top and bottom line
• Improving customer satisfaction
• Accurately measuring business performance against strategic objectives

Finally, the business value that has always been promised is on its way.

What Is the Next-Generation Data Warehouse?

Just as we witnessed the functions of manufacturing and distribution evolve to a just-in-time philosophy, IT and all its disciplines must also make the leap to just-in-time information. The worst offender has been data warehousing, which has been focused on long-term trends and, at best, near-real-time information. The new model will need to integrate real-time metrics with the context of historical information. Business activity monitoring (BAM) infrastructure will need to be integrated with the data warehouse to support this new model. The next-generation data warehouse will advance and align the strategies and objectives of both IT and the business, as well as extend and integrate information across organizational boundaries.

Are You Ready for the Next Generation?

We are experiencing the evolution of software feature/function capabilities made available through business intelligence and data warehousing (BI/DW) technologies. The macroscopic view of the BI/DW technology stack outlined in the table below provides a glimpse of what will be commonplace within the next five years.

Main focus

Early adopters of data warehousing supported business sponsors who sought general information and efficient data delivery. Companies now seek specific information, a single version of the truth, and a trusted source of information.

Data volume/data quality/ education vehicle

The concept of “trusted” information is further complicated by the fact that volumes of data are growing astronomically, and most organizations do not have a formal process or methodology to address data quality. In fact, many organizations are still in denial about the quality of their data. The creation of centers of excellence to support and elevate enterprise information quality is a new concept and one necessary to ensure the ongoing integrity of enterprise information.

Data provisioning

Data provisioning has evolved from scripting processes to the use of extract, transform, and load tools (ETL), and today may also include the use of EAI and EII tools. It is important to understand the implications of each tool and which tools are appropriate for your organization.

Audience

The data warehouse audience is maturing, and the demand for real-time information delivery is mounting. There are many ways to address those needs. Organizations will be forced make “build versus buy” decisions. Viable options include business performance management applications, business activity management applications, or building your own real-time delivery mechanism by leveraging the capabilities of your business intelligence technology stack and ETL stack.

Business intelligence approach

Future business intelligence implementations will need to support thousands—not hundreds—of users. BI suites have matured, and each product has specific capabilities that will be of interest to your organization. Determine which tools can support your flavor of business intelligence and can scale to support your particular needs.

How Do You Get There?

The CIO, with assistance from the business stakeholders, must transform the IT organization so it can reduce cost of ownership and increase contributions to business growth and efficiency. This may sound simplistic; however, advancing and standardizing the technologies within a dynamic organization can be as difficult as changing the tires on a moving 18-wheeler. A successful transformation requires an effective strategy and a multi-year plan, as well as an understanding of current and emerging technologies, operational needs, information needs, and organizational readiness.

Once transitioned to the new model of data warehousing, the next-generation tools will exist effortlessly in this new, extended enterprise. This new model will provide the environment to distribute the information necessary to deliver just-in-time knowledge from various platforms through a single user interface shared across trading partners that is sufficiently fluid in structure and content to be redirected as the challenges of business present themselves.