TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Considering a Cloud-based EDW: Questions You Must Ask

Enterprise data warehousing in the cloud represents a major change in thinking as well as in architecture. Addressing these critical questions can help you judge the practical value of cloud EDW for your enterprise.

• By Jake Dolezal
• June 21, 2016

Once upon a time, running an enterprise-class data warehouse (EDW) in the cloud was the stuff of fairy tales. The choice for most organizations was, and still is, an on-premises data warehouse -- typically an appliance-based platform. However, the costs of scale are gnawing away at the notion that this is always the best practice.

Appliances, for all the power and perceived control they offer companies, are often difficult to expand. Often the real estate required for sufficient power to handle the workload or storage to hold the data is too expensive. Add the complexity and in-house talent required and it becomes a difficult pill to swallow.

EDW managers and proponents are forced to answer the business-value question again; a question they thought they had answered once and for all the day the appliance was acquired. While the footing for big, expensive warehouse appliances erodes, the business need and appetite for data remains.

The cloud, once just a fairy tale, now offers an attractive option with better economics (pay as you go and easy to budget), better logistics (streamlined administration and management), and better scale (elasticity and the ability to expand a cluster with a few clicks). EDW vendor heavy hitters now have cloud offerings. Just search the Amazon Web Service Marketplace for your personal favorites -- Teradata, Vertica, Microsoft, and so on are all there with preconfigured, click-and-launch instances ready to go.

Your enterprise must still look and think before leaping into the cloud. The first challenge is to determine how a cloud-based EDW will fit in the current infrastructure and data ecosystem of your organization. Whenever you adjust your data architecture, you have to consider the impact across the board: platform, data sources, performance, windows for extraction, data integration, and so forth.

When considering and evaluating a data architecture that contains a cloud-based enterprise data warehouse solution, ask yourself:

• How will current data staging and change data capture processes fit?
• How will quality assurance, data validation, profiling, and data quality components work?
• How will our current (or planned) master data management (MDM) solution integrate with the cloud EDW?
• Can we perform the same exception handling processes, load job restarts, rollbacks, audits, and data traces?
• Can we meet data availability service-level agreements and windows for extraction while moving large chunks of data to and from the cloud?
• Can we maintain security, privacy, and encryption requirements and regulations?

These questions are only the beginning. Acquiring an enterprise data warehouse solution in the cloud is a shift in thinking as much as it is a shift in architecture. Despite its promise, skepticism will abound, even (sometimes especially) among IT folks.