Defining Elastic Data Warehousing
Now that clouds are established IT platforms, some users are migrating their data warehouses to leverage the elasticity of clouds.
- By Philip Russom
- March 29, 2016
What is elastic data warehousing?
It's simply your data warehouse (DW) on an elastic cloud platform. This may involve a new DW or an old one migrated to an elastic cloud. Of course, hybrids are possible -- part of your DW goes on the cloud (typically data associated with volatile or disruptive workloads) while the rest remains on premises.
What is elasticity?
A cloud is elastic when it "automagically" allocates and reallocates computing resources. Imagine a DW platform consisting of an elastic cloud and a database management system (DBMS) optimized for such a cloud. The elastic cloud automatically provides the DBMS with the resources it needs when workloads spike, then reallocates those resources as workloads subside and are replaced by others. The automation of an elastic cloud greatly assists scalability and performance while reducing efforts for capacity planning and performance tuning.
Why is an elastic cloud useful for data warehousing?
Elasticity is useful in data warehousing because data volumes fluctuate dramatically in most DW platforms. For example, heavy batch-processing loads kick in after hours. As extracted data lands in a staging area, intense processing occurs and processed data is loaded into targets, most of which are data structures in the DW. Likewise, loads may spike during the business day as managers refresh their dashboards, users read reports, time-sensitive data structures are refreshed intraday, and analysts explore data for analytics.
What are the benefits of elastic data warehousing?
Elasticity provisions server resources when needed, then reallocates resources as other workloads need them. This, in turn, assures performance and scalability but also simplifies capacity planning and tweaking. In most cases, a cloud provider handles software and hardware upgrades, thereby freeing up users' time for more value-adding development tasks. Furthermore, the elasticity and automated upgrades help the data warehouse and development team be more agile and productive.
An elastic DW platform shortens the time-to-use for new or incremental data warehouse solutions. Note that system integration is done for you by the cloud provider, so initial use is mostly a matter of loading data and working with it immediately. Expanding the DW is mostly automatic without the need to wait for IT to install new servers, etc.
Cloud-based platforms and SaaS solutions are known to have relatively low licensing and maintenance costs. For example, users needn't incur capital expenses, real estate costs, and electricity bills typical of a data center because these are folded into the license fee paid to the cloud provider. Elasticity can also reduce the amount of maintenance that DBAs do and performance tweaks that developers do.
What are the common use cases for data warehousing on an elastic cloud?
Most data warehouse use can benefit from elasticity and pre-integrated cloud-base systems, ranging from traditional uses (reports) to new ones (analytics). Related disciplines can also benefit, including data management, integration, and quality.
However, TDWI sees clouds being used more for the most volatile workloads, such as incorporating new data sources (which can be unpredictable in size and processing) and broad data exploration with such new sources. Some users segregate workloads by leaving known, predictable ones on premises (usually standard reports and OLAP) while migrating data for less-known and more disruptive workloads to an elastic cloud (especially discovery-oriented analytics).
Given the affordable nature of cloud-based data systems, TDWI sees common usage by small-to-midsize businesses or budgets (SMBs). After all, an organization of limited size typically has limited resources for data management and warehousing; a cloud-based elastic DW platform provides ample infrastructure without the usual capital expense.
To learn more about elastic data warehousing, replay the archived TDWI Webinar, Demystifying Elastic Data Warehousing.
About the Author
Philip Russom is director of TDWI Research for data management and oversees many of TDWI’s research-oriented publications, services, and events. He is a well-known figure in data warehousing and business intelligence, having published over 600 research reports, magazine articles, opinion columns, speeches, Webinars, and more. Before joining TDWI in 2005, Russom was an industry analyst covering BI at Forrester Research and Giga Information Group. He also ran his own business as an independent industry analyst and BI consultant and was a contributing editor with leading IT magazines. Before that, Russom worked in technical and marketing positions for various database vendors. You can reach him at [email protected], @prussom on Twitter, and on LinkedIn at linkedin.com/in/philiprussom.