New TDWI Research Report Explores Drivers, Challenges, and Opportunities for Unified Data Warehouses and Data Lakes
Report examines the convergence of the data warehouse and data lake and how organizations are using these environments together.
SEATTLE, WA, June 9, 2021—TDWI Research has released its newest Best Practices Report: Building the Unified Data Warehouse and Data Lake. This original, survey-based report explains the modernization and evolution of data warehouse architectures, the emergence of the data lake, and their use together. It also provides best practices to help organizations plan their unification strategies.
The principal author of the report, Fern Halper, is vice president and senior director of TDWI Research for advanced analytics. She explains that enterprises are accomplishing unification in several ways, including physical consolidation and using semantic layers and virtualization. She offers several examples of how enterprises are using these technologies today and explains that data lakes are not being used as a replacement for the data warehouse but as a complementary technology.
According to the report, “The two [technologies] have recently converged to form a new and richer data architecture. The architecture is fairly new, and not many organizations have embraced it yet.” The report explores the advantages of this convergence and how enterprises can maximize its benefits.
The report’s coauthor, James Kobielus, senior director of TDWI Research for data management, adds that nearly two-thirds of survey respondents said the point of the unified data warehouse/data lake is to get more business value from data, whether in operations or analytics.
Report Highlights
This comprehensive report reveals:
- Top value drivers of the new architecture include unifying silos (53%), providing a better foundation for analytics of new and traditional data types (49%), and storage and cost considerations (28%)
- When consolidating the technologies, enterprises may implement tools (such as data pipelines, data catalogs, and business glossaries) as well as data disciplines (including data governance, master data management, and metadata management)
- Artificial intelligence’s many use cases are the principal driver behind the evolution of data warehouses into unified data warehouse/data lakes
- Over a third (36%) of respondents report having either a data warehouse or a data lake in the cloud, indicating their mainstream acceptance
- Modern software tools (even those not specifically designed for data governance) can assist with important practices and help scale data governance functions across the enterprise
Today’s drive to unify data warehouses and data lakes is focused on the need to enable data integration, interoperability, and cross-platform processes. The TDWI report examines these drivers and explores how enterprises view these demands as well as potential barriers that inhibit enterprises from successfully unifying their data warehouses and data lakes.
The report concludes with 8 recommendations for successful unification and comments about why each is important.
Platinum sponsors of this report include Denodo, Dremio, Matillion, Snowflake Computing, Trifacta, and Vertica. Qlik was a gold sponsor of the research.
Download the report here
Watch the webinar on June 22, 2021, 9:00 a.m. PT, 12:00 p.m. ET
About the Authors
FERN HALPER is VP and senior director of TDWI Research for advanced analytics, focusing on predictive analytics, social media analysis, text analytics, cloud computing, and other big data analytics approaches. She has more than 20 years of experience in data and business analysis, and has published numerous articles on data mining and information technology. Halper is coauthor of “Dummies” books on cloud computing, hybrid cloud, service-oriented architecture, and service management and Big Data for Dummies. She has been a partner at industry analyst firm Hurwitz & Associates and a lead analyst for Bell Labs. Her Ph.D. is from Texas A&M University. You can reach her at [email protected], @fhalper on Twitter, and on LinkedIn.
JAMES KOBIELUS is senior director of TDWI Research for data management. He is a veteran industry analyst, consultant, author, speaker, and blogger in analytics and data management. Kobielus focuses on advanced analytics, artificial intelligence, and cloud computing. Previously, he held positions at Futurum Research, SiliconANGLE Wikibon, Forrester Research, Current Analysis, and the Burton Group, and he served as senior program director, product marketing for big data analytics, for IBM, where he was both a subject matter expert and a strategist on thought leadership and content marketing programs targeted at the data science community. You can reach him by email ([email protected]), on Twitter (@jameskobielus), and on LinkedIn.
About TDWI
For over 20 years, TDWI has provided individuals and teams with a comprehensive portfolio of business and technical education and research about all things data. TDWI’s in-depth, best-practices-based knowledge can be quickly applied to develop world-class talent across your organization’s business and IT functions to enhance analytical, data-driven decision making. TDWI offers major conferences as well as topical seminars, onsite education, membership, certification, live webinars, resource-filled publications, industry news, and in-depth research. See tdwi.org or follow us on Twitter @TDWI.
About 1105 Media
1105 Media, Inc., is a leading provider of integrated information and media in targeted business-to-business markets, including specialized sectors of the information technology community; industrial health, safety, and compliance; security; environmental protection; and home healthcare. 1105's offerings span print and online magazines, journals, and newsletters; seminars, conferences, and trade shows; training courseware; and web-based services. 1105 Media is based in Woodland Hills, California, with offices throughout the U.S.