Executive Summary | Modernizing Data and Information Integration for Business Innovation
Executive summary for the TDWI Best Practices Report: Modernizing Data and Information Integration for Business Innovation
- By David Stodder
- November 17, 2021
Data and higher-level information integration are the foundation for reaching data-driven objectives. Getting complete, quality views of data about subjects of interest is essential to nearly every business decision or action. For analytics and data exploration, people across enterprises need timely access to new and diverse data that is integrated, transformed, and managed, not in a one-size-fits-all fashion, but flexibly to fit diverse types of data, knowledge requirements, and workloads. This puts pressure on organizations to modernize data and information integration, including by taking advantage of advances in artificial intelligence (AI) and automation embedded in tools and technologies. Organizations also increasingly need cloud services that supply powerful, modern computational processing for higher speed, scale, and agility.
This TDWI Best Practices Report focuses on strategies for solving data and information integration challenges. Organizations want to increase user satisfaction, support flexibility and innovation, and enable people to collaborate effectively to achieve business objectives such as operational efficiency, richer customer engagement, growth, resilience, and risk mitigation. The report examines business and technology trends and drivers; we analyze research data about challenges in using current solutions and explore where organizations need to modernize.
The report finds that organizations need to address interrelated issues to advance with data and information integration. They need to support expanding data visualization and analytics performed by data-savvy business users who want self-service power and flexibility for data access, blending, and preparation—and want to make decisions based on deeper knowledge of data relationships.
At the same time, organizations are looking for solutions, especially in the cloud, that support self-service expansion but offer cost efficiency, governance, and fewer headaches due to data fragmentation. This report finds that analytics as well as AI and machine learning (AI/ML) development are important drivers behind modernization. Organizations need smart automation to reduce latency in data pipelines and integration processes so they can provision fresher or real-time data for analytics, AI/ML, operational dashboards, and embedded intelligence.
Organizations face challenges stemming from disparate data silos, confusion about data definitions and master data, and difficulty discovering and analyzing complex data relationships. Traditional data and information integration systems, designed for a defined set of business processes, can limit the value people derive from new data types that do not fit easily into standard samples and aggregations. This report looks at common pain points and discusses strategies for solving them and moving beyond legacy constraints.
Relevant solutions discussed include data virtualization, data catalogs, knowledge graphs, master data management, data fabrics, and data silo consolidation into unified cloud data management. Rather than competing, solutions often complement each other within a larger data strategy to enhance agility so that organizations can handle both routine and unanticipated, ad hoc workloads. Building deeper knowledge about the data including its location, lineage, and relationships not only helps people find data faster and gain comprehensive views; the knowledge is critical to data governance and adherence to regulations.
This TDWI Best Practices Report offers recommendations that highlight priorities for modernizing data and information integration for business advantage.
Actian, Denodo, SnapLogic, Stardog, and Trifacta sponsored the research and writing of this report.
About the Author
David Stodder David Stodder is an independent data and analytics industry analyst. Previously, he was senior director of research for business intelligence at TDWI, where he spent more than 13 years. Stodder focuses on providing research-based insights and best practices for organizations implementing BI, analytics, AI, data intelligence, data integration, and data management. He has been a thought leader in the field for over three decades as an industry analyst, writer, and speaker. He was the founding chief editor of Intelligent Enterprise where he also served as editorial director for nine years. Stodder is a TDWI research fellow.