New TDWI Research Report Explores Unified Data and Analytics Pipelines
Report looks at new solutions and practices that support business innovation.
SEATTLE, WA, March 25, 2022—TDWI Research has released its newest TDWI Best Practices Report: Unifying Data Management and Analytics Pipelines. This original, survey-based report focuses on the current state of DataOps and MLOps in modern organizations and highlights the trend toward unifying those two types of pipelines.
The author of the report, James Kobielus, is senior director of TDWI Research for data management. He explains that there is considerable growth potential in the adoption of unified DataOps and MLOps pipelines in today’s enterprise environments.
In the report, Kobielus points out that “both [DataOps and MLOps] leverage and extend enterprise investments in unified analytics, data engineering, data governance and curation, data science and statistical modeling, and metadata management.
Both require a high-performance pipeline in which data and models are prepared, packaged, tested, and deployed into useful applications.” He adds that both derive value from integration with the continuous integration and continuous deployment pipeline tooling often referred to as DevOps.
The report discusses in detail the methods of DataOps and MLOps and examines how these practices align with the continuous integration/continuous deployment methods more closely associated with DevOps, especially in cloud environments.
Report Highlights
This comprehensive report reveals several key findings:
- For more than half of survey respondents, ETL and related processes are the top use case for DataOps
- Roughly the same number think their DataOps pipelines aren’t structured well enough for optimal integration, governance, quality, or compliance
- Lack of standardized processes for data and analytics teams was reported as a top challenge to DataOps and MLOps pipeline management
The report also includes a case study examination of one organization’s successful pipeline unification project, which involves a unified DataOps/MLOps pipeline that integrates with a data lakehouse and implements an automated CI/CD DevOps workflow.
The report concludes with six best practice recommendations to help readers align and unify DataOps and MLOps pipelines in their organizations.
This research was sponsored by Incorta, Matillion, SAP, and Snowflake.
Download the report here.
Watch the webinar live on April 20, at 9:00 a.m. PT, 12:00 p.m. ET, and on demand afterward.
About the Author
JAMES KOBIELUS is senior director of TDWI Research for data management, focusing on data management. He is a veteran industry analyst, consultant, author, speaker, and blogger in analytics and data management. Kobielus focuses on advanced analytics, artificial intelligence, and cloud computing. Previously, he held positions at Futurum Research, SiliconANGLE Wikibon, Forrester Research, Current Analysis, and the Burton Group, and he has served as senior program director, product marketing for big data analytics, for IBM, where he was both a subject matter expert and a strategist on thought leadership and content marketing programs targeted at the data science community. You can reach him by email ([email protected]), on Twitter (@jameskobielus), and on LinkedIn (https://www.linkedin.com/in/jameskobielus/).
About TDWI
For over 25 years, TDWI has provided individuals and teams with a comprehensive portfolio of business and technical education and research about all things data. TDWI’s in-depth, best-practices-based knowledge can be quickly applied to develop world-class talent across your organization’s business and IT functions to enhance analytical, data-driven decision making. TDWI offers major conferences as well as topical seminars, onsite education, membership, certification, live webinars, resource-filled publications, industry news, and in-depth research. See tdwi.org or follow us on Twitter @TDWI.
About 1105 Media
1105 Media, Inc., is a leading provider of integrated information and media in targeted business-to-business markets, including specialized sectors of the information technology community; industrial health, safety, and compliance; security; environmental protection; and home healthcare. 1105's offerings span print and online magazines, journals, and newsletters; seminars, conferences, and trade shows; training courseware; and web-based services. 1105 Media is based in Woodland Hills, California, with offices throughout the U.S.
Media Contact:
Sabrina Bozek
Senior Director of Marketing, TDWI
[email protected]
425-277-9175