DataOps: Industrializing Data and Analytics
January 1, 2019
DataOps is an emerging set of practices, processes, and technologies for building and enhancing data and analytics pipelines to meet business needs quickly. As these pipelines become more complex and development teams grow in size, organizations need better collaboration and development processes to govern the flow of data and code from one step of the data lifecycle to the next—from data ingestion and transformation to analysis and reporting.
DataOps builds on concepts popular in the software engineering field, such as agile, lean, and continuous integration/continuous delivery, but addresses the unique needs of data and analytics environments, including the use of multiple data sources and varied use cases that range from data warehousing to data science.
For large organizations with big development teams, DataOps is an antidote to many of the woes that beset IT and development organizations.