TDWI Executive Program:
The State of Data Management

Prerequisite: None

James G. Kobielus

Senior Director of Research for Data Management

TDWI

The pace of innovation in data management continues to astonish. New generations of data platforms, tools, and applications are being adopted, and enterprises continue to evolve their practices to derive the full value from these innovations.

TDWI’s senior research director James Kobielus will discuss the future of data management methodologies, technologies, platforms, and tools. The key trends that he will discuss include:

• Unified data observability: Enterprises are making unified observability central to their data management practices, using it to identify and resolve data quality issues, detect data incidents at scale, fine-tune alerts to intelligently notify data team owners and impacted downstream consumers when incidents occur, and provide field-level lineage to enable data engineering teams to triage and resolve incidents.
• Automated data engineering: Enterprises are expanding implementation of automated ELT that leverages no-code, visual, and AI/ML-driven tooling involving less-technical business professionals, the “citizen data engineers.”
• Synthetic data preparation: Enterprises are shifting away from traditional data preparation toward synthetic data generation, which is the production of data that is statistically equivalent to actual data and addresses many privacy, anti-bias, and machine learning operationalization challenges.
• Comprehensive data streaming: Enterprises are accelerating migration from batch data management architectures to comprehensive cross-enterprise data streaming architectures in support of continuous, real-time, and low-latency data analysis.
• Adaptive data permissioning: Enterprises are intensifying their use of intelligent data permissioning tools that alert teams to risky data access behavior, support continuous discovery of sensitive data, and enable more adaptive management of attribute-, purpose-, and role-based data access controls.
• Stringent data compliance: Enterprises are implementing more stringent controls on data management auditing, archiving, protection, security, and quality, commensurate with increasingly stiff legal penalties for noncompliance being imposed across more fragmented regulatory regimes associated with local and regional data sovereignty regulations.
• Responsible data governance: Enterprises are expanding their adoption of domain-specific data stewardship practices associated with new responsible governance mandates, including ethics, accountability, safety, and sustainability.
• Flexible data mesh: Enterprises are shifting their data management architectures toward domain-oriented, distributed, and federated approaches that facilitate more flexible cross-domain discovery, sharing, governance, and productization of business data.