November 7, 2011
Data management continues to move closer and closer to real-time operation. This includes data management disciplines for data quality, data integration, master data management, and event processing. Among these, real-time data quality (RT DQ) is the second-fastest growing discipline, after master data management and just before real-time data integration.
The gradual migration toward real-time operation is the strongest trend in data management today, and the growing adoption of RT DQ is part of that trend. But why the rush to real time?
On the technology side, real-time functionality is more viable today than ever before. Moore’s Law continues to yield faster and faster CPUs, now seen in multi-core processors. CPUs aside, everything is faster and more reliable for real-time operation of DQ and other data management disciplines. This includes networks, databases, and data interfaces. In particular, Web services and service-oriented architecture give RT DQ the speed, reuse, and interfaces it needs to be embedded into a wide range of operational applications. Furthermore, the larger data sets become, the harder it is to correct, standardize, and augment them in batch. Whether big data or not, RT DQ lessens the load of traditional DQ batch processing by offloading some of it into real time.
Ultimately, RT DQ must satisfy a business need, or else it’s just a technical exercise. We all know that the pace of business is accelerating, and RT DQ helps automate the new speed of business. Fast-paced business processes demand clean and complete data—as soon as the data is created or altered—to support customer service, overnight delivery, operational BI, financial transactions, cross-sell, up-sell, and marketing campaigns. Similarly, these same fast-paced processes demand the sharing of data in real time across multiple applications with overlapping responsibilities (e.g., a customer record shared among ERP and CRM applications). For these and other situations, RT DQ reduces business risk by enabling continuous data governance and by correcting or improving information while it’s in motion in a business process.
Hence, there are many good technology and business reasons to deploy real-time data quality functions today. This TDWI Checklist drills into the details and desirable use cases for RT DQ to help user organizations understand what’s available and when to use it.