November 7, 2011
Data management continues to move closer and closer to real-time
operation. This includes data management disciplines for data
quality, data integration, master data management, and event
processing. Among these, real-time data quality (RT DQ) is the
second-fastest growing discipline, after master data management
and just before real-time data integration.
The gradual migration toward real-time operation is the strongest
trend in data management today, and the growing adoption of RT DQ
is part of that trend. But why the rush to real time?
On the technology side, real-time functionality is more viable today
than ever before. Moore’s Law continues to yield faster and faster
CPUs, now seen in multi-core processors. CPUs aside, everything
is faster and more reliable for real-time operation of DQ and other
data management disciplines. This includes networks, databases,
and data interfaces. In particular, Web services and service-oriented
architecture give RT DQ the speed, reuse, and interfaces it needs
to be embedded into a wide range of operational applications.
Furthermore, the larger data sets become, the harder it is to correct,
standardize, and augment them in batch. Whether big data or not,
RT DQ lessens the load of traditional DQ batch processing by offloading
some of it into real time.
Ultimately, RT DQ must satisfy a business need, or else it’s just
a technical exercise. We all know that the pace of business is
accelerating, and RT DQ helps automate the new speed of
business. Fast-paced business processes demand clean and
complete data—as soon as the data is created or altered—to
support customer service, overnight delivery, operational BI,
financial transactions, cross-sell, up-sell, and marketing campaigns.
Similarly, these same fast-paced processes demand the sharing
of data in real time across multiple applications with overlapping
responsibilities (e.g., a customer record shared among ERP and
CRM applications). For these and other situations, RT DQ reduces
business risk by enabling continuous data governance and by
correcting or improving information while it’s in motion in a
business process.
Hence, there are many good technology and business reasons to
deploy real-time data quality functions today. This TDWI Checklist
drills into the details and desirable use cases for RT DQ to help user
organizations understand what’s available and when to use it.