LESSON - The Need for Proactive Data Quality Management
By John “JT” Taylor, Chief Technology Officer, iWay Software
Today’s business transactions are highly complex, generating information in multiple ways and causing it to flow into and throughout an organization at a rapid pace. While this has optimized efficiency, it has also created major data quality problems.
Organizations are finally taking data quality seriously, implementing policies and tools to correct the invalid information that plagues their enterprise systems. However, most data quality initiatives take a reactive approach, when in fact only proactive quality control will fully ensure the integrity of all data, at all times.
Where It Starts
There’s no single cause of data quality problems. Although manual data entry used to be a key contributor, new inbound information channels such as automated business-tobusiness (B2B) interactions and Web portals play a major role today. These next-generation sources deliver information that is more sophisticated, yet harder to harness, and only a real-time “data quality firewall” can mitigate the risks.
There Is No Such Thing as a Small Data Quality Problem
It takes just one piece of corrupt information to create monumental issues. Bad data multiplies at an astonishing rate, polluting not only the system in which it originates, but also the many other information sources it touches as it moves across a business. Therefore, the longer a company waits to detect and correct a bad record, the more damage it can do.
This is why taking a reactive approach to data quality, instead of a proactive one, can be an expensive decision. In a recent study, independent analyst firm SiriusDecisions notes what it calls the “1-10-100 rule,” which demonstrates the benefits of proactive data quality. The rule states that it costs only $1 to verify a record upon entry and $10 to cleanse and dedupe it after it has been entered, but $100 in potential lost productivity or revenue if nothing is done.1
What You Need in a Data Quality Management Solution
When choosing a solution to support your data quality strategy, there are two key features to look for. First is the ability to protect agnostic sources. Many companies approach data quality only from the perspective of their own internal systems. However, a lot of data comes from outside corporate walls; it is collected from applications maintained by partners, or aggregated from various Web sites. Failing to consider these sources can leave huge gaps and create an environment fraught with risk.
The second capability—and perhaps, the most critical one—is real-time quality control. Yes, it’s important to identify and correct any bad data that already exists. But quality must also be controlled as data flows instream, moving from system to system during the course of dynamic processes, or as it moves downstream as users access it for reporting and analysis. The key to truly keeping enterprise data “clean” is the ability to stop corrupt information as it heads upstream and enters the environment via various methods and formats, such as e-mail, manual data entry, B2B exchanges, and Web and self-service portals.
Only the most advanced tools on the market today can proactively control quality across all upstream, instream, and downstream data, using predefined rules and localized dictionaries to catch corrupt information as it is generated, stopping it before it reaches a database.
Summary
Data quality issues impact businesses of all types. Regardless of their cause, these problems cost billions of dollars each year. The longer they go undetected, the more damage they can do. Companies must leverage real-time quality control to not only correct existing records of subpar quality, but to also stop bad data from entering the environment in the first place.
1 “The Impact of Bad Data on Demand Creation,” by Jonathan Block, senior director of research, SiriusDecisions.
For a free white paper on this topic from iWay Software, click here and choose the title “Optimizing Data Quality in the Enterprise: How to Tackle Your Bad Information.” For more free white papers, click here.