The Butterfly Effect on Data Quality
By Jim Walker, Director of Product Marketing, Talend
The term butterfly effect refers to the way a minor event, such as the movement of a butterfly’s wings, can have a major impact on a complex system, such as weather. The minor event represents a small change in the initial condition of a system, but it starts a chain of events that have a profound effect on the larger ecosystem.
Enterprise data is equally susceptible to the butterfly effect. When poor-quality data enters the complex system of enterprise data, even a small error can lead to revenue loss, process inefficiency, and failure to comply with regulations. Data issues often begin with a tiny mistake in one part of the organization, but the butterfly effect can produce disastrous results.
How a Small Data Error Becomes a Big Problem
When data moves within an organization, it affects each system in different ways. The more interconnected systems you have, the more potential there is for escalating data quality issues. Without some kind of data management strategy that includes data quality, your company is likely to suffer from multiple ongoing data problems.
Something as simple as a transposed character has lasting effect. For example, if two call center workers enter the same customer address as “25 Main St.” and “25 Mian St.,” it could impact your ability to market to that customer, or lead to a costly mailing mistake.
At the highest level, data quality issues affect:
- Revenue: Without accurate customer data, your organization can’t achieve revenue goals. Poor data quality most often affects your ability to reach customers, meet their needs and desires, and cross-sell and up-sell.
- Efficiency: Untrustworthy data results in wasted time and resources, as you are forced to check and recheck facts and figures before you make decisions. It also prevents data from being easily shared by others in your organization.
- Compliance: Poor-quality data affects your ability to comply with industry and local laws, such as Sarbanes-Oxley, Basel II, Do Not Call, and HIPAA. Lack of compliance can lead to unnecessary fines and levies.
The Pervasiveness of Data
When data enters the corporate ecosystem, it rarely stays in one place. Consider the typo in the customer address as it travels throughout your enterprise. Marketing accesses the data in the CRM system to reach customers. A successful campaign results in orders, which have an impact on shipping, billing, supply chain, support, and other systems. Finally, they are reported on in a warehouse. If the data enters the ecosystem as incomplete, incorrect, or duplicate, many systems are affected.
Benefits Across Systems of Clean Data
All your systems can benefit from clean data. For example, let’s look at CRM, ERP, and a data warehouse:
- CRM: With clean customer data, you can offer personalized customer service and obtain a more complete understanding for cross-sell and up-sell opportunities.
- ERP: With clean supply chain data, you will have a better picture about delivery times on orders, and be more efficient. You can avoid unnecessary warehouse costs by holding the right amount of inventory in stock.
- Data warehouse: Clean data creates effective analytics. Using the clean data in a warehouse can help you find trends, see relationships in data, and understand the competition in a new light.
Components of an Effective Data Quality Program
To be successful with data quality, your solution should address the problem in both real time and in batch. It must also provide these basic functions:
- Profile and monitor: Provide insight into initial problems and monitor quality
- Standardize: Tools to standardize and improve basic quality issues, such as nulls and formats
- Augment: Ability to cross-reference data against a trusted, accurate third-party source for validation or improvement
- Match and survive: Identify duplicate data and remove or survive a quality set of records no matter the domain
This article originally appeared in the issue of .