Revefi Research: Bad Data Wreaking Havoc
Survey shows why data teams need a trusted copilot to manage data quality, performance, usage, and costs.
Note: TDWI's editors carefully choose press releases related to the data and analytics industry. We have edited and/or condensed this release to highlight key information but make no claims as to its accuracy.
New research from Revefi reveals that bad data costs companies significantly and in many ways. Forty percent of the more than 300 IT directors, data and analytics managers, and other IT professionals surveyed said they encounter 11 to 100 data incidents per month. Sixty-five percent of the group said bad data delays processes. More than 75% said it is somewhat, very, or extremely difficult to manage data warehouse spending, which is especially problematic right now as companies work to do more with less.
“Data quality and management issues are on the rise, and that’s a costly problem for businesses,” said Sanjay Agrawal, CEO and co-founder of Revefi. “It leaves them spending too much time manually identifying root causes of data issues, and it creates delays, wastes money, leads to poor decision-making, and reduces customer trust and the accuracy of AI models.”
More than a quarter of those surveyed said detecting most data incidents takes up to eight hours. A tenth of the group said identification can take days or even more than a week. In addition to the time it requires, manually identifying problems is also enormously resource-intensive, because finding the root cause often requires the involvement of multiple people across teams.
That’s just uncovering the source of the problem. You still need to fix it. Yet 43% of survey respondents said it takes more than 48 hours to resolve a data incident after discovering it.
Half of survey respondents from the manufacturing sector and 60% of IT professionals in education admitted that it takes them more than 48 hours to resolve data incidents. All respondents in energy, oil, and gas said they typically must dedicate more than 48 hours to resolve a situation stemming from bad data.
Bad Data and Other Data Challenges Can Have Adverse Consequences
Fifty-eight percent of IT professionals surveyed said that data quality and cleanliness are the most significant challenges they face when working with data. Nearly as many (57%) revealed they have encountered inaccurate data.
The same share (57%) said bad data has led to poor decision-making. Nearly as many (56%) said they believe bad data reduces the accuracy of AI model performance. That is concerning considering the very high use of AI models in recent months.
Half of survey respondents said managing their data warehouse spending is difficult. Bad data also can erode data users’ trust and work against company efforts to build a data culture. Indeed, 40% of respondents said they believe bad data reduces customer trust.
Data Quality Is Critical to AI Model Training, Ensuring Ethical AI Development
As the adoption of AI grows and more organizations rely on AI models to automate more decisions and processes, the need for high-quality data takes on even greater importance.
That said, it’s troubling that 43% of respondents said they have experienced negative consequences due to poor data quality in AI projects. It’s also concerning that more than half (52%) of IT pros only somewhat trust the data sources being used to train AI models.
However, there’s also some good news. A whopping 70% of IT professionals believe that addressing data quality issues is important from an ethical standpoint in AI development.