Study Finds Three Out of Four Executives Lack Confidence in Their Data's Quality
New research from Trifacta reveals what's holding back analytics adoption, quantifies data prep tasks, and examines cloud use.
- By James E. Powell
- January 30, 2020
A new report from Trifacta, a data wrangling vendor, shows that though the cloud has gained wide acceptance in helping organizations modernize their analytics, many problems remain.
"Obstacles to AI & Analytics Adoption in the Cloud" -- which surveyed 646 data professionals across industries and job titles in late August, 2019 -- asked respondents about their organizations moving data to the cloud, what's hampering their data cleaning efforts, and what burdens they face in prepping data for analytics, AI, and machine learning (ML) projects.
Data Quality Inhibits AI Projects
Data prep remains a big burden for survey participants. Almost half (46 percent) of data scientists are spending more than 10 hours preparing data for an analytics and AI/ML initiatives. The problem is that such prep work is critical -- 71 percent of respondents say their organizations rely on data analysis to drive future business decisions and 59 percent say poor data leads to miscalculating demand. More than one in four say bad data will cause an enterprise to target the wrong prospects. Respondents said that deduplication (21 percent), data validation (also 21 percent), and analyzing relationships between fields (20 percent) are the "most likely steps to improving data accuracy."
The survey shows there is good reason for enterprises to be concerned about data prep: only a quarter (26 percent) said their own data was "completely" accurate before preparation and cleaning; 42 percent said their data was "very accurate." That means nearly a third of respondents have serious data quality issues.
The C-suite is taking notice. Although 76 percent of C-suite executives "have AI and machine learning initiatives included in their company's road map," three-quarters of them are not confident in the quality of their data.
Other consequences of poor-quality data cited include analytics and AI/ML projects taking longer (38 percent), costing more (36 percent), or failing to achieve the expected results (33 percent).
Quality isn't the only data issue. Only 14 percent say they have access "to all the data sources they need."
Cloud Adoption Driven by AI, ML
Enterprise adoption of the cloud for AI and machine learning is confirmed in the survey: two-thirds (66 percent) of respondents say that all or most of their analytics and AI/ML initiatives are running in the cloud, and 69 percent are using the cloud for data management. Nearly as many (68 percent) IT pros are storing most or all of their data in the cloud today, a trend respondents expect to grow to 88 percent within two years. Benefits of AI and ML in the cloud include accessibility between environments (51 percent), increased efficiency (49 percent), and cost reduction (48 percent), which is likely why 61 percent of respondents say AI/ML initiatives are part of their company's road map.
Of those using cloud storage, it's a close race between platforms: Google Cloud is the most popular (used by 53 percent of respondents), followed by Microsoft Azure (48 percent) and Amazon Web Services (46%).
"The growth of cloud computing is fundamental to the future of AI, analytics, and machine learning initiatives," said Trifacta CEO Adam Wilson in a company statement. "Unfortunately, the pace and scale at which this growth is happening underscores the need for coordinated data preparation because data quality remains one of the largest obstacles in every organization's quest to modernize their analytics processes in the cloud."
The full report is available here; registration is required.
About the Author
James E. Powell is the editorial director of TDWI, including research reports, the Business Intelligence Journal, and Upside newsletter. You can contact him
via email here.