More Than Two-Thirds of Companies Leave Valuable Data Untapped, Survey Finds
Data professionals, hampered by constant infrastructure challenges, struggle to keep up with demands from the business.
Note: TDWI’s editors carefully choose vendor-issued press releases about new or upgraded products and services. We have edited and/or condensed this release to highlight key features but make no claims as to the accuracy of the vendor's statements.
Although data-driven decision making is the number-one indicator of success for today’s businesses, a new global study conducted by Dimensional Research for Fivetran, an automated data integration provider, indicates organizations are largely unable to leverage the ever-increasing amount of data they have on hand. According to the study, 44 percent of respondents say that key data is not yet usable for decision making and 68 percent say additional business insights could be extracted from existing data if they only had more time. This is largely attributed to inefficiencies faced by data engineers, who are responsible for transforming data throughout an organization into a format that can be easily analyzed.
According to the survey:
- Nearly all participants (98 percent) have problems building pipelines to access their data
- The same percentage also say their pipelines break, with 51 percent stating it happens more frequently than once a month
- Respondents who reported pipelines breaking daily cite source availability and data schema changes as the top reasons
As a result, business suffers with data engineers spending too much time fixing and maintaining pipelines rather than focusing on higher-value tasks that generate a positive impact for the business. Almost two-thirds (66 percent) report reduced operational efficiency and 59 percent report delayed decisions or lost opportunities due to broken pipelines.
For all organizations, having immediate and reliable access to data is critical for decision making. The study indicates that pipeline challenges for data engineers continue to strain the process, and businesses lose out on valuable insights as a result.
Additional findings from the survey include:
- 79 percent of companies plan to hire more data engineers within the next year
- 72 percent of organizations move source data daily or more frequently
- 59 percent of companies use 11 or more data sources
- 91 percent say source availability problems are the reason why pipelines break daily
- 55 percent say it takes longer than one business day to repair pipelines
- 86 percent use multiple solutions to build new data pipelines
- 41 percent say a lack of data strategy (policies, processes, standardized technology) is the top data-related challenge their organization faces
For details, visit https://fivetran.com/.