Pandemic Accelerating Advanced Analytics Adoption, Survey Finds
Data access is more critical for 53 percent of respondents since the pandemic, research sponsored by Starburst and Red Hat shows.
Note: TDWI’s editors carefully choose press releases related to the data and analytics industry. We have edited and/or condensed this release to highlight key features but make no claims as to the accuracy of the author's statements.
Data access became more critical for 53 percent of surveyed organizations throughout the pandemic as analytics workloads and demands increase significantly, according to new market research commissioned by Starburst and Red Hat. The survey, conducted by Enterprise Management Associates (EMA), found bottom-line issues are driving a demand for faster data access, with 35 percent of survey respondents looking to analyze real-time business risks and 36 percent seeking growth and revenue generation through more intelligent customer engagements.
Despite the demand for data analytics, more than one-third (37 percent) of survey respondents are not confident in their ability to access timely, relevant data for decision-making. “The State of Data and What’s Next” survey revealed that this lack of confidence stems from four main challenges, Starburst notes:
- Data is stored on multiple platforms. The survey found that 52 percent of respondents have data in five or more data storage platforms. More enterprises are expected to follow this trend, with 56 percent of respondents anticipating reaching more than five platforms in the next year.
- Moving data across sources is a challenge. Many enterprises are finding the task of building and deploying data pipelines difficult. Some of the biggest obstacles are combining data in motion with data at rest (32 percent), the excessive time it takes to address break and fix (30 percent), data pipeline complexity (26 percent), and the manual coding lift for deploying error-free data pipelines (25 percent).
- Developing a data pipeline is time intensive. Building the right infrastructure for accessing data takes significant time and effort. For 45 percent of survey respondents, it currently takes more than a day to develop a data pipeline with 27 percent saying it can take anywhere from three days to two months. Making that data pipeline operational takes more time -- 52 percent of respondents said that it adds another day or more, and 24 percent say it adds another week.
- Every second counts between querying data and gaining insight. Due to rapidly changing pandemic business conditions, enterprises report having significantly less time to gain insight from their data before it is outdated. Viability of business decisions comes down to a matter of milliseconds, according to 17 percent of respondents, with 39 percent saying business decisions require latency of one second or less.
To meet these challenges, the organizations surveyed indicated they are embracing advanced analytics through a distributed SQL query engine running on a unified analytics platform.
When asked about analytics technologies, the top three answers for the most suitable modern architecture for digital decisions included:
- 34 percent advanced analytics platforms
- 27 percent unified analytics platforms
- 26 percent distributed SQL query engines
The survey found enterprises moving to specific practices and product capabilities to meet data access and analysis challenges including:
- Survey respondents reported 56 percent of their data is in the cloud and 44 percent is on-premises. Those surveyed expect to have 62 percent of their data in the cloud and 38 percent on-premises by the end of 2021. Organizations expect to be working with more than one vendor. The number one criteria for 47 percent of respondents is flexibility to access data from multiple clouds.
- Automating IT and data operations was a data strategy goal for 44 percent of respondents.
- Implementing best practices for search (32 percent) and cataloging data (30 percent) were top priorities for improving quick access to data.
- Modern analytics capabilities for data analysts were important to 42 percent of respondents. They want the capability to run a single query across relational databases, file systems, and object storage. Other sought after capabilities include support for analyzing streaming data or real-time events (38 percent), and running a single query across structured and semi-structured data (37 percent).
“The ability to store data once and query it many different times for all types of analytical workloads is critical to a successful analytics program,” concluded John Santaferraro, the author of the EMA report.
The survey report is available here.