TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Advanced Analytics: A Look Back at 2021 and What's Ahead for 2022

Organizations will improve data literacy and try to address ever-increasing complex environments next year.

• By Fern Halper
• December 22, 2021

In 2021, as part of the move to collect and manage new data types and to grow analytics initiatives (spurred on by the COVID-19 pandemic), organizations began moving to the cloud more rapidly. Multicloud platforms became more popular, and vendors hyped unified platforms, especially in the cloud. The idea of the data lakehouse also gained steam in 2021 as organizations looked to utilize the features and functions of both a cloud data warehouse and a cloud data lake to support diverse data types and some new analytics use cases.

For Further Reading: How to Develop a Data-Literate Workforce How MLOps Delivers Business Value Unifying DW and DL: 6 Key Questions

As organizations become more analytically advanced, they are wrestling with two interrelated goals: dealing with the complexity of their data environments while increasing data literacy in order to grow analytics adoption. According to TDWI research, self-service data visualization and machine learning (ML) continue to be priorities among organizations. Yet the percentage of survey respondents who say that they've implemented self-service analytics such as visual data discovery has increased only slightly. Likewise, if users had stuck to their plans regarding ML, the adoption rate would be about 75 percent, yet we only see about 25 percent of respondents to our surveys implementing machine learning.

Although some of this may be due to COVID-19 and the issues changing data may have had on building models, there are also other factors at work, including the lack of skills and talent, the absence of a trusted data infrastructure, and the lack of resources and support.

In 2022, organizations will need to address these challenges if they hope to move forward in their data and analytics efforts. This will involve both automated and augmented technology and new organizational teams and processes.

Trend #1: Data literacy will remain a key priority in 2022

Companies want to become more advanced in analytics in order to better compete. Yet, they are struggling with both keeping talent in house and building new talent to perform more advanced analytics. In other words, organizations need to build literacy to utilize tools such as self-service BI, and they need to either retain or grow talent to move forward with data science.

Data literacy was a priority in 2021 and it will continue to be in 2022. Data literacy involves how well users understand and can interact with data and analytics and communicate the results to achieve business goals. It includes understanding data elements, understanding the business, framing analytics, critical interpretation, and communication skills. As part of this, we expect to see more organizations building literacy enablement teams to help educate their people.

We expect to see modern analytics tools with more advanced and augmented features such as natural language search and the ability to surface descriptive insights becoming more popular. In terms of data science talent, we expect to see more organizations training their business analysts to become data scientists. This may involve the use of automated tools, such as autoML (tools that automatically generate model features and build machine learning models), but in a controlled way. Explainability features (and other guardrails) will continue to be an important part of vendor solutions in 2022.

Trend #2: Organizations will try to better manage the complexity of their environments

As organizations move to cloud, multicloud, and hybrid environments to support diverse data types for analytics, they will need to manage this complexity. We expect to see more tools on the market that will support cloud data management, security, and governance. Many of these tools will be automated and augmented. For instance, newer tools for managing security across multiple environments (e.g., via a single pane of glass) will become more popular in 2022.

Likewise, tools for multienvironment data governance will become more prevalent. These tools include capabilities for centralizing and updating policies across cloud and hybrid environments and augmented capabilities such as identifying suspicious patterns in data access and generating alerts or halting a query. The complexity of today's environments will demand the use of automation and augmented tools.

Additionally, organizations will continue to try to unify their environments to provide a trusted data source to their users. This may involve physical or logical unification. New skills will be required here, as well. As vendors provide more structure to cloud data lakes, along with the ability to support analytics tools such as SQL or data science notebooks, we expect to see adoption of the cloud data lake growing in 2022 as an important part of this architecture to support modern analytics.We also expect to hear more about the lakehouse and the idea of the data mesh for unifying environments.

Trend #3: MLOps and data ethics will become more important

As organizations begin to build more machine learning models, they will come to realize that it isn't just the front end of the process that is important; the back end (i.e., productionalizing the model) is the part that can make or break the effort. As organizations increase the number of models in production, they will need new roles including MLOps. This is the highly technical team responsible for model management and monitoring. They are responsible for validating, registering, deploying, monitoring, and retraining models in production. Although many data scientists are responsible for this today, expect organizations to look more seriously into hiring for MLOps and utilizing MLOps tools in 2022.

Additionally, as they make more inroads into analytics productivity, organizations will become more interested in issues such as data ethics and model governance. Data ethics involves the right and wrong uses of data; it is especially important in machine learning. Likewise, governing models will become more front and center as organizations grow the number of models in production.

The Bottom Line

The past few years have been hard on many companies. Some have had to rapidly pivot and deploy new data and analytics technologies in order to survive and thrive. Others have managed to maintain their status quo. In 2022, companies will tie up loose ends and put data and analytics in order. We expect to see organizations put more trusted data environments in place, make use of tools that utilize more augmented and automated capabilities, and continue to build their skills in data and analytics.