The Disruption of 2020 and the Implications for 2021
Long-term trends in data management and analytics have kept their integrity but with serious adjustments for COVID-19 and associated recession.
- By Philip Russom
- December 22, 2020
The Disruption of 2020
For professionals in data management and analytics, calendar year 2020 began predictably as a continuance of multiyear trends, namely new and modernized solutions for data warehouses, data lakes, many forms of analytics, business reporting, data integration, and the usual list. That was just the calm before the storm.
In early March 2020, it became clear that the newly emerged novel coronavirus -- AKA COVID-19 -- was far more lethal and infectious than previously thought. The entire world was suddenly facing an existential threat that would demand tremendous agility, discipline, and change from a lot of people. As of this writing, more than a million people have died from COVID-19, and the economies of most nations are in dire straits.
The preservation of human life must be the primary goal of all reactions to COVID-19. However, safety measures -- lock downs, social distancing, avoidance of crowds, and limited travel -- have taken a toll on business in many geographies, resulting in regional recessions. For example, in the U.S., the deepest point of the recession is similar to the dire effects of the Great Depression, especially in terms of unemployment.
Modern business is now facing its fiercest challenges. It must respect COVID-19 safety measures to protect people -- employees, partners, customers, everyone in proximity -- while maintaining operations in a way that will ensure the continuance of the business. Toward this end, businesses -- plus institutions for healthcare, education, and government -- have all made sudden and dramatic changes in operations, products, services, and business models.
Implications for 2021
Ironically, many organizations have been praying for a “disruption” that would enable them to pull off a “business transformation,” so they may become a digital enterprise or more driven by data and analytics. However, be careful what you wish for: COVID-19 is not the kind of disruption people had in mind, and people are suffering. Yet, many of the very recent business changes made for the sake of COVID-19 adaptation are actually in line with the components of a technology-driven business transformation.
In many cases, organizations accelerated their business transformation plans by focusing on transformations that address COVID-19 and its recession, namely migrating more of IT to cloud, doing far more business online instead of in person, using cloud-based applications to work from home, and putting more customer interactions online or over the phone. In fact, these transformations position an organization to continue operations and perhaps even thrive in a COVID-19 world. We can argue that these adaptations also position an organization for success in a post-pandemic world, even after the virus and the recession recede. That’s why much of the “new normal” relative to business will probably become standard operating procedure in the future.
These changes have affected many data management and analytics professionals. Concurrently, these same professionals have been instrumental in effecting positive change in these trying times. Oddly enough, most of the data-driven trends we thought would be played out in 2020 were actually active and impactful. However, they were redirected toward different goals than anticipated, being disrupted by COVID-19 safety concerns and the resulting economic conditions. Just as many of the ad hoc business transformations seen in 2020 will be with us in 2021 and probably beyond, the redirection of data-driven technical trends will, too.
The Impact of COVID-19 Disruption on Data and Analytics
Let’s drill into a few areas within data and analytics practices to see how they were both disrupted and energized to support the urgent business changes needed to adapt to COVID-19 and recession realities.
Data and analytics modernizations. Given changes to business and data itself in recent years -- plus the arrival of new and compelling tools and platforms -- almost every layer of the decision support technology stack has been in need of modernization for almost a decade now. In fact, many data teams and analytics specialists anticipated that their highlights in 2020 would be the modernization of data warehouses, reporting, data integration, metadata management, and so on. Even the “new stuff” had reached a maturity level that demanded modernization, as seen in data lakes and programs for analytics and data science.
Few people realized that a pandemic or recession were coming, which would scrap their plans and push them into a completely different list of modernizations. Yes, data professionals were still modernizing systems and teams, but not according to plan. Instead, they updated key decision support systems -- and especially solutions already built atop them -- to provide information, insight, and operational acumen that would help their employers survive the pandemic and its ensuing recession.
Cloud migration and adoption. The most influential trends in IT has for decades been the slow-moving, multiyear adoption of cloud as the preferred compute platform for, well, everything, including data management and analytics. Organizations adopt cloud for its benefits, namely scalable data management, processing speed for analytics, minimal administration, reasonable cost, and an elastic environment conducive to agile development methods. Note that many modernization projects involve “replatforming,” where an on-premises legacy toolset and/or platform is migrated to a cloud-based equivalent so the system has a modern platform that will elastically scale into the future.
The slow-moving trend of cloud adoption was sped up tremendously by COVID-19 as organizations deployed more cloud-based apps as a way of automating new methods for business operations, such as work from home, distance learning, online customer service, online public service, and every imaginable form of e-commerce, supply chain, and logistics.
Growing analytics programs. If we drill down to data-driven practices, the fastest-growing trend is the creation or adoption of programs for analytics, with a focus on advanced forms such as graph, machine learning, artificial intelligence, and predictive analytics. TDWI saw organizations with a three-year plan for analytics accelerate that into a three-month plan. Why? Analytics proved to be indispensable to understanding the “new norm” of the world and the daily, evolving condition of individual enterprises.
Many analysts and data scientists were asked to develop new “what-if” analytics, predictive models, and sandboxes -- almost daily -- as the pandemic safety requirements, the state of the economy, and customer’s reactions to those churned repeatedly. New analytics were created and older ones revised to provide insight into the COVID-19 world relative to customer base segments, profitability, shipping optimization, online business processes, and difficult, gut-wrenching decisions about which operations to grow, minimize, or close -- plus whom to fire and whom to furlough.
Modern business reports. Even the ancient Phoenicians had accounting and reports, albeit written in cuneiform on clay tablets. Business reporting is engrained and still relevant to the modern world, such that it will never go away. The modernization of business reports began decades ago, which resulted in modern dashboards, scorecards, performance metrics, and key performance indicators (KPIs).
Due to the pandemic and its ensuing recession, many business managers who depend on reports woke up one morning to a business that barely resembled the one they knew. After all, as a business changes dramatically, so do the business entities and processes, which in turn affects the data, metrics, and KPIs found in reports and dashboards. The pandemic and recession forced a frantic update of reports (and the queries and data sets behind them) so reports could faithfully represent what the business needed to track and measure today.
Data Management and Analytics as Guidance for the New Normal
One of the unsung success stories of the COVID-19 pandemic and recession is about how data management, analytics, and reporting helped many organizations make a hard left turn into a COVID-19 and recession reality. For example, TDWI found organizations using these data disciplines to answer urgent business questions:
How many students do we have left? How many students can our systems support concurrently online? Which store locations do we close? Which suppliers can we rely on for personal protective equipment? How do our increased delivery costs and expanded call center impact profitability? Can we make payroll without salary cuts? At the current cash burn, how long can we stay in business? How is “work at home” affecting employee productivity? What resources are not devastated by the pandemic and recession, and how can we quickly build a new business based on those? Who are the essential employees and who can be let go?
To answer these mission-critical questions, TDWI saw data and analytics professionals, along with others in IT, working evenings and weekends, from mid-March into June 2020. Their dedication and innovation helped save many businesses, jobs, customers, whole industries, and sometimes lives. Thank you, data and analytics professionals, for your solutions and contributions to a healthier, more secure, more modern, and, well, normal world.