Don’t Fall Behind: Why Data Usability is Essential to Your Digital Business (Part 1 of 2)
AI can help businesses obtain usable data and achieve their digital ambitions. Here’s why that’s so important.
- By Koert Kuipers
- November 9, 2022
Recent history has shown that digital-native companies have the power to disrupt entire industries and shift the dynamics of market leadership. Think of how Netflix disrupted conventional television with on-demand streaming and intelligent, personalized content recommendations. Netflix leapfrogged the existing competition; now networks -- such as NBC with its Peacock app -- are playing catch-up.
From retail to travel, this situation has happened across multiple industries where traditional businesses look to innovate and become digital businesses themselves to avoid falling behind the curve. They want to be like the digital natives, who are really data-first companies, meaning they can use all of their data to draw actionable insights, offer hyper-personalized products and services, and create differentiated experiences. However, it’s this ability to use all enterprise data where organizations still fall short of their digital ambitions.
Old Wine, New Bottle
Many aspiring businesses see digital natives in the cloud and so buy into the notion that migrating their own enterprise data to the cloud -- along with innovations in artificial intelligence (AI) -- will help them to achieve transformative results overnight. Although the cloud serves a critical step in centralizing and providing greater access to all your data, it still doesn’t give you usable data (which is ultimately what delivers value and fuels digital businesses). In fact, Forrester studies have shown that up to 73 percent of data collected within an enterprise goes unused.
The challenge of data usability for organizations is that on-premises enterprise data warehouses (EDWs) were designed for specific BI reports, mainly because keeping more data than what’s needed for reporting seemed expensive and wasteful. These data warehouses are populated with raw data from transactional databases. However, much of that data ends up being dropped because it’s seen as unnecessary for the BI reports. The only reason data is kept is to fulfill a narrow and specific need rather than for developing potentially transformational, interesting, or sophisticated insights to compete with the digital natives of the world.
By moving their EDWs to the cloud, companies can successfully recreate their existing BI reports, but not much more. To evolve as a digital business, you need to load raw data and reintegrate it into the cloud. This is a highly iterative process, and considering the petabytes of data that exist across the modern enterprise, requires significant manpower, flexibility, and scalability.
Linking All the Bits
Even once you have all of your data in one place, making sense of it all can be intimidating. One reason digital businesses are so successful and inspire customer loyalty is their ability to connect the dots of each customer’s behavior, preferences, and history to serve them as a unique “segment of one.” This is how Netflix personalizes content recommendations and how Amazon customizes product recommendations for each user.
In contrast, businesses tend to have lots of fragmented bits of information floating around in the cloud. The data sets may all be in one place, but they still exist in isolation, just as they were in EDWs. Rather than remain in silos, these records need to be standardized, cleansed, and linked so an enterprise can understand the data relationships and take advantage of them.
An airline, for instance, will have data on passengers from different sources such as their ticketing, luggage, and frequent flier systems. Without record linkage, it’s difficult to understand that the data about “John Doe” in each of these systems is actually related to the same passenger. As a result, if John’s flight is suddenly canceled and he’s rebooked on a different flight, mishaps can happen (such as not rerouting his checked luggage) that negatively impact the customer experience.
With data sources linked in a metadata layer, along with additional enrichment from third-party systems (e.g., a partner car rental or hotel booking service), the airline can more readily rebook the passenger, provide baggage tracking through a mobile app, add the mileage to their frequent-flyer membership, and even provide additional services at their destination. The fragmented data comes together to reveal the full picture of who this customer is and how best to serve them, helping the enterprise win the customer’s loyalty.
A data asset essentially helps you link and make sense of all the bits of information from across the enterprise and readies the data for on-demand use for analytics, products, and services. Some enterprises may try to perform record linkage through traditional means (such as a relational database through SQL) but find it breaks down at the scale required for enterprise data in the cloud. With continuing shortages in data science talent, finding enough people to do the cleansing and integration manually is a challenge.
To Innovate, We Must Automate
When you need to perform a complex, iterative task at scale but have a limited talent pool, you can’t rely on the techniques and processing of yesterday. That’s precisely how you fall behind and when emerging data-first disruptors leapfrog over you. Instead, businesses eager to become more digital must also be forward-thinking in how they attain usable data. Today, the best and most innovative way is through AI and automation.
Rather than applying AI just for the decisioning side of analytics, AI can automate the arduous but essential tasks involved in “getting data ready” from all available enterprise data and do so at scale. With usable data assets as the fuel, companies can propel themselves forward as digital businesses, ready and able to compete and even surpass their digital-native counterparts.
A Look Ahead
In Part 2 I will explain the three steps you can take to develop usable data using AI software.
About the Author
Koert Kuipers is the chief technology officer of Tresata, a company on a mission to enable every company to fulfill the promise of “digital,” beginning with the creation of usable data at scale. Koert holds a master’s degree in applied physics from the University of Groningen. You can contact Koert via email or on LinkedIn.