How to Control Hidden Data Costs
The benefits of using data to make business decisions are well known by now. So why do so many businesses still struggle with becoming data-driven?
- By Ben Herzberg
- January 23, 2023
By now, the benefits of using data to make business decisions have been hammered into our minds: improved efficiencies, more personalized customer experiences, better service, enhanced product innovation... the list goes on.
Still, companies struggle with becoming data-driven success stories.
More than half of all U.S. and Canadian businesses say data management costs prevent them from bringing digital innovations to market. In addition, 82 percent of data decision makers find it challenging to even forecast these costs -- not surprising given that so many of these costs are “hidden.”
When it comes to managing data, some costs are significant. Hardware and software purchases, storage infrastructure or cloud subscriptions, network costs, backup and maintenance fees, BI tools, hiring and training -- these are big line items, but we can anticipate and plan for them. Others, however, are much less obvious. These hidden costs crop up unexpectedly, making a dent not only in your bottom line but also in the potential of your data projects. Companies need a clear understanding of all the costs of managing data if they are to plan properly.
Let’s take a look at three of these hidden costs, how they can undermine your data’s value, and how you can tackle them.
Hidden Cost #1: Wasted time and resources
Think of the time each employee must spend to manually search for data in a sea of records or to update a data catalog. Consider the time it takes a data engineer to manually review and approve data access requests. Now think about how this grows exponentially as organizations expand and the data they collect and store increases -- what a huge bottleneck! Sixty percent of workers surveyed estimate they could save six or more hours per week if the repetitive aspects of their jobs were automated. That’s 312 hours per year per employee that is wasted. Time wasted with manual data processes and delayed approvals can also impact project timelines, which ultimately affects your bottom line.
The solution: Make it easier for the data end user. If you can cut down the time your users spend searching for and requesting access to data, you can allow them to focus instead on using that data in powerful ways. Start by having the right metadata available so data is easily searched, filtered, and found. Consider creating a data portal -- a central spot where data consumers can go to easily find and access data. Put the power of self-service tools into your employees’ hands so they can easily access data and gather insights in real time, without involving IT. These steps can all improve time-to-value, eliminate costs associated with manual tasks, and relieve the burden on IT and data engineers.
Hidden Cost #2: Security and compliance ramifications
From insider threats to hacking to phishing, data breaches are constantly in the news. Yes, the direct cost of the breach alone can be catastrophic. The average global cost of a breach is $4.35 million, according to an IBM report, which doesn’t factor in hard-to-measure costs such as reputational harm.
Think, too, of all the time wasted on manual and inefficient security practices meant to protect sensitive data, such as granting and revoking access and detecting and responding to threats. It is also very expensive and laborious to keep logs across a variety of databases, lakes, and warehouses, making auditing and reporting painful. Plus, the costs of noncompliance -- in the form of regulatory fines, legal fees, and loss of customers -- can be enough to bankrupt a company.
The solution: Set clear security policies and processes and bake them into your data management strategy. Bad actors will find ways to exploit your most sensitive data, but you can reduce exposure and risk, and, in turn, reduce the associated hidden costs. Any data management strategy should include security processes such as identity management, access controls, encryption, and backups. Better collaboration between data, IT, and security teams is also important.
In many cases, a data security platform can automate these processes. For example, a tool that continuously identifies and catalogs sensitive information across all data sets, and automatically applies the right security policies, can reduce data engineering costs. Data privacy and security should always be top of mind, but with the right processes and technologies, it’s entirely possible to keep sensitive data secure without incurring unexpected costs.
Hidden Cost #3: The toll of bad data
It is costly to store, sort, and analyze data. It costs even more to store, sort, and analyze bad data -- to the tune of $3.1 trillion per year, according to a 2016 IBM report. With a data-entry error rate of 400 per 10,000 manual entries, businesses that process large amounts of data will really feel this in their wallet. Plus, data that is rife with errors, is irrelevant, or is duplicated makes it all the more challenging and time-consuming to find the data you actually are looking for.
The solution: Use technology to your advantage and automate what you can. It’s unfeasible for organizations to expect humans to manage the immense amount of data we gather. It’s even harder when you factor in labor shortages and budget cuts. However, technology can help make data management processes more efficient and streamlined, saving both money and resources. For example:
- Fifty-five percent of workers believe that human errors in manual data entry would be eliminated by automatically collecting, uploading, or syncing data. Although it’s nearly impossible to eliminate all data quality issues, automation can certainly help reduce the error rate.
- Companies with huge volumes of data are also finding success with artificial intelligence (AI) and machine learning (ML) to support functions such as data cleansing, cataloging, anomaly detection, and governance. According to Gartner, AI-enabled automation in data management and integration will reduce the need for IT specialists by 20 percent by 2023, further saving money.
A Data-Driven Future
We’ve all heard the adage that data is the new oil. Clichés aside, data is indeed a company’s biggest asset and organizations continue to be pressured to find new ways to make the most of their data. That progress can be stopped in its tracks if companies don’t have a clear understanding of every potential cost involved in managing data. That includes anticipated financial costs as well as less obvious costs such as time delays associated with manual processes or the reputational impact of ineffective security practices and compliance missteps.
With these suggestions offered in this article, however, organizations can set their sights on doing great things with their data instead of worrying about the costs of doing so.