Why Gut Instinct Still Dominates Decision Making (And How to Become Data-Driven)
Trust in data remains a key concern for executives, but two-thirds still don't put data first when making decisions according to a new report.
- By James E. Powell
- October 1, 2020
The biggest companies may say they're increasingly using data to drive their decisions and that they're investing heavily in prioritizing data within their organizations, but in fact, two-thirds of CEOs still rely on gut feel when making decisions.
That's just one of the insights in Alation's new "State of Data Culture" report. Conducted by Wakefield Research surveying 300 data and analytics leaders at enterprises with more than 2,500 employees in the U.S., U.K., Germany, Sweden, Norway, and Denmark, the report assessed enterprise progress in "establishing a data culture within their organizations, the challenges they face in embracing data-driven decision making, and the progress they have made in leveraging data to drive business value for their organizations." Alation, a data catalog pioneer and data intelligence solutions provider, says it plans to update the report quarterly.
The report introduces what it calls a Data Culture Index (DCI), "a quantitative assessment of how well an organization is positioned to enable data-driven decision-making." The index has three component scores (rated from 0 to 4), one for each of the "pillar disciplines of data culture":
- The ability to find data (data search and discovery)
- The ability to properly analyze, interpret, and draw conclusions from data (data literacy)
- The ability to ensure trustworthiness and accountability of data assets, including compliance with policies and regulations (data governance)
How are companies doing? Not great. Only 12 percent of companies scored an A. Not only did two-thirds of subjects rate a grade of C, D, or F, the report cites an important "data culture disconnect" -- data leaders think they're doing better than the researchers do. Over half (58 percent) of data leaders gave themselves a higher score than did Wakefield.
When asked what causes this disconnect, Alation cofounder and CDAO Aaron Kalb told Upside he hypothesizes that the largest factor is "just human psychology. We struggle with estimation in general and particularly with self-appraisal (especially when facing an uncomfortable truth about ourselves or our organizations). Think about trying to estimate the number of seats in a stadium (remember those?) all at once versus if you can roughly count the seats in one section and then the sections in the venue.
"The DCI calculation methodology is a simple ruler -- it even goes to 12 -- to get a more accurate measure of the actual state of data culture in an organization. By looking at each of the three critical aspects separately, and thinking about penetration across departments, you get a more granular (and therefore more accurate) picture than just ballparking."
Success Factors
The report uncovered a few success factors. Of companies scoring an A or B, 59 percent foster a data culture that values collaboration between business and the data and analytics teams. Only 44 percent of companies with lower DCI scores do so. Higher-rated companies are also more likely to manage data governance at the point of use. Ninety-two percent of companies with ratings in the top third have a corporate initiative to become more data driven; overall 78 percent do. When it comes to challenges, 40 percent of those rated in the bottom third cite lack of buy-in from leadership as their top challenge.
The Search for Trustworthy Data
Nine out of ten people polled for the report said that their C-level executives "at least sometimes question the data that they use, with more than half (56%) saying this happens often or all of the time." The higher up in the organization, the more respondents question the data. A third (32 percent) of C-level respondents say questioning data happens "all the time" or "often," but that rises to 53 percent for executives and 72 percent for directors.
That could explain why two out of three data leaders say those same executives "ignore data when making business decisions, relying instead on gut instinct." With some bravado, perhaps, C-level executives are likely to ignore data because they believe their gut instinct is what sets their decisions apart (42 percent). Another third (35 percent) cite insufficient collaboration, the same percentage that said they use gut instinct "because they're used to doing things their own way."
Moving from Gut Feel to Data
That "gut feel" seems entrenched in the report's subjects. We asked CDAO Kalb what best practices can end the belief that instincts are better than data?
"Our 'gut instincts' are actually powerful recommendation engines informed by our evolutionary history and trained on experiential data collected over our whole lives. Gut should trump quantitative data sometimes -- when we need to make a decision in less than a second, when that experiential training data is relevant to the decision at hand, or when the numerical data before us is limited or stale or suspect.
"However, often our instincts point us in the wrong direction for deliberative decisions because the present isn't like the past, and the world has changed in ways that our built-in cognitive biases make it hard to see.
"By investing in data search and discovery, we can make data readily available, so data-driven decisions can be made at the speed of business.
"By investing in data quality and data governance programs, we can reduce the amount of (appropriate) skepticism folks have about data so they take it seriously and have fewer 'excuses' for overriding the story the data tells.
"Finally, by investing in data literacy, we can help folks interpret data well and avoid potentially misleading interpretations -- choosing to believe statistically rigorous results and ignore spurious conclusions rather than just leaning on confirmation basis (embracing data they expect and rejecting data which is surprising)."
COVID-19 Impact
Other report findings examine the impact of COVID-19 on relying on data, how data is used, and changes to use in BI tools, data visualization, and data catalogs since the pandemic began.
Kalb noted that "the research unsurprisingly shows 70 percent of C-level executives are using data more since COVID-19, with organizations using more BI and visualization tools than ever. I think it's interesting that there's also a large uptick in use of data catalogs due to spikes in personnel changes, remote work, and questions regarding our new normal. This may reflect the fact that not only are the answers to old data questions more relevant than ever (because the answers are different and ever-changing), but the crisis is also raising totally new questions which require people in organizations to find, understand, and trust new data sets with which they have no prior familiarity."
Expectations and Surprises
Upside asked Kalb what results were in line with his expectations and what surprised him.
"Ninety percent of organizations reported their decision makers routinely question the data with which they're presented. That was unsurprising and actually reassuring. Healthy scientific skepticism is part of a strong data culture; people should ask about the size of data sets, sampling, methodology, etc. However, I was surprised and a bit disappointed and concerned to read that two-thirds of organizations find their C-suite ignores data sometimes, often, or 'all the time.' Why collect data if you're more likely to disregard it than to leverage it?
"Just a few years ago, many large organizations didn't have a CDO. I was surprised and delighted to see that over 85 percent of respondents' organizations had a C-level data leader and that more than twice as many had the title of CDAO versus CDO. This reflects the speed with which organizations are realizing how important it is to not just manage data but mine it for insights and business value."
Best Practices for Improving a DCI Score
What key steps does Kalb recommend for companies rated C or below to increase their scores?
"First, the companies need to staff for success. They need to appoint a strong, centralized CDAO reporting to the CEO or board so data is seen as powering the whole business. Ensure this person has ambassadors in each function and unit.
"The report shows organizations with such structures have significantly more developed data cultures. Furthermore, the highest-scoring organizations were using data to improve customer experiences and develop new products -- with initiatives led by operations and customer service -- whereas the lower tranches associated data more with risk and compliance.
"Next, these enterprises should invest in software and programs for data search and discovery, data governance, data literacy, and data quality so people can find relevant trustworthy data and interpret it correctly.
"Third, they should foster collaboration between 'data people' and business people, bringing data and metadata to the point of use and impact.
"Finally, set goals and regularly measure progress."
That's solid advice for all companies, no matter what their CDI score.