Predictive Analytics: Slow Adoption Despite Big Benefits
Why is a high-value technology such as predictive analytics so under-represented in the enterprise?
- By Stephen Swoyer
- November 7, 2007
Few technologies provide as much ROI bang for the buck as predictive analytics. Despite its impressive benefits, only 21 percent of respondents to a recent TDWI survey had fully or partially implemented predictive analytic solutions in their organizations, while another 19 percent were in the process of developing them.
Why does adoption continue to lag?
Predictive analytics is a deceptively simple way of describing a set of business intelligence (BI) technologies that help uncover relationships and patterns within large volumes of data. That’s the first part of the predictive analytic value proposition.
The second part (and probably the most important part from the perspective of business decision-makers) is that predictive analytic tools uncover actionable insights. The relationships they discover can, in turn, predict behaviors or events. It’s in this respect that TDWI Research contrasts the forward-looking view afforded by predictive analytic solutions with the historical perspective (that of a "rearview mirror," according to many predictive analytic vendors) afforded by BI tools, which are typically employed less as predictive and more as deductive technologies.
"[O]ther BI technologies—such as query and reporting tools, [OLAP] tools, dashboards, and scorecards—examine what happened in the past," writes Wayne Eckerson, director of TDWI Research, in a recent report.
These tools are deductive, Eckerson argues, because "business users must have some sense of the patterns and relationships that exist within the data based on their personal experience."
Rearview mirrors or no, conventional BI tools enjoy much greater adoption than their forward-looking counterparts, at least right now.
The good news is that 44 percent of TDWI’s survey respondents were still exploring their options with respect to predictive analytic tools. That means about 84 percent of respondents were implementing, investigating, or at least nominally open to predictive analytic deployments; just 16 percent had no plans to deploy the technology at all. That’s still a puzzling adoption rate for a technology that—if case studies, ROI success stories, and marketing anecdotes are correct—has an enviable track record.
"[P]redictive analytics can yield a substantial ROI. Predictive analytics can help companies optimize existing processes, better understand customer behavior, identify unexpected opportunities, and anticipate problems before they happen," Eckerson writes. For six years running, he points out, a majority of TDWI’s annual Leadership Award winners have used predictive analytic solutions to achieve noteworthy business results.
Fear and Trepidation
Which begs a particularly insistent question: Why is a high-value technology like predictive analytics so paradoxically under-represented in the enterprise?
For a number of reasons, thought leaders say, starting with the technology’s esoteric roots in statistical analysis.
"Predictive analytics is also an arcane set of techniques and technologies that bewilder many business and IT managers," Eckerson points out. "It stirs together statistics, advanced mathematics, and artificial intelligence and adds a heavy dose of data management to create a potent brew that many would rather not drink!"
The upshot, experts say, is that it isn’t unusual for business users to experience a kind of fear and trembling when they’re exposed to predictive analytic tools: such tools, for all of their wonder-working potential, can also be intimidating—perhaps even awe-inspiring.
Perhaps one should say were intimidating or awe-inspiring. In fact, proponents say, the current crop of tools eliminates many of the ease-of-use issues traditionally associated with predictive analytics.
"We’ve always seen a lot of interest coming from the business side. If you go back to the early days of predictive analytics, we used to see people with real visions in areas like marketing, where problems could be tackled very well with predictive models, and they were interested, but maybe they were a little intimidated, too," says Colin Shearer, senior vice-president of market strategy with predictive analytic SPSS.
"So, yes, maybe there was this fear of actually using the technology—this fear of something novel or untested. A kind of intimidation [in the face] of the tools. But there’s no longer a fear factor of getting involved in something novel, and the tools are getting easier and easier to use."
Along with rival SAS, SPSS controls a sizeable slice of the analytics market and a disproportionate share—given its comparatively small BI market footprint—of the predictive analytics space. According to market watcher International Data Corp. (IDC), SPSS derives almost 90 percent of its revenues from sales of advanced analytics software, for good reason: both SPSS and SAS started out as developers of statistical analysis software; both remain highly respected analytics vendors to this day.
Predictive analytics is, in a sense, a college professor’s or researcher’s statistical package of choice gussied up—that is, made both safe and relevant—for business users. For this reason, SPSS and SAS have benefitted from growing interest in predictive analytic solutions.
"SPSS’ focus on its concept of the predictive enterprise, which emphasizes forward-looking analysis of customer and operational data, enabled the company to improve its growth rate in 2006 to 12.3 percent," wrote IDC analysts Dan Vesset and Brian McDonough in a recent market research report.
More to the point, Vesset and McDonough said, SPSS is well-positioned to benefit from emerging predictive analytic trends, too—particularly a growing need to access and analyze unstructured content. "SPSS has also emphasized its capabilities for the text mining of unstructured content, a functionality that further enhances its ability to address CRM analytics needs," the IDC analysts observed.
Groundswell Ahead?
Both SPSS and SAS—along with many of the big BI vendors also touting predictive analytic capabilities—are poised to benefit from a predictive analytic groundswell because, they argue, they’re delivering tools that address the very specific expectations and requirements of business users, and not (as was traditionally the case) those of power users.
According to Shearer, for example, the latest version of SPSS’ BI and statistical analysis platform builds on the user-friendliness of its predecessor (which introduced a degree of user self-service capabilities) by delivering fully revamped Mac OS and Linux client versions, too. What’s more, he argues, mounting interest from a non-traditional customer segment—namely, enterprise IT departments themselves (as distinct from individual business units)—provides an indication of just how prevalent predictive analytic technology is becoming.
"What is an interesting phenomenon is more and more we see inquiries coming and serious interest being shown from the IT side as well. More and more we see this as becoming a category that IT are realizing that there’s a lot of interest in the business, [so they’ have] to think about adopting predictive analytics seriously. … [They’ll] have to take a look at it," he says.
According to TDWI’s Eckerson, one of the principal concerns of business managers and other decision-makers isn’t so much the efficacy or the usability of predictive analytic solutions, but their appropriate (or most effective) initial inflection point: namely, where should they start?
"Most have only a vague notion about the business areas or applications that can benefit from predictive analytics," he writes. "[M]ost don’t know how to get started: whom to hire, how to organize the product, or how to architect the environment."
The most common inflection point is in marketing, Eckerson and others say. Other important applications include budgeting and forecasting, fraud detection, demand planning, customer service, quality improvement, surveying, and supply chain management, according to TDWI’s research.
There are further wrinkles here, too, because while SAS, SPSS, and others claim to have licked the usability problem, there nevertheless remain a number of significant barriers to successful predictive analytic deployments. Experts cite the importance of developing (and sticking to) a multi-stage process (starting first and foremost with the codification of a clear project definition), not to mention the half imaginative-artistic, half logical-scientific task of actually building predictive models (for which the hiring of business-savvy analysts is seen as a must), as two particularly notable challenges.
"[M]ost experts agree that predictive analytics requires great skill—and some go so far as to suggest that there is an artistic and highly creative side to creating models—most would never venture forth without a clear methodology to guide their work," Eckerson explains.