Ad Hoc BI Is Killing Us!
by Wayne Eckerson, Director of Research, TDWI
On the Chinese calendar, the year 2003 is known as the “Black Sheep Year.” For business intelligence professionals, 2003 is the year of “reporting.”
It’s uncanny that within the past four months all major BI tools vendors have made serious moves to add high-end reporting functionality to their suites. This summer, Business Objects and Hyperion acquired production reporting vendors (Crystal Decisions and Brio Software, respectively.) In June, Cognos launched ReportNet, a three-year “skunkworks” project that company officials now say is the firm’s flagship product. MicroStrategy this month will launch a new reporting module and Microsoft is beta testing new reporting services that it will bundle for free into the next version of SQL Server.
Click Your Heels Twice
Why is there such a flurry of interest now in such an allegedly tired and unsexy technology as reporting? Haven’t we heard these same BI vendors ridicule reporting technology as out-of-date and out-of-touch with users’ analytical needs? Why are investment bankers and Wall Street analysts now conversant in pixel-perfect report authoring and multi-channel report bursting, among other things?
The answer is simple: the majority of users simply want to analyze data using predefined reports. By reports, I mean a mixture of static production reports and structured interactive reports, which are exception reports with predefined drill paths to detailed data (e.g. a dashboard, scorecard, or parameterized report.)
Like Dorothy in the Wizard of Oz, most BI vendors have been on a long and torturous journey, only recently to discover that “home” was never far away or out of their reach. Starting in the 1990s, most BI vendors sold ad hoc query or OLAP tools, which, as it turns out, have only really been adopted by power users. Now, after more than a decade, these vendors realize that selling reporting tools to business users is their real “home” – the big market for BI.
At the same time, many user organizations are waking up to the harsh reality that they’ve overspent on the current generation of BI tools. Vendors, consultants, and third-party solution providers have pushed them into ad hoc solutions when what they really need are structured reporting environments, with predictably dismal results. The majority of their users find the ad hoc BI products too difficult to use and no longer use them. This is a major reason why there is such a large percentage of BI tools gathering dust on company shelves instead of being put to use(1).
Ad Hoc BI – The Ultimate Timewaster
None of what I’ve said so far is that new or insightful. For years, industry experts have decried the mismatch between BI tools and user analytical needs(2). However, most experts have failed to comment on another pernicious problem of the current generation of BI tools. That is, for every several users who find a BI tool difficult to use, one person develops a destructive, codependent relationship with the tool. Simply put, many users are wasting a heck of lot of time doing ad hoc queries when they should be doing their real jobs! It might not be too big an exaggeration to say that ad hoc query tools have replaced computer games as the biggest productivity sinkhole inside corporate cubicles.
“Ad hoc BI still kills us,” says Ken Kirchner, a data warehousing manager at Werner Enterprises, Inc. in Omaha, NE. “In a worst-case scenario, a senior director who knows Access and has an ODBC connection starts creating ad hoc queries – and before you know it, they’ve spent 17 hours doing their own analysis outside of our formal reporting and analysis groups.”
And Kirchner is not alone. “We don’t want the majority of our users doing ad hoc queries,” says David Norton, a senior vice president of marketing at Harrah’s Entertainment, a casino management company based in Las Vegas. “We know what information our business users need to do their jobs. So, we create a standard set of static reports and then spend time educating them how to use those reports effectively.”
It would be easy to discount Harrah’s “BI blasphemy” – its use of static reports -- as an example of poor practices employed by a “late-adopter” company. However, Harrah’s is not a technology laggard. In fact, it’s revolutionizing the casino industry by leveraging data warehousing and business intelligence technologies to foster high levels of customer loyalty, gain market share, and boost revenues.
The Pendulum Swings Back
Surprisingly, Harrah’s is not alone here. According to a recent TDWI report, users at companies with successful BI solutions are more likely to analyze data via static online reports (47 percent) than any other method! (See chart 1.) The next most popular methods – interactive online reports (39 percent), paper reports (37 percent), and creating reports from predefined selection criteria (parameterized reports -- 34 percent) all fall within the reporting domain.
User Interactivity with Data
| Succeeding | Struggling |
| View paper reports | 37% | 47% |
| View static online reports | 47% | 37% |
| Navigate interactive online reports | 39% | 24% |
| Create reports from predefined selection criteria | 34% | 27% |
| Build “what if” analyses or forecasts | 18% | 12% |
| Create reports from scratch | 17% | 14% |
| Create statistical models | 9% | 8% |
Chart 1. Users in organizations with successful BI implementations view more static online reports than other ways of interacting with data. Based on 540 respondents.(3)
Thus, despite all the BI hoopla about giving users unfettered access to data, we need to openly recognize that static reports still offer significant benefits. It’s clear that leading BI adopters like Harrah’s are sending a signal – it’s time for the BI pendulum to swing from exploratory BI to more structured interactivity with data.
The Historical Perspective
Originally, organizations justified the deployment of data warehouses and BI tools to offload the IT department from spending too much time and resources creating ad hoc reports for users. The rationale was that if they created a self-service BI environment, users could create their own reports more quickly and would never again be held hostage by the IT department and its backlog of requests.
The good news is that this scenario worked well – in fact, too well. Many IT departments have drastically reduced the time they spend creating ad hoc reports. Unfortunately, they have simply shifted the burden to end users, who spend an inordinate amount of time and energy creating ad hoc reports instead of doing their real jobs.
Moreover, this redistribution of work undermines another key rationale for deploying data warehouses and BI tools – the requirement to standardize data and achieve a “single version of the truth” throughout an organization.
When business users create their own reports, they apply custom rules to custom snapshots of data, often portraying their activity as favorably as possible. The result is the proliferation of “spreadmarts” – spreadsheets or custom reports that contain unique views of the business that can’t be easily reconciled with each other. This “spreadmart” phenomenon creates chaos and confusion and is another reason to rein in ad hoc BI.(4)
Practical Steps
Attentive business managers now recognize that it’s time to rethink their BI deployments for the good of the business. Kirchner, for example, is replacing a commercial BI tool with a custom analytic application built using .NET. The new analytic applications will greatly customize and simplify the process of accessing, analyzing, and acting on data warehouse data for its various user constituencies.
Other firms, like the Scotts Company, are splitting the middle between IT- and user-developed reports. They assign “super users” to write reports on behalf of their colleagues and evangelize best practices for interpreting and applying BI data. These super users are technically savvy business users within a department or functional area who receive training and support from IT but are responsible for creating reports in their areas and supporting the analytical needs of their colleagues.
A Time and a Place for Ad Hoc
As organizations add more structure to their BI environments, they shouldn’t go to the other extreme and eliminate ad hoc BI altogether. Ad hoc BI plays an important role within BI deployments, albeit a limited one. The only users who should engage in ad hoc exploration of data using query or OLAP tools are those who have been hired to analyze data for a living. Typically, these are “business analysts” who know spreadsheets and corporate databases inside and out. According to recent studies, business analysts or “power users” comprise only 5 percent of all users.(5)
Given this puny market for ad hoc BI, it’s no wonder that all the BI vendors have finally discovered reporting. Reporting has never been sexy, but it’s pretty darn effective when trying to disseminate and maintain a standard set of information to a broad base of users.
Recognizing the shifting winds, BI vendors who for years have touted the superiority of OLAP and ad hoc queries are now adding reporting capabilities before they get sucked up by the advancing reporting tornado. So, the current flurry of BI vendor activity is a good sign – it shows that the ultimate convergence of reporting and analysis to create both static and interactive exception-based reports is just around the corner.
_______
- According to “The OLAP Survey 2,” the average shelfware rate of OLAP tools is 39 percent. Other factors contributing to shelfware are the way products are sold and priced. See www.olapreport.com/new.htm to learn how to obtain the report.
- In 1998 I wrote an article, still used in some business schools, about the “decision support sweet spot,” which argued that what users really want is exception reports with a degree of interactivity to investigate problems in more detail. See Journal of Business Intelligence, Summer 1998 or Summer 2003. Also, Philip Russom of Forrester Group has written on this subject in several industry trade magazines in the past year.
- Obtain a copy of the report here
- See “The Rise and Fall of Spreadmarts,” DM Review, September, 2003.
- See “In Search of the Ubiquitous Analytic End User: Profiling the Analytic End User,” by Keith Gile, DM Review, September 2003