TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

The Brave New World of Business Intelligence

The next wave of business intelligence will once and for all take BI mainstream, with query and analysis front-ends on every desktop.

• By Stephen Swoyer, Eric Kavanagh
• August 9, 2006

The archetypal sci-fi thriller, Brave New World foretold a reality vastly different than author Aldous Huxley saw around him. Years later, he would comment on just how quickly so many of his prescient predictions came true. How time flies.

The same holds true for the realm of business intelligence (BI). Still a fledgling industry by many accounts, BI stands poised for dramatic change. In fact, according to market watcher International Data Corp. (IDC), we’re currently afloat in new and uncharted BI waters.

IDC says the BI marketscape evolves in 15-year epochs or “waves.” The first of these, the market watcher says, lasted from 1975 until around 1990. This epoch was mostly concerned with production reporting on mainframe systems, but the second wave—which IDC says subsided just last year—saw the beginning of the “modern era” of BI, in which a new breed of end user-oriented, client-server tools supplanted and extended the production-centric roots of paleo-BI with more sophisticated query, reporting, and OLAP capabilities.

Get ready for the next wave of business intelligence, which—by 2020, IDC states—will once and for all take BI mainstream, with query, reporting, and analysis dashboards on every desktop. “When we look back in a few years, we'll see that 2005 was another turning point in the BI market and the beginning of the new wave of investment in BI by organizations in all industries,” writes IDC analyst Dan Vesset. “The current market cycle is expected to last until 2020 and will be focused on expanding the reach of BI to more users both inside and outside the organization and a move to automate more decision processes by combining QRA [query, reporting and analysis] and advanced analytics functionality.”

In its first two epochs or waves, BI was mostly the province of analysts, managers, and other power users. That’s already starting to change, Vesset says, and—thanks to the growing ubiquity of reporting and dashboarding solutions—could be a distant memory in a decade’s time.

“[A]nalysts and managers represent only a relatively small portion of an organization, estimated at about 15 to 20 percent of employees. In fact, IDC believes that the market for reporting and OLAP tools for power users and analysts has reached a level of maturity that cannot sustain the growth rates of the past in terms of new license revenue,” he writes.

This might be one reason why the big BI ISVs are talking more seriously than ever about finally making good on the promise of BI for the masses. But the mainstreaming of BI—and the attendant explosion of BI-related revenues—will attract bigger fishes, too. This could put the squeeze on some smaller best-of-breed players. “Instead, larger IT vendors such as Microsoft, Oracle, and IBM along with the existing specialty BI vendors are now targeting this market. As the market continues to mature, it is highly likely that the larger IT vendors will continue to gain share,” Vesset speculates.

TDWI faculty member Stephen Few, Principal of Perceptual Edge, notes: "BI for the masses will suffer the same problems that currently plague BI for analysts and power users, unless software vendors change their approach. What we have today are BI tools that don't work nearly as well as they should, even for analysts and power users. The reason they haven't reached the masses isn't because vendors haven't attempted to reach the masses, but because most of the tools are so difficult to use and reveal so little about the data that only power users are willing to put up with them."

Meanwhile, Cindi Howson, also a TDWI faculty member and President of ASK, comments: "Just as things had to happen with cell phones before they became mass products or the Compaq Portable II... the same is true with the BI suite. Lowering the price is one thing and I think we are seeing that in per seat licenses. Usability and relevance, though, are other things that need to be addressed."

Former TDWI keynoter and regular faculty member Dan Merriman wrote this in an email sent to BI This Week: "The functionality and usability of the technology is critical. However, success is also dependent upon the ability of companies to 1) define and use sets of metrics that measure actual business results while also providing users with insight into where they should focus to improve results (e.g., leading indicators), 2) establish clear accountability for business metrics and targets, and 3) develop a culture of performance measurement and continuous improvement. If the focus is on the technology alone, it is destined for the shelf."

Mark Madsen, President of Third Nature, another member of the TDWI faculty, writes: "To provide better BI outside of standard reports will definitely take some serious retooling of these products and interfaces. If you can't figure out how to get what you need in a couple of minutes, the product probably doesn't belong in the hands of the front line employees.

"I think we'll see a shift though," continues Madsen. "People are being trained by the simplicity of online interfaces that do a lot behind the scenes to make the experience simpler. As new developers get into BI companies, they'll bring along different expectations and ideas. The products can only get better. They pretty much have to. We're still looking at the online equivalent of greenbar reports."

There’s a lot of work yet to be done, Vesset agrees. “Whether we're talking about information workers with higher levels of freedom to decide about their daily workflows and processes or line-of-business employees who may be restricted by systems and policies in how they perform their duties, a vast population exists whose business intelligence requirements have not been met to their full potential,” he argues. But as vendors gear up to deliver BI solutions optimized for rank and file employees (as well as important organizational stakeholders like suppliers, partners, and customers), they’ll be forced to rethink—and possibly re-architect—many of their existing solutions, Vesset predicts.

“This shift in market focus can be only partially addressed through existing BI software, which as already mentioned was created with the analyst or power user as the intended audience. Clearly a frontline employee will have limited use for an OLAP or an ad hoc query tool,” he says. “In fact, to address the needs of frontline employees and line-of-business managers, organizations must redefine and expand what they mean by BI. The expanded vision of BI must take into account not only the technologies involved but also business drivers and performance management methodologies.”

But just why are businesses increasingly anxious to expose BI capabilities to as many users as possible? Vesset and IDC highlight a number of reasons, including the usual bugaboos (namely, compliance and competitive pressure from without), that are chiefly to account for the mainstreaming of BI. “BI can help drive consistency in decision making. It's important to make correct decisions, but often it is also important that decisions are not made arbitrarily … [and] that different employees followed similar decision processes, which can be audited or monitored,” he points out.

And on the competitive front, companies are increasingly looking to equip users with tools that do more than just deliver information. This is consistent with the Dashboards 2.0 push that some industry prognosticators—such as TDWI’s own Wayne Eckerson—have predicted. “Within the process of performance management, it is important to go beyond simply dashboards and reports that focus only on information delivery,” says Eckerson. “To put BI into an operational context, it's not enough to have dashboards that simply report on what happened. This information is valuable but of limited use. Dashboards should also show context around the information and provide guidance for action. In other words, they need to be in the context of whatever business process the dashboard is built for and support predictive analytics.”