Next-Generation Business Intelligence

Promise and Reality

When end-user reporting and analysis tools made their debut in the early 1990s on Windows desktops, there were high expectations that the tools would liberate end users from their dependency on IT departments to create custom reports. The combination of these tools and newly minted data warehouses caused vendors and pundits to proclaim that the era of “self-service” business intelligence had arrived.

However, reality quickly fell short of promise. It turned out that most users found the tools too difficult to use. Even when the tools migrated from Windows to the Web, simplifying user interfaces and easing installation and maintenance burdens, it was not enough to transform BI tools from specialty software for power users to general-purpose analytical tools for all knowledge workers in an organization.

Even power users abused the early generations of BI tools (and still do!). Most use BI software as glorified extraction tools to download huge data sets to their desktops, clogging networks and bogging down query performance for everyone else. These users then dump the data into Microsoft Excel or Access to do their “real” analysis, creating spread marts that undermine data consistency.

New Wave on the Way

Fortunately, a new wave of business intelligence tools has arrived. This next generation of BI blends the once distinct worlds of reporting and analysis inside a dashboard interface, occupying the business intelligence “sweet spot.” (See Figure 1) Tools or applications in this sweet spot deliver the analytical functionality that a majority of users in an organization—typically around 80 percent—want and need to do their jobs effectively.

The information needs of these users are best summed up in the mantra: “Give me all the data I want, but only the data I need, and only when I need it.” In other words, most users don’t want to be bothered with data unless there is an exception condition they need to examine. And then, and only then, do they want to look at all data that might possibly be relevant to the situation, and they want to do this quickly and efficiently.

Figure 1. Business Intelligence “Sweet Spot”
Performance dashboards blend the attributes of reporting and analysis to create a dynamic or “drillable” report that meets the needs of 80 percent of your workforce.

Key Features

The next generation of BI tools supports the “user mantra” primarily by providing a Web-based dashboard that lets users monitor the health of the processes they manage at a glance. Color-coding, alerts, and easy-to-read charts and tables enable users to quickly see when performance is above or below expectations. Then, if desired, they can click on a metric and get more information about what is driving the exception condition.

Thus, a Web-based dashboard conforms to the ways most users want to work. They provide only the information users need, when they need it. Dashboards don’t overwhelm users with a dizzying array of reports or analytical options; they keep things simple by highlighting anomalies that users need to investigate and providing additional information only as needed. In essence, performance dashboards are “prettified” exception reports with built-in analytical tools that make it easy and fast for users to get to the information they need to do their jobs.

Performance dashboards provide the perfect blend of reporting and analysis because they parcel out information in layers. Like peeling the layers of an onion, dashboard users peel back layers of information to get to the root cause of a problem. Each successive layer in a dashboard provides additional details and perspectives and enables users to move from reactive monitoring to proactive analysis and action.

Top Layer—Monitoring and Exceptions. The top layer of a performance dashboard is a visual exception report. It lets users place a half-dozen metrics on the screen for easy viewing (and sometimes reports or other documents, if the dashboard is constructed in a portal format). The metrics are represented as visual icons (such as stoplights, gauges, or thermometers), charts, or tables and are updated as needed to meet user requirements (e.g., minutes to hours to days). The metrics give users a quick overview of the performance of the processes or people they manage.

Middle Layer—Analysis and Exploration. Below the top layer or dashboard view is an analytical system that users access by clicking on a metric or performance icon. This analytical view, usually delivered via OLAP technology, lets users slice and dice the data with exceptional speed to explore the root causes of problems. OLAP excels at letting users analyze data “at the speed of thought.” Dashboards often provide guided analysis, steering users to the right data with predefined paths or recommendations that structure their navigation so they don’t get lost in the data.

Bottom Layer—Operational Data and Reports. Although a majority of users will be satisfied with information that they find in the analytical layer, some may want to drill further and examine individual transactions. This takes them to the bottom layer of a performance dashboard, which consists of operational data stored in a data warehouse or transaction system. Most OLAP tools built into performance dashboards can automatically submit queries to these systems, retrieve the transaction data, and present it to users in a separate window.

The next generation of BI tools starts users at a much higher level of analysis than previous generations of tools, which presented users with operational reports or analytical tools and expected users to find anomalies and examine their root causes. A performance dashboard takes much of the guesswork and time out of this process by adding a monitoring layer on top of analytical and reporting tools and integrating all three layers into a seamless whole. Next-generation BI tools start with exceptions, move to analysis, and generate detailed reports only when users require them. In short, the tools conform to the way users want to work instead of the other way around.

Twenty Characteristics

Although this list is not comprehensive, many of the characteristics of next-generation BI tools can be summed up as follows:

1. Web-based— Provides easy access, simplifies user navigation (i.e., “back,” “forward,” and “refresh” options), and centralizes data management and administration.
2. Portal-like—Provides a single place on the Web where users can go to get all the information they need.
3. Dashboard Interface—Displays a few key metrics on the front page that let users compare performance to targets at a glance. Groups key metrics for other departments or divisions using tabs or folders.
4. Layered—Arranges data in multiple layers, with each successive layer providing increasingly detailed data about a metric, process, or event. Allows users to drill down from the dashboard front page to successive levels of detail.
5. Interactive—Lets users easily navigate through or “slice and dice” the data, which includes the ability to drill down to more detail, drill across to different subjects (i.e., dimensions), or page through the data by category (Eastern region, Western region, and so on). Users can also insert new data or columns, add new calculations, or search, sort, format, filter, or pivot the data.
6. Guided—Provides guidance to “casual” users about what reports, paths through the data, or actions they should take based on the context of the data they are viewing.
7. Timely—Metrics and reports are updated in a timely fashion—in seconds, minutes, hours, or days—to meet user requirements.
8. Proactive—Provides a rules engine that lets users (or developers) define targets and thresholds for various metrics and specify when and how they should be notified about an exception condition (e.g., an “alert”) and whether to trigger an automated action or series of actions (e.g., an “agent”).
9. Customizable—Customizes the dashboard by user role and level, displaying only the tabs, metrics, reports, and data that each user is authorized to see.
10.  Personalizable—Allows users to select objects from an authorized list and arrange them on the dashboard to suit their preferences.
11.  Flexible Access—Lets users natively access data and reports from multiple front-ends, including Microsoft Office applications and wireless devices.
12.  Collaborative—The tools make it easy for users to share views or reports with colleagues. Users can embed comments in the reports, send or publish the reports to a list of users, or set up a workflow that requires a predefined group of users to review and sign off on critical information.
13.  Flexible Delivery and Formatting—Lets users schedule and publish reports to any channel, including the Web, e-mail, printer, wireless device, and in any format, including Excel, PowerPoint, PDF, HTML, and handheld formats.
14.  Self-defining—Makes it easy for users to find out the origins and nature of any metric or object in the dashboard (i.e., provides business metadata). Hooks into a metadata repository where business and technical people store standard terms, rules, and definitions.
15.  Printable—Lets users print data or metrics in document format with proper page breaks and headings in any order they prefer. This feature can be a make-or-break feature when deciding which commercial tool to purchase to build a dashboard.
16.  Timely—Delivers data to users as soon as they require it (i.e., in “right time”).
17.  Fast—Provides sub-second response times to user clicks and requests for data.
18.  Scalable—Performance doesn’t degrade no matter how many users are on the system at any given time.
19.  Responsive—Developers can deliver new data and functionality within days or weeks, not months or years, thanks to a service-oriented architecture.
20.  Portable—Lets users disconnect from the network and take data with them for further analysis. This can be done by exporting to Excel or creating a replica of the original view or report.

Summary

A new generation of BI tools that blend reporting and analysis capabilities inside a Web-based performance dashboard promises to finally empower all knowledge workers. When combined with a robust data warehousing infrastructure, the new tools provide the right data to the right person at the right time to optimize decision making, improve efficiency, and accelerate results.

Wayne Eckerson is director of research at The Data Warehousing Institute and is currently working on a book titled Performance Dashboards: Optimizing Efficiency and Accelerating Results, due out in October 2005.