Deploying Dashboards and Scorecards (Report Excerpt)
By Wayne W. Eckerson, Director of Research and Services, TDWI
Why Performance Dashboards?
Convergence of Two Disciplines. Dashboards and scorecards represent the convergence of two distinct disciplines in desperate need of each other: performance management and business intelligence. Like long-lost siblings, these two disciplines have struggled on their own to deliver real business value and gain a permanent foothold within organizations. But together they offer an unbeatable combination whose whole is greater than the sum of the parts.
Performance Management. Performance management is the process of measuring progress toward achieving key goals and objectives in order to optimize individual, group, or organizational performance. Performance management encompasses strategy-setting, goal-setting, planning, budgeting, forecasting, and modeling techniques.
Business Intelligence. Business intelligence (BI), on the other hand, consists of the tools, technologies, and processes involved in turning data into information and information into knowledge to optimize decision making. BI encompasses data warehousing, data integration, reporting, analysis, and data mining technologies.
Together, these two disciplines provide a powerful new way to communicate strategy to all employees and monitor and analyze business activity designed to optimize performance. The result is a dashboard or scorecard—a new type of performance management system—that uses BI technologies to apply performance management techniques on an enterprise scale.
Three Major Benefits. When properly deployed, dashboard and scorecard systems offer three main benefits:
- Communicate Strategy. They provide executives with a powerful means to communicate key strategies and objectives continuously by tailoring metrics to each employee based on his or her role and level in the organization. As an agent of organizational change, dashboards and scorecards enable executives to get the entire organization marching in a coordinated fashion toward the same destination.
- Monitor and Adjust the Execution of Strategy. Once goals are established, dashboards and scorecards let executives and managers monitor the execution of the strategy and plans on an hourly, daily, weekly, or monthly basis depending on requirements. These performance management systems enable executives and managers to work proactively and identify and address critical problems undermining progress before it’s too late to fix them.
- Deliver Insights and Information to All. Dashboards and scorecards deliver critical information at a glance using graphical symbols, colors, and charts. The applications graphically highlight exception conditions and alerts and let users drill down into more detailed data to fi nd the root cause of a problem. These tools conform to the way users work and don’t force users to conform to the way BI tools work. This is a major reason dashboards and scorecards are so popular today.
What Are Dashboards and Scorecards?
The many different types of dashboards and scorecards can each look and function slightly differently, and often go by different names depending on the organizations implementing them. To dispel the confusion, here is a definition:
Dashboards and scorecards are multilayered performance management systems, built on a business intelligence and data integration infrastructure, that enable organizations to measure, monitor, and manage business activity using both financial and non-financial measures.
Dashboards and scorecards provide more than just a screen populated with fancy performance graphics: they are full-fledged business information systems designed to help organizations achieve strategic objectives. They help measure the past, monitor the present, and forecast the future, allowing an organization to adjust its strategy and tactics in real time to optimize performance.
The Difference Between Dashboards and Scorecards
Some people use the terms dashboard and scorecard interchangeably, while others use the terms to refer to different types of analytic applications for measuring performance. In my definition, dashboards and scorecards are simply different types of visual display mechanisms, within a performance management system, that convey critical performance information at a glance. In other words, they are the monitoring application, not the system itself!
A good performance management system should be able to deliver either a dashboard or scorecard interface, since both do the same thing: display the status and trends of key performance indicators. The primary difference is that dashboards tend to monitor the performance of operational processes whereas scorecards tend to chart the progress of tactical and strategic goals. Dashboards also tend to display charts and tables with conditional formatting, whereas scorecards use graphical symbols and icons to represent the status and trends of key metrics. (See Figure 1.)
| | Dashboard | Scorecard |
| Purpose | Measures performance | Charts progress |
| Users | Managers, staff | Executives, managers, staff |
| Updates | Real-time to right-time | Periodic snapshots |
| Data | Events | Summaries |
| Top-Level Display | Charts and tables | Symbols and icons |
Figure 1. Dashboards and scorecards are visual display mechanisms—the monitoring layer in a performance dashboard. A good performance dashboard should support both types of displays.
Best Practices in Deploying Dashboards and Scorecards
Much of the advice in this section can be applied to all IT projects, but you will also find targeted tips and techniques for deploying dashboards and scorecards.
1. You Get What You Pay For. You can’t deploy a dashboard or scorecard that delivers real and lasting business value without spending money—unless of course, your company has already invested in a BI and data management infrastructure that can deliver the right information to the right user at the right time in the right format, and has already modeled and loaded a majority of data required in the dashboard or scorecard. Here, companies that have successfully deployed a data warehouse and BI tools have an edge. Those that haven’t incur greater costs.
One IT manager at a large manufacturing and distribution company garnered significant attention internally when his team created an effective (but decidedly low tech) dashboard for an executive vice president to monitor customer service operations. Soon, other business managers approached the IT manager to convert their Excel and Access databases into dashboards.
“They were decidedly uninterested when we told them how much it would cost,” says the IT manager. “They don’t understand the costs involved in cleaning, integrating, and modeling the data and building a bulletproof system that delivers sufficient right-time and detailed data so that they can make accurate and timely decisions.” The IT manager said it cost $400,000 to build the dashboard and that it’s been running for four years without any dedicated IT support.
Of course, you can deploy an inexpensive performance dashboard today, even without the requisite infrastructure. There are many commercial products that you can license for under $50,000 that provide some data integration capabilities. However, you will eventually need to replace these products as word spreads about the success of your solution and you need to scale it up to support more users, more sources, more detailed data, and more frequent updates without compromising performance and response times.
2. Plan for the Long Haul. Always plan for success, because the alternative is disaster. Word about successful dashboard and scorecard solutions spreads like wildfire. If you’ve delivered a successful solution, you’ll be bombarded with requests to deliver performance management systems for other departments and will need to rapidly expand the scope and scale of the existing system. Meanwhile, the number of users may grow rapidly, placing undue burden on processing power, networks, and databases. If you are not careful, response times will plummet and your hard-won reputation will suffer irreparable damage.
Chris Gentry, director of business intelligence at CCC Information Services, offers this sensible advice based on years of delivering BI solutions: “Unless you prepare for 20 percent growth in users, 15 percent growth in queries, and four to five new data sources every year, you will not meet customer expectations. The initial footprint of your solution should be 15 percent more than your most optimistic forecast.”
3. Plan for Real Time.1 The value of a dashboard or scorecard increases exponentially with the freshness of data. This is not to say that dashboards or scorecards that are updated monthly don’t deliver business value—they can! Many successful Balanced Scorecards, for example, are only updated monthly. However, augmenting a dashboard or scorecard with more timely data increases its value even more. A performance management system populated with more timely data lets executives and managers keep their fingers on the pulse of the organization in ways they never could before. They work much more proactively to optimize performance.
So, even if your business people don’t ask for more than daily updates, be prepared to deliver them. Build in hooks to messaging backbones, if they exist, to trickle feed data into your environment. Or select commercial dashboard solutions that support event-driven processing and can prove their scalability across users, sources, and data volumes.
4. Develop on a Single Platform. Because “dashboards are in the air” as one BI manager said, it is very easy for managers to build or buy their own solutions independent of each other. The dashboard silos eventually compete with each other for resources and, more importantly, undermine an organization’s ability to get a single picture of what’s going on in the enterprise.
This is what happened at IBM several years ago. The company had many different dashboard solutions serving different IBM executives in various departments. Each ran on a different platform, contained overlapping data, and duplicated resources, according to Julia Kennedy, senior manager of enterprise BI and reporting in the CIO’s office of IBM. Eventually, IBM executives selected one dashboard among many to standardize on across the company. The winner, a solution called EDGE (Enabling Decisions for Global Execution), now supports 22,000 users throughout the enterprise.
To avoid the disruption of switching to a new platform, it’s best to develop all dashboards and scorecards on a single platform that leverages a unified data integration infrastructure. The dashboard platform should support team-based development with check in/out and version control with seamless support for the deployment of development, testing, and production environments. The data infrastructure ensures the delivery of trusted business data using data integration, data quality, and metadata management tools. These technologies RESEARCHenable business users to examine the lineage of any data element before making critical decisions. Also, administrators can assess the impact of changes to source systems or applications on data integration workflows and reports.
While it’s possible for a single development team to stay ahead of business requests for new performance management systems in a small company, it’s likely that the organization may need to appoint a program office to manage deployment across multiple departments using parallel development teams. The program office needs to establish standards for usage, data, metrics, and functionality to avoid creating non-integrated silos.
5. Develop Effective Metrics. Again, metrics are ultimately the key to the success of any dashboard or scorecard. They are the linchpin between the business and technical architectures. There are many techniques for developing effective metrics, but these are the most important ones:
- Get Buy-In. Metrics won’t propel the organization in the right direction unless people whose performance is being measured understand, accept, and endorse the metrics. The best way to gain this buy-in is to involve workers in the process of defining metrics, targets, and thresholds. Since workers are closer to the business processes, they will better understand whether metrics accurately capture the nuances of a process and whether targets are realistic or not. When you gather requirements from workers, get their feedback on proposed metrics and targets. You’ll design more accurate metrics that have a greater impact on the business and are less likely to be manipulated by individuals who now have a vested interest in making them work.
- Simplify. Humans can only absorb so much information at once. To avoid cluttering a dashboard or scorecard, display only a handful of metrics (four to seven is a good number) on a single screen at a time. If you have more metrics than this—which is not unreasonable if you are measuring an end-to-end business activity—you should create hierarchies of metrics using folders, tabs, or drill downs to preserve the clarity and simplicity of the dashboard display.
- Empower. What’s the point of displaying metrics if you don’t empower users to affect the outcomes? When designing metrics, you need to examine the context in which they are used and give workers license to make decisions—even unorthodox ones—that will improve performance. This requires reengineering business processes and delegating responsibility to people closest to the customer or the process being measured. Concomitantly, organizations need to hold workers accountable for the outcome of the measures. If no one is accountable, the metrics won’t have any impact on the organization. It’s also better to hold an individual accountable even if a team manages the process or task being measured.
- Avoid Perfectionism. It’s difficult to design new metrics in a vacuum. While it’s important to gather comprehensive requirements and map business processes and information flows, you will never really know how well a metric works in practice until you deploy it and see what happens. To avoid analysis paralysis, adhere to the 80 percent rule: develop the metric to a point where you are 80 percent confident that it will have the desired effect, then deploy it, track the results, and refine it as needed.
- Monitor and Revise. All metrics have a natural lifecycle. Over time, metrics lose their impact as workers streamline processes to the point where additional gains are not worth the effort. In addition, the business usually changes, forcing organizations to add new metrics that cause workers to expend their energies elsewhere. To ensure the effectiveness of the metrics and the performance management system as a whole, you need to monitor the usage of the metrics on a continuous basis. Dashboard and scorecard teams should aggressively prune underused metrics (after first consulting with the business, of course), and they should monitor the uptake of newly deployed metrics to quickly identify problems that users may be having with the metrics or views.
- Incentives. It’s very dangerous to attach incentives in the form of bonuses or compensation to metrics that have not been fully vetted and accepted by the organization. It’s important to deploy metrics for a while in order to identify and close potential loopholes, change calculations to more accurately reflect reality, and provide sufficient training so users understand how to impact outcomes. Attaching incentives to metrics prematurely is a recipe for disaster and can catapult organizations into chaos.
- Involve Technical People. One common mistake is to create metrics for which no data exists. To avoid this awkward situation, make sure you assign technical people to the team that gathers requirements and designs the metrics. During discussion, these people can evaluate the existence and condition of data needed to populate proposed metrics.
Conclusion
The future of dashboards and scorecards looks bright. Today, many organizations have deployed performance management systems, but the size and scope of a majority of implementations is still small. This will change as user organizations gain more experience with these systems and vendors begin to offer robust commercial dashboard and scorecard solutions.
Growth is bound to happen because dashboards and scorecards provide business users with all the information they need to make effective decisions and achieve strategic objectives without being overwhelmed. These systems deliver data on demand as users need it—from visual displays that let users monitor KPIs at a glance to dimensional and operational data for conducting analysis and taking action.
Wayne W. Eckerson is the director of research and services for TDWI. He has 17 years of industry experience and has covered data warehousing and business intelligence since 1995. Eckerson is the author of many in-depth reports, a columnist for several business and technology magazines, and a noted speaker and consultant. His book, "Performance Dashboards: Measuring, Monitoring, and Managing Your Business," was published by John Wiley & Sons in 2005. He can be reached at [email protected].
This article was excerpted from the full July 2006 report. TDWI appreciates the sponsorship of ADVIZOR Solutions, Business Objects, Celequest, Corda Technologies, Hyperion Solutions, Microsoft, MicroStrategy, SAP, Oracle, and Visual Mining.
To download the full report, visit www.tdwi.org/research/reportseries.
Next
Previous
Back to Table of Contents