Signals, Not Noise: Performance Management's Vital Role

David Stodder
Director of TDWI Research for Business Intelligence

Topic: Experts in BI

Data gives us signals about what's happening in business processes, finance, customer interactions, and more. Unfortunately, data can also create a lot of noise. One of the chief attributes of good performance management is to separate the signals from the noise, so that executives, managers, and frontline personnel can quickly and easily see what's going on in areas for which they are held accountable and where they can actually use the information to affect outcomes. The challenge comes when performance management systems deliver too many signals: that is, so many that the signals themselves become noise.

IT management knows about this cacophony all too well. When performance or availability threshold alerts start flashing from all the hardware, software, and network components that make up an application's infrastructure, IT administrators have a devil of a time trying to identify the root cause. Without a good map of application dependencies, they have difficulty determining which applications are using which components, which component is to blame, and where technology upgrades could be most beneficial to applications. IT also has trouble analyzing whether alerts are pointing out not component performance failures but design flaws in applications that the original developers need to come back and fix.

Dashboards full of blinking alerts, gauges going red, and other alarming visualizations can draw administrators' attention to urgent problems within particular components. However, they also can leave IT management blind to steps, including targeted investment, that could have a more beneficial impact and be in greater alignment with the organization's strategic objectives.

The solution that leading IT organizations employ today is end-to-end application performance management (APM) -- a direction that executives and managers on the business side should study as one that may be useful for their implementations of performance management. Typically, the end-to-end approach is to look at application performance from the outside in: that is, from the user's perspective of a process that users or customers initiate and which ends with a result (such as a recorded transaction).

Major systems vendors as well as boutique tool providers are addressing IT management's demand for end-to-end APM tools and methods. Are business intelligence and data warehousing vendors doing something like this for performance management in business operations?

Silos: The Bane of Cross-Functional Enterprise Objectives

Just like IT and its component silos, business performance management also tends to live in silos in many organizations. Some implementations are closely aligned with Balanced Scorecard, Six Sigma, or other management methodologies. Others are dashboard-based outgrowths of BI reporting, OLAP, or spreadsheets. The Office of Finance will have its metrics while corporate executives, contact centers, and other departments will have theirs.

Although it might make tactical sense for each function to have its own metrics, from an enterprise perspective, the assortment of metrics and key performance indicators (KPIs) can often add up to a lot of noise, not clear signals. This frustrates organizations that are pursuing objectives requiring cross-functional collaboration and information sharing. In this age of hyper-sensitivity to risk, the inability to get a coherent picture of performance can leave organizations vulnerable to blind- side hits from problems upstream or downstream in their supply chains or even sudden economic changes they are not tracking.

How will organizations climb out of their management silos to provide clear performance "signals" to the enterprise? Technologically speaking, the way out of this situation may come from implementation of "semantic" layers, such as Oracle's Common Enterprise Information Model (introduced in 2010 with Oracle Business Intelligence Enterprise Edition 11g) and similar metadata infrastructure from IBM, SAP Business Objects, and other vendors. Much as application dependency mapping tools have done for APM, these layers could provide the foundation for end-to-end performance management in business functions. Organizations will be able to view information management more from the perspective of the user and consider how to improve that experience.

The key objective of implementing semantic or metadata layers is to keep data definitions straight as users move from one report, metric, or chart to another, or as they access heterogeneous data sources. As these layers evolve, true "semantic" integration will allow organizations to bring coherency to how they define business concepts, such as customers and products. The layers will help organizations map these higher-level concepts to data sources. Ideally, this effort will be integrated with master data management and data quality processes so that mapping is orderly and no new silos of confusion are created.

Performance management, the lead theme for the upcoming TDWI World Conference in Washington, D.C., continues to be a critical goal for organizations that want to move beyond "data dump" reports to make information delivery personalized and actionable. However, as tools and methods are deployed, organizations don't want performance management to become its own worst enemy, crowding out the signals for the noise.

David Stodder is director of research for business intelligence at TDWI. Dave can be reached at [email protected].

---