The Sociology of KPIs
While teaching a course on performance dashboards recently, I had a minor epiphany. The reason key performance indicators (KPIs) are so difficult to create is because you need a degree in sociology to predict the impact they will have on human and organizational behavior. My advice to the class was: Don’t try to design perfect KPIs on the first try; rather put them in play, see what behaviors they drive—both good and bad—and then adjust quickly.
Interpreting KPIs. The first challenge with KPIs is that without adequate training and socialization people will interpret results differently. For example, if a KPI’s status is red but its trend is positive, what should a user think? Perhaps, someone already spotted the problem and applied a remedy which is now working. Conversely, what if a KPI’s status is green but its trend is downward? A green light indicates performance is above average, but should you take action now before the green light turns yellow? Tools that let users annotate KPIs can help users take the right action and teach novices how the company works.
Driving Behavior. The second challenge with KPIs is using them to drive right behaviors. If we apply incentives to KPIs—such as attaching merit pay to performance measured by the KPIs—we in effect are conducting a giant sociology experiment. Since humans are irascible creatures embedded in complex (daresay dysfunctional) organizational systems, understanding the true impact of KPIs is impossible to predict.
The mistake most KPI teams make is focusing on one behavior at the expense of another that is outside their purview. For example, call center executives who want to boost productivity may create a metric that rewards agents for the number of calls taken per hour. Incented by this metric, agents will be tempted to terminate calls or transfer them to other departments without resolving them. This will have a dramatic affect customer satisfaction, which may be the responsibility of another team. Higher-level executives need to intervene and make sure a counterbalancing metric is introduced that rewards agents for both productivity and first-time call resolution.
KPI Ecosystems. KPI design teams need to think of KPI ecosystems. Although two metrics may conflict with each other by driving contradictory behavior, this is ok. In a strange way, this conflict empowers workers to make judicious decisions. Humans have a great capacity to live in and reconcile the tension between two opposites (although not without some anxiety.) With strong leadership and proper training, employees can effectively balance countervailing metrics.
Buy In. It’s also imperative that you get workers’ input before you implement incentive-based metrics. That’s because workers understand the nuances of the processes being measured and can identify whether the metrics are realistic and attainable and any potential loopholes that unscrupulous workers might exploit. Putting metrics out for broad-based review helps ensure the buy in of the people whose behavior you are trying to measure and change.
Defining KPIs is a sociology experiment and your workers are the test subjects. Treat them with respect, and your experiment has a better chance of success. But remember, it is an experiment, and if it fails, that’s part of the process. Refine the metrics and try again until you get it right.
For more information on designing effective metrics, see Wayne’s report titled “Performance Management Strategies: How to Create and Deploy Effective Metrics.”
Posted by Wayne Eckerson on July 2, 2009