The Right Way to Measure Business Value: Lessons Learned From a Failed Gold Prospector
Watch out for conflicts of interest and surface-level KPIs when establishing a plan for measuring value.
- By Nick Pylypiw
- January 13, 2020
My team and I recently went whitewater rafting on the Rio Grande during our annual company retreat. It was a beautiful September day just north of Santa Fe as we approached a rickety old suspension bridge hovering dangerously close to the river -- the landmark Glen Woody Bridge. Here, our rafting guide recounted the legend of the bridge's namesake -- prospector W. M. "Glen" Woody (Ol' Glen Woody).
Ol' Glen Woody was the proprietor of a gold mining camp on the banks of the Rio Grande. By many accounts, he was a slick and successful businessman, though not necessarily an honest one. He discovered some low-grade gold deposits, but not nearly enough to pique the interest of serious investors. Seeking a way to secure capital to begin mining his mountain, Ol' Glen Woody devised a plan. He loaded up some gold samples into shotgun shells and peppered the side of the mine's entrance with them. Visible gold! This gave potential investors an exaggerated assessment of the mountain's yield, and they lined up to invest -- to their detriment.
Unfortunately for Ol' Glen Woody, legend has it that he met an untimely end after conning the wrong business partner. However, in addition to "never commit gold mine fraud in the late 1800's," he left behind some important lessons on evaluating and measuring the true value of your business initiatives.
Set Clear Goals Before Measurement
Establish the threshold for success before you gauge whether "there's gold in them there hills." What amount of "gold" would make the investment worth it? For example, marketers looking to increase traffic to their website should first know what level of traffic increase they would consider a success, given the initiative's cost. Is a 10 percent increase in traffic enough, or does it need to be 20 percent? This might even require some initial analysis to determine typical variation due to seasonal "noise."
Without this due diligence, businesses are analyzing results without context. This puts the decision makers in a dangerous place, forcing conversations about whether the results are "good enough." Setting clear mile markers ahead of time makes this determination easier.
Respect Checks and Balances
If you let Ol' Glen Woody set up the evaluation criteria for determining if his mine is a worthy investment, you can bet his mine will pass the test. This type of obvious conflict of interest is common in the business world. The pricing team, for example, may be in charge of determining whether a recent competitive price calibration initiative was successful. It would be difficult for this team to be impartial. It's tough for anyone so intimately familiar with the inner workings of a project (the assumptions made, the alternatives considered, the pros and cons) to devise a rubric that would penalize their work.
Successful measurement requires a balance of the SMEs on the pricing team, decision makers in the organization, and (often) third-party experts or cross-functional business partners (e.g., finance). If Ol' Glen Woody's investors had brought in a team of experts specifically skilled in assessing the presence of gold in the Southwestern U.S. (rather than just accepting Glen Woody's criteria), they would likely have had a better read on the location's potential.
Don't Stop at the Surface
Surface-level KPIs can only get you so far. True and thorough measurement requires multi-level KPIs and deeper levels of exploration. Trend analysis, hypothesis testing, regression estimates, and other techniques can forecast and measure results, but they can also provide false positives when you consider too few data points.
The investors' method -- looking at the side of the mountain to determine if there were visible gold deposits -- was crude but likely would have been effective if they'd considered additional factors. For example, they might have:
- Looked at a number of core samples 10 feet deep to determine how deep the gold ran
- Tested 20 random sites to determine how widespread the gold appeared
- Asked why there were so many shotgun shells on the ground
By adding literal and figurative depth to this otherwise elementary process, the investors would have had a much more robust measurement method.
Show Me the Money
All business initiatives, however nuanced, are rooted in the same objective: money. When decision makers are looking at results, the question they ask themselves is probably not "What percentage of this random core sample of earth contains high-grade gold ore?" They really want to know something like "Given the labor and equipment costs associated with mining and refining this amount of ore, how much risk will I incur and what is my potential profit?"
These questions are often difficult to answer given the unknowns and assumptions. However, any successful measurement initiative should have some indication, even if just directional, of the potential dollar impact on the business.
About the Author
Nick Pylypiw is director of data science at Elicit, an award-winning consultancy that helps companies transform the way they use customer and employee insight. You can contact the author via email or LinkedIn.