TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Measuring the Chief Data Officer

CDOs have unique skills. Here are five KPIs with which to measure their performance.

• By Jill Dyché
• October 2, 2018

In my last Q&A column, Rohit, a chief analytics officer (CAO), asked for an optimal set of KPIs for his role. His question got me thinking about the differences in how CAOs and chief data officers (CDOs) are measured. That's been a contentious issue in some of the companies I've worked with because senior executives looking to hire CAOs and CDOs understand that analytics and data are both critical sources of competitive advantage that deserve dedicated processes, technologies, and teams.

For Further Reading: So You Want to Hire a CDO? Before Hiring a Chief Analytics Officer, Answer These 3 Questions 5 Steps for CDOs to Transform Data into a Strategic Asset

To their credit, they're willing to invest in the leadership necessary to formalize these programs at the enterprise level. Yet these same executives are often flummoxed at where to draw the boundary lines between analytics and data programs, often leaving it up CAOs and CDOs to figure it out. This can lead to political maneuvering and turf wars that can often sabotage promising analytics, AI, and machine learning efforts and lead to staff turnover and even abandoned projects.

Data and analytics are discrete specialties, requiring domain expertise and technical savvy. They involve different skills, talents, and delivery frameworks. Likewise, their leaders should be measured on different results.

I imagine a subsequent question from the CDO ranks:

Hey, Jill.

Thanks for sharing your set of KPIs for the chief analytics officer. How about giving chief data officers equal time?

Sincerely, Chief Data Officers

It probably won't surprise you that some of the indicators are similar. What differs -- often profoundly -- is what's delivered.

#1: Data delivery

There's that word again: delivery. In my last column I shared my rule-of-thumb from years of judging industry best practices awards, including TDWI's:

Either new data or new functionality every 3-4 months.

Just as a CAO has a delivery road map for analytics projects, so should a CDO have a data delivery road map. After all, the days of forklifting data into a data warehouse are over. To that end, here's another rule of thumb:

The amount of time a team spends loading data is inversely proportional to the business value of that data.

Nowadays, data delivery has as much to do with where the data is coming from as it does where the data is going. Source system data exploration, curation, and preparation are skill sets in their own right and critical to provisioning analytics and AI capabilities. Data Ops is increasingly being practiced by teams of people who understand that data and analytics evolve apace.

#2: Data categorization

This might strike you as a bit "in the weeds," but show me a company that has developed a rigorous set of data categories and I'll show you a company that understands data delivery priorities, constituencies, and privacy policies.

Data categories can take many forms and are often industry specific. One way of categorizing data is based on its shareability, from data that is the exclusive purview of a small team of certified specialists (NPI data and fraud analysts, for instance) to data that is regularly shared outside the company's four walls with customers and business partners.

Other categories can include security tiers, strategic categories that circumscribe data by their ability to address corporate objectives, and even user taxonomies where certain lines of business or analytics programs are prioritized over others. In each case, the categorization can inform skill sets, tools, and deployment time frames.

#3: Mastered data

Full disclosure: I co-authored a book on master data management a few years ago. Of my four books, it sold the fewest number of copies because MDM isn't sexy. It smacks of infrastructure. Executives never really got the difference between an MDM hub and a data repository. When topics such as probabilistic versus deterministic matching algorithms came up in conversation, they'd lower their heads and check their smartphones.

(Hey. Wake up, will ya? I'm not finished discussing MDM!)

Data that has truly been mastered -- defined, aggregated, matched, enriched, and stored -- is sanctioned. It can obviate the need for protracted definitional, standardization, and ownership debates. An authoritative customer or product record can increase speed to market, enable targeted campaigns, and prevent supply chain and shelf space management blunders.

Effective master data management also reveals organizational discipline. It's yeoman's work, often replete with heated platform debates, knotty algorithms, and endless performance optimization iterations. However, when MDM is operationalized, so many other data management challenges are resolved.

#4: A data catalog

Remember data catalogs? Those long inventories of source systems, database and table names, and, if you were enterprising enough, metadata? Remember how protracted those Excel spreadsheets became? I know. It's so 2008, but data catalogs have come a long way.

The velocity and complexity of new data sources and formats outpace our incumbent data correction and standardization capabilities. The new crop of automated catalog tools provides tagging and search functions, rules for usage, and data protection mechanisms. Modern data catalogs can be automatically populated and curated using artificial intelligence while allowing administrators to annotate fields, supply definitions, and assign ownership.

Data catalogs should also grow in lockstep with your business, says tech industry analyst and blogger Jen Underwood. "Data catalogs need to be able to manage a wide variety of data source types (relational, semistructured, or unstructured) residing anywhere (on-premises, cloud, hybrid) and scale with your growing data landscape."

Again, if this sounds in the weeds, it is, but a CDO needs to ensure that data is meaningful in the context of its use or the data will never fulfill its strategic promise.

#5: Data usage

Data used to be a numbers game. These days, how much data you've provisioned to how many users is beside the point. What matters is the data's usability. Broad data usage statistics are relatively simple to calculate, and an effective measurement is to compare the data consumed by business people or by machines to the set of generally available business data.

Hopefully the ratio will be low: available data should be synonymous with useful data. Not the case? Then the CDO hasn't been enabling a data-driven business -- she's been supporting a science project.

A Final Word

At this point you might be wondering why data governance didn't make my list. But data governance is a means, not an end. A solid data governance process ensures the consistent realization of the KPIs described above.

These five KPIs aside, perhaps the best indicator of a CDO's success is a successful CAO.