Why the DIKW Pyramid Is Essential for Your Data Team
The heart of your data strategy should be a hierarchical model you've probably never heard of. Here's why it's so important and how you can apply it in your own organization.
- By Peter Jackson
- October 29, 2021
Data is the lifeblood of the digital economy, which makes it essential that every business should have an effective strategy for optimizing its data. The heart of that strategy should be the DIKW Pyramid.
Never heard of it? The DIKW Pyramid, which has been around in one form or another since the 1940s, ascends through the four levels that give it its acronymic name: data, information, knowledge, and wisdom. The pyramid is used in the knowledge management realm but is fairly new to data science and may be unfamiliar to many leaders in the business and IT communities. Every chief data officer, however, should be conversant with its concepts, which provide the discipline necessary to build a data strategy that can deliver insight and wisdom.
From Raw Data to Wisdom
As Caroline Carruthers and I point out in our book, The Chief Data Officer's Playbook, the DIKW Pyramid's thematic roots may go back to T.S. Eliot's 1934 play, "The Rock".
- Where is the life we have lost in living?
- Where is the wisdom we have lost in knowledge?
- Where is the knowledge we have lost in information?
Turn that cycle from losing wisdom and information to gaining them, and you have the framework for the pyramid, which has been used in fields as varied as military preparedness and nursing informatics. Soon, it will appear in the best and most coherent data strategies.
The DIKW Pyramid delivers collected, curated, and contextualized data to data engineers and data scientists, It provides a dashboard-like view and takes its users on a data journey, with each step answering questions and adding value.
Take customer data, for example. Using the DIKW Pyramid as a framework, raw data is refined into more valuable information about customers, and key elements are added to produce knowledge about their wants and patterns. This eventually allows data engineers and scientists to use insights about the customer to predict what they will want next and, therefore, what products to develop -- a decision that certainly requires wisdom.
Best Practices for Applying the DIKW Pyramid
Many business leaders agree that their organizations have not quite mastered every level of the pyramid. To get started, begin with the data, which should be both plentiful and of high quality. Just as you'd need a lot of fuel to produce a golden nugget, you need a lot of data to produce a small amount of wisdom. You need to identify the right data and the right technology to produce valuable insights into customer behaviors, market trends, or whatever your organization's goals may be.
As you move further up the pyramid, the skills needed to ensure success by people working at higher levels change. As you go from customer support teams collecting the raw data to intelligence and engineering teams that sift through information and derive relevant knowledge, you need to ensure that everyone in the chain is experienced with relevant skill sets. Only then can you support effective data storytelling and get to the top of the pyramid -- where business leaders make decisions that increase revenue, improve services, or foster better relationships with customers.
To deliver pyramid-driven data storytelling, you need an organization-wide effort, beginning with buy-in at the top of your enterprise. C-suite teams and your board of directors need to understand the importance of using the pyramid and to see the cause and effect of making decisions based on this model. Data teams should demonstrate its value with a dashboard depicting customers and the supply chain that details the kinds of decisions that will impact customers and increase revenue or reduce costs. The dashboard should depict the correlations between real life use cases and the DIKW Pyramid.
Awareness of the pyramid must spread through the entire organization. DIKW is not hierarchical; it is an ecosystem in which every part supports all others. Everyone in the organization needs to understand the DIKW concept, how it relates to them and their jobs, and where they fit within the pyramid. The goal of following these best practices is to build an operating model that supports the DIKW Pyramid.
Data Strategies Will Continue to Evolve
As data and data-driven decision making become ever more important to a business's ability to compete and drive revenue, the DIKW Pyramid will become increasingly essential to building an effective data strategy. Data leaders should keep it at the core of their strategies, which will evolve along with the business landscape and the pyramid itself.
About the Author
Peter Jackson is the chief data and analytics officer of Exasol. He is a data evangelist and co-author of two best selling data books: "The Chief Data Officer’s Playbook" and "Data Driven Business Transformation." Prior to joining Exasol, Peter was director of group data sciences at Legal and General, where he was responsible for the group’s data strategy and for driving innovation and digital transformation globally. During this time, Peter also served as the chief data officer for the L&G Investment Management business, an Exasol customer. Additionally, Peter held the first chief data pfficer role at Southern Water, leading the utilities giant’s data transformation and helping it become a data-driven organization.