TDWI Introduces Big Data Maturity Model

TDWI's Big Data Maturity Model helps organizations assess their big data readiness and offers best practices for moving along the maturity path. Its co-author, Fern Halper, explains the five major maturity stages.

[Editor's Note: Fern Halper and Krish Krishnan will be introducing TDWI's Big Data Maturity Model in their keynote presentation at the TDWI World Conference in Orlando, Florida (December 8-13, 2013). We asked Halper to give us a preview of the stages that make up the maturity model.]

TDWI has just launched its TDWI Big Data Maturity Model and Assessment Tool. Krish Krishnan, from Sixth Sense Advisors, and I have been working on it for many months and we're excited to see it released. We are also grateful to our sponsors (Cloudera, IBM, MarkLogic, and Pentaho) for their support!

There are two parts to the Big Data Maturity Model and Assessment Tool. The Big Data Maturity Guide walks you through the actual stages of maturity for big data initiatives and provides examples and characteristics of companies at different stages of maturity. In each of these stages, we look across the dimensions necessary for maturity. These include organizational issues, infrastructure, data management, analytics, and governance. We've identified five stages of maturity and a chasm, as shown below.

Here is a high-level view of each stage; more detail is provided in the guide.

Nascent: The nascent stage represents a pre–big data environment. In this stage, most companies have a low awareness of big data or its value across much of the business. There is no real executive support for the effort, although there are pockets of people spread throughout the company who may be interested in the potential value of big data. At this stage, sometimes, the organization has bought into the concept of analytics and it may have a data warehouse, for instance, but it has not yet started to explore advanced analytics or begun its big data journey. This may also mean that its governance strategy is more IT centric than business-and-IT centric.

Pre-adoption: During the pre-adoption stage, the organization is starting to educate itself about big data analytics. The organization may have invested in some new technology, such as Hadoop, in support of big data. It knows that it will be implementing big data analytics in the near term, although the effort is usually departmental in scope.

Early Adoption: This stage of maturity is typically characterized by one or two proofs of concept (POCs). Organizations tend to spend a long time in this stage, often because it is hard to cross the chasm that leads to corporatewide adoption of big data and big data analytics.

The Chasm: Companies spend a large amount of time in this phase because there are hurdles to get through to get to corporate adoption. There is the obvious challenge of obtaining the right skill set. Hadoop skills and advanced analytics skills may not be present in the organization. There may also be political issues. Governance can also be an issue.

Corporate Adoption: Corporate adoption is the major crossover phase in any organization's big data journey. During corporate adoption, end users typically get involved, gain insights, and transform how they do business. For instance, they may change how decisions are made by operationalizing big data analytics in the organization. Most organizations trying to reach this stage of maturity might have repeatedly addressed certain gaps in organization, infrastructure, data management, analytics, and governance.

Mature/Visionary: Only a few companies can currently be considered visionary in terms of big data and big data analytics. At this stage, organizations are executing big data programs as a well-oiled machine using a highly tuned infrastructure with well-established program and data governance strategies. The program is executed as a budgeted and planned initiative from the company perspective. In the visionary stage, there is excitement and energy around big data and big data analytics.

Of course, very few companies today are in the later stages of big data maturity and moving from one stage to the next is not necessarily a linear progression. You can be mature in one area, for instance infrastructure, but not at all mature in organization or governance.

Where Are You on the Path to Big Data Maturity?

We are excited about the Big Data Assessment Tool, whichallows respondents to answer a series of about 50 questions that examine an organization and its infrastructure, data management, analytics, and governance dimensions. Once complete the respondent receives a score in each dimension as well as some expectations and best practices for moving forward. In addition, a unique feature of the assessment is that respondents can actually look to see how their scores compare with their peers, by both industry and company size. Of course, your scores also feed into the overall averages, so it is important to answer the assessment honestly. You can come back to it in several months and see how you are doing!

To begin an evaluation of your own organization, visit

TDWI Membership

TDWI Members Get FREE anytime access to all reports and publications

Individual, Student, & Team memberships available.