Big Data Maturity: Measuring Your Journey
How can you define the success criteria of your big data program? TDWI's forthcoming Big Data Maturity Model will be able to help.
By Krish Krishnan, CEO, Sixth Sense Advisors, Inc.
[Editor's note: Krish Krishnan is leading several conference sessions at the TDWI World Conference in Chicago (May 5-10, 2013), including Big Data Maturity: Measuring Your Journey on May 10.]
As the buzz and hype cycles about big data settle down, there is a sense of uneasiness about the topic itself and how to define the success criteria for big data programs within enterprises.
On one end of the spectrum we have a departmental- or individual-driven effort to understand and integrate big data, which is akin to a shadow IT-style effort. On the other end of the spectrum we have a line-of-business-driven initiative that captures the attention of the enterprise. Both these steps are right and wrong at the same time.
They are right from the standpoint of your needing to start somewhere and make this happen, but they are wrong from the viewpoint that in most cases the attempts fail due to a lack of understanding or planning, setting incorrect expectations, or not implementing the appropriate architecture.
How do you implement the big data program in the most appropriate manner? Who can guide you in the journey?
One approach to understand the best implementation approach is to use a reference architecture model. The underlying issue here is that the newness of the big data space limits robust reference architectures. The other way to create the best strategy and implementation approach is to follow a maturity model that provides the inputs you need to create a road map-driven architecture for implementing your big data program.
When you hear the phrase big data maturity model, your first reaction may be: "What?" Yes we are definitely talking about a maturity model for big data -- and one that is the first of its kind, oriented towards each aspect of a big data program.
In any measurement process, there are several categories across which the maturity of the enterprise is measured, and with big data there is no difference. Think of three components in a triangle, with technology at the bottom, process in the middle, and people at the top.
As we start drilling down into these three core areas, the following sub-components emerge as areas that need to be measured and organized to provide a meaningful guide to the big data program and an organization's journey.
- Scope: To what extent does the big data program support all parts of the organization and all potential users?
- Sponsorship: To what degree are big data sponsors engaged and committed to the program?
- Value: How effectively does the big data solution meet business needs and expectations?
- Architecture: How advanced is the big data architecture, and to what degree do groups adhere to architectural standards?
- Program governance: Does the organization have a program governance model? How effective is the program governance with the big data program?
- Data governance: Does the organization have a data governance model, and how effective is data governance with the big data program?
- Data: To what degree does the data provided by the big data environment meet business requirements?
- Development: How effective is the big data team's approach to managing projects and developing solutions?
- Skills: Does the organization have the skills needed to support the big data initiative?
- Delivery: How aligned are reporting/analysis capabilities with user requirements, and what is the extent of usage?
- Technologies: What are the key capabilities available within the organization for big data technologies?
- Visualization: What are the key requirements for data visualization and analytics delivery?
- Statistical models: What are the requirements for measuring and monitoring the performance of the enterprise? What are the different techniques used today and expected from big data technologies and solutions?
- Alignment: What is the alignment across the enterprise to the big data effort?
- Measurement: What are the different strategies for measuring success? Who defines these measurements and their associated success criteria?
Why a Maturity Model?
The answer is quite simple: a road map helps you plan a big data program's life cycle, but it does not help in developing a measurement process, which can help you create an adoption model for the program, create a risk management process, and provide organizational value in multiple phases of a program's implementation.
Another significant advantage of a maturity model is that it helps an organization establish a baseline that can be used for one program, create a learning experience and a knowledge base that can be applied to future programs. This is a very significant move, and when combined with a road map, will create a unique strategic advantage for any organization.
At TDWI, we are working on a Big Data Maturity Model that will be generally availability this summer. The model will be available at TDWI's website; further details will be available at the TDWI World Conference in Chicago and on the TDWI website in May, 2013.
The TDWI Big Data Maturity Model serves as a guide that can help you develop milestones and avoid pitfalls. It is not based on just phases or stages, nor is it a technology-focused implementation document. We cover many subject areas including architecture, governance, technology, information management, security, funding, skills, and data, measuring your organization's maturity in these areas and the capabilities discussed earlier in this article. We then create a rubric-based approach to provide you with constructive feedback on your current state and recommendations for how to get to the next phase.
The course to be introduced this year at the TDWI World Conference in Chicago will encompass all the points raised in this article and will provide an overview of the TDWI Big Data Maturity Model. We will discuss in detail what big data maturity is and how you think through the different layers (and their associated challenges). We expect attendees to leave the course thinking about how to implement such an exercise for their big data engagement.
Come, learn, and implement!
Krish Krishnan is the CEO of Sixth Sense Advisors, Inc., an independent management and technology consulting organization, and a TDWI World Conference session leader. He can be reached at [email protected].