How Mature Are Your Big Data Practices, Anyway?
With its new Big Data Maturity Model, TDWI serves up an online tool organizations can use to gauge the progress of their big data practices.
- By Stephen Swoyer
- December 17, 2013
When it comes to big data, how does your organization stack up? Where do you excel? What do you need to improve? Are there any areas that you've overlooked?
More pointedly, how do you know if you're doing it right?
For some organizations dipping their toes into big data, it might still be too early to pose any of these questions. Over time, however, they'll take on greater urgency. Enter TDWI's new "Big Data Maturity Model" -- an online self-assessment tool organizations can use to gauge the progress of their big data practices.
More than a decade ago, TDWI released its "Business Intelligence (BI) Maturity Model," which offered a similar self-assessment capability. One interesting aspect of that tool was that it actually matured over time: as conditions, standards, and baselines changed, so did the model's scoring.
The new Big Data Maturity Model will evolve in the same way, says Fern Halper, research director for advanced analytics with TDWI. Right now, it scores across five different "dimensions": organization, infrastructure, data management, analytics, and governance.
Just because you have big data tech or you're "using" big data apps doesn't mean that you're big data mature. The Big Data Maturity Model was designed to offer a contextual assessment: it scores on the basis of enterprisewide maturity, placing special emphasis on people and supporting or enabling processes. In other words, what you're doing with big data (i.e., the technologies you're using or the applications you've built) isn't necessarily more important than how you're doing it.
Suppose an organization uses the basic Hadoop stack to deliver production applications and makes use of supporting projects such as Apache Zookeeper (a synchronization and scheduling facility for Hadoop); Mahout (a machine learning project for Hadoop); Apache Drill (a project designed to support data-intensive distributed applications); and Cloudera Manager (Cloudera's Hadoop management console). It sure sounds like a "mature/visionary" shop, doesn't it?
Not necessarily. "We look at big data as an extension [of existing practices], not just as Hadoop or NoSQL, [but] what does it do to your [business] processes. When you put Hadoop underneath a Teradata engine, for example, and you're a Teradata shop, what does this mean? We don't want to look at this just from a technological perspective, using these specific tools, but [rather from the perspective of] what kinds of processes can come to Hadoop," says independent analyst and TDWI faculty member Krish Krishnan, who collaborated with TDWI's Halper on the model.
In other words, says Halper, it's possible for an organization to post a very high score in one dimension and nonetheless record very low scores in others. The Maya, after all, were -- by ancient standards -- world-class astronomers; Mayan commerce was highly developed, as was Mayan medicine; as a civilization, however, the Maya lacked the wheel. (The Maya had a concept of the wheel but didn't make practical use of wheels -- e.g., for transportation or production -- in economic activity.) This isn't to pick on the Maya; it's to point out that judgments about "maturity" or "development" are inescapably contextual.
"The maturity model actually scores each dimension separately, so [an organization] could come up as being mature in analytics but not mature in governance. That's the way we chose to do it: if you're not mature in governance and you're mature in some of these other areas, you're going to get dinged for that in terms of overall maturity," she explained.
TDWI's Big Data Maturity Model also scores on the basis of people and/orprocess maturity.
"From a scoring perspective ... we are looking at the importance of process from an architectural or a data management infrastructure perspective. We measure the technology aspects but we also measure the process aspects and the people aspects," Krishnan elaborates.
The model will continue to mature as standards and baselines mature, stresses Halper, who cites the category of Hadoop management as a good example. "Integrated management of Hadoop: the technology doesn't exist at this point to do that, [so] this assessment itself will evolve," she says.
Helpful, Useful, or Distracting?
Maturity models -- like Magic Quadrants -- get people talking. They're conversation starters.
They invite questions about accuracy, validity, and comprehensiveness: e.g., does the model measure the "right" things? Does it weigh them appropriately? More important, does maturity -- as determined by the model -- correlate with success?
A more basic (but no less important) question has to do with the helpfulness or usefulness of the maturity model as such. Industry luminary Claudia Imhoff, president of Intelligent Solutions Inc., thinks that tools such as TDWI's Big Data Maturity Model can be both helpful and useful.
"I think especially in a new area, maturity models help companies kind of figure out 'Am I almost there, am I miles away, am I never going to make it?' I think they can at least help people gauge somewhat their situation -- and figure out what to do next," says Imhoff, who cites TDWI's BI Maturity Model as an example. The perspective, prescription, and guidance that the BI Maturity Model provided helped (over time) to demonstrate its value, she maintains.
"If a maturity model doesn't come with clear steps on how to get to the next level of maturity and what it means to be in this level -- [for example,] is it good, bad, or different -- then it does tend to lose some of its usefulness," Imhoff explains. "Another problem is that most organizations have multiple levels of maturity: they may be very mature in one area and very immature in others," she continues, acknowledging that TDWI's Big Data Maturity Model attempts to account for this. "Again, it's a way of gauging your level of expertise, your level of maturity and technological progress. That has value."
An industry veteran we spoke to -- a person with more than 20 years of experience in data management -- wonders if this kind of tool is actually helpful? Is it useful -- or is the opposite the case?
This person doesn't think so. A maturity model "creates a lazy man's proxy for strategy. There is no 'risk' if you follow the maturity model and [if] other people who are [recognized as] better competitors than you are further ahead on this maturity model," said this industry veteran, who spoke on background because of an existing relationship with TDWI.
"Therefore, the 'good strategy' is to emulate what they did, ignoring all of the others who emulated what they did but failed because they didn't have the same capabilities and thus didn't have the ability to emulate. In other words, [a maturity model] encourages survivor[ship] bias."
This person compared the concept of the maturity model with -- of all things -- the Diagnostic and Statistical Manual of Mental Disorders, or DSM. Both "models" attempt to define standards; both purport to determine what's "normal" and what's "abnormal," "healthy" and "unhealthy;" and both include or exclude people, practices, processes, and so on.
This year, the publication of a new version of the DSM triggered a furor: the revised DSM-5 reclassified existing diseases, eliminated (excluded) others, and added completely new ones. "A maturity model [could be seen] as a way to exclude divergent practices which challenge the conventional wisdom. This creates exactly the situation where a new thing can enter and disrupt the old thing because the old orthodoxy refuses to acknowledge the value of alternatives. It presupposes a 'best' way of doing something," this industry veteran concluded.
Intelligent Solutions' Imhoff isn't buying that argument. She sees the maturity model as a pragmatic tool – and nothing more. You wouldn't and shouldn't base your decision-making (e.g., about where to innovate or what to prioritize) solely on the basis of your place in a maturity model's pecking order, she points out. A maturity model's guidance or prescriptions should, however, factor into your decision-making – considered (and weighted) alongside information from other sources.
"You can't categorically say [that a maturity model is] irrelevant, it's not usable, it's not of value. I think anything that helps to explain or clarify our industry, especially these new areas in our industry, can be useful. How useful it is is a subjective thing, but then, that's always the case," she points out.