Why Enterprises Must Avoid the Master Data Management Divide
Many shops are still sitting tight on aging -- and potentially obsolete -- MDM implementations, at their peril.
- By Stephen Swoyer
- November 19, 2009
In the early days of master data management (MDM), many shops built their MDM projects using a mix of homegrown, kludgey, or ill-fitting commercial technologies. They had to. There just weren't many tools or services available
These days, there's a teeming MDM software and services segment. 3rd-generation (3G) MDM tools are here, 4th-generation (4G) tools are on the horizon, and MDM services should receive a big boost once mid-market shops get their MDM acts together. At the same time, however, many shops are still sitting tight on aging -- and potentially obsolete -- MDM implementations.
Sitting tight, but not sitting pretty. As MDM continues to evolve, shops that tough it out through older implementations will miss out. In just four years, one market researcher predicts, a majority of companies will be practicing enterprise MDM. This means they'll be coordinating master data across multiple lines of business and multiple channels.
Shops that don't invest in next-gen MDM will be left out.
When You Can't Buy, Build
MDM, like many other technology practices, evolved in a surprisingly ad hoc fashion.
Administration and OperationsAnalytic Apps and Dev ToolsBI ServicesXBusiness AnalyticsXData IntegrationDW DesignXInformation DeliveryInfrastructureOpen Source BI/DWKey words: MDM, master data managementESJ Short Title: MDM Divide
First there was "reference data," which data management (DM) pros have been dealing with since the early days of data warehousing, when data marts tended to predominate.
Reference data was helpful, but was -- almost by definition -- siloed: its scope was that of specific business domains or information systems.
This was fine, at least as long as function- or domain-specific data marts predominated. With the rise of the enterprise data warehouse (EDW), however, DM groups started pulling in data from a host of sources dispersed across several business domains.
That's when they first identified a need for MDM -- but few MDM services or software solutions were available. In the absence of commercial (much less shrink-wrapped) MDM technologies, shops shot from the hip.
The result was that custom-built MDM "solutions" could often be as kludgey as they were functional. In some cases, in fact, DM groups fell back on a few old tricks -- such as deploying an operational data store (ODS) to function as a "temporary" repository for master data.
The ugly truth, of course, is that "temporary" solutions frequently have a long life. "It was a situation where, half the time you had this ODS that was being used as a data mart in disguise -- [i.e.] for reporting or operational reporting -- the other time it's being used to overlap the MDM business. And why not? We didn't have MDM when the ODS was being built and we had to have a place to store master data, at least for a while. It was the logical place to put it," recalls Dan Graham, a Teradata Corp. official who logged more than a decade with IBM Corp.'s DB2 Database team, too.
This, too, was fine -- as far as it went. What's vexing is that shops don't always retire band-aid solutions, however temporary they might be.
The not-so-surprising upshot is that organizations are still getting by using homebuilt MDM implementations based -- in many cases -- on an ODS or interim repository: in other words, temporary solutions, deployed in a context (i.e., the ODS) that is by definition temporary, but which in an all-too-familiar movement have nonetheless become permanent.
"The ODS is what you usually turn to when you don't have a database that can do something. It was a perfect fit [for master data]," Graham concludes. "It's usually a case of 'Quick! Get this project over with!' or 'Quick! We have to get this project done!' That's bad enough when it's [i.e., the project is] a data mart. When it's as big [and] as critical as MDM, it's worse."
If You Can Buy, Why Build?
The first MDM pure play offerings appeared more than a decade ago, while MDM itself has been a salient issue for DM practitioners for the last five years.
More to the point, MDM has momentum: large database and application vendors are fully on board. Some -- such as IBM Corp., Oracle Corp., SAP AG, SAS Institute Inc. (via its DataFlux subsidiary), and Teradata -- have been pushing MDM (or MDM-like functionality) for several years now; others, such as Hewlett-Packard Co. (HP), Informatica Corp., and Microsoft Corp. have recently introduced (or, in the case of Microsoft, have promised to introduce) MDM offerings or services, too.
In spite of the disappearance of some MDM players -- e.g., MDM pure-play Razaa was acquired by the former Hyperion Solutions Corp., which itself was acquired by Oracle; IBM acquired MDM power Exeros Inc. earlier this year -- there's no shortage of best-of-breed talent: pure-players Initiate Systems Inc. and Siperian Inc. are two of the most prominent, while a number of business intelligence (BI), data integration (DI) and data warehousing (DW) best-of-breeds -- such as Kalido, MicroStrategy Inc., Silver Creek Systems, and Trillium Software -- market MDM offerings or services, too.
In other words, there's no shortage of creditable MDM suppliers or solutions. Why, then, are some shops staying put on ad hoc or homegrown MDM implementations? After all, most large companies have already transitioned from homegrown reporting or ETL, opting in the first case for the convenience, manageability, and flexibility of shrink-wrapped reporting tools; in the second, for the manageability, scalability, and canned connectivity of commercial ETL or DI offerings.
To a degree, it's an issue of timing: the rise of MDM -- which didn't really hit its stride (with the availability of MDM-themed offerings from IBM, Kalido, Oracle, SAP, and Teradata, just to name a few) until late-2005 -- coincides with a boom-gone-bust implosion that market watchers say amounts to the worst economic setback since the Great Depression. Second, many shops already have made the move to commercial MDM.
More to the point, buy-versus-build has prevailed for half a decade.
"Since 2004, given the choice of building or buying an MDM solution, the trend has clearly been to purchase off-the-shelf MDM packaged applications and customize them to the organization's needs -- similar to prior trends in Enterprise Resource Planning … and Customer Relationship Management," writes industry veteran Aaron Zornes in Enterprise Master Data Management: Market Review & Forecast for 2008-12.
Zornes, a principal with the MDM Institute, foresees steady (if unspectacular) growth in the commercial MDM segment, with revenues increasing from $2.8 billion in 2008 to $4 billion by 2012. "Even after several years of healthy adoption rates, the MDM market is actually just beginning its trajectory toward broad adoption and deep penetration," he says.
What's driving this growth? For starters, Zornes notes, today's MDM offerings are objectively mature: he describes packaged MDM offerings from IBM, Oracle, SAP, and Teradata as 3G products. The 3G topology differs from its 2G predecessor -- i.e., a data-centric hub model in which technologies such as the ODS could thrive -- and is based instead on an application-specific hub model. Packaged 3G tools are also comparatively sophisticated, especially relative to their home-built predecessors: not only are they able to address multiple entities or data domains but they also include a number of BI or DI amenities, such as an integrated data profiling capability or canned dashboards.
What's more, Zornes continues, 4G tools are on the horizon. Next-gen MDM offerings will boast support for multiple master entities and improved support (via technologies such as search or text analytics) for unstructured data, among other amenities. By the time 4G MDM tools become widely available, Zornes argues, master data will have arrived: organizations will be dealing with master data at an enterprise level, coordinating MDM across multiple lines of business and multiple channels.
"By 2012, more than 80 percent of the Global 5000 size enterprises will have committed to enterprise MDM as a core business strategy and have implemented at least one master entity," he predicts.
This could have painful ramifications for some organizations, however.
With the shift away from data-centric and toward application-centric MDM, shops that choose to tough it out on older homegrown MDM technologies -- most of which are (by definition) based on data-centric hub topologies -- could miss out.
"MDM is becoming less of a standalone technology infrastructure as the emphasis is increasingly on relationship between domains, user interface, and integration with other emerging and adjacent technologies such as RFID, entity analytics, [or] business intelligence," Zornes concludes. "This … focus concerns more than integration with existing CRM and ERP systems. It is recognition that MDM is the type of infrastructure solution that will leverage all other systems -- other application stacks, other CRM and ERP instances, [and] other data feeds."