TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Deconstructing the Master Data Management Monolith

Is a microservices-based architecture the right choice for your MDM solution?

• By Pradeep Karpur
• March 2, 2023

Your data must be trustworthy and connected to thrive in today’s economy. Master data management (MDM) helps ensure the uniformity, accuracy, stewardship, semantic consistency, and accountability of an organization's shared master data assets, thereby creating a trusted data set. MDM also plays a critical role in bringing data together from various sources, thereby enabling the development of business-critical insights with connected data sets.

For Further Reading: How the Right Data Management Foundation Fosters Digital Transformation Success Preparing Your Organization for a Move to the Cloud  Executive Q&A: Data, the Cloud, and the Insurance Industry

Traditional MDM Implementations Are Becoming Monoliths

Many organizations across the world have caught on to the value of deploying MDM solutions and have been scaling up the data domains and system integrations within their MDM platforms. This is evident from the size of the MDM market which is valued at $14.83 billion and expected to register a CAGR of 15.5% (according to Emergen Research).

After this rapid growth, organizations are beginning to realize that their MDM platforms are becoming large, monolithic platforms with multiple data domains and system integrations. These days it is not uncommon to find MDM implementations with 20 or more data domains and 100 or more system integrations. As organizations have transitioned away from monolithic mainframes, MDM runs the risk of becoming the monolith of our time.

Monolithic Architectures Are Fraught with Challenges 

As we have learned from the history of software engineering, monolithic software platforms are fraught with challenges. Organizations are beginning to see these problems with traditional MDM implementations as well.

For example:

• Developing integrations with new systems takes time and is challenging
• Scalability and performance challenges grow with increasing data domains and users
• Traditional MDM is not responsive enough to increasing business expectations

A monolithic implementation pattern is at the heart of these challenges.

Modern Software Platforms Are Built on Microservices

Microservices is a popular architecture pattern used within modern applications that focuses on breaking down large applications into a family of interconnected and independent software modules. As an architectural style, it emphasizes:

• Highly maintainable and testable software modules
• Loosely coupled components
• Independently deployable software packages
• Organization around business capabilities
• Ownership by a small team

MDM platforms have started evolving towards this pattern, and vendors have started structuring their MDM software as a family of microservices.

Microservices in MDM

MDM has been successful in creating a single source of truth to serve as a “back office” for data processes, but it has had challenges delivering data to destinations where it can support real-time business decisions and help you respond to customer needs. That’s where microservices come in. A microservices architecture allows you to decompose MDM’s monolithic processes and interfaces to create use-case-specific applications. MDM takes care of the data quality and consistency, while microservices deliver it to where it’s needed, when it’s needed, helping you leverage data as a strategic asset and create a competitive advantage.

For instance: real-time services could be designed and deployed to check whether a customer exists, retrieve the most recent address, or update the billing details, eliminating the need to build expensive data integrations. This architecture also enables MDM to scale in a fit-for-purpose manner. For instance, a material master microservice can be a totally independent package from a business partner microservice, and each can expose different APIs, scale differently, and be enhanced without affecting the other.

Enabling Microservices in MDM

This is a paradigm shift being enabled by several factors beyond just the technology.

Cloud SaaS MDM platforms. Many modern cloud-based MDM platforms make it simpler for MDM teams to construct and deploy microservices that can programmatically invoke a set of integration services to access data strewn across an extended enterprise.

API-based integrations. APIs are at the heart of enabling microservices from a technology standpoint. A well-designed API integration layer in the MDM architecture would turn an MDM platform into a plug-and-play solution while integrating new systems.

Use-case-based services. These APIs would need to be designed to serve a specific use case. This shifts the organization of an MDM solution from being focused on a data domain to being focused on a business use case.

Regression test beds. With the increasing velocity of MDM releases, a robust regression test bed is essential to maintain solution integrity. Test automation has become a de facto standard in software development and needs to be implemented within the MDM solution

Agile delivery approach. Finally, the development processes must embrace agile practices to support frequent releases of small packages of code that drive business value.

Planning Your Organization’s MDM Evolution

Most major MDM products have already embraced the microservices-based architecture in their cloud offerings and are moving away from traditional MDM monoliths. It is imperative that your organization looks into this and assesses if it would make sense to start embracing it. However, the transition does not end with just adopting a new version of the technology. This paradigm shift will also require a change in mindset and a change in your development process with regards to master data management.