By using website you agree to our use of cookies as described in our cookie policy. Learn More


CASE STUDY - Hybrid Data Matching Approaches in Single View of Customer and Master Data Management Applications

Commentary by Jarugumilli Brahmiah, Enterprise Architect, National Life

National Life Group (NLG) is an established Fortune 1000 company based out of Montpelier, Vermont. NLG offers diversified financial products to its customers, including life insurance, annuities, and investments.

High-Impact Business Value

As a part of its enterprisewide data initiative, National Life embarked on a transformational journey to achieve an integrated view of its customers, policies, and producers. Information pertaining to these key business assets was spread across multiple data systems ranging from legacy mainframe systems to modern databases. It was crucial for the company’s top management to analyze and report on key performance metrics across business units.

The Problem with Data Matching

Data matching is a crucial step in achieving a single view of the customer and similar master data management (MDM) applications. The success of this process depends on accurately matching records lacking common identifiers and applying business rules around the match process.

With data spread across multiple data systems in different formats, matching the policy information was a major challenge. After extensive analysis of various data sources, NLG concluded that traditional deterministic matching solutions would not lead to a final solution. NLG needed extensive data standardization, including parsing, tokenization, and custom matching algorithms, to handle fields that had different formats and business logic embedded into them.

Managing the Cost of the Solution

Managing the cost of the solution proved to be an interesting exercise. It was challenging to find a solution with the required flexibility and scalability but minimum licensing requirements and cost. These were the issues faced during the evaluation process:

  • Vendor development costs needed to be low for extending the matching engine to meet custom matching requirements
  • A good majority of the matching solutions were not capable of scaling or did not have the functionality needed for the complex data
  • Additional licensing requirements for standardization, parsing components
A Hybrid Solution that Works

NLG decided to use a commercial ETL engine to handle the data integration and more traditional matching algorithms, and off-load the custom parsing, tokenization, and matching aspect to an independent data matching engine that could integrate with the ETL engine.

NLG used another interesting option in the build/development/deployment phase of the project: while the in-house development team focused on building the processes using the commercial ETL software, a third-party specialist company helped build the custom matching algorithms on an open source platform and integrate the solution with other ETL processes. This helped in two ways:

  • It minimized the development effort and learning curve for the in-house team in developing on a new technology/tool platform.
  • It quickly and effectively reached the business and technical goals of the project by off-loading the development of the custom matching component.

Open source technologies have helped in cost control and customization. Using a hybrid approach could be an interesting option for companies trying to maximize the functionality of a solution while limiting the impact on their budget.

This article originally appeared in the issue of .

TDWI Membership

Get immediate access to training discounts, video library, BI Teams, Skills, Budget Report, and more

Individual, Student, and Team memberships available.