CASE STUDY - Hybrid Data Matching Approaches in Single View of Customer and Master Data Management Applications
Commentary by Jarugumilli Brahmiah, Enterprise Architect, National Life
National Life Group (NLG) is an established
Fortune 1000 company based out of
Montpelier, Vermont. NLG offers diversified
financial products to its customers, including
life insurance, annuities, and investments.
High-Impact Business Value
As a part of its enterprisewide data initiative,
National Life embarked on a transformational
journey to achieve an integrated view of its
customers, policies, and producers. Information
pertaining to these key business assets
was spread across multiple data systems
ranging from legacy mainframe systems to
modern databases. It was crucial for the
company’s top management to analyze and
report on key performance metrics across
business units.
The Problem with Data Matching
Data matching is a crucial step in achieving
a single view of the customer and similar
master data management (MDM) applications.
The success of this process depends
on accurately matching records lacking
common identifiers and applying business
rules around the match process.
With data spread across multiple data systems
in different formats, matching the policy
information was a major challenge. After
extensive analysis of various data sources,
NLG concluded that traditional deterministic
matching solutions would not lead to a final
solution. NLG needed extensive data standardization,
including parsing, tokenization, and
custom matching algorithms, to handle
fields that had different formats and business
logic embedded into them.
Managing the Cost of the Solution
Managing the cost of the solution proved
to be an interesting exercise. It was challenging
to find a solution with the required
flexibility and scalability but minimum licensing
requirements and cost. These were the
issues faced during the evaluation process:
- Vendor development costs needed to be
low for extending the matching engine to
meet custom matching requirements
- A good majority of the matching solutions
were not capable of scaling or did
not have the functionality needed for the
complex data
- Additional licensing requirements for
standardization, parsing components
A Hybrid Solution that Works
NLG decided to use a commercial ETL
engine to handle the data integration and
more traditional matching algorithms, and
off-load the custom parsing, tokenization,
and matching aspect to an independent data
matching engine that could integrate with the
ETL engine.
NLG used another interesting option in the
build/development/deployment phase of the
project: while the in-house development team
focused on building the processes using
the commercial ETL software, a third-party
specialist company helped build the custom
matching algorithms on an open source
platform and integrate the solution with other
ETL processes. This helped in two ways:
- It minimized the development effort and
learning curve for the in-house team
in developing on a new technology/tool
platform.
- It quickly and effectively reached the
business and technical goals of the project
by off-loading the development of
the custom matching component.
Open source technologies have helped in
cost control and customization. Using a
hybrid approach could be an interesting
option for companies trying to maximize the
functionality of a solution while limiting the
impact on their budget.
This article originally appeared in the issue of .