CASE STUDY - Epsilon Improves Client-Facing Data Services: Talend Integration Suite Builds a Demographic, Compiled List File from Multiple Formats
Commentary by Aaron Dix, Senior Technical Manager, Data Engineering, Epsilon
Leading Marketing Services Firm
Founded in 1969, Epsilon is the industry’s leading marketing services firm. Ad Age ranks it the number one U.S. marketing services firm and the number one U.S. direct marketing agency. Epsilon combines the power of the world’s most extensive collections of consumer and business data with world-class creative and proven techniques to maximize marketing success for clients worldwide.
Epsilon’s problem was typical for a company that aggregates data for its clients. Data arrived in many different formats and required tedious hand coding. The company needed to streamline the integration process.
Interfacing with Legacy Systems
Epsilon had prior experience with proprietary tools and didn’t want to experience the restrictions of a closed solution for its projects. “We wanted a solution that was more formally focused on data integration,” said Aaron Dix, senior technical manager of the data engineering group.
Though cost was a consideration in looking for a new solution, Epsilon found that of the products they tested, Talend was one of the easiest to use. Because previous programs were written in Java or Perl, the developers were already comfortable with the technology. “Talend outperformed some of the other products we were testing,” said Dix. “However, the determining factor was that the project involved legacy code that we needed to integrate into our build solutions. With its ease of use, Talend allows us to easily interface with external processes. Basically, we’ve overlaid a lot of legacy technology with Talend and it’s much easier to maintain.”
"Our project involved legacy code that we needed to integrate into our build solutions. With its ease of use, Talend allows us to easily interface with external processes." —Aaron Dix, Senior Technical Manager, Data Engineering, Epsilon
Building a Demographic Database
Epsilon’s project entailed building a compiled list containing geographic and demographic data, including age, ethnicity, occupation, and income—appending more than 800 attributes. This data is sourced from more than a dozen different formats and loaded into the database after processing through Talend. “It’s a very large database,” explained Dix, “containing around 430 million records.
“Initially the data is imported in flat files that are usually fixed or delimited—ASCII, DOS, or UNIX,” he continued. “We actually use the Alterian Integrated Marketing Platform on the back end. Talend handles the integration aspect in between. Typically, Talend also performs file retrieval; it’s got a very efficient set of components that integrates well into what we’re doing.
“The code generation approach is also a plus. We can verify a job by looking at the generated code and then adapt it to our needs. We can also write code that is then invoked from within our Talend processes. By adding user routines, we create components or import them from Talend’s extensive library. And we can reuse roughly 80 percent of it just by calling the routine we developed.”
From Talend Open Studio to Talend Integration Suite
Epsilon invested in Talend Integration Suite. “It was well worth the investment,” Dix said. Beyond value-added features for larger projects, the Talend Integration Suite subscription also includes technical support and IP indemnification.
“If you work on many different systems, even for testing, the product is very efficient,” said Dix. “Instead of manually exporting your code over to many different systems, Talend Integration Suite lets you launch that code and test it on different systems from a single repository. It also facilitates reusability and makes teamwork pretty seamless.”
Talend Integration Suite is also in use elsewhere in the company. “We have a sandbox for ad hoc projects,” explained Dix. “Talend lets us take external data formats and quickly create a database where we can parse and analyze data.”
For a free white paper on this topic from Talend, click here and choose the title “The Top 10 Reasons for Choosing Open Source Data Integration.” For more free white papers, click here.