LESSON - Finally the Right Tools for the Job: Fix the Flat Tire, Replace the Plugs, or Buy a New Car
By Ira Warren Whiteside, Chief Data Quality Evangelist, Melissa Data
Marketplace
Data quality software, as defined by analysts, consists of data profiling, generalized cleansing, parsing, matching, enrichment, and monitoring. Most clients have implemented each of these processes in some fashion. Some lean more toward custom code, and others toward a mix of vendors. We believe the market is ready for an incremental approach to solving a customer’s data quality problems. The primary goal is to provide the customer with the tools to integrate each entity to the specific level required or needed. Whether customers need a one-off quick fix or a permanent production solution, they should have the ability to use the same toolset for either problem.
A Conceptual Data Quality Smack down
All too often we seek an alternative solution instead of repairing or fixing what we already have. From this standpoint, we need more tools and fewer solutions. This may seem counterintuitive in the BI vernacular; however, it is exactly what we do in the real world.
In our current economic market, most clients are interested in fixing their data and leveraging their existing tools with their existing resources—contractors or employees. Let’s look at some real-world examples: You buy a car, it breaks, and you fix it. You build a house, it breaks, and you fix it. Your first option is usually to repair rather than replace. In BI, we take a skewed approach and assume the customer needs a new solution rather than a repair. Perhaps we should look at our customers’ existing applications as their solution (car, house) and concentrate on providing them the means to repair what’s broken.
The fix to a customer’s problem may not be a new solution, but rather a new tool. Most customers have unique requirements and partial solutions in place. By focusing on providing the right tools, customers can begin to repair their existing data quality solutions and gradually improve their processes until they are satisfied. Another issue facing customers in the BI or software world is that they can’t just buy one tool—vendors usually want to sell a complete solution in lieu of a tool or a set of tools. A new approach would be to provide the tools and instructions for customers to help themselves—in essence, a DIY (do it yourself) approach.
In addition, a client may have many platforms in use. If customers take a toolkit approach, they can use the tool in whatever platform they have. For instance, if you need profiling and/or address correction, you want a tool that integrates into all of your platforms (e.g., Microsoft SQL or SSIS, Oracle PL/ SQL or OWB, Informatica). It is true that this approach would involve some custom fitting, but just like the real world, if you need a new carburetor or a new water heater, you simply install it.
For a free white paper on this topic, click here and choose the title “Six Steps to Managing Data Quality with SQL Server Integration Services.”