LESSON - Building an Agile and Trusted Data Foundation
A data competency conter helps resolve data quality and integration issues, so you deliver trusted data across the organization.
By Philip On, Director, Enterprise Information Management, Business Objects, an SAP company
As the volume and sources of your organization’s data grow exponentially, the need for trusted and accurate information accelerates. To effectively gain insight from your enterprise data, you need an enterprise information management (EIM) strategy that simultaneously addresses data integration and data quality.
Problem
Most organizations approach data integration and data quality projects tactically. A team of developers or consultants assembles to address a specific business need, and when the project is over, they disband—with the result that skills and knowledge are not reused. Compounding the inefficiency are enterprise-class data integration tools that are overly complex and that require months of ramp-up time to be proficient. On the flip side are tools that are too limited—tools that, for example, support advanced requirements such as data cleansing with hand coding.
Solution
Think Strategic
Building an agile and trusted data foundation requires a strategic approach to information management. Think big—but start small. Although you can approach your information management strategy with one building block at a time, you must consider the big picture every step of the way. For example, you may want to start with a data quality project to clean up your customer data during the extraction, transformation, and load (ETL) process. Subsequently, you may want to leverage the same cleansing rules to improve your customer data at the operational customer relationship management (CRM) source system.
Create Standards
Consider forming a competency center for data to drive your organization’s data governance and standards. A bank in Belgium faces challenges with getting an accurate view of its business because it uses 15 different terms to describe company revenue, including “total sales,” “net revenue,” and “net income.” A data competency center reconciles the differences and enforces standard terms that ensure more accurate use of information among users. A data competency center helps you resolve data quality and integration issues and, as a result, you deliver trusted data across the organization.
A data competency center helps you resolve data quality and integration issues and, as a result, you deliver trusted data across the organization.
Seek Out the Right Tools
Organizations understandably favor automated tools over hand coding to support data integration and quality projects. These are a few key considerations when selecting a tool.
Prioritize Ease of Use
If effective use of a tool requires hiring specialized consultants, then you should think twice before considering it. A tool should accelerate development cycles, not extend them. Some technologies require separate interfaces for project management, data profiling, ETL design, data quality, debugging, and deployment. Look for a tool that will help you accelerate time to market by allowing you to seamlessly develop an entire data integration or data quality process using a single interface.
Prioritize Data Quality
Data quality is the hardest but most important part of information management. When evaluating technologies, make sure your choice handles rich cleansing processes as well as data in any industry or operational domain, such as customer, product, and beyond. Your data quality solution should allow you to parse, cleanse, complete, match, and consolidate enterprise data anywhere in your IT infrastructure. You should be able to cleanse data at its source, during the ETL process, or during data entry.
Prioritize Metadata Management
Metadata provides the contextual information to help you understand what happens to your data as it passes through various stages of its life cycle—from creation, to transformation, to consumption by the business user. Without metadata, it’s impossible to fully answer the question, “Where did this number come from?” To achieve this kind of insight, you need to ensure that the tool you select provides out-of-the-box metadata integration among your data integration, data quality, business intelligence (BI), and enterprise resource planning (ERP) environments.
Conclusion
Building an agile and trusted data foundation begins with a strategy for information management that promotes standards and reuse of resources across the enterprise. Be sure you evaluate your tools on the merits of ease of use, data quality, and end-to-end metadata management.