LESSON - Data Integration: Consider It an Enabling Platform
Creating a data integration platform enables organizations to provide faster and more flexible mechanism for data integration.
By John Williams, Senior Vice President, Collaborative Consulting
Introduction
Historically, data integration was often considered an activity to pass information between applications. More recently, data warehousing highlighted the need for more substantial data integration capabilities, as it required integration across multiple applications and consolidation into a single environment. While the two types of integration may differ in some ways, they are fundamentally alike, with the same basic requirement: get key elements of information moved from one application to the other and transformed into a format that the receiving application can use.
Data integration also has presented one of the key challenges to any new application implementation. As much as one half of the effort expended on new application implementation is focused on redeveloping the integration points to other applications. Much of this effort is spent on understanding the specific details of the existing integrations before beginning the rewrite process. This occurs regardless of whether the integration is to a data warehouse or an operational application.
The current focus on information as an asset, as well as the rapidevolution of data integration tools, has changed the way we look atdata integration.
The Data Integration Platform
This traditional model is changing rapidly. The current focus on information as an asset, as well as the rapid evolution of data integration tools, has changed the way we look at data integration. As organizations move to new application platforms, implement new data warehouses, or simply struggle with new application implementation, they are beginning to realize that having a robust data integration capability is key to their continued success.
More and more organizations are envisioning and creating a data integration platform. This concept has been evolving over the past decade or so with the advent of data warehousing and ETL tools, which fundamentally have been focused on batch processing. It has been fostered further by the introduction of EAI and messaging architectures, which are more focused on real-time and nearreal- time data movement. It is not simply a technology; it also consists of process, people, and tools.
Creating a data integration platform enables an organization to provide a faster and more flexible mechanism for data integration. This environment can be used across both operational and analytical applications, and its sole purpose is to facilitate information movement between environments. It limits or eliminates redundant efforts to map data from one system to another by creating a documented understanding of the information that is being provided by one system to another, enabling other systems that need that same information to take advantage of its availability. It provides a methodology for making consistent decisions on how to integrate data across existing and new applications and creates institutional knowledge of data integrations so that they can be maintained over time.
Application Platform Vendor Offerings
This capability is fast becoming one of the key selling features of the application platforms. All of the major solution providers have made rapid application integration a crucial selling feature of their particular solution, most recently adding business intelligence integration to their portfolios. The current risk with the platform provider offerings is that they do not consider integration to products outside of their offerings adequately. Unless an organization is vested fully in a single vendor offering, it will be critical for it to create a data integration platform independent of the platform vendor offering to allow for integration of other applications.
Conclusion
Organizations must quickly develop a data integration competency or continue to struggle and spend valuable resources reinventing application integrations every time a new application comes on board or an existing application needs additional data. The tools have evolved and become substantial enablers to creating this capability; but beware of assuming that the application platform vendor offerings are adequate to the task. Focus on creating a data integration platform, made up of people, process, and tools, that provides a capability that is an advantage to the entire enterprise. If done right, this capability will not only accelerate application implementation initiatives but also foster an institutional knowledge of information that can be leveraged across the enterprise now and in the future.