June 27, 2012
New trends in the industry are making data integration more paramount than ever. Essentially, the concept of data integration is being redefined; it is no longer limited to extracting data sets from internal sources and loading them into a data warehouse, but instead focuses on effectively facilitating the delivery of information to the right places within the appropriate time. Data integration goes beyond extract, transform, and load (ETL); data replication; and changed data capture, although these remain key components of the integration fabric.
It is also hard to ignore the “big data” revolution as organizations seek to adapt their information management environments to accommodate massive data volumes coming from a large variety of sources (especially internal ones!) at accelerating speeds. Some of the challenges include absorbing numerous data feeds (both internal and external), moving data to analytical appliances designed for big data analytics, coupling the results with existing data warehouse and business intelligence (BI) architectures, and delivering results to a variety of downstream knowledge workers and information consumers. When coupled with the migration of storage, infrastructure, and business applications to the cloud, we see how these trends require moving large amounts of data to many different places in real time without allowing data replicas to become out of sync with each other.
Mounds of structured data, unstructured data, big data, and advancements in cloud technology are imposing new requirements for data integration. This TDWI Checklist Report will explore some of the key drivers these new requirements are intended to address. Whether you are looking to support the performance needs of big data applications, filter concepts from unstructured data, monitor hundreds of data feeds for unexpected behavior, export data across enterprise boundaries, or provide real-time reporting and analysis, there is a rapidly expanding need for data integration competency that extends well beyond traditional ETL.