Six Ways Data Replication Enables Real-Time Data Integration
As real-time operation becomes a more prominent business requirement, don't forget data replication.
- By Philip Russom, Ph.D.
- September 11, 2012
According to a 2011 TDWI report, almost half of data integration specialists are using some form of data replication today. Yet, we seldom hear much about replication in the IT press. We do hear about the need for data integrated in real time, which, by fortuitous coincidence, data replication can handle easily.
For example, real-time configurations of data replication technologies support fast-paced business practices, such as operational business intelligence and just-in-time inventory. Replication can synchronize 360-degree views of customers and other business entities across heterogeneous applications. Data replicas are an important component of business continuity, and the use cases for replication span both operational and analytic applications.
Here six real-world use cases where data replication enables real-time data integration:
Use Case #1: Operational BI
This is the most widely adopted BI practice of recent years. Operational BI fetches fresh data from operational databases and applications, then presents that data as metrics or key performance indicators (KPIs) in a management dashboard or scorecard. Although many types of tools can fetch operational data, replication is ideal for this use case because it is relatively non-intrusive for the applications, it has interfaces to most application data, and it can operate in real-time.
Use Case #2: Real-time analytics
Reporting has accelerated into real-time data (as seen in operational BI) and analytics is now experiencing the same acceleration. Hence, there's a growing need for real-time data to support time-sensitive analytics such as customer profiling, sales forecasting, price optimization, production yield in manufacturing, fraud detection, and risk calculations. As with operational BI, real-time analytics can be enabled by replication.
Use Case #3: Real-time data warehousing
The real-time data for operational BI and real-time analytics must be managed somehow. For this purpose, a real-time data warehouse integrates and aggregates data that will feed into reports and other BI products that are refreshed frequently or on-demand. The real-time data warehouse serves up time-series data to provide a historic context for real-time data. The "secret sauce" of a real-time data warehouse is a real-time or near-time data integration technique, typically federation, messaging, micro-batch ETL, or replication.
Use Case #4: 360-degree views
Integrating data across business units and applications is on the rise, especially in the form of the 360-degree view. For example, complete views of customers can improve customer service and retention as well as customer analytics. Similar benefits come from views of other business entities (e.g., products, financials, or employees). Again, replication is ideal for 360-degree views because of its fast, non-invasive access to application data plus its support for bidirectional data synchronization across heterogeneous systems.
Use Case #5: Database high availability (HA)
A database management system (DBMS) or data warehouse cannot share data or operate in real time if it is not highly available, which is why database high availability is a requirement for any application involving real-time data or 360-degree views, as well as any time-sensitive or mission-critical application. Data replication is by far the most common enabler of database high availability in use today.
Use Case #6: Big data
Some configurations of data replication tools can handle the trickling and streaming data common with big data coming from Web servers, robotics, and sensors. Although we automatically think of replication's real-time capabilities (due to its long service in HA), modern replication tools can also extract and load large datasets in near time (say, an hour per terabyte or faster), which is imperative for analytics with big data.
For more information, read the 2012 TDWI Checklist Report Data Replication for Real-Time Data Warehousing and Analytics, available online at http://bit.ly/DataReplication.