TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

LESSON - Accelerating Your Business with Real-Time Data Integration

• May 6, 2010

By Dain Hansen and Irem Radzik, Directors of Product Marketing, Data Integration, Oracle

Today’s most innovative companies recognize that integrating information effectively is key to accelerating the business. Those organizations seek opportunities to capture and respond to business events faster and more rigorously. Real-time data integration is ideal for maximizing value from an organization’s information assets.

The good news is that real-time data integration solutions are readily available, proven in their benefit, and present a lower barrier to entry than ever before. To fast-track the adoption and advantages obtained from real-time data integration, organizations must determine not just where it will bring business value, but also the criteria for choosing the best technology option.

Value Drivers

Real-time data integration presents significant opportunities to help the business grow and excel, but it is not without architectural considerations when applied to data warehousing, business intelligence (BI), and service-oriented architectures (SOA).

Next-generation data warehousing and operational BI: More companies are empowering front-line employees with improved operational decision support through the use of fresh, relevant data. Here, day-old data is not acceptable. The data must be complete, accurate, and trusted—and now more than ever, timely. Operational BI is best supported by real-time data integration, typically fed to an operational data store or a data warehouse.

In today’s 24/7 enterprise, traditional data warehousing solutions face a major challenge: moving very large and growing data volumes within shrinking batch windows. Real-time data integration supports a new era for data utilization, by streaming operational data to the warehouse without batch windows while enabling timely data.

Support for SOA: Real-time data integration can contribute to more successful SOA initiatives, providing the heavy lifting of data movement without burdening the business process execution engines. This helps lower costs (by consolidating legacy systems), reduce the risks of outdated data, and shorten the time to deliver new service offerings.

Selecting the Right Technology

Many real-time data integration solution offerings may sound similar on the surface. To make the best choice, there are five important aspects that should be closely evaluated:

1. Performance. The optimal data integration solution is capable of keeping up with the volume of changed data at the determined latency. This could mean moving thousands of transaction operations per second. Spikes during peak times should be handled with little to no additional latency.
2. Impact. Continuous, real-time changed data feeds should not impact the performance of source systems. Look for technologies that move only the necessary data (for example, only committed transactions) and offer features such as filtering and compression to further minimize bandwidth requirements.
3. Flexibility. Beyond support for heterogeneous systems, the technology should be easy to implement, manage, and scale as needs change, and should support myriad topologies. For example, adding new data sources and/or targets should be straightforward and not require major overhauls.
4. Data integrity and recoverability. The real-time data integration solution should be durable and easily recover from human errors and unexpected interruptions, such as network issues, without losing or corrupting transactional data. During the evaluation period, test recoverability given several failure scenarios.
5. Integrated and complete. Look for solutions that combine the core elements of data integration—real-time and bulk-data movement, data synchronization, data quality, and data services—to ensure your investment serves the whole enterprise with timely and trusted information.

Conclusion

Real-time data integration helps to make organizations more profitable and competitive. Technologies that support a comprehensive and flexible approach to data integration deliver timely, accessible, and high-quality data. Companies should focus on solutions with high performance, low impact, and reliability to support today’s 24/7 business.

For a free white paper on this topic from Oracle, click here and choose the title “Real-Time Data Integration for Data Warehousing and Operational Business Intelligence.” For more free white papers, click here.