LESSON - Top Three Methods to Optimize Your Data Warehouse with Data Integration
By Irem Radzik, Director of Product Marketing, Data Integration, Oracle
Data warehouses are becoming increasingly important as the centerpiece of all day-to-day business information. To create a robust and agile decision support solution for the enterprise, IT organizations strive to optimize the data warehouse environment. Data integration infrastructure is a critical component of data warehouse implementations and plays a major role in improving its performance, flexibility, and adoption across the enterprise.
The following methods are the top three ways data integration infrastructure can help maximize the return on data warehousing investments.
1. Increase ETL Performance with In-Database Transformations
Traditional ETL technologies need to use a middle-tier server to perform transformations before loading the data into the data warehouse. This architecture not only increases cost by requiring acquisition and management of additional servers, but it also limits the speed of the data loading process.
Data integration products that perform all transformations within the target database have major advantages both in cost of ownership and performance. This extract, load, and transform (ELT) architecture enables the solution to use the power of the target database engine, and thus delivers significantly higher performance than traditional extract, transform, and load (ETL) architectures. ELT solutions are also faster because they avoid the network hop that traditional ETL solutions have to go through from the use of a middle-tier server.
2. Minimize Batch Windows with Trickle- Feed throughout the Day
As the data warehouse supports global users around the clock, ETL batch windows are shrinking quickly. IT organizations are under major pressure to move ever-growing data volumes within a limited time window.
Increasing ETL speed is one way to reduce required batch windows. A more effective approach is to minimize it with real-time data integration. Throughout the day, trickle-feeding the data warehouse with changed data every few minutes enables the data warehouse to be accessible 24/7. For real-time data integration, choosing a log-based changed data capture method also eliminates the performance impact on source systems, allowing both OLTP and the data warehouse systems to be highly available for the business.
3. Leverage Timely and Accurate Data with Real-Time Changed Data Movement
Organizations strive to respond faster and better to their customers’ needs. One way to enable this goal is by leveraging timely information when making business decisions. Using low-latency data from source systems allows the data warehouse to deliver more up-to-date, reliable information for end users, and increases user adoption. While this capability may not be critical for some of the strategic decisions that the data warehouse supports, for operational users it is a different story. Up-to-date information helps users make better decisions and improves customer service quality and operational efficiencies. Real-time data integration helps eliminate batch window dependencies and enables the delivery of timely and accurate data to end users.
Oracle’s data integration products offer robust, easy-to-use, low-TCO solutions and enable continuous access to timely and high-quality data. The Oracle Data Integration product family includes Oracle GoldenGate, real-time data integration software designed for low impact, high performance, and strong reliability. In addition, Oracle Data Integrator Enterprise Edition delivers high-performance bulk data movement and in-database transformation capabilities with its unique ELT architecture. Finally, Oracle Data Quality provides data profiling and data quality as part of the data warehouse loading process. These products are fully integrated together to lower TCO and speed development. Oracle’s data integration products are also certified with Oracle Exadata and other major data warehouse platforms.
By using Oracle Data Integration together with Oracle Exadata, customers see a high return on their data warehouse investments by gaining multi-fold performance improvements, minimizing batch windows, and leveraging timely data for decision making.
This article originally appeared in the issue of .