TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Q&A: Pair Appliances and Virtualization to Beef Up Performance

Data warehouse appliances and virtualization make logical partners for companies interested in cost-effective solutions to handling massive amounts of data across devices.

• By Linda L. Briggs
• August 25, 2010

Despite the continuing slow economy, demand for data warehouse appliances is growing. Businesses taking advantage of the benefits of this hardware range from a huge worldwide convenience foods business to a baseball league. Data virtualization can act as a strong partner to appliances, providing a single view of information across multiple appliances. Data virtualization is also useful because it provides a stable reporting layer during normal migration exercises, such as when additional data warehouse appliances are added to the information infrastructure.

In this interview, Robert Eve, executive vice president of marketing for Composite Software, Inc., an independent provider of data virtualization software, talks about the growth of data warehouse appliances and data virtualization. He also suggests critical considerations for companies that want to introduce appliances and data virtualization into the enterprise.

BI This Week: Why is demand for data warehouse appliances currently accelerating?

Robert Eve: At a macro level, it’s the confluence of three primary drivers. The first is the well-reported information explosion, and the technical challenges involved in making this information accessible in forms that business decision-makers can easily use. Second, costs per terabyte and for support are coming down, making data warehouse appliances more affordable and therefore more appealing. Finally, recent advancements in analytics technology, particularly in predictive analytics, promise to help make sense of the massive data volumes we’ve been talking about.

As data warehouse appliances continue to advance in functionality, they support the nascent predictive analytics technology, further enhancing their attractiveness as a strategic component to the enterprise information architecture.

What are some examples of the business impact of data warehouse appliances?

I heard many terrific examples of positive business impact among a broad range of industries at Netezza’s recent user conference, Enzee Universe, which took place June 21 to 23. Here are a few highlights from the conference:

• A leading worldwide convenience foods business uses data warehouse appliances and analytic applications to garner major business benefits in two key areas. First, the company optimizes its international network of delivery routes, making the system more efficient and ensuring timely delivery of its products. Second, the company continuously refines its merchandizing mix daily on a retail outlet-by-outlet basis to maximize sales and margins.
• Major League Baseball captures information about every pitch, at-bat, and fielding play within a data warehouse appliance, using this data to predict players’ future on-field performance. This helps teams evaluate current and free-agent talent, refine coaching and development methods, and determine salaries -- thereby maximizing their wins.
• A global freight, transportation, and logistics company uses data warehouse appliances to identify behavioral patterns that indicate potential dissatisfaction within its existing customer base, therefore arming its customer care group to proactively take steps to improve satisfaction before customers defect to another provider.

When we talk about data warehouse appliances, where does data virtualization fit in?

Data virtualization is a strong complement to data warehouse appliance strategies, especially in large enterprises with complex information architectures and multiple data warehouse appliances. That’s because data virtualization quickly and easily provides a single view of information across multiple appliances. There are a number of examples in which this type of data abstraction and data sharing is particularly useful:

• Across departments, such as marketing, sales, and finance, for a single view of customers
• Across distributed geographies, such as the Americas, Europe, and Asia to gain a single, global view of operations
• After a merger or acquisition, for timely and minimal risk when merging information contained in the information systems in merged company A and merged company B
• Over time-partitioned data sources to combine operational, strategic, and archived data

Data virtualization is also beneficial because it provides a stable reporting layer during normal migration exercises, such as when additional data warehouse appliances are added to the information infrastructure.

To drill down into a specific example, let’s take the case of an enterprise that wants powerful marketing campaign analysis across its multiple sales and marketing channels. A typical enterprise information system infrastructure might feature two or more data warehouse appliances acting as analytic data stores for web site traffic and e-mail marketing analytics, as well as another appliance containing customer master records combined with a high-performance data virtualization platform that federates these data warehouse appliances, along with as sources such as sales force automation and customer service applications.

The combined, highly optimized solution of data warehouse appliances and data virtualization delivers the full range of customer information required from agile, 360-degree view queries of customer engagement to wide and deep marketing campaign analyses.

What are some critical considerations when companies begin using data warehouse appliances with data virtualization in their enterprise architecture?

Performance is the most important priority when introducing or expanding the number of data warehouse appliances and complementary data virtualization platforms in the enterprise architecture. To ensure the highest performance possible, IT teams should weigh appropriately how the data warehouse appliance vendors and their data virtualization solution counterparts have maximized performance. Have they teamed up to engineer new algorithms and techniques that specifically leverage and extend the native optimizers within the appliance? Has the data virtualization vendor extended its high-performance federated query algorithms and techniques for the types and mix of queries found when querying across multiple appliances? Has the vendor done only the minimum required, by simply connecting to the source appliances and reusing existing optimizations, without the value-added engineering needed to meet enterprise-scale performance needs?

How does your recent announcement with Netezza about the Netezza Data Virtualizer fit into what we’re discussing here?

At its core, the newly announced Netezza Data Virtualizer powered by Composite Software links two or more Netezza data warehouse appliances into a single, federated environment. In too many enterprises, data marts and analytic appliances operate as islands within their information architectures. Yet, today’s organizations depend on information typically from across multiple data sources and business applications for decision making. Alternatively, data may be copied across databases to allow queries that span across the information islands. The ability of the Netezza Data Virtualizer to run federated queries overcomes this issue in a way that is both agile and relatively inexpensive.

Both Netezza and Composite Software are high-performance leaders in their respective fields, so this was our starting point in our alliance as we developed the new product. As a result, we engineered four new, unique optimizations that enable the Netezza Data Virtualizer powered by Composite to further accelerate queries across Netezza appliances.

[Editor’s note: More information about the Netezza Data Virtualizer is available at http://www.netezza.com/data-warehouse-appliance-products/virtualizer.aspx and http://www.compositesw.com/index.php/resources/white-papers-reports/]