TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

The Appliance Tipping Point: Commoditization

DATAllegro last week unveiled an off-the-shelf data warehouse appliance based on hardware from partners Cisco, Dell, and EMC.

• By Stephen Swoyer
• February 28, 2007

LAS VEGAS, Nevada—Appliances, data warehousing and otherwise, took center stage at last week’s TDWI Winter World Conference, held in Sin City amidst the clamor of Chinese New Year and the NBA All Star Game.

Appliance specialist DATAllegro helped generate much of the excitement, thanks to its in-conference announcement of an all-but-off-the-shelf appliance solution, which it plans to deliver in conjunction with hardware partners Cisco Systems Inc., Dell Computer Corp. and EMC Corp. But speculation about the future of the appliance segment—especially amid news that another pure-play appliance vendor might soon make a splash of its own—also helped contribute to an overall sense of appliance fixe.

DATAllegro kicked things off last Tuesday when it announced its new “v3” series, DW appliances that leverage the commodity hardware, in addition to the company’s bread-and-butter appliance software and the open source Ingres database. DATAllegro’s new v3 series consists of two discrete product lines: DATAllegro Multi-Rack Appliances (MRA) and DATAllegro Single Rack Appliances (SRA). DATAllegro says MRA—which starts at $24,000 per TB—can scale from 15 TB on up to 1 PB; the SRA line-up, on the other hand, consists at the moment of DATAllegro SRA12, a $500,000 data-warehouse-in-a-rack that can support a maximum of 12 TB of user capacity.

CEO Stuart Frost describes DATAllegro’s new v3 systems as “next-generation” DW appliances, based entirely on commodity hardware and software components. The upshot, he argues, is a cheaper overall platform that outperforms its predecessor systems, to boot. “We’re not engineering the hardware, or the storage, or doing any of that, so it’s much cheaper,” he comments.

Frost spins commoditization as an inexorable historical force: the proprietary always gives way to the commodity, he argues, even in cases where proprietary solutions have helped jumpstart markets. “In IT markets, the first mover in any real disruption has been proprietary, and they’ve done quite well,” he says. “But whenever they’ve been challenged by a commodity platform, the commodity players have always won out. Always.”

DATAllegro’s announcement wasn’t much of a surprise, actually: company officials had previously hinted at the likelihood of making a commodity hardware play sometime in the early months of 2007.

Nevertheless, DATAllegro arch-competitor Netezza Inc.—the vendor that not only developed and marketed the first DW appliance, but also demonstrated the viability of the appliance model—was noticeably silent last week. Company officials did not respond to several requests for comment.

Confusion about Netezza’s strategy abounds. For example, some DW industry watchers suggest that its Netezza Performance Server (NPS) systems are architecturally dependent on the PowerPC chips it uses to augment its Snippet Processing Units (SPU). SPUs are individual nodes—ranging from 56 in Netezza’s 6 TB NPS 10050 entry-level appliance on up to 896 in its high-end 100 TB NPS 10800—which provide the query processing muscle for Netezza’s massively parallel processing (MPP) implementation.

At a time when DATAllegro, Kognitio and a host of quasi-appliance competitors—including not just high-end DW powerhouse Teradata (a division of NCR Corp.), but also Hewlett-Packard Co. (HP), IBM Corp., and Sun Microsystems Inc.—are touting the strengths of commodity-based hardware platforms, Netezza’s PowerPC-based SPU architecture could conceivably put that company at a competitive disadvantage, particularly if Netezza is in fact architecturally dependent on PowerPC.

This concern is without merit, however, according to no less an authority than former Netezza CTO—and DW appliance creator—Foster Hinshaw. “From the outset, the thinking was, as [the appliance] evolves, you keep the same shell, but you upgrade your innards,” says Hinshaw, who recently left Netezza to pursue other opportunities. “So Netezza absolutely is not dependent [on Power]. They could use any chip in there.”

But if Netezza isn’t architecturally dependent on Power, why does it continue to use it? There are a number of different reasons, says Hinshaw: for starters, PowerPC itself is a commodity chip—albeit one that isn’t as widely available as x86 processors from Intel Corp. and Advanced Micro Devices (AMD) Inc. The important takeaway, he argues, is that Netezza-running-on-PowerPC not only helped establish the DW appliance’s signature price/performance value proposition but clearly outperforms conventional competitors, too. “The reality is that with PowerPC, Netezza is 100 times faster than an Oracle [real application clusters] system,” he maintains. “We’ve taken a tiny Netezza three-bladed and one Oracle/EMC/Sun solution and Netezza’s performance just blew them away.”

There are other considerations, too. The Power chips Netezza uses in its SPUs are low-power, high-efficiency designs. They consume less energy—and, consequently, dissipate less heat—than competitive designs from Intel and AMD. One upshot of this, says Philip Howard, an analyst with consultancy Bloor Research, is that Netezza’s SPU innards translate into an advantage in high-density environments—particularly in scenarios where data center space is at a premium and cooling requirements are similarly onerous.

Data warehouse architect Mark Madsen, a principal with BI and data integration consultancy Third Nature Inc. (and co-author of Clickstream Data Warehousing, a how-to book about building Web-enabled data warehouses), says he’s heard mostly good things about Netezza—especially from the company’s customers—but suggests that the company’s PowerPC-based strategy might not be feasible in the long-term. “The classic complaint for appliances is that the hardware market makes steady progress. When you go with specialized hardware for something big and complex, you risk slowing progress relative to general-purpose hardware. The inexorable climb of the market means that software on general purpose hardware eventually catches up,” Madsen indicates.

Nor is that all, adds Madsen. “The cost of general purpose solutions stays level or even declines relative to custom. If you keep investing in engineering to maintain your lead over general purpose, your cost is higher and therefore you need higher prices to maintain your margins,” he indicates. “Eventually you are eaten from below because you've positioned yourself to serve only the very high end of the market. The ceiling for large data solutions is moving up, but the market is moving up as well. If the market goes faster you're toast.”

That’s probably why DATAllegro went the commodity route, he maintains.

“They couldn't maintain the cost structure for that engineering versus the performance gains they were seeing. The new partnerships should also help them drive sales into accounts where they currently don't have easy access, a point they stressed,” Madsen concludes.