Infobright Does It Right
I love software that I can download from the internet, implement without reading a manual or attending a training class, use in production free of charge for 30 days, and then purchase with a credit card at bargain basement prices.
I did just that recently with Citrix’s GoToWebinar software. I even conducted a very successful live Webinar for 300 people before the trial period ended. For a fraction of the price and with almost all the same bells and whistles, GoToWebinar puts competing products to shame, such as On24, WebEx, and LiveMeeting.
You might assume, as I did, that this bright new era of Web-based software distribution is suitable for low-end solutions and consumer-oriented software, but certainly would never work for enterprise-caliber systems, such as those we have come know and love (or hate?) in the data warehousing (DW) and business intelligence (BI) market. But you would be wrong.
15-minute Rule
I had a nice chat today with Miriam Tuerk, CEO, and Susan Davis, VP of marketing, at Infobright, an open source provider of a high-end analytical platform by the same name that targets the MySQL market. The company, based in Toronto, was founded in 2005, began shipping product in 2007, and now has 50 paying customers, mostly mid-market companies in the Internet, telecommunications, and financial services markets that want to analyze large volumes of data and expect fast query performance at low cost without much setup or maintenance.
“We have a 15 minute rule,” says Tuerk. “A knowledgeable MySQL database administrator should be able to download Infobright and implement it within 15 minutes.” Tuerk estimates that 10,000 individuals have downloaded the free, community version of the software. Paying customers, which include Xerox, Telus, and the Royal Bank of Canada, pay annual fees of $9,950 and $15,950 per terabyte for additional features, such as faster loading, support for slowly changing dimensions, the ability to insert, update, and delete data from the column-store database, and guaranteed support levels. That’s a bargain as far as I’m concerned.
Although Infobright gives up potential revenue by using this pricing and distribution model, it saves a lot by not having to conduct expensive proofs of concept in competitive deals at enterprise accounts.
Secret Sauce
Infobright is not only easy to do business with, it has some interesting technology that sets it apart from the pack of 20+ analytical platform providers crowding the market these days. It doesn’t require database designers to create indexes, partitions, and aggregates and minimizes schema design and ETL work. This saves time and money and requires fewer experts to get the system up and running.
Data Packs and Compression. It stores columnar data in 64k data packs, each of which can be compressed with different compression algorithms that are best suited to the data within. So, Infobright compresses data to one-tenth to one-fortieth of the raw data size. This lowers cost by reducing the amount of hardware and storage needed to analyze large data volumes, and it accelerates data loading speeds, which Tuerk says are the fastest in the industry per server at 280GB/hour.
Metadata. While most database management systems store two or three times the amount of raw data due to overhead (e.g. indexes, aggregates, partitions), the only extra data that Infobright adds is what it calls a Knowledge Grid, which is usually small enough to store in memory, according to Tuerk. The Knowledge Grid is metadata about the data packs—statistical data (min/max, average, counts, etc.), histograms of the data values in each data pack, and relationships among data packs. Many queries can be answered from the Knowledge Grid alone, accelerating response times to the speed of memory.
Query Optimization. Using statistics and pointers, the Knowledge Grid enables the Infobright optimizer to quickly identify the handful of data packets that need to be decompressed to resolve a query. It also works heuristically, using intermediate results to refine query plans. Rather than spreading queries and I/O across multiple nodes like MPP databases, Infobright’s architectural approach is premised on doing as little disk I/O as possible.
Scalability. From a hardware perspective, Infobright is a single-server system that currently scales to 50TB although Tuerk says that is not a physical limitation and systems can handle more data if necessary. It provides full support for SQL 92 and some SQL 99 extensions but it doesn’t yet participate in TPC benchmarks, so it’s hard to validate scalability claims. Davis says they’ve been reluctant to participate because some benchmarks require full table scans, which is what their architecture tries to avoid.
From a query perspective, each processor core running on a Windows, Linux, or Solaris server can handle a single concurrent query at peak performance. So a 32-core system can handle 32 concurrent queries at peak performance, which is a significant limitation for customers at the high-end of their target market. As a consequence, Infobright has been working on a multi-server option—which it will ship later this year—that will let customers add servers and/or storage to scale out the amount of data or concurrent queries the system can support. This is not an MPP system, just a multi-server system using a shared disk architecture.
Conclusion. Overall, if you want a quick and easy analytic platform that is also powerful enough to meet the analytical needs of 95% of organizations, you should consider Infobright. They do it right!!
Posted by Wayne Eckerson on June 12, 2009