Upstarts Filling Business Intelligence and Data Warehouse Gaps
Upstart BI and DW vendors say their offerings address gaps or pain points established solutions ignore. It's a pitch that resonates with customers.
- By Stephen Swoyer
- December 9, 2009
The influx of new players in the business intelligence (BI) and data warehousing (DW) segments has certainly made things interesting. Unfortunately, some analysts say, that's all these vendors been good for. Once they're asked to show their cards, skeptics allege, they go bust -- the vendors talk a good game, and they have promising technologies, but customer adoption (the yardstick of success) never takes off. Doubters like to point to would-be power-players-gone-bust such as LucidEra, Dataupia Inc., and (on the basis of fewer than half a dozen customers) DATAllegro Corp.
BI and DW upstarts are undeterred. For one thing, every BI or DW company worth its salt can point to a few customer success stories, and that hasn't always been the case. Half a decade ago, for example, DW pioneer Netezza Inc. was hard pressed to identify more than one or two named customers, while nascent rival DATAllegro couldn't produce so much as a single reference.
Furthemore, today's BI and DW entrepreneurs can point to customer success stories that purport to underscore the value -- or the differentiation -- they bring to the table. Their claim isn't just that their technologies add value but that their technologies add value by virtue of being differentiated from established offerings. Their offerings address needs, gaps, pain points, or cracks that established solutions ignore, address imperfectly (e.g., big square peg, not-so-big round hole), or simply omit, chiefly as a result of bloat or scope.
Take Lyza Soft, a BI newcomer that markets Lyza, an end-user-oriented analytic workbench. It's one of a handful of vendors that field "Workgroup BI" solutions. According to CEO and founder Scott Davis, Lyza -- like other end-user-oriented BI offerings -- targets the otherwise unaddressed requirements of real-world users. Davis has little patience for the hemming, hawing, or grousing of the data management (DM) establishment -- the folks who tend to advocate a highly-centralized, single-version-of-the-truth-or-bust approach to BI and DW. In fact, such behavior has kept some kinds of users -- particularly power users, analytic experts, and motivated self-starters -- penned up and effectively constrained for far too long, Davis insists.
Similarly, he argues, the BI suites sold by IBM Corp., Oracle Corp., and SAP AG -- among other big players -- are unsuitable for these classes of users, too. Not only are they far too square-peggish, Davis insists, but -- by virtue of their out-of-the-box deployment and management models -- they defer by default to IT. One upshot of this, he contends, is that highly motivated users -- the constituency Lyza itself targets -- tend to feel constrained by even the most self-serviceable of Big BI tools.
"Pretty much all [the users we're targeting are] doing is opportunistic, long-tailed stuff. For example, they're using it as a prototyping device for how do you collapse the implementation time on a longer project," explained Davis, in an interview at TDWI's World Conference in San Diego. "We have no intention of picking a fight with Google or Microsoft or any such entity. There is a vacuum space, a void, created by the big ship as it cruises through the water. There is a space for a non-[establishment] product, and we think that we can be that product."
It's a message that seems to resonate with users. Lyza touts at least one reference user (Baldwin Mutual Insurance), but other new BI vendors tally half a dozen or more. Consider ParAccel Inc., one of several players that markets a massively parallel processing (MPP) database. One of its customers -- Autometrics, a software-as-a-service (SaaS) firm that markets offerings for the automotive, travel, and telecommunications industries -- tapped the ParAccel Analytic Database (PAD) to address problems of scale that, from its perspective, were simply unprecedented.
Last year, Autometrics decided to develop a new market demand service for its automotive customers, but when the company conducted its requirements analysis, it determined that the proposed application -- Autometrics Pulse -- would both consume and generate data at a pace that threatened to overwhelm Autometrics' SQL Server-based DW infrastructure.
"For our Autometrics Pulse [applications], we currently have about 235 million records, and we're growing at about one million records a day. We needed something a lot more powerful than what we had, but at that point, we didn't know if any [solution] could handle what we planned to throw at it," says Randy Rubin, vice-president of business development and delivery with Autometrics.
"With everybody using Google, we wanted to have the speed of Google when it came to analytics, but we didn't see any way to do that using existing technology, so that's when we decided to look at these new [analytic] database systems."
Autometrics seems typical of the whiz-bang performance stories revealed at post-implementation reviews -- that companies like to tout: it achieved a "900 to 1000 percent performance gain in some of [its] reports," according to Rubin. It's also a textbook case of an upstart vendor adding value -- and achieving success -- by exploiting shortcomings, gaps, or differences in an existing power hierarchy.
"[PAD] allows us to speak confidently [to customers] that we can deliver on bringing a lot of data together and being able to deliver it with rapid [service levels]. We're typically not going to be doing sub-second, but if you can deliver results in 1,2, or 3 seconds, that's Google-like speeds. That's something we just couldn't do with SQL Server. It's something our team [i.e., data warehouse staff] felt we couldn't do with any off-the-shelf database.
"It isn't that we're moving off [SQL Server]. That isn't what we wanted ParAccel to do. What we wanted it to do was to allow us to keep our existing data [infrastructure, on SQL Server] and to really step beyond that, too. Our perspective has sort of expanded with ParAccel. Basically, we're not afraid to handle anything."
WhereScape is another new and agile vendor that -- by virtue of its ability to target areas of asymmetry (i.e., gaps) -- is achieving some degree of success. WhereScape markets Red, a kind of self-service, drag-and-drop, ETL application. It's one of a crop of new ELT-focused vendors (such as Compact Solutions LLC and DVO Software Inc.) that compete in an ostensibly mature (and often commoditized) data integration market.
Where established players tout either good-enough ETL (e.g., Microsoft Corp.'s SQL Server Integration Services) or full-blown data integration platforms (e.g., DI offerings from IBM, Informatica, Oracle, SAP, and SAS Institute Inc.), WhereScape and its asymmetrical kith trumpet tactical offerings. In WhereScape's case, it positions Red as an ideal solution for resource-strapped shops (which can't afford big DI packages and lack extensive DM or DW expertise) or for organizations that need to address tactical, one-off, or similar needs.
One of its customers is United Rentals, a combined Teradata, Oracle, and IBM Corp. shop. With more than 600 U.S. locations, United Rentals runs a good part of its business on top of System i midrange servers from IBM. System i ships with its own built-in DW -- by virtue of a bundled version of Big Blue's DB2, which has been optimized expressly for System i -- but United Rentals runs its EDW on top of Teradata. It also maintains HP-based servers running Oracle, which support its financial systems.
United Rentals had been using a changed-data-capture (CDC) tool from the former DataMirror -- which IBM acquired last year -- to support its Teradata-based ELT process. Dan Mosher, who directs United Rentals' enterprise data warehousing practice, says this approach worked well -- for the most part.
"As we've been going through development, we're using DataMirror for CDC and loading it into a landing area … and [then we] had Teradata consultants writing stored procedures to take that CDC data and load it into the warehouse," he explains. Teradata consulting expertise isn't inexpensive, so nstead of having his Teradata gurus creating stored procedures, Mosher would've preferred to assign them to more high-value tasks.
That's why he decided to look for something to automate this process.
"I got to looking around and I stumbled across this product quite by accident. I saw it and said, 'I've got to give it a try.' I downloaded it and I didn't believe that it could really do what they claimed. But it did. It did exactly what they claimed," he comments.
"WhereScape is a really nice fit for us. We haven't really changed our approach. We're still doing the ELT. What [WhereScape] gives me is the ability to take some of my other people who aren't necessarily programmers, but they understand data and they understand [WhereScape] as well. They can step through this and create a stored procedure for loading the data. It's very powerful in that it allows them to do it, and I don't have to add additional staff."
IBM, of course, markets best-of-breed DI, CDC, and BI technology, but Mosher says he's quite happy with his Teradata-based DW solution. He says he didn't need or want a best-of-breed DI tool. "[WhereScape] is just such a neat fit. When I found it, I thought, if this works, it'll be perfect. I'd done some research on a bunch of different solutions, just basically reading, and from talking with people I know, I determined that in addition to IBM, Informatica has a big presence, too, but they were just overkill for what I wanted to do."