Most Enterprises Aren't Ready for Big Data
Most businesses are still getting their business intelligence houses in order, focusing on dashboards, self-service, and other priorities. They aren't yet ready for big data.
- By Stephen Swoyer
- October 9, 2012
Big data may be a big game changer, and interest is certainly high about how companies will be able to big data to transform their operations or uncover insights.
The fact is, most businesses are still getting their business intelligence (BI) houses in order, focusing on priorities such as dashboards, self-service BI, mobile BI, and the (endless) effort to eliminate information silos and integrate data sources. They aren't yet ready for Big Data 101.
"Big data is ... way down the list [of priorities] for most [companies]," says industry veteran Cindi Howson, a principal with BIScorecard.com, referring to (still unpublished) findings from her company's "Secrets of Successful Business Intelligence" survey.
With all of the hype surrounding all things "big data," you wouldn't know it. Feed the content of any BI industry website into a word frequency visualization tool and the words "big data" will likely be front and center -- in a large font, no less. Terms such as "dashboards" or "self-service" will likely be much smaller. On this basis, you'd reason that big data was a Very Big Deal, at least to companies who buy or sell BI solutions.
But BIScorecard.com's findings, along with the findings of an earlier survey, suggest otherwise.
Survey Surprise
As it's done every year since 2007, Howson's "Secrets of Successful Business Intelligence" survey polled respondents as to how they use their BI tools, how satisfied they are with their BI vendors, which best practices are most integral to their BI programs, and which technologies or initiatives are most important to them. "Big data" polled near the bottom of the list.
Which issues trumped big data? BIScorecard.com's findings haven't yet been published, but Howson gave BI This Week a sneak peak. Number one on her list: dashboards.
Yes, dashboards, and this isn't anomalous, Howson asserts. "In terms of top innovations companies expect to deploy in the next year, dashboards were rated No. 1, closely followed by self-service and mobile BI. The much hyped 'big data' lagged significantly."
Howson is an acknowledged BI expert. She knows BI tools: at TDWI's quarterly conferences, she'll host "BI bake-offs" pitting tools from IBM Cognos, Information Builders Inc., Microsoft Corp., MicroStrategy Inc., Oracle Corp., SAS Institute Inc., and SAP BusinessObjects against one another. She also works closely with companies as they implement BI technologies. As a result, she stresses, it's a mistake to think of the latest "dashboards" as related to the executive information systems (EIS) of the 1980s, the crude dashboard-like views of the late-1990s, or the glossy dashboards of half a decade ago.
What's changed? Discovery, among other factors.
"I think historically dashboards would only let you investigate so far, and that investigation might [take the form of] drill[ing] into a cube and it might drill into a report," she explains, adding that vendors such as QlikTech Inc. and the former Endeca (which Oracle acquired last year) specifically targeted these limitations with their self-described "discovery" tools.
Over the last 24 months, she adds, all of the big BI players have introduced "discovery" offerings of their own. "This is where I position QlikTech and Oracle with Endeca Information Discovery: they call themselves data discovery tools, but to me they really are a dashboard with exploratory capabilities," Howson concludes, stressing that -- from the perspective of business users -- this is a vast improvement over the status quo.
"So somebody builds a dashboard and that [somebody] is usually an IT person, and that's where [traditional dashboards] are different from the QlikViews. The degree to which you can explore is so much further than what you would historically be able to do with a Business Objects Xcelsius."
Big Data in the Real World
Howson and BIScorecard.com's survey isn't an outlier. Consider an earlier survey from open source software (OSS) BI specialist JasperSoft Inc. It found that most self-described big data adopters are ... using traditional database and ETL tools to power their "big data efforts."
According to the JasperSoft data, for example, fewer than one in five (18 percent) respondents said that they were using Hadoop -- the archetypal platform for big data -- while slightly more (19 percent) said they were using MongoDB, a NoSQL data store that's also touted for use with big data projects.
Other well-known NoSQL solutions included Apache Cassandra (used by just 7 percent of respondents), CouchDB (3 percent), and DynamoDB (4 percent). Elsewhere, analytic database platforms such as those marketed by Teradata Inc., IBM Netezza, and ParAccel Inc. (among others) were used in 11 percent of big data projects.
Elsewhere, an overwhelming majority of respondents in JasperSoft's survey cited data from operational applications -- i.e., traditional data sources -- as central to their big data efforts. Almost four-fifths of respondents said they were piping enterprise application data -- from e-commerce, financial, ERP, CRM, SCM, PLM, and other applications -- into their "big data" projects.
This even came as a surprise to JasperSoft. "It's interesting that the number-one source was application data, number two was machine-generated, and ... number three [was] human-generated," conceded Mike Boyarski, director of product marketing.