TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

The 2011 BI Bake Off at TDWI

The BI market has changed a great deal since our first bake off. Here's a preview of what to expect at TDWI's World Conference Bake Off in August.

• By Cindi Howson
• July 19, 2011

Next month at TDWI in San Diego, I’ll be facilitating the 2011 BI Bake Off with IBM Cognos, Microsoft, and SAP Business Objects. With all three vendors recently releasing major new versions of their BI platforms, the bake off will be the usual exciting, nerve-wracking, and enlightening session.

When I conducted the first BI bake off at TDWI back in 2005, I kicked off the session with dry-ice-emitting smoke, a bow to the smoke-and-mirrors in many vendor demonstrations. I promised attendees that we would try to cut through that smoke, with a candid look at side-by-side differences. Sadly, I have had to abandon the smoke as it violated the hotel’s fire code (killjoy, right?).

In the bake off, I give vendors a carefully scripted demo of the most critical features and differentiators that customers often look for. Attendees then vote on which product best met their requirements for that capability. Everyone knows that it is indeed just a demo and that a real-world test may yield different results. Still, what is clear is that the TDWI attendees are a sophisticated audience, equally tolerant and unforgiving. Attendees give high marks to vendors who candidly say “we don’t do that out of the box, but let me show a workaround.” Conversely, one vendor who started with a marketing video was unilaterally panned, thank goodness, with a lack of votes and not out-and-out booing.

Surprising to me is when a vendor sells their product short. I’ve tested all of them, so I know their strengths and weaknesses. As the goal is strictly education -- but some fun in the process -- I do point out when a demo just didn’t go well or when I disagree with something a vendor said. I have sometimes chalked up these incidents to a vendor’s not being prepared (another thing attendees are unforgiving about), but frankly, I also think it’s that all the product lines have become more complex. Marketers may be able to articulate product differences well, but it is often the technical consultant who knows the product nuances. Hence, many vendors now bring two participants to the bake off.

The Changing BI Market

Just a couple of years ago, few companies had BI standards, and the market was highly fragmented. Today, the BI platform market is more mature. The market has evolved into mega vendors that have BI platforms, numerous smaller vendors that focus predominantly on BI platforms, and dozens of specialty vendors that focus on a single segment within the BI space. With greater market segmentation, it might seem that BI buying has become easier, but it hasn’t. Smart companies make BI investments wisely, and yet business users often demand the newest innovations, regardless of who is touting them.

Most large companies now have a BI platform standard. Even though customers have made an initial investment, these companies continue to evaluate solutions because:

• Vendors constantly release new versions, begging the question: is it better to upgrade or to make a switch?

• Customers are looking for opportunities to lower licensing costs

• Some customers are not satisfied with support from the incumbent vendor

• Vendors were acquired or merged with another company, with a different BI standard

• Customers have outgrown the capabilities of the product initially purchased

• Budgets tightened in 2009 and 2010 have loosened; there’s pent-up demand to expand BI capabilities

For small and mid-sized companies, BI may be a new endeavor. They’ve outgrown their fixed, operational reporting solution and need something more robust. For many of these companies, BI platform breadth may be less important than ease of deployment and time to value.

For companies large and small alike, an ongoing challenge in managing your BI tool portfolio is in knowing when to standardize and when to mix and match. Standardization holds the promise of lower cost of ownership, better support for business users, and integration across modules. These features are attractive to the enterprise, and although reality varies by vendor, there are dozens of emerging vendors bringing often-times superior functionality than what may be available from a BI platform vendor. This is true of dashboards, visual discovery, mobile, search, and SaaS offerings, to name a few.

To help you better evaluate these specialty solutions, in the last year, we’ve added BI Scorecard for a number of specialty products and vendors. At TDWI, we introduced the Dashboard Bake Off in February 2011. This class is a deeper dive into an increasingly important module; BI platform vendors and specialty vendors compete side-by-side on dashboard capabilities. Dashboards are also part of the BI platform bake off in August.

We also give vendors a chance to showcase something cool and innovative that most differentiates them. It’s the one topic the vendor chooses; the rest of the scripted demos are topics I and the attendees choose. With Cognos previewing their new visualization product in the labs and Microsoft whispering about project Crescent, I suspect visualization may be what these two vendors showcase during the “cool topic.” With SAP being one of the few vendors to natively support the BlackBerry Playbook, perhaps they will choose Mobile BI for their cool topic ... but there has been a fair bit of noise about collaboration as well, so we’ll see.

The BI Bake Off provides side-by-side product insights for three vendors; the other goal of the course is to help companies improve their own selection and bake off process.

To make the most of your own software evaluation:

• Define your requirements and prioritize them, differentiating between near and long-term needs

• Understand the trade-offs between a solution from a single-vendor and pursuing a predominant standard

• Get educated about BI pricing and packaging early in the process

• Ask vendors to follow scripted, not canned, demos

• Gather feedback from a broad constituent of BI stakeholders

• Recognize that requirements change over time as users and software capabilities evolve; never assume your BI tool strategy will stand still

Making smart software investments and managing your BI tool portfolio is important to BI success, but as I often say, ultimately the question of which tool is less important than how you use your BI tools.

Regards,
Cindi Howson, www.BIScorecard.com

Copyright © 2011 by Cindi Howson and BI Scorecard; reprinted by permission of the author