Q&A: Managing and Evaluating Your BI Tool Portfolio
Cindi Howson explains best practices for evaluating (and re-evaluating) your BI tool set and looks at what's cool in BI.
- By James E. Powell
- November 12, 2013
[Editor's note: Noted analyst and TDWI faculty member Cindi Howson, founder of BI Scorecard, is conducting a session at the TDWI World Conference in Orlando (December 8-13, 2013) entitled "Managing and Evaluating Your BI Tool Portfolio." We spoke with Cindi to get a preview of the key issues involved in BI tool management.]
TDWI: Who should be part of the team that evaluates BI tools? IT only? A mix of IT and business users? What role does each group play? For example, I assume business users set the functionality criteria and IT sets the tech requirements, but then how do you prioritize the combined requirements and resolve conflicts? What role should the product sponsor play?
Cindi Howson: Any BI evaluation should be done jointly by both business and IT. It can be hard to prioritize requirements, particularly if a vendor is stronger on the business requirements and weaker on the IT requirements. However, all requirements are ultimately determined by the business.
Criteria related to security may be a good example. If a vendor doesn't support authentication against existing security systems, that may require additional work for IT to implement a new solution, or it may mean an increased risk of people getting access to the data who shouldn't. It could also result in delays in granting access. Therefore, IT may identify the security requirement, but when a vendor can't fulfill it, the business has final sway over the priorities, risks, and associated costs.
In your session, you talk about looking at solutions from the Big 4 [SAP, IBM, Oracle, and Microsoft] and BI pure-plays [MicroStrategy, Information Builders, SAS, QlikTech, and Tableau] and the importance of understanding the strategic and functional differences between the solutions. Do those differences fall into two
or three main categories that you can share with us?
We publish an annual report that clearly describes these considerations and categories but to list briefly here:
- Strategic: Vendor growth and leadership, breadth of BI/IM solutions, quality of account management and technical support, pricing and packaging, degree of innovation
- BI functional modules: business query, visual data discovery, interactive reporting, mobile, dashboards, production reporting, OLAP, admin and architecture, Office integration, scheduling and information delivery
What are the biggest problems enterprises face in managing their BI tool portfolio? What best practices can they follow to successfully manage their portfolio?
They want a one-size-fits-all solution from a single vendor. It's a fallacy. A pipe dream. Instead, companies should pursue the idea of a predominant BI standard in which they buy several modules from one vendor but are willing to supplement with superior capabilities from another provider for a particular module.
How often should an enterprise reassess its BI tool portfolio? Do you recommend a regular interval (say, every six months), only when a major event occurs (for example, when an enterprises is involved in a merger), or a combination of both (or something completely different)?
There's a difference between re-assessing and re-implementing. BI leaders and experts need to keep a constant pulse on the BI tools market. Many vendors have adopted agile development and are releasing new versions as often as three times a year, with significant improvements and capabilities.
Of course, some (usually the megavendors) are only issuing releases once or every other year. As the market innovates, the BI team needs to assess which innovations should be embraced and when. If those innovations aren't coming from the predominant BI incumbent, then the BI team needs to communicate requirements with their preferred BI vendor and/or be prepared to supplement BI capabilities from other vendors.
There also are other events -- a merger or acquisition or a major new product release with significant migration costs -- that are ideal times to re-assess if a company has made the right BI tool investments or if they need to modify direction.
With constant innovation, how do companies know what BI changes to invest in? Is cloud and big data as hot as they sound?
This is a challenge and a topic of the afternoon "Cool BI" course. I encourage companies to think of the MVP framework -- maturity, value, and positioning of an innovation. Big data and cloud in BI are still emerging innovations, so less mature, for example, than predictive analytics. But they clearly have a role to play and are driving a lot of the recent releases and BI start-ups. (
Which innovations will you focus on in Cool BI? What's cool?
I like to refer to the class as a wine tasting of BI innovations. We take about eight innovation areas and look at why it matters, the value it offers, and how mature are the capabilities. I also invite three guest vendors to discuss the trends and demonstrate the capabilities. We will have Neutrino BI as a panelist to talk about the role of search and natural language processing and search, Birst about Cloud BI, and MicroStrategy about Mobile BI.
I saw that you also just released a new book focusing on some of these topics.
Yes, it's a second edition to the Successful BI book, subtitled Unlock the Value of BI and Big Data. It's an update with new case studies from Medtronic, Netflix, and others, as well as updated material on the role of cloud, mobile, and big data.
How many tools should an enterprise consider evaluating against at one time? At some point, it doesn't make sense to evaluate 10 or 20 products because of the time involved, so is there a magic number you recommend?
There's no magic number.
What's the most efficient way you know to whittle down the essential differences between tools?
Focus on the differentiators of the most critical requirements. We provide a list of just 10 in the class, but if you really want all of them, the latest version of the detailed BI Scorecard spreadsheet (over 300 criteria) will flag them. [Editor's note: Access to the spreadsheet is free, but a short registration is required.]
When working with clients, what seems to be the biggest bias their teams must overcome when evaluating tools?
Fear of change or job loss. Change can be hard, but I hope people view it as an opportunity to freshen their skills and improve upon a current deployment. Fear of job loss shouldn't be a concern for most BI analysts unless they are lazy or unethical, but I've never met a deadbeat BI worker! The BI industry has a shortage of talent.