Enterprise Business Intelligence: Strategies and Technologies for Deploying BI on an Enterprise Scale (Report Excerpt)
Excerpted from the full August 2005 report. TDWI appreciates the sponsorship of Actuate, Business Objects, Cognos, Hyperion, Information Builders, SAP America, Siebel Systems, Inc., and Temtec.
By Wayne W. Eckerson, Director of Research and Services, TDWI, and Cindi Howson, President, ASK LLC
Moving BI to the Enterprise
Business intelligence (BI) is an umbrella term that encompasses the processes, tools, and technologies required to turn data into information, and information into knowledge and plans that drive effective business activity. BI encompasses data warehousing technologies and processes on the back end, and query, reporting, analysis, and information delivery tools (that is, BI tools) and processes on the front end.
This report and survey focuses largely on the front-end BI environment and BI tools. However, it often steps back and puts BI tools into the context of the entire “BI environment” or “BI solution”—in other words, both the back and front ends of business intelligence.
Proliferation of BI Tools
Scattershot Deployments. Most organizations have not deployed BI solutions in a systematic or consistent manner. They have allowed individual workgroups, departments, and divisions to build their own data warehouses and data marts, purchase their own BI tools, create their own BI applications, and define key metrics, data elements, and business views in unique, nonstandard ways. Thus, although BI usage has increased overall, BI deployments remain small and disconnected. Many organizations today are riddled with these “analytic silos.”
This proliferation threatens to undermine the promise of BI to deliver business insight and value to the enterprise. Invariably, each analytic silo uses a different set of BI tools, leading many BI professionals to complain that their organizations have one of every kind of BI tool imaginable. TDWI research shows that organizations have an average of 3.2 BI tools from different vendors. (See Table 1.)
| Average Number of BI Tools by Company Size |
| Revenues | <$500M | $500M to $5B | $5B+ |
| Average number of BI tools | 2.3 | 3.1 | 3.7 |
Overall Average = 3.2 |
Table 1. Organizations have an average of 3.2 BI tools from different vendors. The larger the company, the more BI tools it has. Based on 594 qualified respondents.
BI Tools by Category. The proliferation of BI tools is more obvious if we count BI tools by category instead of by vendor. The TDWI survey shows that organizations average almost three production reporting tools, three OLAP tools, two dashboard applications, two end-user query and reporting tools, 1.5 data mining tools, and 1.5 planning/modeling tools. Combining these figures with the previous chart, organizations have an average of 13 BI tools, with almost four BI tools or modules per vendor. The good news is that organizations plan to standardize on one BI tool per category within three years (see Table 2).
| Today | In Three Years | Change |
| OLAP | 2.7 | 0.99 | -63% |
| Production reporting | 2.8 | 1.4 | -50% |
| Dashboards/scorecards | 2.1 | 1.1 | -48% |
| Query & reporting tools | 2.1 | 1.2 | -43% |
| Data mining tools | 1.6 | 1.2 | -25% |
| Planning/modeling tools | 1.5 | 1.3 | -13% |
Table 2. Organizations plan to consolidate OLAP tools the most, followed by production reporting and dashboards/scorecards. Based on 594 respondents.
Expanding the BI User Base
While organizations are trying to reduce the number of BI tools they possess, they are also looking to expand the number of users who leverage BI tools to make decisions. Expanding the use of BI from power users to all users is an important priority for organizations that want to empower knowledge workers with relevant and timely information to make quality decisions and improve performance. The irony of BI tools is that although they have proliferated in most companies, their penetration into the user base has been minimal.
On average, organizations plan to boost the number of BI licenses they purchase for potential BI users from 41 percent today to 60 percent in three years, a sizable increase. At the same time, organizations also want to increase adoption rates among existing BI users. Today, only 45 percent of licensed BI users use the tools on a regular basis (i.e., weekly). In three years, organizations plan to expand regular BI usage to two-thirds (65 percent) of licensed BI users. (See Illustration 1.)
Illustration 1. Organizations plan to deploy BI tools to a majority of their employees in three years. Based on 594 respondents.
Doing the math, this means that currently an average of 18 percent of potential BI users actively use BI tools, but this will more than double to 39 percent in three years. Even if our respondents are overly optimistic (which is usually the case), this is still significant growth in the penetration and use of BI tools.
Characteristics of Enterprise BI Tools. The key to increasing the penetration of BI tools is multifaceted. In the past, the lack of adequate training, a perception that BI tools are for power users only, and BI tools that are too complex for casual users inhibited the growth of BI tools in many organizations. In the future, organizations need to make sure they deploy BI tools and applications that are fast, intuitive, and customized to a user’s role in the organization. The BI tools and applications must also provide access to timely, relevant, and accurate information and be able to reach into operational systems, if required, among other things.
Turning Business Intelligence into an Enterprise Resource
Critical Success Factors. TDWI research shows that two-thirds of organizations (66 percent) are now trying to transform BI from a departmental solution to an enterprise one. Only 17 percent of organizations have completed the task, while the remaining 17 percent will continue to deploy BI departmentally. This data shows that BI has a ways to go until it reaches maturity in most organizations. (See Illustration 2.)
Illustration 2. Based on 594 respondents, who were asked whether their organizations were trying to transform BI from a departmental to an enterprise initiative.
Although a discussion of all the steps required to achieve BI maturity is outside the scope of this report, we will highlight a number of tasks that organizations must undertake to transform BI from a departmental resource to an enterprise one. Much of the advice below involves aligning the BI solution with business strategy and making sure the business, not the IT department, owns the solution and guides the IT project teams during implementation and beyond.
- The business must recognize the need
- There must be strong executive support
- The business must own the initiative
- Implement a BI strategy, not just a tool
- Standardize and validate data
- Standardize architecture, processes, and success criteria
- Make BI tools conform to users
- Leverage dashboards and scorecards
- Develop functionality and reports rapidly
- Deliver fast response time performance
- Deliver right-time data
- Monitor usage
- Deliver continuous iterations of the project
Standardizing BI Tools
Drivers of Standardization. There are two main reasons why organizations are intent on standardizing their use of BI tools and delivering an enterprise solution: costs and information consistency. Other, less pressing factors include the desire to better integrate transactions and analytics, support an expanding user base, and reduce the number of suppliers. (See Illustration 3.)
Illustration 3. Based on 594 respondents, who were asked to rank the top three drivers of their BI standardization effort.
Costs. The overhead costs of supporting multiple BI tools add up quickly. Organizations must pay software licenses and annual maintenance fees for each BI product they own. In addition, many BI tools require dedicated servers, separate security approaches, and proprietary metadata, thus increasing hardware, operating system, and administration costs. End-user training and support costs climb with each additional BI product. Organizations must assign and train different technical experts to configure each BI tool and develop custom reports.
Consistent Information. But costs are only part of the desire to standardize. Without a common BI toolset, it is very difficult to deliver consistent reports and information. Invariably, groups using different BI tools create overlapping reports with different calculations and formats. Even if the reports run against the same data warehouse, the information they contain will not be consistent.
Given the strategic and tactical costs of multiple BI tools, chief financial officers and BI executives are now working closely together to reduce the burden of supporting multiple toolsets. According to TDWI research, almost one-quarter of respondents (24 percent) have already standardized a single set or small number of BI tools, whether from one or multiple vendors. Almost one-third (32 percent) plan to standardize toolsets within two years, and another 22 percent plan to do so gradually over many years. A minority (22 percent) have no plans to standardize their BI tools. (See Illustration 4.)
Illustration 4. A third of organizations plan to standardize on a single set of BI tools within two years. Based on 594 respondents.
Patience Is Key. Most large organizations are in no rush to standardize BI tools. They realize it takes a long time to undo years of neglect, change long-ingrained habits, and convert users to new BI tools. “We chip away at the old applications,” says Russ Vaughn, former senior vice president of data warehousing at Bank of America.
Selecting Enterprise BI Tools
One of the first steps in standardizing on enterprise BI tools is to select a product upon which to standardize. This is not as easy as it may sound. The requirements for enterprise BI tools are more stringent and comprehensive than departmental BI implementations, which serve a homogeneous user population. What organizations really need is not more BI tools, but a BI platform.
A BI platform provides functionality and modules designed to meet the unique information and analytical requirements of a broad range of users within the enterprise. It lets developers rapidly customize and extend the BI platform’s core BI components and services to create new analytic applications or embed BI functionality within existing applications. It also supports metadata linkages to data integration tools and other upstream and downstream systems to create an integrated BI environment that is easy to adapt to evolving business requirements. BI platforms are built on a scalable and extensible infrastructure that meets data center requirements for reliability, availability, and performance.
In short, a BI platform delivers the right information to the right person at the right time, creating a nimble, fast-moving organization that is much more responsive to market changes and customer requirements. Given these new capabilities, organizations should take the opportunity to reevaluate their business needs, user requirements, and technical architecture before purchasing new BI tools or extending existing BI licenses.
Evaluation Criteria. When selecting BI tools, organizations need to marry strategic and technical criteria. One approach is to let the IT department narrow down the list of BI tools that meet technical requirements and fit within the company’s technical architecture. Then, a committee that represents a cross section of current and future users of the BI tool should evaluate the finalists and select the vendor most aligned with their needs.
According to survey respondents, there are many important strategic criteria. The key ones are: total cost of ownership (67 percent), quality of vendor support (61 percent), pricing (52 percent), vendor viability and leadership (52 percent), and legacy investments in existing tools (51 percent). (See Illustration 5.)
Illustration 5. Based on 490 respondents who rated the above criteria as “highly important” and whose organizations are trying to transform BI from a departmental to an enterprise resource.
Technical Criteria. There is a range of technical criteria that organizations need to consider. Most important is meeting user requirements, selected by 83 percent of respondents. (See Illustration 6.) This is why involving users in any selection process is key. In addition, a majority of respondents selected a half-dozen other critical requirements. These include Web delivery (75 percent), data scalability (i.e., the ability to query large volumes of data) (72 percent), response time performance (69 percent), user scalability (68 percent), and integration with existing applications (54 percent).
Illustration 6. Based on 490 respondents who rated the above criteria as “highly important” and whose organizations are trying to transform BI from a departmental to an enterprise resource.
Challenges to Standardizing on Enterprise BI Tools
There are many challenges involved in migrating to a standard BI toolset. Before your organization launches headlong into a BI standardization project, it should understand the potential problems and pitfalls that often afflict such endeavors. There are numerous technical, political, and cultural problems that lie hidden behind a rosy cost-benefit analysis and may cause organizations to rethink their strategy for sweeping their BI environment clean.
A BI standardization effort can also backfire in many ways. How do you select one platform out of many currently in use within your organization? How do you avoid political and cultural battles that can cost more to resolve than the BI standardization effort will save? How can you be sure the new BI platform will scale up to support all users and scale out to support their diverse sets of analytical needs? In most cases, the answer is to tread carefully, plan thoroughly, and go slowly.
According to the TDWI survey, the top challenges for standardization efforts are individual resistance to change (48 percent), followed by departmental autonomy (43 percent), high switching costs (28 percent), and training and support costs (26 percent). (See Illustration 7.)
Illustration 7. Individual resistance to change is the biggest obstacle. Based on 460 respondents.
Strategies for Standardizing BI Tools. Given the difficulties associated with standardizing BI tools, most take a go-slow approach. “It has actually been harder to shut down a legacy BI tool than consolidate our independent data marts,” says Dirk de Wilde, data architect at Canadian National Railway Company. “The tool drives hundreds of complex reports, not all of which are being used, but which would take us an immense amount of time to convert.”
To make progress toward enterprise BI, many organizations adopt different strategies based on the situation. Most firms grandfather existing BI tools, but with some exceptions: 23 percent remove support for the existing BI tools; 22 percent redirect the queries to a new data/metadata source; and 41 percent continue to support the tools but require new projects to adopt the standard. Only about a third aggressively mandate a cutover to new tools by a certain date. (See Illustration 8.)
Illustration 8. Most organizations grandfather existing BI tools rather than replace them. Based on 594 respondents.
The Future
If deployed properly, enterprise BI tools begin to fade into the background. Like the electrical grid that we can plug appliances into, we will soon expect the BI utility to deliver insight on demand in response to our queries. Using Web services and service- oriented architectures, our application developers will plug core applications into the BI utility and give users the critical information they need to manage the processes for which they are responsible on a day-to-day basis. The day that BI becomes invisible is the day it finally succeeds.
Wayne W. Eckerson is TDWI's director of research and services. Eckerson has 17 years of experience in the IT industry, and he has covered data warehousing and business intelligence issues since 1994. He is the author of "Performance Dashboards: Measuring, Monitoring, and Managing Your Business" (John Wiley & Sons), published in October 2005. [email protected]
Cindi Howson is the president of ASK LLC, a BI consultancy focusing on BI tool selection and optimization. Howson is the author of the newly launched BIScorecard.com for independent BI research, numerous articles for Intelligent Enterprise and TDWI FlashPoint, and "Business Objects: The Complete Reference" (Osborne McGraw-Hill). In addition, she teaches TDWI's course on Evaluating BI Toolsets. [email protected]
This article originally appeared in the issue of .