Developing a BI Strategy for CRM/ERP Data
Excerpted from the full October 2004 report. TDWI appreciates the sponsorship of Hummingbird Ltd., Hyperion Solutions Corporation, Informatica Corporation, SAP America, Inc., Siebel Systems, Inc., and Teradata, a division of NCR.
By Colin White
Growing Marketplace. Following a hiatus after the year 2000, the use of application packages is once again growing rapidly across all business areas, especially in front-office CRM systems and back-office ERP systems. This growth is occurring not only for application packages that handle business transaction (BTx) processing, but also for packaged solutions that support business intelligence (BI) and data warehousing (DW).
The Application Suite. Several vendors offer complete suites of products that contain a set of integrated applications that support both BTx and BI processing for a wide range of business areas from the front office to the back office.
A Complete IT Infrastructure. Application package suites frequently come with development tools for building BTx and BI applications, middleware for integrating other applications and data, and an application server environment for running and managing packaged and custom-built applications.
Using Business Intelligence to Drive the Business. In parallel with the growth in the use of BTx and BI application packages, the BI industry is also going through a period of significant change. BI is no longer used just for doing strategic and tactical reporting and analysis, but also for driving and optimizing daily business processes and workflows. BI is no longer nice to have, but essential to business success. The rapid evolution of application packages, application package suites, and business intelligence raises several important questions.
- In what business areas are companies deploying BTx and BI application packages today? How successful are application suites, what benefits do they offer, and what strategies are being used by organizations to deploy them throughout the enterprise?
- How does the use of packaged application solutions in business units impact existing and future BI projects? What strategies should be used to integrate data between application packages and with an existing data warehousing environment? How does the use of application packages and application suites affect BI/DW product selection? Should companies focus on buying their solutions from a single applications vendor, or should they buy best-of-breed products?
- How do evolving BI/DW technologies such as in-line and real-time BI, performance management, predictive analysis, and support for XML and Web services affect the selection and use of packaged application solutions?
The objective of this report is to answer these questions and offer suggestions for addressing the issues they raise. The goal of the report is also to suggest possible strategies for building a BI system and underlying data warehouse for processing CRM and ERP data managed by application packages.
The Landscape for Packaged Application Solutions
This report compares and contrasts different approaches to developing business intelligence applications using transaction data managed by application packages from vendors such as Oracle, PeopleSoft, SAP, and Siebel. To put this topic into perspective, this section of the report discusses how business users run, optimize, and communicate about business processes. It then looks at how application packages have evolved to support those processes.
At the heart of any corporate data processing system are the BTx processes for handling day-to-day business operations such as front-office customer relationship management (CRM), middle-office finance and human resources, and back-office enterprise resource planning (ERP). The techniques and tools for developing and deploying these BTx processes and their underlying applications have evolved over several generations of computer hardware and software.
Custom Coding. Initially, BTx applications were custom developed using programming languages such as Assembler, COBOL, and PL/I. Over time these languages were superseded by object-oriented languages such as C++ and Java.
BTx Application Packages. As BTx application usage grew, development groups looked for faster ways of building applications. This need led to visual programming tools and packaged BTx application solutions. Initially, packaged BTx applications provided stand-alone solutions for specific business areas. More recently, however, application vendors have been marketing complete suites of integrated packages that meet customer needs for deploying a wide range of related and interconnected BTx applications.
BTx Application Package Evolution. The use of BTx application packages increased as companies struggled to replace and reengineer their aging legacy systems in anticipation of year-2000 issues. This was especially the case in ERP deployments. At the beginning of 2000, the pace of migration to packaged solutions slowed. Both ERP and CRM packages suffered a number of well-publicized failures. As both types of BTx packages have matured, however, the rate of adoption has once again increased. The move to reduce IT costs and standardize on products has also encouraged continued growth in the use of BTx application packages.
Using BTx Application Package Suites. BTx application package suites are used by organizations to replace in-house legacy applications and also stand-alone application packages. The pros and cons of employing an application package suite are well documented. Their main benefits are business models based on best industry practices, faster deployment time, reduced IT resources, and greater integration across applications.
When selecting an application package suite, you should consider issues like reliance on a single vendor, and the customization and maintenance effort required to modify the suite to match the organization’s business processes. The cost of application suites is another common issue. Given the objective of a suite is to reduce development and maintenance costs, then the total cost of ownership (TCO) is a better cost comparison against other approaches.
Several companies interviewed for this report said their senior management had decided to adopt a single vendor suite of BTx packaged solutions because they wanted to reduce IT expenditure. The level of investment organizations had made in a suite was the driving force behind this decision. Executives wanted to offset the cost of deploying a suite with a corresponding decrease in IT resources and software overhead. Moving from multiple dispersed application packages to an integrated product set from a single vendor has obvious financial benefits, but this strategy will take time to implement and has key implications for existing and future BI projects.
Many Organizations Have Deployed Multiple BTx Packages. We asked respondents in our survey about the number of vendors who supply BTx application packages to their organizations: 29 percent are using packages from one vendor, 26 percent from two vendors, 16 percent from three vendors, and 5 percent from four vendors (see Illustration 1). Some 16 percent are using packages from five or more vendors. In total, more than 50 different packages are listed in survey responses. It is likely that some respondents answered this question for their own division or business unit, and in reality the number of packages used by organizations may be higher than the numbers shown.
Business unit autonomy and company mergers and acquisitions are two reasons why companies have multiple packages. Also, individual business units have deployed application solutions at different points in the BTx application package lifecycle, and the maturity of packaged solutions for particular business areas often determined the development approach and products used.
Applications Areas Supported by BTx Packages. The main business areas supported by BTx application packages (see Illustration 2) include finance (78 percent of respondents), sales (54 percent), and human resources (49 percent). It was very common for organizations to support the front office and back office with CRM and ERP packages from different vendors. Also, middle-office human resource and finance departments frequently have their own stand-alone packaged BTx solutions.
When comparing the number of packages used with corporate revenue, 50 percent of companies with three or more packages have revenues in excess of $1 billion. Surprisingly, 19 percent of companies with three or more packages have revenues of less than $100 million, which shows that even smaller companies have to deal with the issue of handling multiple packages.
BI application development mirrors the evolution of BTx applications. Organizations began initially by coding their own BI applications, but rapidly progressed into the use of interactive BI productivity tools and packaged BI applications.
BI Application Package Use Is Growing. Like the BTx applications market, the BI area is also seeing growth in the use of application packages. The worldwide BI application package market is forecast to reach over $4.8 billion in 2007, according to IDC. All three of the IDC BI application package market sectors are projected to enjoy revenue growth through 2007. CRM analytics will grow the fastest with a compound annual growth rate of 12.9 percent, followed by financial analytics/performance management at 10.3 percent, and operations/production analytics at 7.4 percent. Financial analytics/performance management, however, is by far the largest market segment at present.
Survey results show (see Illustration 3) that 30 percent of companies are not using BI application packages from their BTx application vendors, while 37 percent are using just one package. The remaining 33 percent are using two or more of these solutions. Leading business areas being addressed by BI packages (see Illustration 4) include finance (70 percent of respondents), sales (56 percent), and marketing (45 percent). The top two business areas, finance/accounting and sales, are the same as those for BTx application packages. Marketing, however, is more dominant in BI processing than it is in BTx processing.
BI Strategies for BTx Package Data
Having outlined how companies use BTx application packages, we now move on to look at different BI strategies for reporting on and analyzing the data from those packages, and for solving the data integration issues involved.
There are three main BI/DW approaches for processing data managed by BTx application packages. These can be thought of as the Adopt, Avoid, and Accommodate strategies.
Some of the main business and technology requirements that affect product choice include political issues, existing BTx and BI environments, IT skills and resources, product features and functions, integration capabilities, performance and scalability, total cost of ownership (TCO), and vendor relationships and support.
The most popular BI/DW strategy for handling BTx package data according to TDWI survey results is Accommodate, followed closely by Avoid. Of the three strategies, 32 percent of respondents rated the Accommodate strategy as being a “very high” or “high” match to their corporate BI/DW approach. The Avoid strategy matched 29 percent of corporate BI/DW approaches, while the equivalent figure for the Adopt strategy was 24 percent (see Illustration 5).
Another way of looking at these results is 56 percent of respondents are using BI/DW solutions from BTx applications vendors (Adopt or Accommodate strategies), and 61 percent are using third-party BI/DW products (Accommodate or Avoid strategies). These figures clearly show the direction of companies toward incorporating BI/DW solutions from BTx applications vendors in their BI environments.
Although 29 percent of respondents rated the Avoid strategy as being either a “very high” or “high” match, nearly half of the same companies have purchased BI/DW products from their BTx application package vendor. One explanation for this anomaly is that individual business units may have a different BI/DW strategy from that of the central IT group. A business unit may implement a BI/DW solution from a BTx application package vendor, even though the central IT group may have adopted an Avoid strategy.
Other Factors That Influence Product Choice. The survey also looked at some of the factors that influence product choice. Two key factors are integrating best-of-breed products for gaining maximum functionality, and the total cost of ownership (TCO) of the BI/DW application solution. A best-of-breed approach is used by 62 percent of respondents, which is close to the figure of 61 percent for people employing the Avoid or Accommodate approaches (i.e., people using third-party BI/DW products). Nearly half of the respondents rate TCO as “very high” or “high” in importance. TCO requirements should be independent of strategy.
Conclusion
The TDWI survey and follow-up interviews show that most companies have a complex mixture of applications and data stores for handling CRM and ERP data, and for reporting on and analyzing this data for corporate decision making. Providing executives and managers with a consistent and integrated view of business operations in such an environment is a difficult task, and companies use a variety of BI and DW techniques and products to support this task. However, some common patterns and trends can be seen in the survey findings.
1. The deployment of BTx and BI application packages and suites is growing, especially in business areas such as finance, sales and marketing, and human resources.
2. Many companies are content to allow individual business units to deploy their own BTx and BI application packages. These packages offer many advantages to the individual business units, but they can complicate the creation of an enterprise data warehouse and cross-business-unit BI processing. Data integration in this environment is often done by copying warehouse data between a business unit application package data warehouse and a central enterprise DW.
3. To increase the ROI of their IT software investments, some senior executives are mandating the use of an integrated BTx and BI product set from a single vendor across all business units. Whereas this approach helps eliminate the data warehouse and BI issues outlined in item 2 above, it is unlikely in the short term that these companies can eliminate the use of legacy applications and individual application packages completely across all business units.
4. Although the creation of an enterprise DW is still the goal, for the foreseeable future many companies will need to employ a hybrid DW architecture that can accommodate BI tools and application packages from both BTx application and third-party vendors. Such an environment requires an open BI/DW framework that can easily absorb new BI/DW products and technologies as the marketplace evolves.
5. Key requirements for BI/DW for application packages are an open and integrated framework for handling data and metadata, scalability and performance, total cost of ownership, ease of use, and analysis and reporting power.
6. Many companies are nominating business process owners in each business unit to improve communications with IT about the business use of application packages, and to identify business and information requirements for new BI applications.
7. Business intelligence is now being used not only for strategic and tactical decision making, but also to drive day-to-day business operations. In some cases, this is being achieved by integrating BI processes into the business transaction workflow.
8. Many companies are focused on reducing IT costs, rather than using modern BI technology to build smarter businesses. This is making the creation of an enterprise data warehouse for cross-business-unit planning and action difficult to create. IT organizations need to work closely with senior executives to explain the business benefits of integrating business intelligence from the demand side of the organization, with that of finance and the supply side of the organization.
Colin White is president of BI Research. With more than 34 years of IT experience, he has been a consultant for dozens of companies around the world, and is a frequent speaker at leading IT events. For more information, write to [email protected].