Uncorking the Data Bottleneck with Operational BI
Lessons from Web search and Web 2.0
By Dr. Larry Harris
At the strategic level, operational business intelligence (also known as pervasive BI) is about spreading the benefits of Business Intelligence to a wider community of users who are more involved with the day-to-day operations of an organization. On the surface, this would seem to be a simple process of employing existing Business Intelligence tools to users. Unfortunately, at a tactical level, it isn't that straightforward. The same BI tools that served a highly skilled analyst community are rarely appropriate tools for the broader base of operational users. Similarly, the kinds of operational data that these users want may be intrinsically different; frequently more detailed, and more transactional. It may demand more real-time insight.
This article discusses a new approach to operational business intelligence that addresses the differences in the user community, and the different aspects of the data intrinsic to operational requirements.
The Operational BI User Community
Perhaps the greatest failure of traditional BI has been its inability to extend beyond approximately 20 percent of its potential audience. While many users benefit from the reports and dashboards generated by BI systems, at most institutions only a relatively small cadre of analysts actually make direct use of the BI systems to create new reports and perform analysis.
The problem with this approach is that the resulting reports and dashboards often merely identify that a problem exists and serve as the stimulus for more reporting needs or further analysis. Unfortunately, the business user is not trained to perform this analysis alone, and therefore relies on technical analysts to generate the "new" report that likely raises yet another reporting request. This process continues and the result is that "answers" may take weeks or months to materialize. It is a rare situation indeed where the set of standard reports meets all the needs of each member of the business user community.
Search-Oriented BI
The business user community has voted with their feet for the style of user interface they prefer. It isn't a heavyweight, drag-and-drop multidimensional metaphor, but the clean, simple search metaphor (epitomized by Google) that they have become comfortable with on the Web. In fact, it's common in many organizations to hear users explicitly ask, "Why can't we just use Google for this?"
Traditional BI technology handles these requests by layering a search capability on top of the set of standard reports currently generated or across the underlying data store directly. This clearly has some virtue in helping a wider group of users find existing reports or the specific content of a row or column -- either of which may be helpful to them. It's limited to merely "finding" what exists today and doesn't create a new report or conduct futher analysis to answer a question. Consequently, although "search" provides the appropriate user interface (UI) for ubiquitous adoption, it falls short as a BI tool that the business user needs.
Connecting Search to Ad Hoc Query
Logically, it may seem that the "right" solution to this problem is to use the familiar search metaphor to both a) search the repository of existing reports; and b) provide a new customized report that has been generated based on the user's input.
It is both interesting and surprising that a simple search request can be sufficient input to create an ad hoc BI query. For example, a simple input "customer returns" is sufficient to generate a report subtotaling the return dollars of each customer. In the meantime, a full sentence question generates a more powerful and complex report. For example, a question such as "Which customers bought tables but not chairs?" is sufficient to generate a SQL query with a sub-selects command, which is required to answer the question.
Such a query would be hard, if not impossible, to answer even with the most sophisticated drag-and-drop interface, yet it is easy to express as a simple English question. Today's natural language processing technology coupled with a translation to a language that can query the underlying database (such as SQL) is actually an "old" answer to a relatively new problem.
While it may appear that the simple ability to generate all reports "on demand" would obviate the need for a search of existing reporting assets, in some instances, access to an existing standard report is better. Specifically, it is likely that a report generated by an analyst covers the subject matter deeply across dimensions that the user may not have considered initially (i.e., in the "customer returns" example above, the actual dollar returns may be what's interesting from a value perspective, but it is more meaningful to look at this from a percentage-of-orders perspective if trying to identify a "problem client."
The "New" Usability Challenge -- Knowing the Terminology
Leveraging a search-like user interface overcomes the inherent complexity of traditional BI tools that has limited their broad deployment across entire organizations (in addition to financially prohibitive licensing models) but there remains a significant "knowledge gap: that exists: a lack of knowledge of the underlying data. The problem with the search box as an input metaphor is that users often don't know what to type, or more precisely, they often don't know the official name of the data item in the data mart.
This is not surprising, given that it is quite common for there to be five to ten different types of "sales" information in a single data mart, such as dollar sales, unit sales, and cost of sales. Unlike Web search, where a user has an expectation that someone somewhere has created something similar to their query, the world becomes a great deal smaller when focused upon an organization's data. It is, therefore, more important to assist users as they structure their search so that they are exposed to the information that will better enable them to perform a successful "search."
Winning the Popular Vote
The graphic below defines three types of BI (strategic, analytical, and operational), and it also serves to underline the statistic cited earlier -- the 20 percent of the organization that have determined the applications and requirements for BI are at the top of that pyramid. A case could be made that these upper echelons of the organization represent those who have access to the business intelligence data they need.
Although few have embraced Web 2.0 approaches to better harvest the collective knowledge of the masses such as what we see from those humorous clips rated popular on YouTube, a must-read news article forwarded to you from CNN.com or a musician on MySpace that has found fame and fortune thanks to the overwhelming support of the masses.
By empowering the business user community to create and publish the reports they deem most useful, they are able to vote on the most meaningful and actionable reports. Exposing and analyzing the requests and reports of users to others enables insight into the data needs of the organization at every level.
Conclusion
The behavioral change of business users to embrace search and be willing to perform their own analysis has converged with the maturity of natural language processing technology that can extend search's capabilities from merely finding to actually creating new content. This convergence represents an opportunity to finally complete the BI 'picture' and make its benefits pervasive across the entire extended organization while also enabling the user community to make explicit their needs while highlighting their most useful and valuable insights.
- - -
Dr. Larry Harris is the vice president and general manager of the EasyAsk division of Progress Software. You can reach the author at [email protected].