Q&A: Disruptive Trends and Technologies Shaping Our Data Use
Major disruptive factors reshaping how we view and use data include the volume, variety, and velocity of big data, along with social computing, more pervasive BI, and advanced analytics, says Vitria CTO Dale Skeen.
- By Linda L. Briggs
- May 7, 2013
What's reshaping how we use data? According to Vitria CTO Dale Skeen, the key disruptive factors include the volume, variety, and velocity of big data, social computing, more pervasive use of BI, and advanced analytics.
Skeen is the co-founder and CTO of Vitria, whose software combines complex event processing, business process management, and real-time business analytics in one product. Skeen, who holds a Ph.D. in computer science from the University of California, Berkeley, is credited with inventing distributed publish-subscribe communication, and holds over a dozen patents in this and related technologies. Skeen has contributed to 10 books and written numerous journal articles on distributed computing and integration technologies. Prior to Vitria, Skeen co-founded TIBCO Software, where he served as chief scientist.
BI This Week: If we define a disruptive technology as a new technology that unexpectedly displaces an established one (per Harvard Business School professor Clayton Christensen), then what are some of the disruptive technologies you see in BI right now?
Dale Skeen: I see five disruptors that are fundamentally changing how analytics can be applied to business problems and how businesses consume analytics. These disruptors relate to changes in the production and consumption of analytics, and are enabled by new, emerging technologies.
The first disruptor, not surprisingly, is big data. Big data represents an opportunity to analyze more data in a broader context and to a degree of precision that we've never had before. Studies have shown that data-driven businesses that compete on analytics consistently outperform their rivals.
Note, however, that big data also represents the threat of a business becoming caught up in a quagmire of sorts -- a sea of irrelevant facts and inconsequential correlations. For the business, this can lead to the wrong action or, in some cases, lack of action. However, this threat can be managed. In general, the potential benefits posed by big data greatly outweigh its risks.
Why is big data such a disruptive trend or technology?
Big data is influential in three of the disruptors I see -- namely, volume of data, variety of data, and velocity of data. The advent of big data is creating an unprecedented opportunity to use better and more insightful analytics, which consequentially leads to improvements in business operations and to new business opportunities.
What else is on your list of disruptors?
The second disruptor I see is the increasing variety of data. While data variety is often considered a defining aspect of big data, it is important enough to consider it as a separate disruptor.
Capturing data from a wider variety of sources enables panoramic views of data. That leads, in turn, to a more complete understanding of the factors driving the business. For example, combining competitive surveillance data with social data can help the business understand whether a sudden drop in sales of a hot product was driven by a product rumor or a competitive price cut.
These new and novel varieties of data come in a wide variety of data formats -- rigidly structured, semi-structured, unstructured, variably structured, richly structured, and so forth. Analytic software must have the capability to accommodate these data forms and their graceful evolution -- a capability referred to as "data agility." True data agility allows schema and semantics to be discovered either automatically or interactively with user assistance and can gracefully accommodate schema changes. Analytic tools must support data agility in order to provide panoramic insight.
The continuous acceleration of business velocity is the third disruptor. Cycles times that took days or weeks now are performed in hours and minutes. To keep up, analytics must move from daily to hourly to near-real-time. Note, however, that business velocity shouldn't be confused with "data velocity," which is an aspect of big data. The increase in business velocity is driving the need for continuous, real-time analytics -- which is an emerging technology.
Apart from big data, what about the changes being wrought by self-service BI?
The fourth disruptor is what I call user empowerment, which is sometimes called self-service BI. New tools are empowering business users and data analysts to create their own key performance indicators (KPIs), along with dashboards and reports, with little or no help from IT. This enables users to experiment with new forms of analytics. Users can easily try out and refine new analytic models, and discard them if they don't work. This access to analytics by actual business users is by far the most effective way to improve a company's analytic capabilities.
Finally, the fifth disruptor I see is social computing, which enables sharing and collaboration. Enabling users not only to create analytics but also to share analytic models, insights, and results exponentially accelerates the use of intelligence within a company, often leading to an increased rate of innovation. In fact, studies have consistently shown that the most innovative companies tend to use analytics more widely than their competitors. User empowerment, together with social computing, is the fastest way to create a "culture of analytics."
Regarding advanced analytics, where do you see that technology heading?
Advanced analytical techniques, especially in the areas of predictive analytics and optimization, together with the five disruptors I've named, are fundamentally reshaping business analytics. When applied to the rich new sources of data, advanced analytics are enabling us to predict customer preferences with more accuracy, identify latent market demands, and micro-segment markets to achieve competitive advantage through personalized products. In many industries, advanced analytics will differentiate winners from losers.
One of the most exciting opportunities for the use of advanced analytics is in enabling the real-time business through real-time analytics. As I mentioned, the need for business speed is increasing, especially in areas of one-to-one marketing and customer engagement, and this is driving the need for real-time analytics. Emerging technologies (such as complex event processing, or CEP) are enabling the widespread use of real-time analytics for a variety of business problems.
How important is pervasive BI going to be in changing how ordinary users contribute to business decisions?
I prefer to use the term "pervasive intelligence," which is more inclusive than pervasive BI. The prerequisites for pervasive intelligence are the disruptors we've discussed. Perhaps most important in that list is the empowerment of the extended user community -- from ordinary users to data scientists -- and the ability to share insights, actions, and outcomes through social computing. This is the essential foundation on which intelligence can truly become pervasive.
That is to say, pervasive intelligence must be a bottom-up phenomenon supported by the "wisdom of the crowd" -- it cannot be ordained from the top down. Once the foundation of user empowerment and sharing is established, then pervasive intelligence becomes an emergent capability -- user sharing of models, insights, and results lead to improved business results, which encourages more user contribution and more sharing, and thus creating a virtuous cycle driven by analytics.