Best Practices Report | Real-Time Data, BI, and Analytics
September 22, 2014
User organizations continue to push their applications for business intelligence (BI), analytics, and
data management closer to real-time operation. This is because fresh information can support fast-paced,
time-sensitive business processes, such as operational BI, real-time management dashboards,
just-in-time inventory, high-yield manufacturing, facility monitoring, call center information
delivery, self-service information portals, recommendations in e-commerce, and so on.
According to this report’s survey, the leading benefits of implementing real-time technologies for data,
BI, and analytics include better actions based on BI and analytic information, improved customer
service and customer experience, and automated decisions made by software, not people. The barriers
to achieving such benefits include cost, difficulties in designing real-time systems, the state of data
management infrastructure, and inadequate staffing or skills.
The term real time has become an umbrella concept encompassing multiple time frames, speeds, and
execution frequencies. BI and analytic practices are progressively rarer as we come closer to true real
time (milliseconds and nanoseconds). For example, most reports and analyses today operate on data
that’s refreshed daily on a 24-hour cycle. Even so, BI and analytics based on hourly cycles is well
established, and so-called near-real-time cycles (seconds and minutes) are fairly common.
A variety of technologies are in use today, enabling real-time and near-real-time data, BI, and
analytics. Common ones include data federation, replication, data sync, message buses, and
microbatches. A number of functions designed for high performance are increasingly applied to near-real-time uses, namely changed data capture, columnar databases, in-database analytics, in-memory
processing, and solid state drives. Organizations in need of true real time are ramping up the use of
complex event processing (CEP) and continuous stream mining.
The proliferation of operational BI is the leading driver for real-time usage in BI. Most operational BI
users interact with near-real-time data via management dashboards and other types of reports.
However, users are aggressively adopting visual data discovery tools because of their ease of use,
strong self-service functions (which empower many user types), and the tools’ ability to process large
data sets with near-real-time performance.
As user organizations move deeper into analytics (as a complement to reporting), they also move
deeper into real time. For example, this report’s survey ranks operational BI and reporting as leading
practices today, with 43% of respondents using these with some level of real time. However, 34% of
respondents are already practicing operational analytics in real time, with an additional 34%
planning to adopt the same within three years.
Other areas that should experience strong adoption within three years include real-time data
warehousing (and similar practices such as active and dynamic warehousing), predictive analytics,
visualization, social media analytics, text analytics, CEP, and stream mining.
Most of the data handled in real time today is structured (or more specifically relational), followed by
application logs and semi-structured data, but survey respondents anticipate more real-time handling
for social media data, Web logs and clickstreams, and unstructured data.
This report helps users understand the many available technologies and practices for real-time data,
BI, and analytics, plus how these enable fast-paced business processes and yield a wide range of
organizational advantages.
Content Provided by
TDWI, IBM, Actian, Cloudera, Datawatch, Striim, Tableau Software, Treasure Data, Vertica