Best Practices Report | Real-Time Data, BI, and Analytics

September 22, 2014

User organizations continue to push their applications for business intelligence (BI), analytics, and data management closer to real-time operation. This is because fresh information can support fast-paced, time-sensitive business processes, such as operational BI, real-time management dashboards, just-in-time inventory, high-yield manufacturing, facility monitoring, call center information delivery, self-service information portals, recommendations in e-commerce, and so on.

According to this report’s survey, the leading benefits of implementing real-time technologies for data, BI, and analytics include better actions based on BI and analytic information, improved customer service and customer experience, and automated decisions made by software, not people. The barriers to achieving such benefits include cost, difficulties in designing real-time systems, the state of data management infrastructure, and inadequate staffing or skills.

The term real time has become an umbrella concept encompassing multiple time frames, speeds, and execution frequencies. BI and analytic practices are progressively rarer as we come closer to true real time (milliseconds and nanoseconds). For example, most reports and analyses today operate on data that’s refreshed daily on a 24-hour cycle. Even so, BI and analytics based on hourly cycles is well established, and so-called near-real-time cycles (seconds and minutes) are fairly common.

A variety of technologies are in use today, enabling real-time and near-real-time data, BI, and analytics. Common ones include data federation, replication, data sync, message buses, and microbatches. A number of functions designed for high performance are increasingly applied to near-real-time uses, namely changed data capture, columnar databases, in-database analytics, in-memory processing, and solid state drives. Organizations in need of true real time are ramping up the use of complex event processing (CEP) and continuous stream mining.

The proliferation of operational BI is the leading driver for real-time usage in BI. Most operational BI users interact with near-real-time data via management dashboards and other types of reports. However, users are aggressively adopting visual data discovery tools because of their ease of use, strong self-service functions (which empower many user types), and the tools’ ability to process large data sets with near-real-time performance.

As user organizations move deeper into analytics (as a complement to reporting), they also move deeper into real time. For example, this report’s survey ranks operational BI and reporting as leading practices today, with 43% of respondents using these with some level of real time. However, 34% of respondents are already practicing operational analytics in real time, with an additional 34% planning to adopt the same within three years.

Other areas that should experience strong adoption within three years include real-time data warehousing (and similar practices such as active and dynamic warehousing), predictive analytics, visualization, social media analytics, text analytics, CEP, and stream mining.

Most of the data handled in real time today is structured (or more specifically relational), followed by application logs and semi-structured data, but survey respondents anticipate more real-time handling for social media data, Web logs and clickstreams, and unstructured data.

This report helps users understand the many available technologies and practices for real-time data, BI, and analytics, plus how these enable fast-paced business processes and yield a wide range of organizational advantages.

Content Provided by TDWI, IBM, Actian, Cloudera, Datawatch, Striim, Tableau Software, Treasure Data, Vertica

Your e-mail address is used to communicate with you about your registration, related products and services, and offers from select vendors. Refer to our Privacy Policy for additional information.

TDWI Membership

Get immediate access to training discounts, video library, BI Teams, Skills, Budget Report, and more

Individual, Student, & Team memberships available.