Q&A: Advanced Data Visualization: From Atomic Data to Big Data
How is information overload converging on real-time decision making? Andrew Cardno, a data visualization expert at AmericanKiwi LLC, shares his perspective.
- By James E. Powell
- September 2, 2014
[Editor's note: Andrew Cardno is leading (with Stephen Brobst) a session about data visualization at the TDWI World Conference in San Diego (September 21-26, 2014). Overcoming Information Overload with Best Practices in Data Visualization examines pitfalls in and best practices for BI visualization. In this question and answer session, we asked Andrew about how information overload is converging on real-time decision making.]
BI This Week: Why is data visualization relevant to operational decision making, especially given the use of real-time automated decision engines?
Andrew Cardno: According to IDC estimates, the volume of data in the world is doubling every two years. [Editor's note: See Source at end of Q&A.] A decade ago, the largest databases in the world were populated by transaction data. In the majority of cases, the finest grains of the data in these transactions were line items in a financial purchase. Today, the largest databases are driven by interactions in which transactions happen at a ratio of 1:10,000; in other words, for every transaction there are often 10,000 interactions.
These enormous volumes of interaction data is accompanied by a need to make decisions in real time, but in real time with the full knowledge of history in the data. Many real-time interactions in a business can be automated. However, human judgment is critically important in many situations. To enable this human judgment, the real-time decision-maker needs to both understand the real-time predictions and the sea of data around their decisions. This need drives the requirement for real-time data visualization.
How do traditional data visualization techniques combine with advanced data visualization in the real-time business intelligence environment?
Traditional data visualization -- meaning the line, bar, box, and scatter plots we've used for years -- is an important part of an operational business intelligence system and is best applied to discover or monitor known patterns in the data. Advanced data visualization displays far larger data volumes and is best applied to discover patterns in the data.
Consider the spatial map, a canonical example of an advanced data visualization technique. It can show thousands of data points and the user can monitor these data points.
Why is it that the casino industry is known for its leadership in the advanced data visualization space?
The casino industry is awash with detailed data. In fact, the industry has been generating "inside retail space" game interactions for almost 20 years. This information has been used extensively to generate advanced analytics, ranging from views of the gaming floor to high-intensity analysis of their customer-behavior patterns.
How do you distinguish between data exploration and reporting in the context of data visualization?
Data visualization is an extremely broad term. It applies to everything from the correct practice in drawing line graphs to extremely high-density graphics showing tens of thousands of data points. Data exploration is the art of discovering unknown relationships in the data and this can be effectively accomplished using high-density graphics. Reporting, on the other hand, shows analysis on known relationships in the data. These known relationships are effectively communicated using traditional data visualization.
Why do you hold Willard Brinton in such high esteem? His work was published nearly one hundred years ago.
Brinton operated in a world without (and, in many ways, unconstrained by) computer graphics. His two books, Graphic Methods for Presenting Facts (1914) and Graphic Presentation (1939), are remarkable collections of graphics and graphic design methods that are still relevant today and in many cases considered advanced. I would argue that because the work Brinton illustrates was built by cartographers rather than computers, there was extraordinary attention to detail. When considered in its entirety, the work of Brinton stands as a powerful reminder of the importance of learning from history -- in this case, the history of graphic methods.
In your workshop you have an example of how the "war room" practice has been applied to decision making. Why do you place such importance on this practice?
Consider an organization that has the most advanced tools and most accurate data but little inquisitive behavior. This organization is likely to drive little data exploration. In fact, it is likely that the advanced tools will be applied only to reporting functions. To encourage this inquisitive behavior, a practice of data war rooms creates a collaborative environment where those entering the room are tasked with finding the right questions to ask by exploring the data. It is my experience that, done correctly, these war rooms can become a central part of creative growth-seeking behavior required in so many businesses today.
How are real-world examples applied in your workshop at the TDWI World Conference?
Real-world examples have tremendous value in seeing how advanced data visualization is applied to overcome information overload. The real-world examples come with stories of how difficult problems were tackled, how the data visualization was applied, and often times the results of this work. These stories are fun to share, especially where the experience itself gave real meaning to the graphic methods applied.
Source: Gantz and Reinsel , The Digital Universe in 2020.