It Takes Guts and Insight to Make a Good Decision
We explore decision making in the big data world and how humans can be empowered to handle the information overload.
By Andrew Cardno, BIS²
[Editor’s note: Andrew Cardno will be co-presenting (with Stephen Brobst) a session on Overcoming Information Overload with Best Practices in Data Visualization at the TDWI World Conference in Las Vegas, February 23-28, 2014.]
When to Make a Decision
Organizations must continually face up to the challenge of change, and in order to make change, human decisions are typically required. Decision makers are dealing with ever-increasing amounts of data in a fast-moving world. This ever-increasing amount of data creates endless opportunities for analysis; the challenge is not getting more information. Rather, it is knowing when you have enough information to act.
Colin Powell said, “When you have no less than forty percent and no more than seventy percent of the information you need to make a decision.” (See Note 1.) As his decision-making rule implies, acting before you have complete information is the key to good decision making. Acting before you have complete information requires courage.
Those organizations with the courage to act with alacrity, and the foresight to make the right decision, are rewarded with great business; the tardy are destined to be buried in data.
Growth in the Money in Big Data
“IDC predicts that the market for big data will reach $16.1 billion in 2014, growing 6 times faster than the overall IT market.” (See Note 2.) Big change nearly always involves winners and losers. Looking at the retail book industry and the impact of Amazon, the winners are the online book companies, the losers are the brick-and-mortar companies. We could argue that the book market has grown as a result of the online offering, but isn’t it fair to say that the majority of revenue from online came from the traditional retail stores losing their market share?
The change bought about by big data is unlikely to be the exception to the winners-and-losers pattern, and with a predicted $16.1 billion size in 2014, the effects will be substantive.
In addition to the massive economic effect of big data, there is a massive shift in the volumes of this data. As these volumes change, perception of “big changes” also changes. The following three “ages” help us understand the change in perception in the definition of big data:
Age of Transactions: Until around 2000, big data was mostly about transaction volume. Think of the massive retail databases of last century, which held one line of data for each product purchased by a consumer (with a number of items purchased). The majority of this transaction data involved a human processing a financial transaction. Companies such as Wal-Mart and Harrahs drove their business success from the insights extracted from this transaction data.
Age of Interactions: From the turn of the century to 2010, we have seen the explosion of interaction data -- data that involves measurement of interaction of a human with computerized systems. In this age of interactions, there are tens of thousands of lines of data for each transaction record. Interactions include social media data as humans interact with each other. Companies such as Google, Facebook, and Amazon drive their business success from insights extracted from this interaction data.
Age of Automation: The next big data age is astonishingly big because now the data is generated by automated systems monitoring and sensing people. In short, data is no longer even touched by a human. Data comes from sources such as Google glasses, automated factories, license plate readers, facial recognition systems, personal health monitors, drone monitoring systems, and vehicle monitors. These systems generate data without requiring humans to even move. It is hard to predict who -- or even if -- companies will build themselves around this data to the size and success of the age of interactions or transactions. There are many moves to capture and business decisions to be made that are driven from this information.
In summary, data is growing from places that are sometimes unexpected, yet this data has become a central part of many wildly successful companies. Despite the huge data growth so far, as we enter the age of automation, the future holds astonishing volumes of data.
The Role of Humans in Decision Making
As humans, we make our own future. As Abraham Lincoln stated, “The best way to predict the future is to create it.” (See Note 3.) Now as we think about the ever more dynamic world we live in, we have to ask: What are the humans going to do? The short answer is, humans drive change, humans act with limited information, and humans take risks. It is this human ability to have the guts to make a decision and take a risk that in many ways defines the entrepreneurial spirit.
Yet, how can humans make decisions in a world where the data volumes are so massive and the structures are so dynamic that it is difficult to even count the amount of data generated? Furthermore, there are automated systems making massive numbers of decisions; for some at the forefront of automation, simply tracking and measuring the decisions is a massive task.
Humans Think Outside the Box
Computers are good at thinking inside the box -- they can interpolate, infer, and reason within given parameters. Humans think outside the box. We can extrapolate, make judgment calls, and take calculated risks. The challenge is that the growth of data and automation has the potential to create a world of pure information overload.
Enter Data Visualization
Humans have an enormous capacity to absorb data; the eye has over 100 million receptors continuously absorbing and processing information. This amazing ability to absorb information provides the key for humans to move to the next level. Through the correct application of visual techniques, we humans can unlock this massive data absorption capacity and step to the next level of understanding about the information around us.
Bringing It All Together
It is human decisions that drive businesses and humans that build the machines and automation that will shape the future of the world. The great challenge is to overcome information overload in the world. To do this, we need powerful and new visual techniques and methods that enable humans to make decisions that often require courage and entrepreneurial spirit. With the insight from advanced visualization, we have the best chance of getting it right.
Huge macro-economic shifts drive the need for decisions to stay ahead of the market. Nearly every business needs to rethink and reinvent the decision-making process for the big data-enabled world.
Notes:
- Refer to http://integratedleader.com/articles/40-70rule.pdf, December 2013, Steven Anderson, Ph.D., MBA
- Extracted from http://www.forbes.com/sites/gilpress/2013/12/12/16-1-billion-big-data-market-2014-predictions-from-idc-and-iia/, December 2013.
- Extracted from http://www.goodreads.com/quotes/328848-the-best-way-to-predict-your-future-is-to-create, December 2013. Peter Drucker is also attributed with this statement.
Andrew Cardno is an established thought-leader in visual analytics, with nearly 20 years of experience in the field. Andrew has led private Ph.D. and Masters research teams into visualization/development for more than 13 years; this leadership has resulted in his winning two Smithsonian Laureates and more than six innovation awards. He is an inventor with more than 60 patent applications. He has published more than 60 industry and academic articles and is co-author of a book, published in 2012. Often asked to speak on the future of analytics at a variety of venues across the world, Andrew currently serves as CTO of BIS². He holds a bachelor of surveying from Otago University, New Zealand, and a diploma of computer science from Victoria University, NZ. You can contact the author at
[email protected].