Using Data to Drive Innovation
To move enterprises forward, you need to support innovation with both technology and organizational culture.
- By David Stodder
- January 26, 2017
"I can't understand why people are frightened of new ideas. I'm frightened by the old ones." This inimitable quote is from the American composer John Cage, whose musical pieces challenged audiences and the classical music orthodoxy of the twentieth century.
One of his most famous compositions featured musicians who did not play a single note! The idea was to focus on the ambient sounds made by the audience during the performance. He was also one of the first composers to employ a "prepared" piano, meaning that it had objects like forks and spoons stuck in the between the strings and hammers to make the piano sound anything but normal.
Although still not aligned with most audiences' tastes, Cage's revolutionary compositions and performance art continue to provoke thought and stretch minds to consider wider musical parameters. Avant-garde as they are, though, his compositions still rest on structure and some well-honed techniques for performance and composition, not to mention all the processes necessary for readying the concert hall for a performance, selling and collecting tickets, and so on. Even with the most spirited innovations, structured and well-practiced processes are essential to delivering the final product.
The same balance is necessary for innovating in business and technology.
Without doubt, innovation in these fields at times requires John Cage's kind of disruptive thinking to break through. Interdependent processes and practices build up over time to support traditional approaches, so changes to one process usually require changes to multiple processes.
Business innovations often focus on high-level "big idea" changes, but at some point, the ideas need to be executed at a practical level for innovation to have an effect. Innovative ideas become more powerful and influential if they fit into the existing nest of processes and practices or address any needed changes.
Innovation and Data Technologies
Analytics, business intelligence (BI), and data management can help organizations innovate, including by making interdependencies between humans, institutions, entities, and processes more apparent through study of data relationships. Organizations can use these tools to gain a better perspective on how making changes in one process or function will affect another process or function.
Data visualization is critical to seeing data relationships and shortening the decision makers' path to actionable intelligence. With visual BI and analytics, users are able to go beyond simple tabular reporting to look at comparisons, patterns, trends, and outliers. Visualizations are also effective for presenting ideas using "data storytelling" techniques that integrate data findings with narration.
Presentation and storytelling are essential to innovation; if your audience of busy managers and executives cannot grasp ideas easily and understand their potential, they are unlikely to be persuaded.
In most industries, leaders want faster decision cycles; that is, they want to reduce delays in the process of researching situations, arriving at decisions, and putting them into practice. Unfortunately, as organizations move toward being more data driven and less gut-feel and hunch driven, potentially innovative ideas can get buried in the mass of data and other content research sources.
In medical research, for example, it can take forever for specialists to sift through research, which can include data, doctors' text reports, and images. Good ideas can take years to form and then implement.
BI and analytics are being used effectively today by medical research firms to improve decisions about financial and planning considerations that are vital to sustaining research projects. Organizations can use data insights to align resources and finances behind innovative ideas, which can accelerate their implementation. With advanced, cognitive analytics maturing, we are seeing firms beginning to use tools to improve medical research itself.
Innovation and the Cloud
Cloud computing can give organizations greater flexibility to spin systems up to test ideas, do "what if" simulations, create scenarios, and perform data discovery. We found in the survey research for our TDWI Best Practices Report: BI, Analytics, and the Cloud (Q4 2016) that flexibility for just such requirements is a major driver behind use of cloud computing for BI and analytics.
Many organizations are using cloud platforms to create data sandboxes for users to experiment in so they do not have to wait for IT to acquire and configure on-premises resources for specialized data marts to support experimentation.
Cloud is also proving to be a space for organizations to experiment with open source tools and platforms running the gamut from Apache Hadoop and Spark to analytics tools and languages such as R and Python. Limitations users have with their in-house BI and analytics tools and data management platforms can be overcome by combining cloud and open source.
If nothing else, the combination is enabling more users to work with analytics, especially those who had not previously had access to the right tools and platforms.
Inventing with Data
"The best way to predict the future is to invent it," said computer science pioneer Alan Kay famously. As organizations increasingly turn to data as a critical asset for innovation, BI and analytics must help organizations realize Kay's notion. With data visualization and easier-to-use analytics and data preparation, technologies are evolving toward supporting "speed of thought" innovation.
Innovation, however, does not come without mistakes. Another great quote about innovation comes from the Hall of Fame hockey player Wayne Gretzky: "You miss 100 percent of the shots you never take." As organizations pursue innovation, they must allow for errors and learn from them. That takes managerial fortitude, not just analytics.
About the Author
David Stodder is director of TDWI Research for business intelligence. He focuses on providing research-based insight and best practices for organizations implementing BI, analytics, performance management, data discovery, data visualization, and related technologies and methods. He is the author of TDWI Best Practices Reports on mobile BI and customer analytics in the age of social media, as well as TDWI Checklist Reports on data discovery and information management. He has chaired TDWI conferences on BI agility and big data analytics. Stodder has provided thought leadership on BI, information management, and IT management for over two decades. He has served as vice president and research director with Ventana Research, and he was the founding chief editor of Intelligent Enterprise, where he served as editorial director for nine years.