TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

The Importance of Being IoT

To win in the marketplace, an organization must incorporate IoT data and analytics, as well as artificial intelligence, into its strategic vision.

• By Barry Devlin
• May 16, 2017

The World Economic Forum has christened it the "Fourth Industrial Revolution" -- this digital technology embedded in everyday items (from phones to automobiles, production lines to city streets, and even in our own bodies) and its implications. In IT, we talk of big data, the Internet of Things (IoT), and edge analytics, but we often miss the broader significance of the ongoing change for business strategies, human interactions, and society itself.

Even the phrase Fourth Industrial Revolution underestimates the scope and complexity of the radical transformation on which we are now embarking. This fourth iteration will dwarf the previous three.

For those of us who have grown up in the world of data warehousing and BI, the change has already been dramatic. Traditional data is an output of running the business. In the new world, data comes first and business is the result of leveraging it. The IT implications of this shift range from collection and preparation, through governance, to data usage patterns. The divergence is evident in the data warehouse versus data lake debate, but as I wrote earlier this year, you will continue to need both. They are, in reality, complementary.

However, managing traditional data well is becoming merely the table stakes for doing business. The real success and money will emerge from utilizing the new IoT data for innovation in existing business models and for invention of completely new ones.

IoT Challenges Business Strategy Everywhere

Compared to the changes encountered so far by IT, those faced by business fall into the superstorm category. Innovation and subsequent disruption will be widespread, affecting every industry and every business function. The implications are discussed at length in "The Age of Analytics: Competing in a Data-Driven World," McKinsey Global Institute's December 2016 report.

The authors point out that

Leading companies are using their capabilities not only to improve their core operations but also to launch entirely new business models. The network effects of digital platforms are creating a winner-take-most situation in some markets. The leading firms ... are actively looking for ways to enter other industries. These companies can take advantage of their scale and data insights to add new business lines, and those expansions are increasingly blurring traditional sector boundaries.

Winners will be those organizations that can incorporate IoT data and analytics, as well as artificial intelligence, into their strategic vision.

At Teradata Universe in April, chief business development officer Mikael Bisgaard-Bohr shared examples of how leading incumbents in various industries have stepped up to the challenge. From manufacturing to retail, transportation to entertainment, the common themes were a focus on IoT data sources as the raw material for innovation and the widespread use of analytics to experiment with new process and product directions.

The Role of Data Architecture

Taking these themes to the next level of detail and implementation requires a deep understanding of data architecture -- an ability to distinguish between different classes of data and the conditions for gathering, managing, and using them. Bisgaard-Bohr offered three classes: non-coupled, loosely coupled, and tightly coupled.

The last category corresponds to traditional data warehouse data, used for BI and reporting. Non-coupled is the raw data from IoT (and big data) sources where, in principle, storing everything is the goal. In practice, of course, decisions on just how much of everything is cost-justified will be required.

Loosely coupled data, lying between the other two classes in both size and structure, is where analytics will drive maximum innovation and business value. The challenge here is to draw the boundaries with the other classes both reasonably and flexibly. Making this category too broad will lead to prohibitive data management costs. Narrowing it too far will limit the extent of analytics and thus its value.

McKinsey's report clearly enunciates the enormous value and potentially disruptive power of IoT data for business. However, data management professionals have been slow to undertake extensive architectural thinking about this data. Teradata's three classes offer a good starting point but beg deeper analysis. Key questions include: Is coupling a valid basis for categorization? Are three classes sufficient? How and why does data move between them?

Today, data architecture commonly starts with the twin towers: data warehouses and data lakes. However, their evolution as alternatives to relational databases has been more in response to technological developments -- such as the emergence of Hadoop and NoSQL -- than to fundamental thought about data characteristics.

In Business unIntelligence, I proposed a tri-domain information model based on a broader set of core data characteristics as another basis for a complete architecture. Given the business impact and the speed of evolution of IoT -- or the Fourth Industrial Revolution -- it behooves the data architecture community to provide further insight and guidance.