TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Big Data and IoT: The Privacy Dilemma

Security will remain a huge problem for the Internet of Things, but technical solutions are available. IoT's privacy dilemma is more fundamental and will require action on three fronts.

• By Barry Devlin
• February 8, 2016

This year's Consumer Electronics Show in Las Vegas provided its usual quota of gadgets and gizmos to steal the hearts of tech-obsessed consumers, covering every aspect of life from measuring your heartbeat and sleep patterns to keeping you endlessly entertained at home or in your autonomous automobile, managing your home to optimizing your city, brewing beer to finding fish to catch. The common thread in all of this is a proliferation of miniature sensors combined with pervasive connectivity: the Internet of Things (IoT).

Although the focus has been on all the wonderful things consumers can do and vendors can sell, there has been a lot less emphasis on the data -- big, big data -- challenges and changes emerging as this market matures. Data is what will make the IoT work or not. Value and market disruption will emerge largely from how data is used. Disaster will ensue if data is poorly managed.

Data is slowly emerging from the shadows. Perhaps the best indication was the keynote at CES by IBM's CEO, Ginny Rometti; it's the first time IBM has participated at this level. Rometti used the time to link Watson directly to wearables, healthcare, and robotics. Her key comment was "Going digital is not a destination, it's a foundation. What will differentiate you is understanding all that data."

I have long considered artificial intelligence approaches from Watson and others as more of a revolution in business intelligence than other much-touted improvements in visual analytics or data wrangling, for example. The value and disruption that emerges here is based on the in-depth abilities of machine learning approaches to collate and link data across a wide range of data types and to interpret the variety of ways such data can be interrogated.

Unfortunately, little attention is being paid to the numerous data management aspects that must be addressed if we are to avoid a debacle around misuse of personal information garnered from the IoT in the near future. Fundamental security failures in IoT devices have already been exploited. Furthermore, most security experts are of the opinion that IoT device vendors are ignoring security best practices in order to get their devices to market and that extensive hijacking is both feasible likely. With Gartner's estimate of 6.4 billion IoT devices by the end of 2016, the potential for breaches is growing fast.

Although security will remain a huge problem, its possible solutions are largely technical in nature. The privacy dilemma is more fundamental, requiring action on three fronts:

1. Ethical decisions on what use should be made of personally-identifiable information (PII).

Data warehousing and more recently big data have long been used to drive marketing from generic segmentation strategies to more personalized targeting, with Nirvana seen as the "segment of one." There is much talk about anonymized and aggregated data, but the truth is that detailed IoT data is highly personal, and as the number of devices used by any one person multiplies, "deanonymization" becomes increasingly straightforward. As a result, businesses must decide, at the highest level, the ethical bounds of use of personal data, and their compatibility with marketing and financial models in highly competitive markets.

This topic covers ground that is likely very unfamiliar to the majority of business executives, and has received scant attention in the market. A good starting point for exploration is A Primer on Ethical Principles in an Information Governance Framework.

2. Organizational and policy changes

Because of its direct relationship to the physical (as opposed to online) world, IoT data has real operational uses that drive value to the consumers and/or suppliers. Such uses clearly demand PII, to which the consumer must agree. Organizational and policy changes will be required to ensure that such access is not abused by, for example, individualized marketing or sharing of data with business partners.

3. Technical solutions

Storage and/or access to PII must therefore be carefully regulated and segmented by the application of the appropriate technical solutions

A Final Word

Much of the focus for IoT in the coming year or two will be on building the market and, hopefully, on defining and implementing standards for communication and security. However, a parallel emphasis on what needs to change within the business will be essential to avoid the real privacy pitfalls that IoT creates.