TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

The New Ethics of Data Management

The hunger for profit is driving the convergence of the digital and physical worlds, which raises some deeply disturbing privacy implications.

• By Barry Devlin
• July 10, 2017

Two recent press stories sent shivers up my spine: "Google now knows when its users go to the store and buy stuff" in The Washington Post and "Surge pricing comes to the supermarket" in The Guardian. The reason? Both point directly to the naked profit-taking orientation driving at least some of the convergence of the digital and physical worlds and the deeply disturbing privacy implications evident in this trend.

For Further Reading: AI in the Crosshairs The Shortcomings of Predictive Analytics Data Collection: Ethics and Governance

In a mid-May announcement, Google revealed its use of billions of credit card transaction records -- both online and off -- to analyze the impact of digital advertising campaigns. Google was at pains to point out the lengths to which it went in order to protect its users' privacy: years of effort to develop "a new, custom encryption technology that ensures users' data remains private, secure, and anonymous," via a double-blind data-matching approach that prevents Google from seeing real-world credit card details and retailers from seeing Google user IDs.

However, the move still raises significant privacy concerns, from questions of customers' consent to just how impenetrable the double-blind wall is against both hackers and marketers in pursuit of the holy grail -- the "segment of one" -- via reidentification techniques.

Are All Prices Created Equal?

The Guardian article shows similar thinking, this time from the retail side. Similar to the way airline seats or hotel rooms bought online already work, the pricing of everyday goods could be made continuously variable and subject to your willingness (or ability) to pay, determined by big data analytics, of course.

Fixed pricing in stores was introduced only in 1861 in Philadelphia by John Wanamaker with the slogan: "if everyone was equal before God, then everyone would be equal before price." This notion of equality (and it is limited at best) has taken a battering in modern customer targeting across all industries, especially as customers have willingly sacrificed privacy about their social networks and financial circumstances for convenience and the lure of often minuscule discounts.

Wanamaker's slogan thus caused me pause for thought: what do we mean by the phrase in the U.S. Declaration of Independence: "We hold these truths to be self-evident, that all men [not excluding women and children, of course] are created equal?" Consider two identical loaves of bread, sold in the same store at the same time. Their costs of manufacture, distribution, and sale are clearly the same. Do you consider it fair and equitable that two customers should pay different prices?

The widespread acceptance and use of loyalty cards shows that most people probably do. After all, by signing up, you get lower prices as a reward for providing your contact information and, in more sophisticated schemes, your shopping history. The act of signing up is a declaration of consent, however unconscious, to some level of variable pricing.

However, if the retailer were to include additional financial considerations -- based, perhaps, on Google's credit card data -- such that you get only a 5 percent discount because you recently bought an iPhone contract, whereas your "less fortunate" Android-using neighbor is getting the full 10 percent, how would you feel?

Playing the Game

At a broader, socio-economic level, if you are the lucky person paying the least, do you wonder what and who is making up for the retailer's decreased profit on your purchase?

Under neoliberal capitalism, these questions are seldom if ever asked. The only consideration is that business should maximize profit by any means possible. If one customer is judged to be more willing or able to pay more, he or she is fair game!

Of course, customers must be willing to play along, which they often are. For example, when purchasing airline seats, most of us hunt around looking for the best deal, even though the data scientists among us understand that our behavior is being mined to set the offered price. If we knew how to game the system more effectively, would we?

John Naughton, professor of the public understanding of technology at the Open University in the UK, observes: "Without really thinking about it, we have subjected ourselves to relentless, intrusive, comprehensive surveillance of all our activities and much of our most intimate actions and thoughts. And we have no idea what the long-term implications of this will be for our societies -- or for us as citizens. One thing we do know, though: we behave differently when we know we are being watched."

With the retail world increasingly driven by "surveillance" data, we have embarked on an all-encompassing, uncontrolled socio-economic experiment, in which we are the laboratory rats for Amazon and Google. Is this the role you want your children to have?

The Responsibility of Ethical Data Use

Such questions should not be considered the sole preserve of philosophers or ethicists, and certainly not economists. As citizens and customers, we must decide what we accept. Our answers will determine the character of the world we live in.

As data management professionals, these questions must become part of the justification and deliberations for all sorts of big data analytics, Internet of Things, and artificial intelligence projects. Over the past year, we have seen some recognition that security, privacy, and transparency are coming to the fore as fundamental considerations for such projects. This is good news and, indeed, a necessary change. Nicholas Diakopoulos' "Accountability in Algorithmic Decision Making" (Communications of the ACM Vol. 59, No. 2, 2016), offers relevant considerations.

The next -- and most difficult -- challenge for us in the data industry is to begin to consider the ethics of the projects we undertake. How does the data we collect broadly impact the lives of our customers even when it's used according to the aims of the project? Will it decrease well-being? Will it affect employment or job satisfaction? If it is misused, what are the consequences?

Does bias -- intended or unintended -- exist in the training data set or algorithms, and what are the consequences? Sufficient examples of such bias have already emerged in areas as diverse as image recognition and law enforcement to leave no doubt that there are significant societal consequences already emerging.

The time has come for us, as data management professionals, to make difficult choices. The need now is to apply ethical judgement to the entire analytics process, from requirements through to design, development, ongoing use, and eventual decommissioning.