TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

New Architectures and Enhanced Roles for IT

The current explosion of data and tsunami of analytics make this the perfect time to re-evaluate the measures you think are vital in your enterprise.

• By Barry Devlin
• February 5, 2018

Exactly 30 years ago (in February 1988), Paul Murphy and I published the first data warehouse architecture in the IBM Systems Journal.

The first data warehouse architecture.

It was a much simpler world in BI then. The decision making we supported extended little further than understanding and managing sales across two lines of business in the company. Data was ingested from no more than a couple of internal platforms. The impact of actions taken was limited to the objectives and results of -- at most -- a few thousand salespeople and, we hoped, IBM's bottom line.

Fast-forward to 2018 and the scale of data has grown by at least seven orders of magnitude, sourced from the entire spectrum of social media and the physical world. Decisions -- or recommendations for them -- emerge from algorithms and AI models with rationales invisible or incomprehensible to many decision makers. Actions taken may ripple across the entire social and economic landscape.

For Further Reading: The New Ethics of Data Management Algorithms or Democracy -- Your Choice Prejudice in the Machine Can Jeopardize Your Enterprise

These are substantial changes in scope for both information and process. They are of immense significance to people both within the enterprise and beyond its walls. The BI community pays too little attention to a myriad of implications. I have previously written about the ethical consequences for BI providers and the economic challenges raised by such changes.

In this article, I challenge you to consider an important opportunity presented by the current explosion of data and the tsunami of analytics: to revisit and re-evaluate the measures you consider vital in your enterprise.

The Economic Growth Myth

In almost every industry and enterprise, the measure most often described as vital is some variation of growth -- typically in market share, revenue, or profit. This belief in endless growth as a key performance indicator (KPI) is so pervasive that its value goes unquestioned and its source invisible. However, the natural world shows unequivocally that endless growth is impossible. Summer always turns to fall. A population explosion in any species is always followed by a collapse as resources are exhausted.

The blame for this myth lies squarely with economics -- aptly labeled "the dismal science" by Thomas Carlyle -- which, over many decades, divorced income from capital and, more tragically, production from real physical science. The result, post-World War II, was a growing and near-exclusive reliance on a single metric for economic progress: growth of gross domestic (or national) product -- GDP/GNP. For over a century, economists have missed the scientific reality that production/consumption can grow forever only if new energy and resources become continuously available. It beggars belief that the majority of politicians and economists have failed to see that we are bumping against the boundaries of Earth's closed energy/resources system and have been for some decades already.

Radical new thinking in this field is exemplified by Kate Raworth, who has defined a set of social and ecological measures that must be balanced to provide a safe and just space for human existence and, by extension, business and governmental activity. Twelve foundational social measures (from food and water to equality and political voice) combine with nine ecological measures (climate change to biodiversity) to set boundaries within which humanity must operate if we are to survive. Any business claiming to value social responsibility must surely set some or all of these measures among its KPIs.

Money for Nothing and the Kittens for Free

Within the technology industry, our record of considering the bigger picture is equally dismal. For example, the technical solution for the decentralized currency bitcoin demands a mining process based on wasteful and otherwise unnecessary energy use. As of December 2017, mining a single bitcoin consumed energy equivalent to nine U.S. households for a day.

At current run rate, by mid-2019, the bitcoin network will consume more electricity than the entire U.S. does today. The implications for climate change amelioration should be obvious. Meanwhile on the Ethereum blockchain, a $1 million run on virtual kittens has left the network struggling to commit transactions and caused mining costs to rise dramatically.

Despite these concerns, these two examples emphasize just how central technology has become in business. These businesses simply would not exist without the IT infrastructure that has been built over the past three decades. This is the biz-tech ecosystem I have previously described as "The evolving environment where business is fully or heavily dependent on information technology for all activities. IT adopts the role of partner and co-creator in business innovation." At its fullest extent, IT literally is the business. As a consequence, it falls to IT to look beyond its traditional computer-related concerns.

New Role, New Architecture Picture

Thirty years ago, the first data warehouse architecture drove IT practitioners to raise their sights beyond tightly bounded systems that record business transactions to consider the needs of their enterprises for consistent, integrated information. Now, with information technology at the heart of much business, it behooves IT professionals to raise their sights again, deploying their architecture and engineering skills to support and drive sustainable technology and business models.

It is time for us in IT to adopt a new -- broader and more ambitious -- architecture picture that looks beyond IT's traditional concerns. This architectural diagram of an embedded economy is not mine, but comes from Kate Raworth's seminal book Doughnut Economics. George Monbiot provides an excellent introduction in his April 2017 article, as well as an assessment of how we rate globally against the measures.

The embedded economy. (Source: Kate Raworth and Marcia Mihotich)

This architecture emphasizes that the past (and current) business focus on the market and growth in income measures is but one part of the bigger picture. The commons (of which we have some limited experience already through open source software), state, and household (the unpaid work, mostly by women, that contributes immeasurably to our wealth) are all part of the economic process. Furthermore, economic success can exist only within the bounds of a healthy and fair society within the safe bubble of Spaceship Earth.

Here are concepts that make sense to the architects and engineers of IT. IT, more than most business departments, understands the issues and solutions implied here. We now have the data and the analytics tools to set and track KPIs for holistic business and societal good. We probably have more data and tools than we need. We need only the courage to begin.