Choosing Augmentation Over Automation in AI
On the AI journey, automation is often the default, depopulated destination. We must consciously choose to empower humans via augmentation.
- By Barry Devlin
- May 31, 2019
"Airplanes are becoming far too complex to fly. Pilots are no longer needed. ... Often old and simpler is far better ... and the complexity creates danger. All of this for great cost yet very little gain." Donald Trump's tweet , a hasty response to the fatal Ethiopian Airlines crash on March 10, offers a typically visceral response to computerization when it is perceived to cause a catastrophe. Unfortunately, it ignores the subtle issues surrounding the adoption of technology to both support and replace human involvement in decision making.
Early investigations into the cause of this and an earlier crash point to complex, computerized flight-control software -- the Maneuvering Characteristics Augmentation System (MCAS) -- introduced on the Boeing 737 MAX 8 to correct the angle of attack of the aircraft if it becomes too steep under certain flying conditions. A single source -- faulty sensor data -- has been blamed for the crashes. However, the algorithm also considers additional inputs, including the state of the autopilot, the flaps position, air speed, and steep turning. The software apparently also resists repeated overrides by the pilot, information which should, conceivably, have influenced the algorithm's behavior.
Although labelled an augmentation system, MCAS is a fully automated system, invisibly compensating for redesigned characteristics of the plane -- more forward and heavier engines -- that tend to tilt up its nose. Contrary to Trump's claim of "great cost yet very little gain," automation appears to have been easier and cheaper to implement than major physical rebalancing of the plane.
AI -- When Augmentation and Automation Collide
Discussions about artificial intelligence (AI) often present solution design as a simplistic, binary choice between automation and augmentation. However, we must first understand the difference between the two. Only then can we decide which approach -- or combination of approaches -- to choose in different circumstances.
Simply put, automation of processes replaces human decisions and action by technology, increasingly a combination of hardware and software. Augmentation, on the other hand, proposes that technology be used to support and improve human behavior, both in making decisions and taking action.
In the case of autonomous or semi-autonomous real-world systems such as automobiles or airplanes, automation is often the chosen approach to hide or compensate for technological complexity, especially where rapid decisions are needed. Despite decades of experience, such systems are not foolproof.
Although not (yet) fully AI-based, flight-control systems show how automation can consistently operate in complex technological systems. However, they also presage how AI automation can dramatically fail when sensors send faulty data, as in the MAX 8 crash. If and when the algorithm cannot cope, for this or other reasons, the handover from fully automated behavior to augmented or full human control becomes critical. This is another area where MCAS was found wanting.
The lesson here is that automation alone is insufficient. Involving human skills to augment automated systems and augmenting these skills with deeper understanding of the technical complexity of the process are vital design considerations.
Business Decisions Face Similar Choices
Business decision making is seeing a rapid growth in the application of analytics and machine learning. Many businesses expect these AI technologies will provide an easy and rapid path from today's manual decision making to fully automated processes producing "better" decisions.
Chuck Densinger, co-author of Geek, Nerd, Suit and COO at Elicit, in a recent article dismisses this belief -- and I fully agree. He describes a "decision automation continuum" where decision making moves from unassisted to automated in four conceptual stages: data science, BI/reporting, machine recommendation, and machine action.
Placing data science as a first step before BI/reporting may seem odd. However, this is not an evolutionary timeline. His point is that problem analysis and data exploration -- today, the preserve of data science -- must precede the deployment of standard reports and dashboards. Furthermore, only after you have completely understood the problems and possibilities in the first two stages can you contemplate machine recommendations (augmentation) and automated machine actions.
Despite this logical progression ending with automation, Densinger proclaims augmentation more important: "While machines can and will magnify our thinking and apply it to data faster and more accurately than we can without assistance, they still can't think for us. Advantage: human." True though this is, he misses one important consideration -- return on investment (RoI).
Automation Follows the Money
Machine recommendations are great. They keep humans in the decision-making loop and allow us to apply intuition and ethics as appropriate. Sadly, such thoughtful and skilled humans come at a cost. They also represent a bottleneck in the process. The traditional capitalist focus on the bottom line inevitably seeks to remove such expense and speed up the response. If the recommendations are proving good enough, why not move to full automation? This is essentially the same logic seen in the Boeing example.
As AI improves, the urge to shift from human-made decisions, through augmentation, to human-free decision automation becomes more attractive financially. We must therefore define rules and conditions -- based on ethical, philosophical, and human skills reasons -- for choosing augmentation rather than automation or for adding explicit layers of augmentation to automated processes.
Trump's tweet concluded, "I want great flying professionals that are allowed to easily and quickly take control of a plane!" As unaccustomed as I am to agreeing with his comments, I must agree in this case. As data management and analytics professionals, we must urgently step up to the task of figuring out how and when in highly automated AI systems to hand oversight and control to humans despite RoI-based arguments to the contrary.
About the Author
Dr. Barry Devlin is among the foremost authorities on business insight and one of the founders of data warehousing in 1988. With over 40 years of IT experience, including 20 years with IBM as a Distinguished Engineer, he is a widely respected analyst, consultant, lecturer, and author of “Data Warehouse -- from Architecture to Implementation" and "Business unIntelligence--Insight and Innovation beyond Analytics and Big Data" as well as numerous white papers. As founder and principal of 9sight Consulting, Devlin develops new architectural models and provides international, strategic thought leadership from Cornwall. His latest book, "Cloud Data Warehousing, Volume I: Architecting Data Warehouse, Lakehouse, Mesh, and Fabric," is now available.