Humans in the Loop for Machine Learning
Integrating people into machine processes will have a significant influence in how ML is employed in business.
- By Brian J. Dooley
- July 9, 2018
Machine learning (ML) is gaining an increasing share of the public imagination, but its limitations are also becoming apparent. ML solutions can provide important new capabilities across a wide operational space, but we are still nowhere near creating an artificial general intelligence.
Current ML solutions are sophisticated and may be combined to create broader applications, but they lack the real-world knowledge and human experience needed to create valid and acceptable outcomes on their own. An increasing part of the ML solution is human-in-the-loop capabilities where the machine matches a pattern but human input determines its validity and helps to refine the result. Such human-in-the-loop interactions will probably define most business ML uses in the years to come.
Integration with Human Thought
The addition of humans to an ML solution is not necessarily as straightforward as it might appear. ML processes contain a range of phases from data exploration to model creation, from testing and verification to actuation and experience. Most phases can be enhanced through intervention from data scientists, subject matter experts, or the general public.
Humans contribute by providing knowledge and capabilities that are impossible or inefficient for an ML solution. At the same time, ML is able to handle enormous data sets that are far beyond the reach of expert observation. Typical examples are security and classification problems, where the task is too ambiguous for a purely mechanical solution and too vast for even a large team of human experts.
A human-in-the-loop solution embraces the best possibilities of both the human expert and the machine. Such a solution extends human expertise in understanding and acting upon data across a much wider and more diverse input. It aids ML by accessing human experience that is either inaccessible or difficult to consume. Required knowledge may be from human society, external world conditions, hidden data, expectations, implicit rules, and behavioral constraints.
Another advantage of a combined approach is the possibility of removing bias. Human experts have implicit biases from experience; ML can help you identify these biases, but it may also need to be checked for spurious inferences.
Current Experience
Human-in-the-loop is already being utilized by many firms as an extension of the original model of ML operation, in which data scientists were always on hand to evaluate and correct a result. The main difference today is democratization of human input. The people who will interact with ML in the future will not always be data scientists; they will be subject matter experts or the general public. Indeed, one promising opportunity lies in integration with crowdsourcing, where individuals can bring collective experience to bear on questions being solved by ML programs.
Examples of human-in-the-loop ML today include Pinterest's use of automated human evaluation to filter out certain types of images based on crowdsourcing; start-up StitchFix's systems that train fashion classifier routines using a trained crowd; Google's use of humans in building intelligent search with ML, Facebook's image tagging through a combination of human classification and ML; and most current semi-autonomous cars.
Impact on Business and Jobs
Integration of people into machine processes will have a significant influence in how ML is employed in business as well as in defining its impact on the job market. We will need hybrid processes that incorporate the most efficient ways to integrate human and machine capabilities. In a greater sense, this extends beyond ML to other forms of AI and to robotics. Human chip implants are a somewhat distant possibility, but finding the sweet spot for human-machine interaction is of increasing concern as autonomous processes and "smart" machines become ubiquitous.
The interfaces between human intervention and ML need to be further examined. To create a rich ML ecosystem, it is clear that there must be defined roles and expectations for people employed in this way.
The impact of crowdsourcing on ML also needs to be better understood. Democratization of human input becomes imperative as data scientists are less likely to be available. The same pressures underlying democratization of BI several decades ago now apply to ML; processes need to be driven by the business rather than by engineering alone.
As we enter into this new and unexplored territory, it is important to bear in mind that new situations can create unexpected results. Expect more developments soon.
About the Author
Brian J. Dooley is an author, analyst, and journalist with more than 30 years' experience in analyzing and writing about trends in IT. He has written six books, numerous user manuals, hundreds of reports, and more than 1,000 magazine features. You can contact the author at [email protected].