Machine Learning Practitioner Survey Reveals Strong ML Community Support for AI Bill of Rights
More than one-quarter of ML practitioners believe bias will never be truly eradicated from AI products according to a survey commissioned by Comet.
Note: TDWI’s editors carefully choose press releases related to the data and analytics industry. We have edited and/or condensed this release to highlight key information but make no claims as to its accuracy.
Comet, provider of an MLOps platform for machine learning (ML) teams from startup to enterprise, has released the results of its second annual survey on the state of ML. The all new State of MLOps Industry Report | 2023 Machine Learning Practitioner Survey, which includes responses from 503 ML practitioners, sheds light on several important issues affecting the adoption of ML as well as project and initiative success.
“Our latest survey comes as ML practitioners are facing a new reality, with its own unique set of challenges ahead,” said Gideon Mendels, CEO and co-founder of Comet. “Organizations may tighten their budgets during these unstable economic times, but because leaders recognize ML’s potential to unlock incredible business value, there may be a push for ML practitioners to tackle more complex problems quickly. This may include addressing bias or adhering to a new AI Bill of Rights, making it imperative for ML practitioners and the success of an organization’s ML projects, to have the right tools in place.”
Among the key topics that emerged in this year’s report are the AI Bill of Rights, bias in AI products, and operational challenges amid tightening budgets.
AI Bill of Rights
The U.S. White House Office of Science and Technology Policy (WHOSTP) recently published a blueprint for an AI Bill of Rights, setting out five principles that should guide the design, use, and deployment of automated systems. The document provides a framework for how government, technology companies, and citizens can work together to ensure more accountable AI. In terms of the reaction within the ML community:
- Almost three-quarters (73%) of ML practitioners agree that the AI Bill of Rights should be mandatory by law versus opt-in.
- About 2 in 5 (39%) respondents think the AI Bill of Rights will affect their approach to ML deployment and development by slowing down the process.
- Similarly, 37% think the AI Bill of Rights will make the ML deployment and development process more difficult.
- Over a third (35%) believe it will make the process more expensive.
- On a positive note, almost 2 in 5 (38%) believe the AI Bill of Rights will make the ML deployment and development process safer, and a similar percentage (37%) think it will reduce the possibility of privacy violations.
- Over a third (35%) think the AI Bill of Rights will reduce the frequency of unsafe or ineffective ML systems.
Bias in AI Products
In fact, one of the reasons the WHOSTP created the initial blueprint for the AI Bill of Rights was due to the prominence of bias in AI products. Bias has been one of the leading topics related to AI in recent years. Some view bias as overhyped, with ML practitioners capable of implementing best practices to mitigate it; others think it is a problem that will continue to plague AI systems.
The latest survey reveals how ML practitioners are approaching bias.
- Over a quarter (27%) of ML practitioners surveyed believe that bias will never truly be removed from AI-enabled products.
- Of organizations surveyed, almost 2 in 5 (38%) have a designated point of contact or support team looking out for bias when planning the design and/or launch of an AI-enabled product.
- A third (33%) of respondents think reducing the risk of bias occurring is one of the main benefits of explainable AI, which might indicate this could be a solution (though not without its own challenges).
Additional ML Challenges
The state of the economy clearly weighed on respondents’ minds as they considered its impact on their business and how that could trickle down to affect investments in ML. They also identified other areas of stress or challenges they anticipate facing in 2023.
- All (100%) machine learning practitioners surveyed said the economic situation will impact their business. The most common way, according to respondents, will be redundancies in the tech team (40%), an impact on budgets (37%), and a hiring freeze (36%).
- Almost a third (32%) say innovation will slow as a result.
- Over the next year, ML practitioners surveyed anticipate the biggest to be sustainability (41%), retention (39%), hiring staff with correct institutional knowledge (36%), and explainable AI (36%).
The complete survey can be found here. Registration is required.
Survey Methodology
The research was conducted by Censuswide with 503 U.S. 18 years or older machine learning practitioners between December 20, 2022 and December 29, 2022. Censuswide abides by and employs members of the Market Research Society which is based on the ESOMAR principles and are members of The British Polling Council.