Verified by 
Artificial Intelligence
How Synergistic Is the Combo of AI and Humans?
A study examines the efficacy of AI and human collaboration versus either alone.
Posted November 12, 2024 Reviewed by Kaja Perina
When does it make sense to use artificial intelligence (AI) with human intelligence versus either alone? The answer may seem intuitive, however given the rapid adoption of AI in the workplace worldwide, it merits scientific evaluation. Researchers at the Massachusetts Institute of Technology (MIT) Center for Collective Intelligence at the Sloan School of Management published a new study in Nature Human Behaviour that examines the performance impact of AI combined with humans that provide insights with implications for the future of work.
AI is impacting the future of work. By 2025, an estimated 85 million jobs will be replaced by AI automation according to the World Economic Forum. An estimated two-thirds of U.S. occupations are exposed to partial or full AI automation, according to economists at Goldman Sachs Research.
Organizations are rapidly deploying AI worldwide and transforming how work is conducted. The adoption of artificial intelligence has experienced a surge in the past year according to the 2024 McKinsey Global Survey on AI. In most major geographic areas around the world, artificial intelligence is being adopted by over two-thirds of survey respondents, and 50 percent of survey respondents report that their organizations are using AI in two or more business functions per McKinsey.
“A large body of work suggests that integrating human creativity, intuition and contextual understanding with AI’s speed, scalability and analytical power can lead to innovative solutions and improved decision-making in areas such as health care, customer service and scientific research,” wrote corresponding author Thomas Malone, PhD, MIT professor and founding director of the MIT Center for Collective Intelligence and his research co-authors. “However, a growing number of studies reveal that human–AI systems do not necessarily achieve better results than the best of humans or AI alone.”
Malone, together with his co-authors, MIT assistant professor and computational social scientist Abdullah Almaatouq, PhD and MIT PhD candidate Michelle Vaccaro, point out in their paper that ethical issues, trust, and communication barriers may impede the process of the joint collaboration of AI and humans.
To shed light on what may seem a paradox on the surface, the MIT team aimed to take a large-scale systematic meta-analysis and literature review spanning three years to quantify and evaluate the synergy of AI and human systems.
To achieve this, the team established a criterion to only include scientific papers that had AI and humans working collaboratively on an original experiment and quantified the performance of the collaboration as well as by AI alone and humans by themselves.
The MIT researchers screened 5,126 papers published during January 2020 to June 2023. From this vast pool of papers, they found 74 papers that met their study criteria. These papers contained 106 experiments that yielded information on 370 unique effect sizes that measure the performance results of AI and human collaboration. Decision-making tasks where the participants select a predefined set of options comprised 85% of the effect sizes. The researchers considered three approaches of task performance: 1) with only AI, 2) only humans, or 3) a combination of AI and Human.
When the MIT researchers compared the combination of AI and human scenario to a baseline of either the human-only or AI-only, they found that the performance of the combo was "significantly worse overall” than the baseline.
In the case of comparing the combination of AI and human solution to a human-only baseline, the researchers validated human augmentation and discovered that the combo outperformed the scenario of humans working alone.
“On average, we found evidence of human augmentation, meaning that the average human–AI systems performed better than the human alone,” the researchers wrote. “But we did not find human–AI synergy on average, meaning that the average human–AI systems performed worse than at least one of the human alone or the AI alone."
For decision-making tasks such as medical diagnosis, classifying deep fakes, and demand forecasting, AI working by itself outperformed the combo of AI and human collaboration.
Despite the fact that their principal findings suggest that on average the combination of AI and humans lack synergy, the researchers “do not think this means that combining humans and AI is a bad idea.”
“On the contrary, we think it just means that future work needs to focus more specifically on finding effective processes that integrate humans and AI,” the MIT researchers concluded.
Copyright © 2024 Cami Rosso All rights reserved.
