Journal of Mechanical Design

companion website

FEATURED ARTICLES

ADVISE: Accelerating the Creation of Evidence Syntheses for Global Development Using Natural Language Processing-Supported Human-Artificial Intelligence Collaboration

Authors: Kristen M. Edwards, Binyang Song, Jaron Porciello, Mark Engelbert, Carolyn Huang, Faez Ahmed

Abstract

When designing evidence-based policies and programs, decision-makers must distill key information from a vast and rapidly growing literature base. Identifying relevant literature from raw search results is time and resource intensive, and is often done by manual screening. In this study, we develop an artificial intelligence (AI) agent based on a bidirectional encoder representations from transformers (BERT) model and incorporate it into a human team designing an evidence synthesis product for global development. We explore the effectiveness of the human–AI hybrid team in accelerating the evidence synthesis process. To further improve team efficiency, we enhance the human–AI hybrid team through active learning (AL). Specifically, we explore different sampling strategies, including random sampling, least confidence (LC) sampling, and highest priority (HP) sampling, to study their influence on the collaborative screening process. Results show that incorporating the BERT-based AI agent into the human team can reduce the human screening effort, i.e., the number of documents that humans need to screen, by 68.5% compared to the case of no AI assistance and by 16.8% compared to the industry-standard case of using a frequency-based language model and support vector machine-based classifier for identifying 80% of all relevant documents. When we apply the HP sampling strategy, the human screening effort can be reduced even more: by 78.3% for identifying 80% of all relevant documents compared to no AI assistance. We apply the AL-enhanced human–AI hybrid teaming workflow in the design process of three evidence gap maps for U.S. Agency for International Development and find it to be highly effective. These findings demonstrate how AI can accelerate the development of evidence synthesis products and promote timely evidence-based decision-making in global development.

Read More

SHARE: 

Featured Articles Subjects

Additive Manufacturing
Ancient design
Artificial Intelligence
Associate Editors
Awards
Bioinspired Design
Complex Engineered Systems
Compliant Mechanisms
Composites
Data Driven Design
Data Mining
Data-driven Design
Design Automation
Design Communities
Design Education
Design Fixation
Design Innovation
Design of Mechanisms and Robotic Systems
Design Optimization
Design Research
Design Theory
Design Theory And Methodology
Digital Twin
Direct Contact Mechanisms
Double-Blind Review Option
Dynamics
Editors' Choice Award
Energy
Engineered Materials And Structures
Ethics
Fluids
Gears
Generative Design
Guest Editorials
IDETC
Industry
Information Design
International Perspectives
JMD History
JMD Review Process
JMD Statistics
Kinematics
Leadership
Machine Learning
Manufacturing
Mechanisms
Mechanisms Robotics
Memoriam
Neural Networks
Optimization
Origami
Orthotics
Piezoelectric Actuators
Power transmission gearing
Product Development
Robotics
Simulation-based Design
Smart Structures
Special Issues
Sustainable Design