In a recent article in the Journal of the American Medical Association, BIDS/BCHSI Data Science Health Innovation Fellow Stephanie Eaneff, with Ziad Obermeyer (Associate Professor of Health Policy and Management in Berkeley's School of Public Health) and Atul J. Butte (Director of UCSF's Bakar Computational Health Sciences Institute), advocate for increased oversight and quality control in the implementation of predictive algorithms in applications for human health.

Predictive algorithms are currently used in a variey of clinical applications, and more than 50 AI/ML algorithms have already been cleared by the US Food and Drug Administration. Used appropriately, these algorithms can help clinicians and healthcare providers diagnose and manage disease.  However, if not implemented appropriately, algorithms can exacerbate existing systems of structural inequality.

According to the article, "As the US Food and Drug Administration reassesses its regulatory framework for AI/ML algorithms, health systems must also develop oversight frameworks to ensure that algorithms are used safely, effectively, and fairly. Such efforts should focus particularly on complex and predictive algorithms that necessitate additional layers of quality control. Health systems that use predictive algorithms to provide clinical care or support operations should designate a person or group responsible for algorithmic stewardship. This group should be advised by clinicians who are familiar with the language of data, patients, bioethicists, scientists, and safety and regulatory organizations."

While increased algorithmic stewardship will not eliminate all unintended consequences, it will "help to ensure that these new technologies are used safely, effectively, and fairly, and to the benefit of diverse patient communities."

The Case for Algorithmic Stewardship for Artificial Intelligence and Machine Learning Technologies
September 14, 2020  |  Stephanie Eaneff, Ziad Obermeyer, Atul J. Butte  |  Journal of the American Medical Association (JAMA)