AI Case Study

IBM Health's Watson for Oncology is criticised for providing inaccurate and potentially dangerous treatment recommendations for cancer patients

IBM Health has claimed its Watson for Oncology has been implemented in 230 hospitals globally, but internal documents suggest the system has made serious errors in its treatment recommendations. Furthermore, the company has been criticised for the way the system was trained, on a small portion of doctors from one American hospital.

Industry

Healthcare

Healthcare Providers And Services

Project Overview

From Gizmodo: "Internal company documents from IBM show that medical experts working with the company’s Watson supercomputer found “multiple examples of unsafe and incorrect treatment recommendations” when using the software, according to a report from Stat News. Stat reviewed documents that were included in two presentations given in June and July 2017 by IBM Watson’s former deputy health chief Andrew Norden. The documents were reportedly shared with IBM Watson Health management. According to Stat, those documents provided strong criticism of the Watson for Oncology system, and stated that the 'often inaccurate' suggestions made by the product bring up 'serious questions about the process for building content and the underlying technology.'

According to the report, the documents blame the training provided by IBM engineers and on doctors at MSK, which partnered with IBM in 2012 to train Watson to “think” more like a doctor. The documents state that—instead of feeding real patient data into the software—the doctors were reportedly feeding Watson hypothetical patients data, or “synthetic” case data."

Mass Device reports that "IBM has defended its Watson for Oncology software, releasing a statement to STAT indicating that it has 'learned and improved Watson Health based on continuous feedback from clients, new scientific evidence and new cancers and treatment alternatives,' and that it has released 11 software updates to improve functionality over the past year.

But internal documents indicate that training and effectiveness of the Watson for Oncology system was flawed due to the small number of cases and the inclusion of artificial cases with only one or two doctors supplying recommendations for each type of cancer it was designed to work with, according to the report.

The internal presentation included an example case in which a 65-year old man with lung cancer and evidence of severe bleeding was recommended chemotherapy and a drug called bevacizumab, which includes a “black box” warning advising that it shouldn’t be administered to patients experiencing severe bleeding, according to the report."

Reported Results

According to Mass Device: "The documents also indicated that internal studies of the Watson for Oncology product were designed to generate favorable results, according to the report. Customer feedback from the internal documents include comments reflecting serious dissatisfaction from customers, according to the report."

Technology

Function

Operations

General Operations

Background

Gizmodo reports that the company claims "Watson for Oncology is trained to help oncologists treat 13 cancers and is being used by 230 hospitals around the world, and has 'supported care for more than 84,000 patients.'" However, according to Healthcare Dive, a "STAT review of internal IBM documents suggests the company’s Watson supercomputer wrongly advised doctors on how to treat patients’ cancers."

Benefits

Data

Trained on potentially synthetic patient data and cases based on doctor recommendations from Memorial Sloan Kettering Cancer Center.