AI Case Study

DeepMind achieves human specialist accuracy in diagnosing retina disease based on scans using machine learning

DeepMind and Moorfields Eye Hospital have developed a convolutional neural network-based model for automating the diagnosis of different retinal diseases based on optical coherence tomography scans. The machine learning method performed as well as opthamologist specialists to triage scans based on presenting diseases.

Industry

Technology

Software And It Services

Project Overview

DeepMind conducted the research in collaboration with Moorfields Eye Hospital. As stated in the research paper: "We developed our architecture in the challenging context of OCT imaging for ophthalmology. We tested this approach for patient triage in a typical ophthalmology clinical referral pathway, comprising more than 50 common diagnoses for which OCT provides the definitive imaging modality... Our framework can triage scans at first presentation of a patient into a small number of pathways used in routine clinical practice with a performance matching or exceeding both the expert retina specialists and optometrists who staff virtual clinics in a UK NHS setting."

From the DeepMind blog: "On top of this, our technology can be easily applied to different types of eye scanners, and not just the specific type of device it was trained on at Moorfields. This might seem inconsequential, but it means that the technology could be applied across the world with relative ease, massively increasing the number of patients who could potentially benefit. This also ensures the system can still be used in hospitals and other clinical settings even as OCT scanners are upgraded or replaced over time. If this technology is validated for general use by clinical trials, Moorfields’ clinicians will be able to use it for free across all 30 of their UK hospitals and community clinics, for an initial period of five years. These clinics serve 300,000 patients a year and receive over 1,000 OCT scan referrals every day – each of which could benefit from improved accuracy and speed of diagnosis."

Reported Results

According to the research paper, the "framework achieved and in some cases exceeded expert performance... Performance of our framework matched our two best retina specialists and had a significantly higher performance than the other two retinal specialists and all four optometrists when they used only the OCT scans to make their referral suggestion. When experts had access to the fundus image and patient summary notes to make their decision, their performance improved but our framework remained as good as the five best experts and continued to significantly outperform the other three."

Technology

From the research paper: "Our framework uses an ensemble of five segmentation and five classification model instances... The referral decision recommended by our framework is determined by the most urgent diagnosis detected on each scan. Patients may also have multiple concomitant retinal pathologies. These additional pathologies do not change the referral decision, but may have implications for further investigations and treatment. Our framework was therefore also trained to predict the probability of a patient having one or more of several pathologies. A key benefit of our two-stage framework is the device independence of the second stage. Using our framework on a new device generation thus only requires retraining of the segmentation stage to learn how each tissue type appears in the new scan, whereas the knowledge about patient-to-patient variability in pathological manifestation of different diseases, which it had learned from the approximately 15,000 training cases, can be reused.

The first stage of our framework consists of a segmentation network that takes as input part of the OCT scan, and outputs a part of a segmentation map. That is, it predicts for each voxel one tissue type out of the 15 classes. We applied four variations over it... The first variation allows us to control the receptive field for z separately and is furthermore less computationally intensive. The second and third variation aimed at improving the gradients flow throughout the network, which makes the training process easier. The last variation extends the receptive field such that each pixel in the output effectively has the whole input contained within its receptive field.

The classification network learned to map a segmentation map to the four referral decisions and the ten additional diagnoses... We found using dense convolution blocks to be critical for training classification networks on large three-dimensional volumes. For both of these networks we trained five instances."

Function

R And D

Core Research And Development

Background

From the research paper: "Medical imaging is expanding globally at an unprecedented rate, leading to an ever-expanding quantity of data that requires human expertise and judgement to interpret and triage. In many clinical specialities there is a relative shortage of this expertise to provide timely diagnosis and referral. For example, in ophthalmology, the widespread availability of optical coherence tomography (OCT) has not been matched by the availability of expert humans to interpret scans and refer patients to the appropriate clinical care. This problem is exacerbated by the marked increase in prevalence of sight-threatening diseases for which OCT is the gold standard of initial assessment. Automated diagnosis of a medical image, even for a single disease, faces two main challenges: technical variations in the imaging process (different devices, noise, ageing of the components and so on), and patient-to-patient variability in pathological manifestations of disease."

Benefits

Data

Three-dimensional optical coherence tomography scans; from the research paper: "Data were selected from a retrospective cohort of all patients who attended Moorfields Eye Hospital NHS Foundation Trust, a world renowned tertiary referral center with 32 clinic sites serving an urban, mixed socioeconomic and ethnicity population centered around London, United Kingdom, between 1 June 2012 and 31 January 2017, who received OCT imaging...

Clinical labels for the 14,884 scans were assigned through an automated notes search with trained ophthalmologist and optometrist review of the OCT scans. The presence or absence of choroidal neovascularization, referable macular edema, normal and other pathologies visible on the OCT scan were recorded. In addition, patients with choroidal neovascularization or macular edema confirmed through treatment were labeled directly from the Moorfields OpenEyes electronic health record. Gold standard labels were acquired for 997 patients that were not included in the training dataset. We then tested our framework on this dataset. For each patient, we obtained the referral suggestion of our framework plus an independent referral suggestion from eight clinical experts, four of whom were retina specialists and four optometrists trained in medical retina."

Furthermore, the research has resulted in a database prepared for machine learning applications. From the research blog: "The original dataset held by Moorfields was suitable for clinical use, but not for machine learning research. So we’ve invested significantly in cleaning up, curating and labelling the dataset to create one of the best AI-ready databases for eye research in the world. This improved database is owned by Moorfields as a non-commercial public asset".