AI Case Study
Google Research detects diabetic eye disease as well as leading ophthalmologists with machine learning
Diabetic retinopathy (DR) is the fast growing cause of blindness globally and impacts 20% of the nearly 500m living with diabetes globally. Early detection is critical but there are limited doctors. Google trained convolutional neural network (CNN) models on 128,000 labelled retinal images graded by ophthalmologists. The model achieved 96–97% sensitivity which is comparable to some of the most elite specialists.
Industry
Technology
Internet Services Consumer
Project Overview
"Working closely with doctors both in India and the US, we created a development dataset of 128,000 images which were each evaluated by 3-7 ophthalmologists from a panel of 54 ophthalmologists.
This dataset was used to train a deep neural network to detect referable diabetic retinopathy. We then tested the algorithm’s performance on two separate clinical validation sets totalling ~12,000 images, with the majority decision of a panel 7 or 8 U.S. board-certified ophthalmologists serving as the reference standard. The ophthalmologists selected for the validation sets were the ones that showed high consistency from the original group of 54 doctors."
Reported Results
"...the algorithm achieved 96–97% sensitivity and 93% specificity! That is comparable to, if not slightly better than, the results of the 8 ophthalmologists grading the images. Since these 8 ophthalmologists were already elites of the initial group of 54 specialists (they were selected based on high rate of self-consistency), the model is doing extremely well."
"In the setting of disease screening, we want to aim for high sensitivity (allowing us to confidently rule out the negatives; i.e. very little false negative)."
Technology
Convolutional neural network
Function
R And D
Core Research And Development
Background
"Diabetic retinopathy (DR) is the fastest growing cause of blindness, affecting more than 20% of the 488 millions of people living with diabetes worldwide. High blood sugar can cause damage to blood vessels of the retina (tissue covering the back of the eye, made up of light-sensitive cells.) Vision is not affected initially but irreversible blindness will occur in time without treatment. Early detection is therefore crucial in order to administer timely treatment and prevent the disease’s progression."
"Unfortunately, medical specialists capable of detecting the disease are not available in many parts of the world where diabetes is prevalent."
Could machines learn to spot the disease from images?
Benefits
Data
"Training Data Set: 128 175 retinal images from EyePACS (electronic medical record) in the US and 3 eye hospitals in India. All images were graded by 3 to 7 different ophthalmologists, from a panel of 54 US-licensed ophthalmologists and senior residents. Gradings included DR, DME and image quality. These serve as labels to the images.
Validation Sets x2: After this training, the algorithm is put to the test with 2 validation sets: 1) A random sample of 9963 images taken from EyePACS, not overlapping with the previous training set. 2) A publicly available data set called Messidor-2, with 1748 images."