AI Case Study

Amazon stops using machine learning program to evaluate job candidates based on historical resume datasets due to gender bias

Amazon shelved a human resources project which used machine learning trained on historic data to analyse and reject job applicant resumes. Because it was trained on data from which human hiring decisions had been made, it perpetuated those engrained biases, and it has been revealed that in 2015 it stopped analysing resumes in this way.

Industry

Technology

Internet Services Consumer

Project Overview

"The group created 500 computer models focused on specific job functions and locations. They taught each to recognize some 50,000 terms that showed up on past candidates’ resumes. The algorithms learned to assign little significance to skills that were common across IT applicants, such as the ability to write various computer codes, the people said.

Instead, the technology favored candidates who described themselves using verbs more commonly found on male engineers’ resumes, such as “executed” and “captured,” one person said."

Reported Results

In addition to the evident gender bias as candidates who used language in line with resumes coming from male employees, the outcomes generated by the system seemed random.

Parts of the system are still used for mundane tasks like duplication detection, however.

Technology

Function

Human Resources

Recruitment

Background

"Amazon’s experiment began at a pivotal moment for the world’s largest online retailer. Machine learning was gaining traction in the technology world, thanks to a surge in low-cost computing power. And Amazon’s Human Resources department was about to embark on a hiring spree: Since June 2015, the company’s global headcount has more than tripled to 575,700 workers, regulatory filings show."

Benefits

Data

Training data was resumes of past candidates.