AI Case Study

New York Times increases article commenting capacity by 3x using machine learning to automatically flag inappropriate content for moderator review

The New York Times (NYT) is using the machine learning API Perspective to flag online comments as potentially toxic, making human moderation review more efficient. This has allowed the NYT to offer commenting on 3x more of its articles.

Industry

Consumer Goods And Services

Media And Publishing

Project Overview

The New York Times devotes considerable resources to moderating its online comments but "moderation takes time, and in 2016 The New York Times was only able to enable comments on about 10 percent of articles. Eager to find a solution that would let them support comments across a greater swath of articles, they looked to machine learning to help. They turned to Perspective, a free tool developed by Jigsaw and Google that uses machine learning to make it easier to host good conversations online." (Google)

According to Perspective AI: "Perspective is an API that makes it easier to host better conversations. The API uses machine learning models to score the perceived impact a comment might have on a conversation. Developers and publishers can use this score to give realtime feedback to commenters or help moderators do their job, or allow readers to more easily find relevant information."

Reported Results

From Google: "Using Perspective, The New York Times was able to triple the number of articles on which they offer comments, and now have comments enabled for all top stories on their homepage." However, as the developers themselves say on Medium: "The key to leveraging early-stage machine learning is not to use it as a standalone solution, but as an assistant that can help people work more efficiently in their efforts to expand and improve community discussions".

Technology

Function

Risk

Audit

Background

From The New York Times: "The Times’s comments section, widely considered one of the best on the Internet for its insightful, civil and robust discussion, is managed by a team of 14 moderators who manually review approximately 11,000 comments each day."

Benefits

Data

NYT's archive of moderated comments for training