AI Case Study

"Los Angeles-based Second Spectrum Inc. uses artificial intelligence and computer vision techniques to extract a trove of data from video footage of NBA games. With their software, teams can better understand tendencies and probabilities of complex game strategies by running data-driven analysis that takes seconds—work that previously could take months of human analysis.
Before the Warriors started using Second Spectrum about two years ago, the team had loads of video but no easy way to make it into actionable data, says Kirk Lacob, assistant general manager for the Warriors and son of the team’s owner, Joe Lacob.

“If we’re looking at how certain plays ended, we can use the [Second Spectrum] algorithm to watch those 15 plays,” says Lacob, who praised the company’s product and team. “That’s something that’s really useful to coaches, as opposed to a spreadsheet with numbers.”

In addition to the Warriors and Cavs, the technology is used by 14 other NBA teams, according to the company. Broadcast media outlets, which increasingly fuse analytics into their sports journalism, also are customers.

Second Spectrum’s software “learns” precise movements of players and identifies variations on plays and basketball moves. For example, in a pick-and-roll play, the software can identify the ballhandler’s “take” or “reject” of a screen, the screener’s “roll” or “pop,” and the defenders’ responses such as “show,” “soft,” “switch” or “blitz.” The software can then show shooting percentages of each type against each defender—to determine which provides better chances of success.

For example, a team defending a LeBron James pick-and-roll may want to know if a defense works better when James’s defender goes over the top of a pick, or if the other bigger defender should switch onto James—and when to make that choice differently under different conditions.

Coaches can also grab information they want using their own language to describe specific variations of plays, says Ben Alamar, director of sports analytics at ESPN, who formerly worked for the Cavs. “To find all the data to support that—you couldn’t do it before,” Alamar says. “It was way too time-consuming to go through every game and tag for that occurrence. Something that was a monthslong project now takes about 30 seconds.”

The technology is based on machine learning and computer vision techniques that “teach” a computer algorithm to identify things that would take people too long unassisted. “We’ve built machine understanding of sports at a level that a professional coach would trust,” says Second Spectrum Chief Executive and co-founder Rajiv Maheswaran.

NBA teams have used the software to make personnel and transaction decisions, such as signing free agents and making trades, in addition to game strategy, Maheswaran says.

“Moneyball”-inspired advanced metrics are common in baseball, but NBA teams have only recently begun using advanced analytics, drawing on artificial intelligence and computer vision more commonly seen in Silicon Valley startups in sectors such as health care, robotics and enterprise software. Baseball sabermetrics, at least in its basic form, is based on existing statistics, not extracting data from player movements. In basketball, data extracted from video is useful because the free-flowing movements of each player on the court can be important and used to go unrecorded in statistics. Football and soccer also can benefit from these types of analyses, Maheswaran says, and the company is starting to work with teams in those sports.

Second Spectrum can also identify what really is a good shot or bad shot, based on variables such as the location on the floor, shot type, or how the player is moving.
Teams have also used Second Spectrum for the fans. Steve Ballmer’s Los Angeles Clippers have used the company’s data on its Jumbotron screens.

The first time Maheswaran felt convinced the company had something special was in the 2014 NBA playoffs, after teams began using it in 2013. A team told him that Second Spectrum helped lead to winning a playoff series that year. “They’d discovered a strategy they used to win a playoff series,” Maheswaran said of the team, which he declined to name."