AI Case Study

"The aim of traditional game-playing AIs is to calculate the possible results of a game as far as possible and then rank the strategy options using a formula that searches data from other winning games. The downside to this method is that in order to compress the available data, algorithms sometimes group together strategies that don’t actually work. Poker AI, DeepStack, avoids abstracting data by only calculating ahead a few steps rather than an entire game. The program continuously recalculates its algorithms as new information is acquired. When the AI needs to act before the opponent makes a bet or holds and does not receive new information, deep learning steps in. Neural networks, the systems that enact the knowledge acquired by deep learning, can help limit the potential situations factored by the algorithms because they have been trained on the behavior in the game. This makes the AI’s reaction both faster and more accurate ... In order to train DeepStack’s neural networks, researchers required the program to solve more than 10 million randomly generated poker game situations.

To test DeepStack, the researchers pitted it last year against a pool of 33 professional poker players selected by the International Federation of Poker. Over the course of 4 weeks, the players challenged the program to 44,852 games of heads-up no-limit Texas Hold ‘em, a two-player version of the game in which participants can bet as much money as they have. After using a formula to eliminate instances where luck, not strategy, caused a win, researchers found that DeepStack’s final win rate was 486 milli-big-blinds per game . A milli- big-blind is one-thousandth of the bet required to win a game. That’s nearly 10 times that of what professional poker players consider a sizable margin, the team reports this week in Science."