Chalk up another victory for machines.
In artificial intelligence, it's a milestone when a computer program can beat top players at a game like chess. But a game like poker, specifically six-player Texas Hold 'em, has been too tough for a machine to master — until now.
Researchers say they have designed a bot called Pluribus capable of taking on poker professionals in the most popular form of poker and winning.
Four-time World Poker Tour title holder Darren Elias helped test the program's skills, playing Texas Hold 'em against a whole table of the bots.
"It's just me and then five versions of this AI poker bot, which I would play against every day, thousands of hands," Elias said. He'd take on four tables full of them at a time, and he'd alert the computer scientists who designed the bot when it made a mistake. Pluribus — so-named because it takes on many opponents at once — learns by playing against itself over and over and remembering which strategies worked best.
The bot quickly became a stronger player. "It was improving very rapidly, where it went from being a mediocre player to basically a world-class-level poker player in a matter of days and weeks. Which was pretty scary," Elias said.
Ultimately, the machine came out ahead of Elias. So scientists tried a different experiment in which they put one bot up against a full table of pro players. And it won again. Their research was published today in the journal Science.
The easiest way to quantify how well Pluribus did was by how much money it was winning. "If this were for live money, the bot would be winning at a rate of about $1,000 an hour," says Facebook AI research scientist Noam Brown, who designed Pluribus with Tuomas Sandholm, his adviser at Carnegie Mellon University.
A bot playing better than top humans in multi-player Texas Hold 'em is seen as an AI milestone — like when IBM's Deep Blue supercomputer beat chess legend Garry Kasparov.
There are a couple of reasons why multi-player poker has been a challenge for AI programs. Previous groundbreaking poker bots focused on a few different kinds of two-player poker. Because only two players are involved, the bots could figure out a perfect strategy for each game – one that would eventually win every time.
That didn't work in six-player poker games like no-limit Texas Hold 'em, Brown says. Those games have a lot more variables. But even though there's no fail-safe strategy to win, Brown says, Pluribus still outperforms the best humans.
Regardless of how many players are involved, poker is simply a really tough game to program for because there's so much information that isn't out in the open, like it is in chess, for example. The cards are hidden.
"When you go to a game like poker there is hidden information involved, where you have access to information that your opponents don't see, and that greatly complicates things," Brown says.
As Pluribus taught itself to play, some of the tactics it came up with were surprising. "Because it was developed completely from scratch without any access to human data, the strategy that it's developed is very different from how humans play poker," Brown says.
The bot learned to pick its moments and then make huge bets and bluffs — bigger than most humans would make. That's what Elias found when he played against it.
"The bot was not afraid to make these kind of plays often," he says. "Which is something that humans could probably do a little more." Elias says he's starting to incorporate bigger bets into his own game.
The bot was excellent at varying its strategy even when dealt the exact same hand, Elias says, "which is pretty tough to play against because you can't really pick up a pattern."
Pluribus also confirmed something that many poker pros already suspected — "limping" is almost never a good strategy. Limping means making the smallest bet possible to stay in the hand, instead of raising or folding. Pluribus eventually stopped limping as it became a stronger player.
Ultimately, Elias says Pluribus could spell the end of high-stakes online poker. "I don't think many people will play online poker for a lot of money when they know that this type of software might be out there and people could use it to play against them for money." Poker sites are actively working to detect and root out possible bots.
A superhuman poker bot is also humbling and a little sad, Elias says. "There's no going back. The bots will always be better than us."
Brown, Pluribus' developer, says it's exciting that a bot could teach humans new strategies and ultimately improve the game. "I think those strategies are going to start penetrating the poker community and really change the way professional poker is played," he said.
And the bot's success has implications beyond poker. Brown says their AI technology could eventually be useful in other situations where there are multiple people involved and a lot of unknown variables, like getting a self-driving car through traffic.
Copyright 2019 NPR. To see more, visit https://www.npr.org.
