Carnegie Mellon AI Ups the Ante in 20-Day Poker Fest

• By Dian Schaffhauser
• 01/12/17

So far, in the poker tournament being heralded as "Brains Versus Artificial Intelligence: Upping the Ante," AI has collected more chips. In 2015, the outcome was reversed; three of four human players came out on top, besting Claudico, that year's model of the Carnegie Mellon University AI poker-trained program. This year's version, named Libratus (Latin for "balanced" and "powerful"), however, appears to have a bit more power under the hood.

The outcome isn't just for the sake of science. This kind of AI could come in handy in situations where important decisions need to be made with incomplete and misleading information.

The event in which Libratus is being tested is a 20-day set of poker games — potentially, 120,000 hands in total — against four of the world's best poker players. The goal is to figure out how well a machine can be trained "to make extremely complicated decisions based on incomplete information while contending with bluffs, slow play and other ploys," said Tuomas Sandholm, professor of computer science, in an article about the endeavor. Sandholm and Ph.D. student Noam Brown are developing the algorithm that computes the playing strategy.

The game being played is Heads-Up No-Limit Texas Hold 'em, considered one of the world's toughest. The challenges for the computer program are many. First, there are, according to the experts, 10160 information sets or possible paths of play as perceived by the player whose turn it is. Second, the AI must make decisions without knowing all the cards in play while also trying to sniff out human bluffing.

The pros are vying for their share of a $200,000 prize purse, whereas the prof is vying to set a new benchmark for AI.

"Since the earliest days of AI research, beating top human players has been a powerful measure of progress in the field," noted Sandholm. That was the case with chess in 1997, "Jeopardy!" in 2009 and Go last year, he said.

The 2015 competition used a program built from two to three million hours of supercomputing computation. In comparison, Libratus has used around 15 million core hours and will continue racking them up throughout the tournament.

Although Claudico collected fewer chips than three of the pros that year, the research team decided that the 80,000 poker hands it played weren't sufficient to absolutely determine with statistical significance the "superiority" of human or computer. Therefore, the latest match increased the number of hands by half as much.

"I'm very excited to see what this latest AI is like," said Jason Les, a professional poker player based in Costa Mesa, CA. "I thought Claudico was tough to play; knowing the resources and the ideas that Dr. Sandholm and his team have had available in the 20 months since the first contest, I assume this AI will be even more challenging."

On the other hand, the humans haven't been sitting still either. The pros, explained Les, have "embraced publicly available game theory tools that have elevated game play."

The program's algorithm, which was written from scratch, uses new technology to sort out what game theorists call a "Nash equilibrium." That's a scenario in which neither player in a pair will benefit from changing strategy so long as the other player's strategy remains the same.

Libratus exploits a faster equilibrium-finding method that identifies certain paths for playing a hand as not promising and then, over time, begins to ignore them.

"We found that this is not just faster, but that the answer is better," Sandholm said.

Also, during the last tournament, the professional players figured out some obvious bluffs Claudico churned through, which they were able to take advantage of in their own play. This time, the researchers will perform some live computations using a new approach and algorithm to try to influence the endgame.

Refining AI through this kind of open-ended, nuanced game play has practical application in realms such as cybersecurity, military scenarios and medicine.

"Extending AI to real-world decision-making, where details are unknown and adversaries are actively revising their strategies, is fundamentally harder than games with perfect information or question-answering systems," said Nick Nystrom, senior director of research at thePittsburgh Supercomputing Center, where the computation has taken place. "This is where it really gets interesting."

Andrew Moore, dean of the university's School of Computer Science suggested future applications for AI might result in such things as smartphone apps that a user could pull up to handle negotiations for buying a new car. "A lot of people throughout the AI community are watching this event carefully," he said.

The gaming is taking place in Pittsburgh's Rivers Casino, where the public can watch the action. It's also being live-streamed on Twitch, a social video platform for gamers. (Scroll down in the menu of live feeds and look for the "Libratus VS..." games.) Play runs between 11 a.m. and 7 p.m. Eastern time.