Solver Algorithms

Hello and Happy Halloween!

Welcome to the October 30th edition of the Solver School Newsletter.

In the last newsletter, I explored the basics of game trees — visual representations of the sequential decisions made in poker. While game trees can help us conceptualize the game, one that accurately and comprehensively models a hand would be far too complex for us to work with without technology.

Solvers use algorithms to traverse these enormous decision trees and calculate strategies. Today, I’ll dive deeper into how they work to generate an output.

Nash Equilibrium

I introduced Nash equilibrium in the September 30th newsletter issue, but let’s briefly reintroduce the concept here as it relates to poker solvers.

A solver’s goal is to find each player's optimal strategy in a given scenario. This optimal point is the Nash equilibrium — the set of strategies where neither player can unilaterally improve their expected value by altering their actions.

At Nash equilibrium, each player’s strategy played is a best response to the other players' strategies. There are no profitable deviations that any one player can make. It represents the solved game state.

For most practical poker scenarios, finding the precise mathematical equilibrium is effectively impossible. The game trees are too large and complex for computers to solve. As a result, we must work with approximations.

Approximating Equilibrium

Given the vast size of most game trees, solver algorithms generate outputs that approximate equilibrium with an acceptable margin of error. This error term is usually represented as a percentage of the pot and is adjustable in most solvers.

The solver begins with an initial strategy for each player. Based on the output, it will then iterate, continuously adjusting each player’s strategy in response to each other until neither player can exploit the other.

While a true equilibrium can require endless computing time, the solver will stop calculating when the exploitability drops below a configured threshold - for example, when neither player can unilaterally improve their EV by more than 1% of the pot size.

Types of Solver Algorithms

Solvers are grounded in intricate mathematical models and algorithms. Two of the most prominent algorithms are the Counterfactual Regret Minimization (CFR) and the Best Response (BR) algorithms.

Counterfactual Regret Minimization (CFR): CFR operates by playing out countless game scenarios. With each iteration, the algorithm evaluates the "regret" or loss incurred by not choosing one of the other strategic options. This regret metric is then used to adjust and refine the algorithm's strategy. As the number of these simulated game rounds skyrockets, the CFR algorithm progressively inches closer to the elusive Nash equilibrium.

Best Response (BR) Algorithm: While CFR seeks the Nash equilibrium, the Best Response algorithm has a different modus operandi. It aims to find the most potent strategy against a given opponent's strategy. Instead of aiming for mutual optimization, it focuses on identifying and capitalizing on an opponent's vulnerabilities and maximally exploiting them.

A Simplified View of What Solver Algorithms Do

Solver algorithms execute calculations on a scale far beyond human capabilities. We can talk about algorithm types and how the solvers generate outputs, but at the end of the day, all you need to understand to work with them effectively is the following workflow.

For each iteration, the solver:

• Starts with a base strategy for each player

• Calculates EV for every decision point in the tree under current strategies

• Identifies the most profitable deviations for each player

• Updates the strategy based on these profitable deviations

• Recalculates EV with updated strategies

• Repeats the process continuously, with strategies gravitating toward an equilibrium

• Stops iterating when the EV difference between each player’s strategy and the maximum exploit that strategy is within that margin of error I noted above

Again, there’s a lot more complexity in how the algorithms work. They mix individual hand combinations between multiple actions to balance ranges across nodes within the game tree. Advanced techniques like clustering allow partitioning of the game tree to enhance performance further.

But at their core, solver algorithms are powered by computational brute force applied with nuance. They leverage speed to achieve what human minds cannot — rapidly calculating EV through massive game trees in response to evolving strategy adjustments.

The Value Lies in the Understanding

While solvers contain complex math under the hood, there is no inherent intelligence or poker skill programmed within. The outputs are only as good as the inputs and scenarios the user configures. If your inputs don’t reflect the situation you are trying to model, your outputs won’t make much sense.

The actual value of solvers lies in comprehending the fundamentals yourself. With deep understanding, you can model any poker scenario, interpret outputs accurately, and answer almost any strategic question — as long as you account for any simplifications and recognize the limits of abstraction.

These tools aren’t magic. They won’t give you “the perfect way to play” — if that even exists. Using them properly requires a human perspective. Let the solvers handle number crunching at scale while you provide the framework and lens through which the outputs are analyzed. That combination unlocks immense potential.

Looking Ahead

In the next newsletter, I will explore the input part of the equation. Understanding the configuration options and how they can be manipulated to test different hypotheses is crucial for applying solvers effectively.

For more free solver content, check out the rest of the Solver School website. It is filled with video and written content about solvers and analyzing poker with data.

I have several training courses for sale on the Solver School website. If you want to take the next step in using solvers, check out my flagship product, The Solver Masterclass. It contains everything you need to know to become an expert at using solvers to analyze poker data.

You can also follow me on Twitter and YouTube, where I share more solver-focused and poker strategy insights.

I appreciate you following along and reading the newsletter today.

Until next time.
Michael Lukich

P.S. As always, I'm eager to hear your thoughts. If you’d like to reach out, you can reach me at [email protected].