Of Statistical Significance

In lieu of poker strategizing, let's talk statistics.

(I hate statistics. Numbers ain't my thing. And I'm definitely guilty of what follows myself, so don't think I'm chunking rocks from my glass house.)

I'm not sure that everyone really grasps what "the long run" means exactly, when used to talk about poker results. One of the early mantras that we all absorb is that correct play is rewarded in the long run, and that over time the lemurs will pay for cold-calling with A7o from MP, while we will be rewarded for having the good sense to instantly muck that same hand as if it were riddled with smallpox.

And I'm not knocking the mantra. It's a good mantra. And it's true. If you put yourself in +EV situations more often than you put yourself in -EV situations, your final result will be +EV, given a large enough sample size.

It's that last bit that's the kicker, and the source of potential trouble.

As is the case with many things, there's no magic number, written in stone, that equates to "large enough sample size" when discussing poker results. Depending on your mathematical bent and how conservative or aggressive you might be, people toss out all sorts of numbers, as far as what sample size will give you an accurate measure of your true BB/100 or BB/hr.

What do you think a decent sample size is to get an accurate analysis of whether you're a winning or losing player, and what your estimated profit/loss should be? 1,000 hands? 10,000 hands? 50,000 hands?

The truth of the matter is that nearly every one of us underestimates the sample size needed to get a statistically accurate indication of how we're playing. Compiling threads from assorted forums and posts, the general consensus is that you need anywhere from 100,000 to 500,000 hands at any given level to get a statistically accurate read on your actual performance.

For the sake of this argument, we'll be slightly generous and say you need to play 200,000 hands before you can get an accurate dipstick reading on your mad poker skillz.

Yeah, I know. That's an absolute shit-ton of hands. And therein lies the rub.

The vast majority of the poker addicted don't play full time. We won't even come close to playing the necessary 548 hands, each and every day, just to play 200,000 hands in a year. Even getting in "only" 100,000 hands per year is next to impossible for most people with jobs, lives, kids, etc. Generally speaking, none of us will play enough hands in any given year to give us an accurate idea of how well or how poorly we're actually playing. If we move up (or down) in levels, or focus on other games, the water gets even muddier.

Which isn't the most reassuring of conclusions, when followed to its logical end. From the point of view of the validity of large numbers, we'll never really know where we're at, if our positive or negative results are due more to variance and sheer randomness than our actual level of skill. Or, at best, it'll take most people four or five years of playing before they can truly gauge where they're at on the poker skill-o-meter.

But that's not really the point of this, as it's actually a pretty useless working conclusion. The only time where having an accurate gauge of your potential win rate is truly useful is if you're considering quitting a job that pays X dollars with the hopes of playing poker full time and making Y dollars, and Y must be greater than/equal to X. Otherwise knowing your actual expectation from playing poker is pretty useless.

Wait, what? How can knowing that you're a losing player be useless, since you'd know that it's not variance or fickle luck and that you actually need to work hard to improve your game, so that you can instead become a winning player?

Because, simply, you should never stop working on your game, winning or losing. There's no magic point where you say okay, that's it, I'm good enough, I'm playing at a level where I make the money I hope to make, that's it, no more thinking about poker or getting any better. Even if you wanted to do that, it's largely impossible. The experience itself of playing more hands and encountering more situations makes you a better player, even if you fight it tooth and nail.

Great players don't show a 2BB/100 win rate over 500,000 hands by being lucky; they post results such as that because they work hard at poker and play better than the majority of their opponents. You don't achieve that level of success by playing a lot of hands and then looking at your stats, feeling warm and fuzzy by the fact that the numbers back up the fact that you're a winning player, and not just a luckbox. You achieve that level of success by playing the best you can and always working to get better and to wring more expectation out of the system, bet by single bet.

That said, I do think there are practical applications of the above, beside always striving to get better. I may be in the pessimistic minority here, but I actually think the tried and true mantra of being performance oriented, not results oriented is actually somewhat dangerous advice. I completely agree with it if you interpret it as "always work to get better despite the results you encounter", but only if you add in "...and understand that until you play hundreds of thousands of hands that variance can and will skew your results all over the place."

Or, more simply, I hope people realize and compensate for the fact that in poker the "long run" can take years and years and years of playing to achieve, especially if you do it part-time as a hobby.

The danger (and I've fallen prey to it many times myself and still do) is to approach poker with a certain sense of entitlement, an expectation that your hard work and study will eventually pay off. If you're better than the other lemurs at the table and work hard, study hard, and think hard about poker, it naturally follows that you will, in the long run, be rewarded for making the correct plays. Because you will. That's exactly right. You just have to also understand how long the long run will take to achieve, and realize that variance can play a starring role in short term results.

I've tried to shift my thinking of late to a more hybrid mantra, to take all of the above into account. I still most emphatically subscribe to the larger mantra that consistently making the +EV play will, over time, produce positive results. It's metaphorically up there on the wall, in a really nice frame.

When I sit down to actually play, though, I don't view the situation in the same way. I definitely still strive to make skillful, +EV decisions, but I try to remove any sort of expectation or sense of entitlement. If a year of those sessions won't produce a stastistically sound sample size, then I feel like I have to approach it slightly differently, while retaining the need to strive to always make the best play.

What works for me is to treat it in my monkey brain exactly as if I were playing double deck blackjack, adequately bankrolled, and allowed by the house to count cards (and adjust my bet size accordingly) with impunity. In those conditions, you will encounter situations which are highly, highly +EV, where you will bet the table limit. Even under highly, highly +EV conditions, though, you'll still sometimes lose any individual hand due to the fickle nature of games that involve cards and chance. You'll also lose five hands in a row a some point, despite the absolute best conditions possible. So you'll often shove much money out there, with every expectation and right to win, the odds overwhelmingly on your side, and get soundly thumped, over and over and over.

Over time, however, you'll make tons of money, as all of the +EV situations add up, especially since you're allowed to adjust your bet when the count changes and you're faced with a -EV situation. But the chance for your skill to shine is limited, and often vetoed by that bastard variance. Because the nature of blackjack itself largely takes skill out of the equation (especially in the above scenario), I could play in those conditions until the end of time and never have a crisis of entitlement, as each action is straightforward and determined solely by the underpinning maths. Individual negative results occur at every point but in the end the results are overwhelmingly positive, and I would sit down at a blackjack table under those conditions with a large smile on my face each and every time. Losing weeks and losing months would occur but I'd never stop smiling, as I would know at every point in time, without a doubt, that I would absolutely crush the game over time.

In the end, it's really just silly psychological trickery. For me, treating the short run as if it's something other than poker helps, so that I don't get frustrated and shake my fists at the poker gods, screaming "Why won't you reward me for all of the blood, sweat, and tears I've put into poker? I'm better than these emus but somehow they keep sucking out on me, damnit!" That's not to say that I pretend to ignore the larger mantra of superior performance ultimately trumping all, as I definitely don't. In the long run (when I get there), I fully expect for the work I've put into poker to result in positive results. I just try to segregate the two phenomena in my head to make it easier to deal with on a day to day basis at the poker tables.

Poker is, obviously, much more complicated than counting cards at blackjack. Just to be clear, the last thing I'm advocating is to compare the two. All I'm really suggesting is that you be aware of what variance means in the short run, how long the long run really is, and explore different ways of dealing with the simple fact that your skill and hard work won't always be reflected in your short term results.