How Our NFL & College Football Pick’em Pool Picks Did In 2013

This post provides data on how our picks for season-long football pick’em pools, as provided by our Football Pick’em Pools product, did during the 2013 season.

Product Overview

For those unfamiliar, here’s a quick background:

  • We’ve built proprietary analytics to figure out your best picking strategy for winning various types of NFL and college football “pick’em” style office pools (season prize pools, weekly prize pools, or both)
  • Our data-driven approach is based on thousands of computer simulations of pools similar to yours, factoring in details like the size of your pool, its prize structure, and your current position in the standings
  • We use publicly available national picking trends data to forecast how your opponents are likely to pick each game, and then we identify the picks that give you the best chance to beat them

If you haven’t done it yet, we invite you to go set up your pool to see how the product works.

Strategy Overview

The strategy involved winning a football pick’em contest is downright fascinating. A deep dive into pick’em strategy isn’t the point of this article, but in short, optimal strategy for these contests is much more complex than a lot of people think.

Truly maximizing your odds to win requires, at a minimum:

  • Accurate game predictions
  • Knowledge of which teams are being overrated and underrated by your opponents
  • A precise assessment of the risk/reward profile of every possible pick, given your current position in the standings and the number of picks left in the season

An understanding of game theory can also come into play, especially over the final weeks.

All of this is simply too much for a human brain to process, which is why we built software to figure it out. For just a little taste of some of the strategy elements we evaluate, you can check out a few articles we wrote, keeping in mind that this is just the tip of the iceberg:

How you should approach weekly vs. season prize pools
How your position in the standings should affect your picks
Why picking an expected loser can make total sense

How We Measure Success

Every pick’em pool is different, especially since rules, scoring systems, and the overall skill level of opponents can vary greatly from pool to pool. So tracking a vanilla metric like the percentage of picks we get right, which ignores the specific context of each user’s pool(s), doesn’t tell us much at all about the level of success our paying customers are actually having.

Put another way, the same score that wins a prize in a $5 buy-in corporate office pool may not even come close to cashing in a $1,500 buy-in contest run by a Vegas sports book.

Furthermore, because our product customizes weekly picks for every single user based on the dynamics of his or her specific pool, we don’t have one universal set of recommended picks to track. So there’s really only one good way to measure the effectiveness of our football pick’em picks: Ask users how they are doing in their pools.

So that’s what we do:

Standings Questions
Last year, the debut of our new Football Pick’em Pools product, we asked users for some pool standings information toward the end of the season. They told us details like what place they were currently in, how many points they had, and how many points the pool leader had (unless they were in first place, of course). They had already given us other details like how many entries were in their pool, and what the prize structure was.

As an additional filter, we asked our customers if they used our pick recommendations for the previous week (a) exactly as-is or with minimal changes, or (b) with significant changes. We only count category (a) in our reported performance results. For reference, it accounts for approximately 95% of users.

Aggregating all this user-reported pool standings data gives us a clear picture of how we’re doing overall. Users have no reasonable incentive to report inaccurate standings information, since they have nothing to prove to us, and we make it clear that standings information can directly impact the pick recommendations we give them for the upcoming week.

2013 Season Results

The table below shows the final data we collected in 2013 from users in pools that offered an end-of-season prize. (Pools that only offered only week-to-week prizes are excluded, as strategy tends to be much different for those.)

First, though, it’s important to clarify one thing. Since we ask users for standings data before they get their picks for a given week, the data below represents how our users stood entering the final week of their 2013 football pools.

This data is still likely to be representative of how users actually ended up in their pools last year, since the final week in a football pick’em typically makes up only 6-7% of the total picks made for the season. The true end of season numbers almost certainly didn’t exactly match the ones posted below, but in all likelihood they were either just a bit better or just a bit worse.

To get the baseline expectations that we list in the table below (the “Expected” columns), we assume a scenario where all entrants in our users’ pools users are equally skilled, and no one uses our picks. For example, in a 100-person pick’em pool, we assume that each entrant’s chance to come in first place is 1% (a 1-in-100 chance). The “Reported” columns show the aggregated numbers submitted by TeamRankings users.

2013 Pick'em Results

Interpreting The Data

As is hopefully clear from the table, on balance last year was pretty outstanding for our football pick’em pool picks, and phenomenal for NFL game winner pools.

  • Across all pool types, users were a minimum of 80% more likely than average to win their pools, and a whopping 12 times as likely to win NFL game winner pools. (NFL game winner pools also happen to be the most popular types of pools, which is nice.)
  • In fact, entering the final week of their pools, over 46% of our users in NFL game winner pools (both confidence and non-confidence) were in first place, and over 75% of them were in a prize-winning position. Whoa!
  • College football game winner pools performed next-best, with our users landing in first place 7-9 times as often as expected, and in a prize position 6-10 times as often as expected. Over 75% of our users in college football game winner based confidence pools also were sitting in a prize position entering the final week of picks.

While our game winner pool picks provided the most dominant edge last season, our point spread pool results were actually the more fascinating case. As background, consider this: Our NFL and college football point spread picks, as made by our algorithmic prediction models, both had pretty bad seasons in 2013. As a result, one would instinctively expect our picks for point spread based pick’em pools also to do poorly.

Not so. Our users in NFL spread pools were 3-4 times as likely as expected to be in first place. Users in college football spread pools with confidence points were almost five times as likely as expected to be in first place, and over half of them were in a prize winning position heading into the final week. More on this in the section below.

Finally, the standings percentile information rounds out the picture. The average customer who used our picks for the 2013 season in NFL game winner pick’em pools beat about 94% of their opponents, while average standings percentiles in other pool types ranged between 80-85%. Our worst performing pool type was non-confidence based college football spread pools, but even there, our average user still beat almost 75% of their opponents.

Learnings & Conclusions

  • 2013 was a very good year for our football pick’em picks. Overall it was an outstanding inaugural year for our Football Pick’em Pools product. It remains to be seen what performance levels this product can help users achieve long term, for example in NFL game winner based pools where we just had a lights-out 2013. It’s probably safe to assume that’s not going to be the case every year. Still, given that we delivered users a minimum of a 2x edge to finish in the money across almost all popular types of football pick’em pools, there’s a quite strong case already for the value of this product.
  • Skill absolutely matters, even above and beyond predictions. Great strategy makes a big impact in football pick’em pools, and can even be enough to overcome game predictions that don’t do particularly well. Case in point: Our performance in spread based pools last year. Including all of our low-confidence picks — which we advise bettors to avoid, but most pick’em players don’t have the luxury of passing on games — both our NFL and college football point spread picks against closing lines ended up being under 50% accurate in 2013. Yet we still delivered a big edge to people playing in spread based pick’em pools. How is this possible? Because in pick’em pools, data driven strategies like picking for value, fading the public, and taking advantage of “stale” betting lines in spread based pools often play just as big a role in our pick recommendations as our game predictions do.
  • Season prize pools are a great format for sharp players. Multi-week pick’em pools provide fantastic opportunities for skilled players. Many football pick’ems involve picking hundreds of games over 14-17 weeks, with lots of strategy decisions to be made along the way. The more games you need to pick, and the more strategy decisions you need to optimize, the less likely it is for an unskilled player to just get lucky and beat you. However, it is worth noting that point spreads do serve as an “equalizer” of sorts in pick’em pools, and are likely to increase the proportional role of luck in final results. Our guess at this point is that we’ll generate the biggest edge in game winner based confidence pools, while our long-term edge won’t be quite as big in non-confidence point spread pools.
  • But you need to stick to the strategy for maximum effect. The corollary to the point above is that in a war of attrition, you need to stick to your guns. After our NFL spread picks got off to a terrible start in the opening weeks of 2013, for instance, a number of users in spread based pools gave up on the product. But users who stuck with the product despite falling far behind early on ended up having at least a 40% chance of winning a prize, and on average beat about 80% of their opponents. While it can be tough to stand firm even when a cold steak hits early, the long term payoff is pretty clear.

In closing, it is important to remember that these are aggregated numbers, and there were undoubtedly outliers in both the positive and negative directions. For example, we’d be naive to think that there weren’t any users of our Football Pick’em Pools product last year that did use it for the whole season, and didn’t finish particularly well, and weren’t thrilled with the results. We understand that.

Given the dynamics of football pick’ems, though, this is bound to happen every year. Every pool is different, and even picking 250 games in a season isn’t a huge sample size. So there will always be the chance that lots of people in your specific pool get lucky and do really well in a particular year, or that your pool just so happens to include a specific set of games for which our picks do especially poorly in a given year.

That type of bad luck evens out over time, though, and the overall numbers from 2013 provide solid early evidence that using our approach in the long run should deliver great results in football pick’em pools. We’ll see what happens this year, and provide updates in future posts.