October 1, 2014 - by Tom Federico
This post provides data on how our picks for season-long football pick’em pools, as provided by our Football Pick’em Pools product, did during the 2013 season.
For those unfamiliar, here’s a quick background:
If you haven’t done it yet, we invite you to go set up your pool to see how the product works.
The strategy involved winning a football pick’em contest is downright fascinating. A deep dive into pick’em strategy isn’t the point of this article, but in short, optimal strategy for these contests is much more complex than a lot of people think.
Truly maximizing your odds to win requires, at a minimum:
An understanding of game theory can also come into play, especially over the final weeks.
All of this is simply too much for a human brain to process, which is why we built software to figure it out. For just a little taste of some of the strategy elements we evaluate, you can check out a few articles we wrote, keeping in mind that this is just the tip of the iceberg:
How you should approach weekly vs. season prize pools
How your position in the standings should affect your picks
Why picking an expected loser can make total sense
Every pick’em pool is different, especially since rules, scoring systems, and the overall skill level of opponents can vary greatly from pool to pool. So tracking a vanilla metric like the percentage of picks we get right, which ignores the specific context of each user’s pool(s), doesn’t tell us much at all about the level of success our paying customers are actually having.
Put another way, the same score that wins a prize in a $5 buy-in corporate office pool may not even come close to cashing in a $1,500 buy-in contest run by a Vegas sports book.
Furthermore, because our product customizes weekly picks for every single user based on the dynamics of his or her specific pool, we don’t have one universal set of recommended picks to track. So there’s really only one good way to measure the effectiveness of our football pick’em picks: Ask users how they are doing in their pools.
So that’s what we do:
Last year, the debut of our new Football Pick’em Pools product, we asked users for some pool standings information toward the end of the season. They told us details like what place they were currently in, how many points they had, and how many points the pool leader had (unless they were in first place, of course). They had already given us other details like how many entries were in their pool, and what the prize structure was.
As an additional filter, we asked our customers if they used our pick recommendations for the previous week (a) exactly as-is or with minimal changes, or (b) with significant changes. We only count category (a) in our reported performance results. For reference, it accounts for approximately 95% of users.
Aggregating all this user-reported pool standings data gives us a clear picture of how we’re doing overall. Users have no reasonable incentive to report inaccurate standings information, since they have nothing to prove to us, and we make it clear that standings information can directly impact the pick recommendations we give them for the upcoming week.
The table below shows the final data we collected in 2013 from users in pools that offered an end-of-season prize. (Pools that only offered only week-to-week prizes are excluded, as strategy tends to be much different for those.)
First, though, it’s important to clarify one thing. Since we ask users for standings data before they get their picks for a given week, the data below represents how our users stood entering the final week of their 2013 football pools.
This data is still likely to be representative of how users actually ended up in their pools last year, since the final week in a football pick’em typically makes up only 6-7% of the total picks made for the season. The true end of season numbers almost certainly didn’t exactly match the ones posted below, but in all likelihood they were either just a bit better or just a bit worse.
To get the baseline expectations that we list in the table below (the “Expected” columns), we assume a scenario where all entrants in our users’ pools users are equally skilled, and no one uses our picks. For example, in a 100-person pick’em pool, we assume that each entrant’s chance to come in first place is 1% (a 1-in-100 chance). The “Reported” columns show the aggregated numbers submitted by TeamRankings users.
As is hopefully clear from the table, on balance last year was pretty outstanding for our football pick’em pool picks, and phenomenal for NFL game winner pools.
While our game winner pool picks provided the most dominant edge last season, our point spread pool results were actually the more fascinating case. As background, consider this: Our NFL and college football point spread picks, as made by our algorithmic prediction models, both had pretty bad seasons in 2013. As a result, one would instinctively expect our picks for point spread based pick’em pools also to do poorly.
Not so. Our users in NFL spread pools were 3-4 times as likely as expected to be in first place. Users in college football spread pools with confidence points were almost five times as likely as expected to be in first place, and over half of them were in a prize winning position heading into the final week. More on this in the section below.
Finally, the standings percentile information rounds out the picture. The average customer who used our picks for the 2013 season in NFL game winner pick’em pools beat about 94% of their opponents, while average standings percentiles in other pool types ranged between 80-85%. Our worst performing pool type was non-confidence based college football spread pools, but even there, our average user still beat almost 75% of their opponents.
In closing, it is important to remember that these are aggregated numbers, and there were undoubtedly outliers in both the positive and negative directions. For example, we’d be naive to think that there weren’t any users of our Football Pick’em Pools product last year that did use it for the whole season, and didn’t finish particularly well, and weren’t thrilled with the results. We understand that.
Given the dynamics of football pick’ems, though, this is bound to happen every year. Every pool is different, and even picking 250 games in a season isn’t a huge sample size. So there will always be the chance that lots of people in your specific pool get lucky and do really well in a particular year, or that your pool just so happens to include a specific set of games for which our picks do especially poorly in a given year.
That type of bad luck evens out over time, though, and the overall numbers from 2013 provide solid early evidence that using our approach in the long run should deliver great results in football pick’em pools. We’ll see what happens this year, and provide updates in future posts.
Printed from TeamRankings.com - © 2005-2024 Team Rankings, LLC. All Rights Reserved.