College Bowl Pick’em Performance: How Our Picks Did Last Year

December 6, 2016 - by David Hess

This post reports on how our college football Bowl Pick’em Picks did during the 2015-16 bowl season.

It also provides a detailed explanation of how we measure the performance of our customized picks for college bowl pick’em pools.

2015-16 College Bowl Pick’em Performance Results

Key Takeaways

Across all pool types, our customers were 3.1 times as likely to finish in first place and 2.6 times as likely to finish in the money compared to the average entrant in their bowl pools.
Over the long term that means winning a 20-person pool once every 7 years if you use our picks, compared to once every 20 years if you pick on your own and are about average at doing it. (Quite a difference.)
Our best performance last bowl season was in game winner based pools that did not use confidence points. In those pools, our customers were 6.6 times as likely to win the pool, and 4 times as likely to finish in the money, compared to the average entrant in their pools.
Our worst performance was in spread-based pools that used confidence points, where our customers were 10% less likely to finish first, but still 70% more likely to finish in the money.

So even in our worst performance area, our customers were still more likely than the average pool entrant to win something, and picks in basic game winner pools delivered a huge edge.

(Note: The more you divide up a data set like this, the more likely you are to see some extreme performances — good or bad — just on account of luck. Even if our picks had finished in the money in 80% of pools, for example, there would likely be some type of pool where performance wasn’t spectacular.)

Results Data & Performance Splits

The table below shows data collected from our college football Bowl Pick’em Picks customers after the 2015-16 bowl season ended.

The data refers to individual pools, not individual users, since many of our users enter multiple pools.

Note that we’ve broken out the data by relative level of success (a “finished in first place” table followed by a “finished in the money” table, which indicates winning any type of prize), as well as by pool pick type.

FINISHED IN FIRST PLACE
Pick Type	Scoring	Expected Win Rate	Reported Win Rate	Win Rate Multiplier
Game Winner	1 Point Per Correct Pick	4.2%	27.7%	6.6
Game Winner	Confidence Points	5.9%	16.1%	2.7
Point Spread	1 Point Per Correct Pick	5.2%	11.1%	2.1
Point Spread	Confidence Points	4.7%	4.0%	0.9
Across all pools		5.2%	16.4%	3.1

FINISHED IN THE MONEY
Pick Type	Scoring System	Expected Win Rate	Reported Win Rate	Win Rate Multiplier
Game Winner	1 Point Per Correct Pick	10.1%	40.4%	4.0x
Game Winner	Confidence Points	13.8%	34.4%	2.5x
Point Spread	1 Point Per Correct Pick	11.2%	25.0%	2.2x
Point Spread	Confidence Points	11.6%	20.0%	1.7x
Across all pools		12.2%	32.3%	2.6x

Further Explanation

As you can see, we included both reported results data and expected results data for comparison purposes.

The ratio of those two numbers also provides a rough measurement of the positive or negative impact of our picks (the “Win Rate Multiplier”).

“Expected Win Rate” column. To get a baseline expectation for success in college bowl pick’em pools, we assume all entrants in customer pools users are equally skilled, and no one uses our picks. For example, in a 100-person bowl pick’em pool, we assume that each entrant’s chance to come in first place is 1% (1-in-100).
“Reported Win Rate” column. This column shows the aggregated numbers reported to us by TeamRankings customer survey respondents. These are real, paying customers who used our Bowl Pick’em Picks product to get picks for specific bowl pools they defined in our system.
“Win Rate Multiplier” column. This column shows the Reported Win Rate divided by the Expected Win Rate. If the multiplier is above 1, that implies that our picks increased our customers’ chance to win their pool (or to finish in the money).

Putting Our 2015-16 College Bowl Pick’em Performance In Context

One theme we always stress when it comes to office pool picks is that you need to take a long term view.

When you’re playing against 50 or 100 or 500 people in a pick’em contest, you’re never going to be expected to win, even with the smartest picks on your side.

But winning pools 1.5 or 2 or 3 times as often as expected should bring fantastic returns over the years.

On the surface, it’s reasonable to look at our bowl pick’em results and say, “You guys charge money for bowl pool picks, but in almost 2/3 of bowl pools last year, your customers didn’t even win anything!”

That’s absolutely correct, but you need to consider that result in context.

Why Context Matters

Our customers performed far better than one would expect, winning prizes in 32% of bowl pools they entered, compared to an expectation of 12%.

That edge is what matters, and if it stays at a similar level over time, it should result in quite handsome profits over the long term — as many of our longtime customers have already enjoyed.

It’s also worth noting that our results last college bowl season lined up quite nicely with our general performance expectations for our algorithmic office pool picks.

In internal testing, our office pool simulations typically project that our picks will be 2-10x as likely to win a given pool than the average player, depending on pool size and rules. However, most types of pools fall into the range of a 2-4x Win Rate Multiplier.

Why We Measure Our College Bowl Pick’em Success This Way

Tracking a vanilla metric like “percentage of bowl picks we got right” — a question users ask us fairly frequently — wouldn’t tell us much at all about the level of success our paying customers are actually achieving.

Why? Because it also ignores critical context, such as how many favorites won in a given year, and how many upsets it made sense to pick in different types of pools.

Consider that:

Picking a certain percentage of winners right might be good or it might be bad, depending on how upsets end up occurring in a given bowl season. Some years, picking 67% winners may win you a pool; but in favorite-heavy years it may not even come close. Stats like this may sound good if the number seems high, but still tell a very misleading story about actual customer success winning pools.
Optimal pick strategy can vary greatly from pool to pool. The same final score that wins a prize in a 20-person, $5 buy-in corporate office pool may not even come close to finishing in the money in a 1,000-person, $100 buy-in contest run by a Vegas sports book. As a result, a set of picks optimized to win a prize in that first pool will almost certainly be sub-optimal (too conservative) for the second.

We Don’t HAVE One Set Of “Official” Bowl Pick’em Picks

As a result, having the smartest bowl pool picking system also means not recommending that everyone under the sun use the exact same picks.

Because our product optimizes picks for each specific type of bowl pool, we don’t have a universal set of recommended picks to track. We don’t tell customers what picks to make until they first tell us some critical information about their pool(s), plus any additional intelligence they may have about how their opponents are likely to pick certain games.

Then our pick optimization algorithms go to work.

Finally, even discounting all the reasons above, common sense dictates that if you’re telling people your picks can help them win college bowl pick’em pools, there’s really only one ultimate measure of success:

How did people actually do in real bowl pools using your pick recommendations?

How We Measure Customer Performance In Pools

Somewhat amusingly, despite the rise of an increasing number of competitors over the years, we are the only site publishing bowl pick’em picks or advice that actually measures customer pool performance at a reasonable scale.

We use an email survey to ask all of our customers to tell us how they did in their pools. It’s definitely not a perfect method, but we have to rely on it because our users really have no other need to come back to our college Bowl Pick’em Picks product after bowl season is over.

(They got their picks from the site weeks earlier and submitted them to their pool administrator, and typically they don’t need anything else from us after that.)

A downside of this approach is that email surveys generally have fairly low response rates. So to encourage participation, we run a prize raffle that includes anyone who fills out the survey, no matter if they did well or poorly with our picks.

Hopefully this approach also helps us obtain a more representative sampling of results, rather than only having surveys filled out by users who were either extremely happy or extremely unhappy with the product.

Overall, 148 of our college bowl pick’em customers filled out the survey this year, which is admittedly a small fraction of our customer base. But as long as it’s a representative sample, the numbers quoted in this post should be directionally trustworthy.

College Bowl Pick’em Picks Product Overview

If you’re not familiar with our office pool picks products, here’s a quick summary of our unique approach to helping people win college bowl pools.

The strategy and analysis involved in maximizing your odds to win a specific college football bowl pick’em contest is really just an extremely complex math problem.

With over 40 picks to be made, plus the potential added complexity of confidence points, plus the necessary considerations of pool size and payout structure, optimally solving that problem is far too complicated a job for a human brain.

Up until around 2010, we’d personally been applying the general concepts of value-based picking, game theory, and contrarianism to pool picks for years, with great success.

But a truly precise solution that we could quickly apply across the vast range of pool types and rules and payout structures required the design and implementation of comprehensive, data-driven simulation models. So we built them.

The TeamRankings Process Of Making Bowl Pick’em Picks, In Brief

At a high level:

We use objective computer ratings and Vegas odds to simulate the results of the current bowl season thousands of times. In other words, all of our bowl game predictions are driven by 100% objective data that’s proven to be the most predictive over time.
For each bowl season simulation we run, we also use national picking trends data to forecast how your each of your bowl pool opponents will pick each bowl game, ending up with a diverse set of fictional opponent entries.
In other words, we not only simulate bowl season thousands of times, but we also simulate thousands of bowl pools similar to yours, and each pool simulation features a unique set of forecasted opponent entries that you’ll be playing against. Nobody else does this but us.
We’ve developed algorithms that analyze the results of these bowl season and bowl pool simulations, and they identify the picks that give you the best chance to beat your opponents.

Attacking The Source Of Greatest Edge

People often think that getting an edge in bowl pools is all about analyzing one or two matchups better than anyone else. That’s very rarely the case.

Using good data-driven predictive models (like ours) along with Vegas odds, it’s relatively straightforward these days to get unbiased and solidly accurate win odds predictions for individual games.

Unless you’ve got some inside information that no one else has, it’s doubtful you know more than the betting markets and years of historical data do about the prospects for a specific game.

Rather, it’s the extremely complex process of optimizing dozens of independent picks you need to make, in context of how your opponents are expected to pick those same games, that is where the magic happens — where the greatest edge is available for the taking.

We’ve spent years building the most sophisticated solution to solve that big, hairy problem, and real customer results demonstrate its value. Our customers don’t win every year, but they win much more often than expected in the long term, by using data and game theory to exploit an inefficient market.

If you’d like to learn more and/or get our picks for this bowl season, please check out our Bowl Pick’em Picks page.