Season Record Projections: The Methodology

August 9, 2011 - by Austin Link

Last Wednesday we added a unique new feature to the site:  end of season record projections for college football and the NFL.  At this point it’s entirely based off the pre season ratings, but as the season gets started the power ratings will be used to project games.  Here are a few unique attributes of our season projections.

Direct Calculation

The obvious way to come up with end of season projections is to randomly simulate a season thousands of times and then take the average.  Unfortunately, even with 10,000+ simulations there are sizable error bars on the results, as a few chance iterations can make a team’s odds of going undefeated much higher (or lower) than they should be.

We’ve set out to solve this problem by directly calculating the odds that a team ends up at each win total.  Although the complete win total distributions are not currently up on the site, we do displaying the 10th, 30th, 50th, 70th, and 90th percentile outcomes, to give you a rough idea of the distribution.  Plus, we show the odds for each team to end the season undefeated*, which tells us, among other things, that the Boise State Broncos are the most likely team to go undefeated this season.

*in currently-scheduled games, which means that conference championship games and bowl games are NOT included in the undefeated odds.

Playoff Simulation

Unfortunately, teams are not independent so to calculate certain numbers we do have to rely on Monte Carlo simulated odds.  The simulation is used to calculate odds to win the division, conference, and to get into the playoffs.

Once teams are in the playoffs, we are again able to directly calculate how that particular bracket will turn out.  When we average it over 10,000 brackets we get fairly accurate odds not just on making the playoffs, but also on reaching any given round, including wining the Super Bowl.  This year’s mostly likely Super Bowl champion?  The Green Bay Packers.

Acknowledging Our Error

The last point where our ratings go beyond a simple simulation is in acknowledging our own error.  Especially early in the season the ratings we have for certain teams should probably be qualified with error bars.  If the Indianapolis Colts lose an early game to the Browns, does that mean their playoff odds should drop dramatically?  No, it probably just means we overestimated an aging Peyton Manning.

To account for these imperfect predictions, we use our calculated error to assign a new randomly generated rating to each team for each of the simulations.  So, during one simulation, the Colts may be the best team in the league. In another, they may be only mediocre. Over a large number of trials these average out to a team’s power rating.  This error correction is done both for the Monte Carlo simulation and the direct calculation.  So when you see that a team has 5% odds of going undefeated, those don’t assume that our power ratings are exactly correct.

The season projections aren’t perfect yet, and we’ve already had a couple of small errors pop up. In fact, they’ll never be perfect. There’s always a decent amount of error and chance in sports; it’s part of why they’re so exciting.  We do feel comfortable, though, that the season projections give fairly accurate percentages of each outcome happening.  We hope the system will be an interesting and valuable tool to follow throughout this football season, and beyond.