Under The TeamRankings Hood, Part 2: Defining Each Rating

March 10, 2011 - by David Hess

[This post is the second in a four part series on our ratings and models. Part 1 is here. Today we present more detailed descriptions of the differences between the various power ratings that we publish. Part 3 examinesstrengths and weaknesses of the system. Part 4 covers our statistical models that combine the ratings with outside information in order to make game winner, spread, totals, and money line predictions.]

In Case You Missed Class Yesterday

Our power ratings system is based on the idea of splitting one win credit between the two teams involved in a game. The amount of credit the winner gets is determined by the final score, and by the formula used to convert that score into win credits, which varies depending on which of our ratings is being calculated.

Given the game results from a season, and the win credit conversion formula, an iterative computer algorithm assigns ratings to all teams so that the number of predicted win credits for each team (based on the team’s rating and the rating of its opponents) matches the actual number of credits.

Changing the win credit conversion formula will change the ratings calculation, and allows us to create different ‘flavors’ of our ratings that each emphasize or ignore certain aspects of play. Below, we’ll go over how each of our individual ratings are calculated.

Predictive Power Ratings

The Predictive power ratings are designed to be our best predictors of future performance, and emphasize margins of victory far more than win/loss results. These are the ratings we use when we refer to the Team Rankings Top 25, and they are the ratings that power our conference tournament and NCAA tournament survival odds simulations.

The win credits in the Predictive ratings are based largely on what a game’s final score margin tells us about likely results of a rematch, assuming we know nothing else about the teams in question.

A team that ekes out a narrow win would be only slightly favored in a rematch. So despite winning the game, that team would receive barely more than 0.5 win credits., while the loser would get slightly less than 0.5. A team that wins comfortably would be heavily favored in a rematch, so they would receive nearly a full credit.

Despite emphasizing margin of victory, this rating strongly discounts the effects of extreme blowout games. The difference between a 15 point win and a 5 point win tells us a lot more about a team than the difference between a 50 and a 40 point win. In the latter case, the winning team’s credit may have only moved from, say, 0.998 to 0.999, because those extra ten points didn’t really influence our opinion of the team.

Because of the decreasing marginal returns, the ratings do not simply average margins of victory. The points on either side of a 0 margin are the most valuable. For example: Team A wins one game by 30 points, and loses another by 6. Team B wins two games by 12 points each. While both teams have a combined margin of victory of 24 points, Team B is rated higher than team A because both of their margins are positive. The difference between a 30 point win and a 12 point win isn’t worth as many win credits as the difference between a 12 point win and a 6 point loss.

By discounting actual win/loss results in favor of margin of victory, yet also discounting the marginal value of blowouts, we create a forward-looking rankings system. If the goal is to predict future scores, the Predictive ratings are our best tool.

To calculate a basic margin of victory prediction for an upcoming game, simply subtract the Predictive power rating of the higher ranked team from the rating of the lower ranked team, and add or subtract the home advantage if it applies.

Wins Power Ratings

Our Wins power ratings are the at the other end of the spectrum compared to our Predictive power ratings. Win credits in our Wins power ratings completely ignore margins of victory. You simply get one full credit for a win and no credit for a loss.

This structure creates a backward-looking ratings system that is more analogous to how most humans tend to evaluate team performance, primarily by looking at records and schedule strength and not weighing margins of victory very highly.

If the goal is to simply reward a team for accomplishing its goal of winning games the Wins ratings are our best tool. We caution, however, that this is not a predictive ranking. Without accounting for margin of victory, a team’s past performance should not be used to model its future performance.

Overall Power Ratings

The Overall power ratings are a blend of our Predictive ratings and our Wins ratings. This means they take into account both margin of victory and win/loss results, in order to create a rating that rewards teams both for winning games and for doing so comfortably.

The win credits in the Overall rating are midway between those in the Predictive and Wins ratings. Teams do get a sizable bonus for winning a game (even if it’s a very close win), but winning by a large margin is still worth more than winning a squeaker. By considering both margins of victory and win/loss results, this rating balances rewarding accomplishments with predicting the future.

Situational Power Ratings

The Last 10 Games power ratings, Away power ratings, and all of our other situational power ratings are based on the Overall power ratings, meaning they incorporate both win/loss results and margins of victory.

We used the Overall ratings, as opposed to the Predictive ratings, as the foundation for our situational ratings in order to apply more balanced criteria. Since the sample size for each rating is artificially reduced, we wanted to avoid letting a single blowout skew the results.

When calculating our situational ratings, we first calculate season-to-date ratings for all teams, which are used primarily to determine schedule strength. We then look at a subset of each team’s results and calculate the answer to a simple question: If this subset of games were all we knew about a team’s performance, what would be our best guess for their overall rating?

In the case of the Last 10 Games rating, the subset of games we’re looking at consists of (surprise) each team’s last 10 games played; think of this rating as a schedule-strength adjusted momentum indicator. It is particularly useful when a recent major event has affected a team’s play, such as an injury or a starting lineup change.

Our Away power ratings take a similar approach but only consider road game performance, measuring a team’s ability to perform away from the friendly confines of their home court. All of our situational ratings use this same approach to measure exactly what’s indicated in their names: the Conference power ratings look at only conference games, the Neutral power ratings measure games on a neutral court, and so on.

Strength of Schedule Power Ratings

The Strength of Schedule power ratings measure the quality of opponents that a team has faced over the course of the year. Essentially, this is an average of the Overall power ratings of all of a team’s prior opponents.

This rating does not take results into account, and is provided as mainly a descriptive index, not an analytical tool. Strength of schedule tells you who a team has played. To see how they have fared against those opponents, use our other rankings.

Home Advantage Power Ratings

The Home Advantage power ratings measure the difference between a team’s quality of play at home versus away from home. Essentially, this is a team’s Home rating minus a weighted average of their Away and Neutral ratings.

Because the number of home, away, and neutral games for each team is relatively small, the margin of error on the home court advantage values is large. We publish them mostly as a descriptive tool, and don’t actually use them in calculating the other ratings.

Choosing A Rating

By making small changes within the same basic framework, we can create ratings tailored to measure different aspects of team performance. The appropriate rating to use for any given task depends on your goals.

Do you want to make accurate predictions of future games? Use our Predictive power ratings.

Do you want to reward a team for winning games, without worrying to much about which team is truly better (as may be the case in the NCAA tournament selection process)? Try the Wins power ratings.

Do you want to balance rewarding accomplishments with predicting the future (as perhaps the selection committee should do when seeding the bracket? Check out the Overall power ratings.

Curious how a team has played in specific circumstances? Select one of our situational power ratings.

Whatever your purpose, one of our ratings ought to fill your needs.

Coming Tomorrow: A discussion of the pros and cons of our power ratings approach. [Edit: Tomorrow has arrived. View Part 3: Pros And Cons.]