TeamRankings.com
Get Smart About Sports
February 11, 2012 - by David Hess
ESPN revealed a new college basketball rating system on Saturday, which they’ve dubbed the BPI.
They’ve hyped it up as a significant improvement over Sagarin, Pomeroy, and the RPI, and they’ve included a handy little table that outlines the differences, plus some info on how well their rating has done at predicting the NCAA tournament the past few years.
Let’s talk briefly about all of the BPI’s main selling points, and what its plusses and minuses are.
Winning by 30 isn’t twice as good as winning by 15. We’re totally on board here, and our ratings are exactly the same. The descriptions of our power ratings go into more detail, but suffice it to say that this is something that BPI nailed. Of course, so did we, and so did Sagarin.
Pomeroy is the most respected ratings system we know of that does not account for blowouts. And, well, that’s essentially why Wisconsin is all the way up at fourth in the Pomeroy ratings, as opposed to 12th in our predictive power ratings, and 11th in the BPI.
This could be good or bad, depending on your perspective.
Research has shown that when home teams win the first game of a home-and-home series by single digits, they actually have a losing record in the rematch. Home court advantage in college basketball is huge, and escaping by 1 is simply a bad result for the home team. If that were all you knew about the two teams, you’d expect the visiting team to win a rematch on a neutral court.
So, for a predictive system, this aspect of the BPI doesn’t seem like the greatest idea. However, if you’re trying to design a system that rewards teams for winning, this can definitely be a good thing.
Our Overall Power Ratings treat wins and losses the same way. Any win is better than any loss, but big wins are better than big losses. If you’re looking for a rating system that sacrifices a bit of predictive power in order to reward teams for actually, you know, winning games, then the BPI and our Overall Power Ratings both fit the bill.
In theory, this could be a good thing, but it seems like it could be very tricky to implement in a way that actually improves the predictions. ESPN’s method involves minutes played: if a player that averages a lot of minutes is missing in a game, the importance of that game is decreased by roughly 15%.
While that seems like a reasonable method, this adjustment doesn’t seem to be making much difference.
Pittsburgh (the example they highlighted) is #63 in the BPI ratings. Guess where they are in our predictive power ratings? That’s right, 63rd. We weight more recent games slightly heavier than older ones, which should help account for early season injuries. And we’re not sure you want to de-emphasize recent injuries, as often the player isn’t 100% when they return.
Beyond that, depth is important. Having a good enough bench that you don’t miss a beat when a player is injured could be a sign of a strong team. And the reverse is true — if you tank when one player is missing, that seems like a bad sign. What if he gets in foul trouble?
Still, it’s tough to argue that this conservative adjustment hurts BPI in any way. And in theory, it could do some good.
Removing pace from the equation is great when you’re splitting up overall score margin into offensive efficiency and defensive efficiency, to try to figure out how a team wins. That’s what makes the Pomeroy ratings so useful — they spit out a power rating for offense and defense separately, and they’re able to do so because of the pace adjustment.
But BPI doesn’t do that. It lumps offense and defense together, and just outputs a pace-adjusted scoring margin. The problem is, when you’re trying to predict future game, pace is important.
Think about two games that both feature identical pairs of teams, where the favorite has about a 5 point advantage over the course of 100 possessions. Now imagine one of the games will be played at a 50-possession pace, and the other one at an 80-possession pace. The favorite in the slower game will be expected to win by 2.5, and the favorite in the faster game will be expected to win by 4. That means the underdog in the slower paced game will, on average, be in a position to win on a fluky last second three. The underdog in the faster paced game won’t.
Playing at a faster pace makes it more likely that the true better team wins, and less likely for random bounces of the ball to lead to an upset. Just look at Pomeroy’s own “Luck” rating — there is a positive correlation between pace and luck. Yes, it’s small, but it was there previously, and it’s there again now (the correlation for this year’s ratings is 0.12).
So while the pace adjustment at first *seems* like a good idea, we’re not sure it’s actually improving the rating.
The BPI ratings are the second big analytics splash ESPN has tried to make in the last six months. Their first try, the QBR quarterback rating, wasn’t a smashing success among stat geeks. The BPI ratings seem poised to be a bit more well received.
QBR included an adjustments for clutch play that seemed to significantly reduce its predictive power. That adjustment seemed designed to appeal to people who wanted to see a rating that explained previous games, rather than predicted future ones. The only similar “cool tweak” in BPI is the injury adjustment, and it seems to be weak enough to make little difference in the end ratings.
ESPN published some info about how well BPI has performed over the past 5 NCAA tournaments — it topped Sagarin by 4 games total, and the RPI by 8 games total. For comparison, over the same period our predictive power ratings topped Sagarin by 3 games, and the RPI by 7 games, falling 1 short of the claimed performance of BPI.
[Our guess is that ESPN intentionally picked that 5 year window to paint BPI in the best possible light. Look at the last 4 or the last 6 years, and it could be a different story. Of course, we’re not saying we wouldn’t have done the same thing …]
As we discussed above, BPI has its strong points and it weak points. In the end, because this is a rating that is at its core based on score margin, it will look very similar to many other predictive ratings. Ultimately, it will be judged by stat geeks on its performance, and by casual fans on whether it matches up to the eye test. It does a good job at the latter. Whether it will live up to its billing performance-wise remains to be seen.
Printed from TeamRankings.com - © 2005-2024 Team Rankings, LLC. All Rights Reserved.