April 16, 2015 - by David Hess
Last week, FiveThirtyEight published a post reviewing the accuracy of their 2015 NCAA tournament projections.
In that post, they compared their round-by-round survival odds against those from other analytically-oriented sites using a technique called Brier scores.
Since FiveThirtyEight didn’t include TeamRankings in the comparison (no hard feelings, as our advancement odds weren’t publicly available prior to the First Four play-in round, since they are part of our premium NCAA Bracket Picks package), we went ahead and added our 2015 results to the mix.
What is being measured here is not the actual bracket picks we advised our customers to make; coming up with optimal picks for a bracket pool requires the analysis of several other types of data, such as pool size and public picking trends, in addition to team survival projections. (We’re going to publish another post soon on how our recommended brackets did this year.)
The table below simply compares how accurate various sites were at predicting how far teams were likely to make it in the 2015 tournament.
As you can see, our projected survival odds would have placed us tied with NumberFire for second place in the overall accuracy rankings, and barely (0.001) behind The Power Rank.
We think it’s fair to say that the three of us constituted the top tier in accuracy this year, at least when measuring by Brier scores.
However, our path to that top tier performance was fairly different:
Overall, we’re happy with this result. While ranking first obviously would have been nice, finishing in the top tier with consistently solid showings in every round is a sign that our score wasn’t overly reliant on a couple fluky results. That should also bode well for future tournaments.
For reference, here were our 2015 NCAA Tournament round by round survival odds, before the First Four play-in round began, sorted by odds to win it all:
Seed | Team | Region | Make R32 | Make S16 | Make E8 | Make F4 | Make Final | Win It All |
---|---|---|---|---|---|---|---|---|
1 | Kentucky | Midwest | 99.2% | 92.7% | 86.5% | 73.9% | 52.8% | 40.7% |
2 | Arizona | West | 99.2% | 78.1% | 62.9% | 40.5% | 18.8% | 12.3% |
1 | Wisconsin | West | 96.9% | 81.0% | 60.2% | 33.1% | 13.7% | 8.3% |
1 | Duke | South | 97.1% | 80.7% | 57.9% | 36.1% | 21.5% | 8.3% |
1 | Villanova | East | 97.7% | 78.5% | 60.2% | 36.4% | 20.5% | 7.6% |
2 | Gonzaga | South | 96.6% | 71.2% | 46.6% | 25.5% | 14.1% | 4.9% |
2 | Virginia | East | 94.1% | 59.9% | 39.5% | 22.7% | 12.2% | 4.2% |
3 | Iowa State | South | 92.8% | 66.6% | 32.1% | 14.9% | 7.0% | 2.0% |
3 | Oklahoma | East | 92.1% | 64.1% | 29.3% | 14.3% | 6.4% | 1.8% |
2 | Kansas | Midwest | 87.5% | 53.9% | 30.6% | 7.4% | 2.8% | 1.3% |
5 | Utah | South | 75.5% | 51.9% | 22.0% | 10.7% | 4.9% | 1.3% |
4 | N Carolina | West | 81.2% | 58.1% | 23.0% | 8.9% | 2.5% | 1.1% |
7 | Michigan St | East | 67.0% | 29.6% | 17.3% | 8.6% | 3.9% | 1.1% |
3 | Notre Dame | Midwest | 88.9% | 52.6% | 28.1% | 6.4% | 2.3% | 1.0% |
3 | Baylor | West | 78.3% | 47.0% | 13.9% | 5.4% | 1.3% | 0.5% |
10 | Ohio State | West | 73.4% | 19.0% | 11.2% | 4.6% | 1.2% | 0.5% |
4 | Louisville | East | 77.7% | 46.9% | 16.0% | 6.1% | 2.1% | 0.4% |
7 | Wichita St | Midwest | 67.5% | 32.7% | 16.6% | 3.4% | 1.1% | 0.4% |
11 | Texas | Midwest | 53.8% | 25.3% | 11.7% | 2.2% | 0.6% | 0.2% |
5 | N Iowa | East | 75.9% | 39.1% | 11.9% | 4.0% | 1.2% | 0.2% |
7 | Iowa | South | 57.8% | 17.8% | 8.1% | 2.8% | 1.0% | 0.2% |
4 | Georgetown | South | 80.5% | 34.0% | 10.3% | 3.7% | 1.2% | 0.2% |
9 | Purdue | Midwest | 53.9% | 4.1% | 2.2% | 0.8% | 0.2% | 0.1% |
6 | Xavier | West | 48.3% | 21.9% | 4.6% | 1.4% | 0.2% | 0.1% |
6 | Butler | Midwest | 46.2% | 20.2% | 8.6% | 1.4% | 0.4% | 0.1% |
6 | S Methodist | South | 55.0% | 18.6% | 5.5% | 1.6% | 0.5% | 0.1% |
8 | NC State | East | 56.8% | 13.1% | 6.5% | 2.1% | 0.6% | 0.1% |
8 | San Diego St | South | 54.2% | 10.8% | 4.2% | 1.3% | 0.4% | 0.1% |
6 | Providence | East | 58.6% | 21.8% | 6.6% | 2.2% | 0.7% | 0.1% |
5 | W Virginia | Midwest | 66.4% | 37.6% | 4.2% | 1.6% | 0.4% | 0.1% |
4 | Maryland | Midwest | 68.0% | 36.8% | 4.0% | 1.5% | 0.3% | 0.1% |
10 | Davidson | South | 42.2% | 10.5% | 4.0% | 1.2% | 0.3% | 0.1% |
10 | Georgia | East | 33.0% | 9.7% | 4.1% | 1.4% | 0.4% | 0.1% |
11 | BYU | West | 32.7% | 16.4% | 4.1% | 1.4% | 0.3% | 0.1% |
5 | Arkansas | West | 76.6% | 30.2% | 7.8% | 2.0% | 0.3% | 0.1% |
9 | Oklahoma St | West | 52.0% | 9.9% | 4.0% | 1.0% | 0.2% | 0.0% |
16 | Hampton | Midwest | 0.1% | 0.0% | 0.0% | 0.0% | 0.0% | 0.0% |
10 | Indiana | Midwest | 32.5% | 10.7% | 3.7% | 0.4% | 0.1% | 0.0% |
7 | VCU | West | 26.6% | 2.9% | 1.1% | 0.2% | 0.0% | 0.0% |
9 | St Johns | South | 45.8% | 7.9% | 2.8% | 0.8% | 0.2% | 0.0% |
11 | Boise State | East | 15.9% | 4.3% | 0.9% | 0.2% | 0.0% | 0.0% |
11 | Dayton | East | 25.4% | 8.3% | 2.1% | 0.6% | 0.2% | 0.0% |
8 | Cincinnati | Midwest | 46.1% | 3.0% | 1.5% | 0.5% | 0.1% | 0.0% |
11 | Mississippi | West | 19.0% | 8.0% | 1.5% | 0.4% | 0.1% | 0.0% |
11 | UCLA | South | 45.0% | 13.3% | 3.5% | 0.9% | 0.2% | 0.0% |
8 | Oregon | West | 48.0% | 8.5% | 3.3% | 0.7% | 0.1% | 0.0% |
12 | Wyoming | East | 24.1% | 6.6% | 0.9% | 0.1% | 0.0% | 0.0% |
12 | Buffalo | Midwest | 33.6% | 13.7% | 0.8% | 0.2% | 0.0% | 0.0% |
12 | Wofford | West | 23.4% | 4.2% | 0.4% | 0.0% | 0.0% | 0.0% |
12 | Ste F Austin | South | 24.5% | 10.9% | 2.3% | 0.6% | 0.1% | 0.0% |
13 | Harvard | West | 18.8% | 7.5% | 1.1% | 0.2% | 0.0% | 0.0% |
13 | Valparaiso | Midwest | 32.0% | 11.8% | 0.6% | 0.1% | 0.0% | 0.0% |
13 | E Washingtn | South | 19.5% | 3.2% | 0.3% | 0.0% | 0.0% | 0.0% |
13 | UC Irvine | East | 22.3% | 7.4% | 1.0% | 0.2% | 0.0% | 0.0% |
14 | UAB | South | 7.2% | 1.5% | 0.1% | 0.0% | 0.0% | 0.0% |
14 | Georgia St | West | 21.7% | 6.8% | 0.7% | 0.1% | 0.0% | 0.0% |
14 | Northeastrn | Midwest | 11.1% | 1.9% | 0.3% | 0.0% | 0.0% | 0.0% |
14 | Albany | East | 7.9% | 1.6% | 0.1% | 0.0% | 0.0% | 0.0% |
15 | N Dakota St | South | 3.4% | 0.5% | 0.1% | 0.0% | 0.0% | 0.0% |
15 | TX Southern | West | 0.8% | 0.1% | 0.0% | 0.0% | 0.0% | 0.0% |
15 | Belmont | East | 5.9% | 0.8% | 0.1% | 0.0% | 0.0% | 0.0% |
15 | N Mex State | Midwest | 12.5% | 2.7% | 0.5% | 0.0% | 0.0% | 0.0% |
16 | Manhattan | Midwest | 0.7% | 0.1% | 0.0% | 0.0% | 0.0% | 0.0% |
16 | N Florida | South | 1.9% | 0.4% | 0.1% | 0.0% | 0.0% | 0.0% |
16 | Rob Morris | South | 1.1% | 0.2% | 0.0% | 0.0% | 0.0% | 0.0% |
16 | Lafayette | East | 2.3% | 0.3% | 0.0% | 0.0% | 0.0% | 0.0% |
16 | Coastal Car | West | 3.1% | 0.6% | 0.1% | 0.0% | 0.0% | 0.0% |
9 | LSU | East | 43.2% | 8.0% | 3.5% | 0.9% | 0.2% | 0.0% |
Printed from TeamRankings.com - © 2005-2024 Team Rankings, LLC. All Rights Reserved.