Don’t let the blue-blood Final Four of Kansas, Duke, North Carolina, and Villanova fool you: this year’s NCAA tournament was one of the most chaotic and unpredictable in recent history.
It featured one of the greatest Cinderella runs of all time — Shaheen Holloway’s Saint Peter’s squad became the first ever 15-seed (or 14-seed, or 13-seed, or 12-seed) to ever make a run to the Elite Eight, knocking off two dark-horse championship contenders along the way in 2-seed Kentucky and 3-seed Purdue. Their literally-unprecedented path to the Elite Eight overshadowed a pair of other awesome performances by lower-seeded squads: 10-seed Miami also made it to the Elite Eight, and 8-seed North Carolina is still dancing; after picking up marquee wins over 1-seed Baylor and 4-seed UCLA and finally vanquishing Cinderella in their 20-point win over Saint Peter’s, Hubert Davis’s squad is playing their best basketball of the season headed into a rubber match with Duke in the Final Four.
In addition to this year’s underdog runs, a few favorites have also exited the tournament uncharacteristically early. It’s not all that uncommon for only one 1-seed to make the Final Four, but it’s pretty shocking to see three 1-seeds eliminated before even the Elite Eight. Gonzaga, the top seed in the country, looked uncharacteristically vulnerable in the early rounds (they trailed both Georgia State and Memphis for significant portions of those games) before falling to Arkansas in the Sweet Sixteen. Baylor stumbled in the second round against a North Carolina team that led by 25 points before Brady Manek’s ejection. It took a career game from Bennedict Mathurin for Arizona to take down TCU in overtime in the second round, and they were soundly defeated by Houston in the Sweet Sixteen.
The chaos of this season’s tournament can be quantified by how challenging it has been to predict. While ShotQuality’s model was able to accurately predict the winner of 76.5% of this season’s non-NCAA-tournament games just by looking at the quality of shots each team took in the game, it called the winner of 41 of 60 NCAA tournament games so far, just a 68.3% rate. The average error in final score margin also increased from about 8.1 points to about 10.6 points. While this model performance is still very accurate in absolute terms, it suggests that perhaps March Madness games might have some unique element that makes them deviate from ShotQuality scores more than expected.
Looking at the distribution of model errors might uncover any systemic biases that make March Madness inherently more unpredictable. The density plots and histograms below show the difference between ShotQuality predicted point differential and actual point differential for NCAA tournament and non-NCAA-tournament games.
These charts make the trend seem a bit fuzzier. There has definitely been a slight increase in variance for this season’s tournament as opposed to the regular season, as evidenced by the spread on the density curve. However, it’s not quite as massive as one might expect if there was a substantial difference between regular season and tournament games.
Let’s approach this question from another angle, and look at how some team-level statistics that could potentially increase or decrease variance in final scores such as possessions per game and three-point attempts per game changed in the tournament this season.
No significant differences jump off the page here, either. Two potential hypotheses for increased variance — NCAA tournament games having more possessions than regular season games, and NCAA tournament teams taking more threes than teams in the regular season — actually run counter to what we would expect if the hypotheses were true: the average team playing in the NCAA tournament has actually attempted 0.7 fewer threes per game and recorded 0.5 fewer possessions than the average team playing in a regular season game.
The only somewhat major change that could contribute to variance comes in the area of ball security. Teams in the NCAA tournament protect the ball better than teams playing in regular season games, averaging just 10.3 turnovers compared to a season average of 11.5. That adds up to two or three additional possessions ending with shots instead of turnovers per game; in a game where the smallest differences can turn a victory to a defeat, those extra possessions could be extremely important.
However, the general trend seems to be that there isn’t much of a fundamental difference between team statistics in NCAA tournament games and non-tournament games, which means there isn’t a great explanation for why tournament games have deviated from ShotQuality scores so much this season. The underwhelming answer seems to be simple random variance.
Some simple statistical tests back up this conclusion: that we’ve just lucked into an especially fun and unpredictable tournament this season.
Performing a chi-squared test (a common statistical practice for analyzing differences between proportions) on the ShotQuality model’s decline in accuracy for NCAA tournament games returns a p-value of 0.182, suggesting that if the model is consistent across regular season and tournament games then we should expect to see this much variance (or more) in accuracy about 18% of the time, or once every five years. Performing a two-sample t-test (a common statistical practice for concluding if two samples have the same mean) on the model’s average error between tournament and non-tournament games yields a p-value of 0.562, suggesting that we should expect this much variance in standard error (or more) about 56% of the time, or once every two years.
This answer to the question is, from a basketball fan’s perspective, the most satisfactory. There is no missing statistic that could have foreseen Kentucky’s loss to Saint Peter’s (a game that, based on the quality of shots taken, Kentucky was expected to win 98% of the time) or projected three 1-seeds falling before the Elite Eight. There’s a reason it’s called March Madness; the randomness is what makes this tournament the most intriguing event in all of college athletics.
By Ben Wieland