By: Graeme Thomas
The “Jamal Crawford” Effect:
On April 9th, 2019, Jamal Crawford played arguably the greatest game off the bench of all time. Scoring the most points by a bench player in a single game, the three-time Sixth-Man of the Year winner put up 51 points in 38 minutes. Unfortunately, his efforts went unrealized in the end as the Phoenix Suns fell to the Dallas Mavericks 109-120. With 9:19 remaining in the first quarter the Mavericks took a lead over the Suns and never looked back. By the time Jamal Crawford entered the game with 5:50 remaining in the first quarter the Suns already trailed 10-18, and by the time he scored his first points with 2:25 remaining in the first, the Suns’ deficit had increased to 12. ESPN placed the Suns win probability before Crawford’s first points at a measly 4.2%. Their win probability continued to drop as the Mavericks opened up a 68 to 38 halftime lead. At no moment over the course of the game did the Suns win probability rise above 17% after Jamal Crawford came on in the 1st quarter.
So why does this matter? Jamal Crawford did the unthinkable in scoring 51 points off the bench, but did he impact the game? The Suns were almost predestined to lose by the time he began his scoring streak, and while his feat should be appreciated, none of his efforts were able to change the outcome of the game. While Crawford had an exceptionally “good” game, his 51 points had almost no impact on the Suns chances of winning. This I believe to be a real problem with looking at simple metrics like points per game or even field goal percentage, for while useful in terms of individual accolades, are far from telling the whole story on the impact a player has on their team’s chances of winning. Whether you like it or not, that is what basketball is all about: winning.
Measuring Impact:
How then, can we determine how impactful a player’s performance actually is for their team’s success? The answer can come about in part with the help of a new metric titled Expected Win Probability Above Replacement (EWPAR). EWPAR measures how a player impacts their team’s chance to win the game compared to a replacement-level player. Each shot in a game has a ShotQuality expected value, which considers shot context and the shooting player, and an average ShotQuality expected value, the expected value of that shot with a league-average player. As an example, when Giannis Antentekoumpo attempts an at the rim shot out of a pick and roll he has an expected point value of 1.56, while the average NBA player only has an expected value of 1.36 if in the same situation. Taking this data we developed a model using expected score difference to calculate expected win probability.
Expected Score Difference=Score Difference Before Shot+ShotQuality Expected Value of Shot
Using this calculation of Expected Score difference, we can see how much a player is expected to add to (or eat into) the actual score difference. Using these expected score differences as the primary impetus in our trained expected win probability model, we can find the expected win probability added (EWPA) at each play by taking the difference in expected win probability between each play. This model was produced using a binary generalized linear model, as it accounted for the most amount of variation in win probability. EWPA is an extremely interesting metric on its own but it has a few flaws:
- That players that play for teams that are constantly in back and forth games and will thus have the largest impact on win probability have highly leveraged win probabilities that may overestimate how good they are just based on their teammates
- EWPA can be a confusing statistic to actually absorb as some players have an EWPA above 1, which while useful in comparison to other players, is not all that telling on its own as there are too many confounding factors present that could overestimate for certain players (i.e. usage rate, overtime occurrences, etc.)
This is why EWPA needs to be better refined to capture just how impactful a player really is for their team’s performance. EWPAR produces a more comprehensive and comprehensible metric to use than the raw EWPA. By using our trained model on average expected score difference (using the same equation as above with average SQ value) we can create a new metric of replacement win probability added (RWPA). RWPA is thus the win probability added if a replacement level player was taking every shot. Then by taking the difference between the two (EWPA-RWPA) we get our finished EWPAR.
What EWPAR is:
EWPAR then measures how much a player is expected to add to his teams win probability above a replacement level player if that replacement level player took the same amount and type of shots as them. We have effectively isolated the possible confounding impact of usage on EWPA as we are controlling for the number of plays by describing how much better (or worse) a given player is than an average player would be with the same amount and type of shots. Thus EWPAR tells us just how much better and impactful a player is and how valuable their shots are to their teams success as a whole. EWPAR does a very good job at isolating impact from raw point totals, where it gives a new depth to player impact that was simply not possible to see beforehand. We have disregarded the possibility of stat padding in garbage time, because win probability added will be miniscule in these situations, and have isolated the real importance of a given player on their team’s overall success.
Looking at the top 10 and bottom 10 EWPAR players in the NBA over the last 3 seasons leads us to some interesting insights. The top 10 is flooded with names NBA fans would expect with Kevin Durant leading the line with a EWPAR of .0858. This means that Durant adds 8.58% win probability for his team on average above a replacement level player. This matters because Kevin Durant is thus expected to score when it matters the most for his team, and so because he is a great scorer and he is scoring in high impact moments (i.e. changes in lead late in games) then he has the highest EWPAR. This tells us that Kevin Durant is an irreplaceable player, and over the course of the season he is likely to outperform everyone else in terms of scoring the most in crunch time. The rest of the best are all perennial all-stars, which is to be expected, but maybe more interesting is the bottom 10 for EWPAR, which features a few big names.
What is clear here is that rookies do worse in terms of EWPAR. Take Jalen Suggs’ EWPAR of -.045. An interpretation of this tells us that Jalen Suggs loses his team 4.5% win probability below a replacement level player on average. This makes sense in the case of Suggs as he would be the worst NBA player in terms of field goal percentage (36.1%) if he had enough games to qualify him for the official NBA leaderboards. Suggs then has clearly not adjusted to the scoring difficulty in the NBA. Intuitively though, this is not the death knell to Jalen Suggs as he is a rookie, and still has plenty of time for development. Thus the interpretation of EWPAR has to come with a story based perspective, as taking shots away from rookies now because of EWPAR could come at the future loss of that rookie becoming your most impactful scorer. EWPAR thus needs to be taken in the context of the player themselves, as the coach needs to decide if it is worth the loss in win probability currently in order to develop the player further. That being said, this narrative interpretation does not change the fact that in his rookie year, Jalen Suggs harmed the Magic the most out of anyone on the team in terms of underperformance in key shooting moments.
Another interesting player from the bottom is John Wall with an EWPAR of -.0324, or that on average John Wall loses his team 3.24% win probability below a replacement level player. This means that Wall is being outperformed by replacement level players in the most impactful moments, where he is expected to perform worse in the most important situations, meaning that his presence in shooting scenarios is harmful to his team’s success. Wall received a four-year supermax contract extension from the Washington Wizards in 2017, costing the Houston Rockets $47 million for Wall’s services during the 2021 season. It is clearly no secret that Wall’s downfall has been meteoric, and mostly caused by relentless injuries, but what is more interesting is that it seems that not only is John Wall not the superstar he once was, but that he is actually worse than the average NBA player. This is because Wall is a below average shooter in the league as of late, but his former superstar status seems to entitle him to taking more impactful shots than the more inexperienced players on the Rocket’s roster. What EWPAR does mean then is that Wall should not be the go to man when it comes to scoring in the most important moments, and while he can still have a large impact on the court, other players should be the ones taking the crucial shots on average. Thus, EWPAR can be an exceptionally useful coaching metric for weaning out which players are expected to be the most impactful scorers and which players are putting up points for the sake of putting up points.
What EWPAR is Not:
I want to make it clear that EWPAR is not a measure of out and out “clutchness,” because it is an aggregate calculation. This means that it does not measure the actual points scored by a player in a given moment, but only the ShotQuality historical shooting averages of that player’s shot selection. This means that it should be used to interpret aggregate utility of a certain player, and their impact on their team as a whole rather than for individual moments. So it tells us more about the expected value of the player as a whole to the team’s scoring in terms of impact of that player’s opportunities and average success, rather than clutch moments when that player actually scores. It is also important to keep in mind that because ShotQuality is a historical average tool then sample sizes must be large enough to use EWPAR effectively, where a player’s expected value for a given shot selection is statistically significant based on a large enough sample to control for variance. This means that EWPAR is exceptionally good at looking back on a given player’s season performance and determining how impactful they were, but not as good at measuring the instant impact of new players.
A possible concern is that players that do not get minutes or scoring opportunities will tend to have EWPAR values around 0 because of a lack of play time. This is taken into account by the metric itself then because a value of 0 doesn’t mean that a player is better than a player with a negative EWPAR, but instead that they are either not called upon in important situations or that they are just average in those moments. A negative EWPAR then means that a player is underperforming on average when the moments are most important, so these are the players that coaches expect to have an impact and are actually harming their teams. As such, much of the NBA will have an EWPAR of 0 if they have little to no impact (positively or negatively) because they come on in garbage time, or aren’t expected to take any important shots.
One final note is that while we solve problem 2) in situations of overemphasizing usage rates, we can never truly control for problem 1) regarding the team a player is on (although the effect is far more muted with EWPAR than with EWPA). This could be seen as a drawback, or it could be considered a part of the game of basketball. It makes sense that good players on average basketball teams will be called upon the most and will be the best and most impactful at bringing their teams to victory. It also makes sense that good players on superteams will have much less impact on their team’s outcome because of how good everyone else is around them. This is less the product of a statistical concern of overestimation for players on average teams, but more so the nature of the games that the players have found themselves in. These players are going to be more impactful because they ARE more impactful on average teams. This of course doesn’t make them necessarily better or worse than players on better teams, but it does mean that they are stepping up and hitting shots when it counts the most for their teams. In the end EWPAR is a measurement of how impactful a player is to their team, and this team-first mentality is always going to be biased towards the impact a player has on team success first before it tells us anything about individual greatness.
This I believe to be the beauty of EWPAR, because it strips much of the eye test away from players and takes a no-nonsense approach to player utility. EWPAR provides context to narratives that were not possible before, so that while we can appreciate the “Jamal Crawford” effect in games, we should not take that as a surface level measurement of how much game time Jamal Crawford should get. Jamal Crawford was great, but he did nothing to change the course of the game, which was certainly not his fault, but his 51-point performance needs to be taken with a grain of salt. Meaningless points are good for the stat sheets, but are misleading to coaches. Seeing how good your players are for your team’s real success is much more useful than raw point totals for building a winning team and a winning program.