Public all-in-one impact metrics, the RAPTOR / EPM / DARKO family, try to boil a player down to one number: how much better his team does per 100 possessions with him on the floor, after accounting for teammates and opponents. They're some of the most carefully built numbers in basketball analytics, so feeding them into a prediction model sounds like an obvious upgrade.
We tested that idea twice, once in each of our two models. Both times the answer was the same: no measurable improvement. Here's what we tried, and why we think it didn't help.
Test one: player projections
Our player projections predict counting stats (points, rebounds, assists and so on) for every rotation player each night. We added each player's previous-season impact rating as extra inputs. Using last season's numbers keeps the test honest: nothing from the future leaks into a prediction. About 87% of players matched to a rating each season; the rest were mostly rookies, who simply have no prior season.
We compared the model with and without the ratings across three separate test periods, each retrained with three random seeds, using the paired, game-by-game tests described in our post on signal and noise. The bar for "keep": a significant improvement in every test period, for at least two of the three seeds.
Not one of the 11 stats cleared it. The biggest change anywhere was on points: an average error 0.007 points lower per game, well inside the noise. The odd part is that the model clearly used the new inputs. They ranked fairly high among its inputs by importance. It just got no better at predicting anything with them.
Figure 1. Player model: change in average error after adding impact ratings
Source: HoopModel player-model experiment: three test seasons (2023-24 to 2025-26), three seeds each.
Show the data
| Stat | Avg change in error | Best run | Worst run |
|---|---|---|---|
| MIN | -0.030% | -0.388% | 0.279% |
| PTS | -0.138% | -0.329% | 0.018% |
| REB | -0.020% | -0.324% | 0.261% |
| AST | -0.002% | -0.217% | 0.298% |
| STL | -0.042% | -0.168% | 0.077% |
| BLK | 0.119% | -0.315% | 0.395% |
| TOV | -0.002% | -0.165% | 0.125% |
| 3PM | -0.013% | -0.125% | 0.082% |
| 3PA | -0.026% | -0.128% | 0.087% |
| FTM | -0.017% | -0.299% | 0.116% |
| FTA | -0.010% | -0.269% | 0.160% |
Test two: the team model, where it should matter
That first null had an obvious objection. Impact metrics measure effect on the scoreboard margin, not individual box scores, so a player model is the wrong place for them. The team model, which predicts winners and margins, is their natural home.
So we built a team-level version. Before each game we reconstructed each team's likely roster (players who'd appeared recently, minus anyone listed out or doubtful on the official injury report) and took a minutes-weighted average of their prior-season ratings. On its own, the signal looked strong. The difference between the two teams' ratings correlated with the final margin at 0.28, and when the home team had the higher rating it won 65% of the time.
Figure 2. Home win rate by the two rosters' prior-season impact ratings
Source: 6,249 games, 2022–2026, with a prior-season rating for both teams.
Show the data
| Rating gap (fifth) | Home win rate | Games |
|---|---|---|
| Away much better | 39% | 1250 |
| Away better | 50.6% | 1250 |
| About even | 54.7% | 1249 |
| Home better | 63.4% | 1250 |
| Home much better | 70.8% | 1250 |
Added to the model, it did nothing. On the held-out test month: no significant change in accuracy, margin error or ranking quality, for any model. Pooled across 21 further test windows: still nothing. We also checked the case where we expected the most help, October through December, when this season's box scores are thin and last season's knowledge should be most valuable. There the ratings did slightly worse.
Figure 3. Team model: change in ranking quality (AUC) after adding the ratings
Source: HoopModel team-model experiment; RF and XGBoost, 10 seeds on the held-out month, 3 seeds per monthly window.
Show the data
| Test | Model | Games | Δ AUC | 95% CI | Δ margin error (pts) | 95% CI |
|---|---|---|---|---|---|---|
| 1 month | RF | 230 | -0.0052 | -0.0117 – 0.0018 | 0.020 | -0.015 – 0.054 |
| 1 month | XGB | 230 | 0.0021 | -0.0088 – 0.0125 | 0.161 | -0.050 – 0.366 |
| 21 months | RF | 4,032 | -0.0009 | -0.0037 – 0.0020 | -0.005 | -0.018 – 0.009 |
| 21 months | XGB | 4,032 | 0.0017 | -0.0039 – 0.0075 | -0.034 | -0.105 – 0.042 |
A strong signal isn't the same as new information
A 0.28 correlation with the margin is real. But a model doesn't need a feature that predicts the outcome. It needs one that predicts the part of the outcome its existing inputs miss. By the time the ratings arrived, the model already had:
- a running team strength rating that carries over from one season into the next,
- each team's recent form across dozens of box-score statistics, and
- information about which players are actually available tonight.
Between them, those cover most of what a prior-season impact rating knows: which teams are good, and how much of that quality is suiting up. The ratings mostly told the model things it already knew, in a different unit.
This is a common pattern, and worth knowing if you build models yourself. You can often find a feature that correlates nicely with the outcome. The test that matters is whether the model gets better when you add it.
What we haven't ruled out
Both tests used static ratings from the previous season. Some impact metrics update daily during the season, which could help exactly where a frozen number can't: a breakout player in November, or a team reshaped by a mid-season trade. That's still untested, and it would have to clear the same bar. For now, last season's ratings stay out of both models.