A model that picks 71% of regular-season winners sounds like it should pick about 71% of playoff winners too. Same teams, same sport, same model. It doesn't. When we tested ours, winner accuracy fell by about 7 percentage points once the postseason started. Yet one thing held up almost perfectly: how honest the models were about their own uncertainty.
How we tested it
For each of five seasons, we trained the models only on regular-season games played before that season's playoffs. Then we scored them on three sets of games that all came after training: the last three weeks of the regular season (the control group), the play-in tournament, and the playoffs. Pooled across seasons, that's about 630 late-regular-season games and 400 playoff games.
Picking winners gets much harder
On the late-regular-season control games, our random forest picked 71% of winners and our gradient-boosting model 69%. In the playoffs: 64% and 62%.
A second measure fell even harder. AUC measures how well the win probabilities rank games, from "home team almost certainly wins" down to "home team almost certainly loses." It runs from 0.5 (no better than guessing) to 1.0 (perfect). On the control games both models scored about 0.78–0.79. In the playoffs they dropped to 0.60–0.64, giving up more than half their edge over guessing.
But the uncertainty stays honest
Alongside each margin prediction, the models produce a range meant to contain the real result 90% of the time. In the playoffs those ranges still contained the real margin 88–90% of the time, right on target. The average margin error barely moved either, from 11.9 points to 12.1.
That combination tells you something about playoff basketball. The final scores aren't any wilder than in the regular season; they spread out about as much as usual. What disappears is the ability to tell in advance which side of that spread a game will land on.
Figure 1. Same models, three report cards: regular season vs playoffs
Late regular seasonPlayoffs
Winner accuracy
Ranking quality (AUC × 100)
90% margin ranges: share containing the real margin
Source: HoopModel experiment, five seasons (2022–2026): models trained only on regular-season games before each season's playoffs, five seeds. Late regular season = the last three weeks before the playoffs.
Show the data
| Model | Games | Accuracy | AUC | 90% range coverage | Avg margin miss |
|---|---|---|---|---|---|
| Random Forest · Late regular season | 631 | 71.3% | 0.790 | 89.9% | 11.9 |
| Random Forest · Playoffs | 403 | 63.5% | 0.637 | 88.3% | 12.1 |
| XGBoost · Late regular season | 631 | 69.1% | 0.781 | 89.9% | 11.9 |
| XGBoost · Playoffs | 403 | 62% | 0.600 | 89.6% | 12.2 |
What changes when the playoffs start
We compared the inputs the models see in late-season games against playoff games, and a few shifts stood out:
- The matchups are closer. Only the best teams are left, so the gap in team strength between the two sides is significantly smaller. Most of a model's accuracy comes from spotting mismatches, and the playoffs remove most of them.
- The pace slows. Playoff basketball is a half-court, grind-it-out game. Teams play differently from the regular-season games the models learned from.
- Series change the game. By Game 4, both coaching staffs have spent a week adjusting to one specific opponent. Rolling averages built over months of games against 29 different teams don't capture that.
Figure 2. How lopsided are the matchups? The favorite's pre-game win probability
Source: HoopModel experiment, five seasons (2022–2026): models trained only on regular-season games before each season's playoffs, five seeds. Late regular season = the last three weeks before the playoffs.
Show the data
| Favorite's win probability | Late regular season | Playoffs |
|---|---|---|
| 50–60% | 25.9% | 32.3% |
| 60–70% | 25.9% | 35% |
| 70–80% | 23.2% | 23.8% |
| 80%+ | 25.1% | 8.9% |
None of this means the model is broken. A coin flip between two evenly matched contenders is genuinely less predictable than a contender visiting a lottery team in February, and the models are correctly telling you less.
What about the play-in?
The play-in looked even less predictable, which makes sense for single-elimination games between middling teams. But it's only six games a year, so 30 games across five seasons. That's far too few to say anything with confidence, so we don't.
How to read playoff picks
Treat the confidence numbers with extra skepticism in the postseason. A "65%" pick in April is a weaker statement than the same number in January. The margin ranges, on the other hand, can be taken at face value. Keep it in mind when the track record dips in the spring: a lower hit rate in the playoffs is what this study predicts, not a sign the model has gone wrong.