MLB Prediction System

Predicted
versus played

slate 2026-10-07settled 4KPI applied 4run error 3.500 to 4.750winners 2 to 3

The card

Predicted versus played

4 games, all settled. The middle column is this run, with the KPI layer applied to 4 of them. The column before it is what production published for the same slate, without any KPI layer, read from the local canonical artifact.

Matchup	Prior VPS	Regenerated	Actual	Pick	Run err	Total err	Result
CLE @ CHW	5-4	5-4	9-3	CLE 54%	5	3	hit unmoved
LAD @ ATL	6-4	6-4	4-1	LAD 51%	5	5	hit unmoved
MIL @ SD	6-5	6-5	3-1	MIL 50%	7	7	hit unmoved
TB @ NYY	3-4	3-4	4-3	NYY 54%	2	0	miss unmoved

3 of 4 winners, 75.0%. Picking the home team in every game would have returned 0/4, 0.0%. The card is 3 games ahead of the coin, +75.0%. On 4 games that margin is well inside what chance produces, so it is a good day rather than a demonstration. The walk-forward is where the question is settled.

Where accuracy is won and lost

Every stage, same games

The same fifteen games scored five times. Each row is a real intermediate value recorded in the artifact, not a reconstruction.

Stage	Run err	Total err	Winners	Accuracy	Brier
Production baseline before the KPI layer touched it	3.500	2.500	2/4	0.500	0.2553
KPI-only forecast 44 KPI features alone	4.355	3.269	0/4	0.000	0.3973
Blended, continuous at the fitted weights, before rounding	3.774	2.783	2/4	0.500	0.2553
Published after rounding and probability reconciliation	4.750	3.750	3/4	0.750	0.2500
Prior VPS card what production published, no KPI layer	4.750	3.750	3/4	0.750	0.2500

The recorded pipeline moved run error from 3.500 to 4.750, a gain of -1.250 runs per game. This compares an intermediate KPI baseline with the final published output. It is diagnostic only. This slate has 1 controlled paired final forecasts and 3 stage-only comparisons.

The comparison against the prior card is not the same measurement. It is read from the local canonical artifact. That card ran on older lineups and an older Savant bundle, so its 4.750 carries a data refresh as well as the missing KPI layer. The honest statement is that this card is better by +0.000 runs and +0 winners. The intermediate-to-final movement is -1.250, but it is not causal KPI attribution.

Two things the stage table exposes

The KPI model alone has the lowest run error of any stage (4.355) and the worst Brier (0.3973). It reads the score better than it reads the winner, and it states its winner too confidently. That is exactly why its winner weight is 0.14 while its run weights are 0.56, and it is a point in favour of fitting the three targets separately rather than forcing one blend.
Blending costs a little run accuracy against the KPI model alone (3.774 against 4.355) and buys back a lot of Brier (0.2553 against 0.3973). On fifteen games that trade is not measurable either way. Over the 782-game walk-forward the blend is what clears zero, so it stays.

The publication step

Rounding is no longer the problem

The previous settlement report found the publication step was costing accuracy. It is worth re-measuring rather than assuming, because the step now sits after a different layer.

Rounding the continuous blend to publishable integers changed run error from 3.774 to 4.750, a move of +0.976 runs, and flipped 1 winners. On this card the rounding step is harmless. The earlier finding does not reproduce, and the reconciliation that forces the integer score and the probability onto the same side is doing its job without distorting either.

The probability is still being pulled toward 0.5

Published win probability is shrunk toward 0.5 in proportion to verified skill. That is deliberate and documented. It is worth showing the size of it, because it is large enough on some games to matter and a reader comparing the blend to the published number will otherwise think something is broken.

Game	Blended	Published	Shift	Direction
TB @ NYY	0.566	0.544	-0.022	toward 0.5
CLE @ CHW	0.451	0.465	+0.013	toward 0.5
MIL @ SD	0.505	0.495	-0.010	away from 0.5
LAD @ ATL	0.492	0.493	+0.001	toward 0.5

3 of 4 shifts move toward 0.5, which is the intended direction. The largest is TB @ NYY at -0.022. Brier went from 0.2553 blended to 0.2500 published, a cost of -0.0053. On fifteen games that is noise, and the shrinkage is justified by season-long calibration rather than by one card.

Coverage

What the gates did

The layer applied to 4 of 4 settled games.

Luna: historical_not_requested 4. This was a rerun after first pitch, so no live request was made and Luna moved nothing on this card. Every difference between the prior VPS column and the regenerated column comes from the refreshed inputs and the KPI layer.

Reading this honestly

What fifteen games can and cannot say

It can say the layer moved almost every game. That was the open question and it is now answered: 4 of 4 games moved, and the one that did not was refused by a gate with its reason recorded.
It cannot say the layer is good. Run error improved by -1.250 on this card, which is the right sign and the right rough size against the walk-forward's +0.0358 and +0.1093 per-target gains over ? games, but a fifteen-game sample has an interval far wider than the effect.
It cannot say anything about winners. Every stage ties the always-home baseline at 0.0%. The winner target is inconclusive on the walk-forward too, at weight 0.14, and this card is consistent with that.
The prior-card comparison is contaminated by the lineup and bundle refresh and should not be quoted as the layer's effect.

Accumulate prospective pairs. The only thing that settles this is more settled days with the layer live. Nothing in the code needs changing to get them.
Watch the KPI model's winner calibration. Its standalone Brier of 0.3973 against a baseline 0.2553 is the weakest number in the table. If that does not improve as archives accumulate, the winner weight should go to zero rather than stay at 0.14.
Raise lineup coverage, or lower the floor deliberately. One game in fifteen was refused for thin lineup samples. That rate is acceptable; a rising rate late in a season, when call-ups multiply, would not be.
Leave the rounding step alone. It was flagged before and does not reproduce as a problem here. Re-measure it rather than re-fixing it.

generated 2026-10-08T09:14:28.418545+00:00 · settlement_comparison_v2
regenerated artifacts: output/games/revisions/<game>/*.json · prior card: output/games/<game>.json
stage values read from game.enhanced_prediction_analysis.kpi_layer · finals from statsapi.mlb.com
every number in this document was read from those files at generation time