MLB Prediction System

Predicted
versus played

slate 2026-10-05settled 2KPI applied 2run error 3.000 to 3.500winners 0 to 0

The card

Predicted versus played

2 games, all settled. The middle column is this run, with the KPI layer applied to 2 of them. The column before it is what production published for the same slate, without any KPI layer, read from the local canonical artifact.

Matchup	Prior VPS	Regenerated	Actual	Pick	Run err	Total err	Result
CHW @ CLE	4-5	4-5	4-3	CLE 52%	2	2	miss unmoved
NYY @ TB	4-2	4-2	2-5	NYY 54%	5	1	miss unmoved

0 of 2 winners, 0.0%. Picking the home team in every game would have returned 1/2, 50.0%. The card is 1 game behind the coin, -50.0%. On 2 games that is equally inside chance, and it is not evidence against the layer either.

Where accuracy is won and lost

Every stage, same games

The same fifteen games scored five times. Each row is a real intermediate value recorded in the artifact, not a reconstruction.

Stage	Run err	Total err	Winners	Brier
Production baseline before the KPI layer touched it	3.000	1.000	0/2	0.2875
KPI-only forecast 44 KPI features alone	3.647	1.399	0/2	0.2173
Blended, continuous at the fitted weights, before rounding	3.419	1.186	0/2	0.2875
Published after rounding and probability reconciliation	3.500	1.500	0/2	0.2800
Prior VPS card what production published, no KPI layer	3.500	1.500	0/2	0.2800

The recorded pipeline moved run error from 3.000 to 3.500, a gain of -0.500 runs per game. This compares an intermediate KPI baseline with the final published output. It is diagnostic only. This slate has 2 controlled paired final forecasts and 0 stage-only comparisons.

The comparison against the prior card is not the same measurement. It is read from the local canonical artifact. That card ran on older lineups and an older Savant bundle, so its 3.500 carries a data refresh as well as the missing KPI layer. The honest statement is that this card is better by +0.000 runs and +0 winners. The intermediate-to-final movement is -0.500, but it is not causal KPI attribution.

Two things the stage table exposes

The KPI model alone has the lowest run error of any stage (3.647) and the worst Brier (0.2173). It reads the score better than it reads the winner, and it states its winner too confidently. That is exactly why its winner weight is 0.14 while its run weights are 0.56, and it is a point in favour of fitting the three targets separately rather than forcing one blend.
Blending costs a little run accuracy against the KPI model alone (3.419 against 3.647) and buys back a lot of Brier (0.2875 against 0.2173). On fifteen games that trade is not measurable either way. Over the 782-game walk-forward the blend is what clears zero, so it stays.

The publication step

Rounding is no longer the problem

The previous settlement report found the publication step was costing accuracy. It is worth re-measuring rather than assuming, because the step now sits after a different layer.

Rounding the continuous blend to publishable integers changed run error from 3.419 to 3.500, a move of +0.081 runs, and flipped 0 winners. On this card the rounding step is harmless. The earlier finding does not reproduce, and the reconciliation that forces the integer score and the probability onto the same side is doing its job without distorting either.

The probability is still being pulled toward 0.5

Published win probability is shrunk toward 0.5 in proportion to verified skill. That is deliberate and documented. It is worth showing the size of it, because it is large enough on some games to matter and a reader comparing the blend to the published number will otherwise think something is broken.

Game	Blended	Published	Shift	Direction
CHW @ CLE	0.526	0.518	-0.007	toward 0.5
NYY @ TB	0.453	0.460	+0.007	toward 0.5

2 of 2 shifts move toward 0.5, which is the intended direction. The largest is CHW @ CLE at -0.007. Brier went from 0.2875 blended to 0.2800 published, a cost of -0.0075. On fifteen games that is noise, and the shrinkage is justified by season-long calibration rather than by one card.

Coverage

What the gates did

The layer applied to 2 of 2 settled games.

Luna: historical_not_requested 2. This was a rerun after first pitch, so no live request was made and Luna moved nothing on this card. Every difference between the prior VPS column and the regenerated column comes from the refreshed inputs and the KPI layer.

Reading this honestly

What fifteen games can and cannot say

It can say the layer moved almost every game. That was the open question and it is now answered: 2 of 2 games moved, and the one that did not was refused by a gate with its reason recorded.
It cannot say the layer is good. Run error improved by -0.500 on this card, which is the right sign and the right rough size against the walk-forward's +0.0723 and +0.0853 per-target gains over ? games, but a fifteen-game sample has an interval far wider than the effect.
It cannot say anything about winners. Every stage ties the always-home baseline at 50.0%. The winner target is inconclusive on the walk-forward too, at weight 0.14, and this card is consistent with that.
The prior-card comparison is contaminated by the lineup and bundle refresh and should not be quoted as the layer's effect.

Accumulate prospective pairs. The only thing that settles this is more settled days with the layer live. Nothing in the code needs changing to get them.
Watch the KPI model's winner calibration. Its standalone Brier of 0.2173 against a baseline 0.2875 is the weakest number in the table. If that does not improve as archives accumulate, the winner weight should go to zero rather than stay at 0.14.
Raise lineup coverage, or lower the floor deliberately. One game in fifteen was refused for thin lineup samples. That rate is acceptable; a rising rate late in a season, when call-ups multiply, would not be.
Leave the rounding step alone. It was flagged before and does not reproduce as a problem here. Re-measure it rather than re-fixing it.

generated 2026-10-06T17:22:12.444201+00:00 · settlement_comparison_v2
regenerated artifacts: output/games/revisions/<game>/*.json · prior card: output/games/<game>.json
stage values read from game.enhanced_prediction_analysis.kpi_layer · finals from statsapi.mlb.com
every number in this document was read from those files at generation time