The cardPredicted versus played
2 games, all settled. The middle column is this run, with the
KPI layer applied to 2 of them. The column before it is what production published for the same slate, without any KPI layer, read from the local canonical artifact.
2 of 2 winners, 100.0%.
Picking the home team in every game would have returned
1/2, 50.0%. The card is 1 game ahead of the coin, +50.0%. On 2 games that margin is well inside what chance produces, so it is a good day rather than a demonstration. The walk-forward is where the question is settled.
Where accuracy is won and lostEvery stage, same games
The same fifteen games scored five times. Each row is a real intermediate value recorded in
the artifact, not a reconstruction.
The recorded pipeline moved run error from
2.500 to 2.500, a gain of
+0.000 runs per game. This compares an
intermediate KPI baseline with the final published output. It is diagnostic only. This slate has
2 controlled paired final forecasts and 0
stage-only comparisons.
The comparison against the prior card is not the same measurement. It is read from the local canonical artifact. That card ran on older lineups and an older Savant bundle, so its 2.500 carries a data refresh as well as the missing KPI layer. The honest statement is that this card is better by +0.000 runs and +0 winners. The intermediate-to-final movement is +0.000, but it is not causal KPI attribution.
Two things the stage table exposes
- The KPI model alone has the lowest run error of any stage
(1.800) and the worst Brier (0.3921). It reads the
score better than it reads the winner, and it states its winner too confidently. That is
exactly why its winner weight is 0.14 while its run weights are 0.56, and it is a point in
favour of fitting the three targets separately rather than forcing one blend.
- Blending costs a little run accuracy against the KPI model alone
(2.001 against 1.800) and buys back a lot of
Brier (0.2104 against 0.3921). On fifteen games that
trade is not measurable either way. Over the 782-game walk-forward the blend is what clears
zero, so it stays.
The publication stepRounding is no longer the problem
The previous settlement report found the publication step was costing accuracy. It is worth
re-measuring rather than assuming, because the step now sits after a different layer.
Rounding the continuous blend to publishable integers changed run
error from 2.001 to
2.500, a move of +0.499 runs, and
flipped 0 winners. On this card the rounding step is harmless. The
earlier finding does not reproduce, and the reconciliation that forces the integer score and
the probability onto the same side is doing its job without distorting either.
The probability is still being pulled toward 0.5
Published win probability is shrunk toward 0.5 in proportion to verified skill. That is
deliberate and documented. It is worth showing the size of it, because it is large enough on
some games to matter and a reader comparing the blend to the published number will otherwise
think something is broken.
2 of 2 shifts move toward 0.5, which is the
intended direction. The largest is MIL @ SD at -0.012.
Brier went from 0.2104 blended to 0.2164 published, a cost of
+0.0060. On fifteen games that is noise, and
the shrinkage is justified by season-long calibration rather than by one card.
CoverageWhat the gates did
The layer applied to 2 of 2 settled games.
Luna: historical_not_requested 2.
This was a rerun after first pitch, so no live request was made and Luna moved nothing on this
card. Every difference between the prior VPS column and the regenerated column comes from the
refreshed inputs and the KPI layer.
Reading this honestlyWhat fifteen games can and cannot say
- It can say the layer moved almost every game. That was the open question and it is
now answered: 2 of 2 games moved, and the one that did
not was refused by a gate with its reason recorded.
- It cannot say the layer is good. Run error improved by
+0.000 on this card, which is the right sign and
the right rough size against the walk-forward's +0.0358 and +0.1093 per-target gains over ? games, but a fifteen-game sample has an interval far wider
than the effect.
- It cannot say anything about winners. Every stage ties the always-home baseline at
50.0%. The winner target is inconclusive on the walk-forward too, at
weight 0.14, and this card is consistent with that.
- The prior-card comparison is contaminated by the lineup and bundle refresh and should
not be quoted as the layer's effect.
NextWhat to fix, in order
- Accumulate prospective pairs. The only thing that settles this is more settled days
with the layer live. Nothing in the code needs changing to get them.
- Watch the KPI model's winner calibration. Its standalone Brier of
0.3921 against a baseline 0.2104 is the weakest number in the
table. If that does not improve as archives accumulate, the winner weight should go to zero
rather than stay at 0.14.
- Raise lineup coverage, or lower the floor deliberately. One game in fifteen was
refused for thin lineup samples. That rate is acceptable; a rising rate late in a season, when
call-ups multiply, would not be.
- Leave the rounding step alone. It was flagged before and does not reproduce as a
problem here. Re-measure it rather than re-fixing it.