KPI learning cycle: 20261006

Settled pregame forecasts against official finals, a rolling self-assessment, the calibration it produced and the actions it took · 2026-10-07 09:14 UTC

Yesterday: where the forecast went wrong

Games settled

0 excluded · finals data/games.csv

Club-run MAE by stage

1.25 → 1.00

baseline → blend · KPI alone 0.90 · published 1.25

Layer moved toward result

3/4

club totals the layer moved

Player lines from

statsapi.mlb.com box scores

Game	Club	Actual	Baseline	KPI alone	Blend	Published	Card right	SP outs err	Pen share err	Feature that pushed hardest the wrong way	Dominant cause
LAD@ATL	LAD	3	4.00	3.31	3.46	4	–	–	–	away_off_contact (+0.60)	Ordinary one-game variance
LAD@ATL	ATL	1	3.00	3.50	3.22	3	–	–	–	home_off_power (+0.57)	Ordinary one-game variance
MIL@SD	MIL	3	4.00	3.57	3.66	4	–	–	–	away_off_contact (+0.49)	Ordinary one-game variance
MIL@SD	SD	4	5.00	4.22	4.66	5	–	–	–	away_pen_share (+0.42)	Ordinary one-game variance

Causes are checked in a fixed order and the first that applies is named: layer refused, wrong card (3+ of nine), starter length (±6 outs), bullpen share (±0.20), KPI moved away by 0.3+ runs, outlier (5+ runs from the baseline), then variance. Every cause that applied is kept in the settled document.

Bullpen: who the forecast expected against who pitched

Relievers who pitched, in the pregame bullpen

0.929

14 relievers over 4 club-games

Closer named = arm who finished

0.250

Probability it pitches

0.44 / 0.36

arms who pitched / stayed idle · Brier 0.231

Batters faced error per arm

1.97

Game	Club	Pitched and in pool	Pool	Pitched but not in pool	Closer right
LAD@ATL	LAD	2/2	7	-	no
LAD@ATL	ATL	4/4	7	-	yes
MIL@SD	MIL	4/4	7	-	no
MIL@SD	SD	3/4	7	Michael King (8 BF)	no

Bullpen usage model: kept current · challenger holdout log loss 0.4483 vs current 0.4456 vs usage-only baseline 0.5100

Luna KPI review

Luna skipped_no_pregame_evidence. KPI-level faults 1, player-level faults 1, players flagged 6. New KPIs admitted 0, active candidates 10, promoted 0. Full review: 20261006_kpi_luna_review.html.

Per-player, per-club and per-KPI failures: 20261006_kpi_failures.html, every hitter and pitcher on the card scored against his real line, with the reason and size of each miss and the rolling ledger.

Self-assessment: 20 settled slates in the last 21 days

Blend gain over baseline

+0.029

90% [-0.079, +0.139] inconclusive

KPI alone gain over baseline

-0.045

90% [-0.225, +0.145] inconclusive

Club totals scored

228

moved toward 115/228

Blend weights: what the settled games imply

Target		Club totals	Model weight	Implied by evidence	Clamped 0–1
away_runs		114	0.69	0.35	0.35
home_runs		114	0.44	0.67	0.67

The implied weight is the least-squares fit of (actual − baseline) on (KPI − baseline) over the window's settled club totals. It is evidence for the next walk-forward refit, never applied directly.

Why club totals missed

Cause	Owner	Share	Count	Mean published error
KPI layer did not apply	operator: fix the missing input named in the reason	11%	26	2.19
Card named hitters who did not start	data feed: posted lineups arrive after the pregame run	11%	26	2.46
Starter length missed by two innings or more	model: starter workload projection	15%	36	2.14
Bullpen share missed by a fifth of the game or more	model: bullpen exposure method	3%	8	2.00
KPI layer moved the total away from the result	KPI model: weights, features or recipe	17%	40	2.65
Outlier game, five or more runs from any mean forecast	none: not forecastable	7%	17	5.24
Ordinary one-game variance	none	35%	81	1.80

Lineups, starters and bullpen exposure

Measure	n	Bias	Mean absolute error
Card hitters who did not start (of nine)	226	–	0.96
Starter outs (projected − actual)	210	+2.11	3.89
Bullpen share, published staff (used by the KPI model)	181	-0.055	0.137
Bullpen share, full relief corps (effects document)	213	-0.063	0.133

Per-hitter KPI matchups

Metric	Hitter-games	MAE published	MAE adjusted	Direction right	Rate	Chance	Lift	Tough per PA	Favourable per PA	Separation	SE	Learned β	β status
H	1662	0.6837	0.6835	729/1427	0.511	0.493	+0.018	0.199	0.233	+0.035	0.012	1.000	measured scale confirmed: the interval includes 1.0
SO	1818	0.6795	0.6879	914/1790	0.511	0.498	+0.012	0.187	0.256	+0.069	0.012	0.449	learned: the interval excludes 1.0
BB	1818	0.4670	0.4628	887/1615	0.549	0.537	+0.012	0.078	0.091	+0.014	0.008	0.250	learned: the interval excludes 1.0
HR	1818	0.2193	0.2167	762/1164	0.655	0.626	+0.028	0.027	0.038	+0.010	0.005	1.750	learned: the interval excludes 1.0

Chance is the direction rate expected from the sign mix alone. Separation is the favourable third's rate per plate appearance minus the tough third's, bucketed by the whole-staff multiplier. Learned β scales the matchup adjustment in the effects document from the next slate on.

KPI features ranked by net runs pushed toward the result (worst first)

Feature	Seen	Scored	Pushed away rate	Net runs	Drift ±3σ
home_off_power_spread	77	77	0.58	-11.71	2
away_off_discipline	58	58	0.69	-7.82	2
home_platoon_barrel	64	64	0.66	-7.30	0
home_platoon_whiff	79	79	0.58	-7.21	0
home_off_ground_spread	69	69	0.58	-4.82	0
home_sp_suppress	59	59	0.58	-4.01	0
away_x_fastball	24	24	0.67	-3.95	7
away_sp_ground	62	62	0.61	-3.83	2
away_sp_whiff	36	36	0.67	-3.08	2
away_off_contact_spread	99	99	0.53	-3.03	0
home_x_fastball	36	36	0.61	-2.98	6
away_x_suppress	33	33	0.55	-2.54	8

Winner decision. Over 117 settled games the published winner was right 0.590 and the trained classifier 0.641. They disagreed 30 times; the classifier was right 0.600 of those (advantage +0.200, interval -0.103 to +0.500). Verdict: no_signal; the policy changes only on 40+ disagreements with an interval clear of zero.

What the system learned and did

Calibration snapshot for 20261007, built from 20 settled slates: H β 1.000 (measured scale confirmed: the interval includes 1.0), SO β 0.449 (learned: the interval excludes 1.0), BB β 0.250 (learned: the interval excludes 1.0), HR β 1.750 (learned: the interval excludes 1.0). Bullpen exposure method preferred: undecided.

KPI model: decision retrain (away_runs: settled evidence implies weight 0.35 against the model's 0.69) — retrained, new candidate 20261007T091423Z_47b9db81. Rollback check: kept.

Needs an operator

Finding	Owner	Priority	Action
starter length biased by +2.1 outs over 210 starts	model	medium	recalibrate the starter workload projection toward real outs per start
away_off_discipline pushed the KPI forecast away from the result in 69% of 58 scored club totals, net -7.8 runs	KPI model recipe	low	candidate for removal or rework in the next kpi_model version
home_platoon_barrel pushed the KPI forecast away from the result in 66% of 64 scored club totals, net -7.3 runs	KPI model recipe	low	candidate for removal or rework in the next kpi_model version
away_x_fastball was beyond ±3 standard deviations of its training range in 7 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
home_x_fastball was beyond ±3 standard deviations of its training range in 6 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_x_suppress was beyond ±3 standard deviations of its training range in 8 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_off_power_spread was beyond ±3 standard deviations of its training range in 8 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_x_ground was beyond ±3 standard deviations of its training range in 6 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_x_breaking was beyond ±3 standard deviations of its training range in 6 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
pitcher_strikeout_rate (elite) on SO: adjustments made SO worse over 210 hitter-games in 11 days, mean gain -0.0347, interval (-0.05984, -0.00582)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_home_run_rate_allowed (poor) on HR: adjustments made HR worse over 234 hitter-games in 11 days, mean gain -0.0148, interval (-0.01867, -0.01212)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
batter_strikeout_rate (average) on SO: adjustments made SO worse over 922 hitter-games in 13 days, mean gain -0.0095, interval (-0.01697, -0.00262)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_home_run_rate_allowed (weak) on HR: adjustments made HR worse over 271 hitter-games in 8 days, mean gain -0.0054, interval (-0.0069, -0.00353)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_strikeout_rate (average) on SO: adjustments made SO worse over 822 hitter-games in 13 days, mean gain -0.0032, interval (-0.00605, -1e-05)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_walk_rate (average) on BB: adjustments made BB worse over 880 hitter-games in 13 days, mean gain -0.0007, interval (-0.00159, -3e-05)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
ATL: KPI blend worse than the baseline over 11 games (MAE 2.462 vs 2.455, bias +0.299)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
HOU: KPI blend worse than the baseline over 8 games (MAE 2.902 vs 1.875, bias -2.073)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
MIA: KPI blend worse than the baseline over 8 games (MAE 1.74 vs 1.5, bias +0.134)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
ARI: KPI blend worse than the baseline over 8 games (MAE 3.175 vs 3.125, bias -1.975)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
COL: KPI blend worse than the baseline over 7 games (MAE 3.165 vs 2.857, bias +0.775)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
SF: KPI blend worse than the baseline over 8 games (MAE 1.861 vs 1.75, bias +1.054)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
LAD: KPI blend worse than the baseline over 10 games (MAE 2.776 vs 2.3, bias +1.513)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
WAS: KPI blend worse than the baseline over 6 games (MAE 2.081 vs 2.0, bias -0.588)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
STL: KPI blend worse than the baseline over 7 games (MAE 1.834 vs 1.571, bias +0.569)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
BOS: KPI blend worse than the baseline over 7 games (MAE 2.185 vs 1.714, bias +2.185)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
Yuki Matsui (SD): KPI adjustments hurt BB (gain -0.72 over 7 games, bias +0.57), HR (gain -0.01 over 7 games, bias +0.16), H (gain -0.12 over 6 games, bias +0.44)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jeremiah Jackson (BAL): KPI adjustments hurt SO (gain -0.69 over 5 games, bias -0.29)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Trea Turner (PHI): KPI adjustments hurt SO (gain -0.61 over 8 games, bias +0.34)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Kyle Schwarber (PHI): KPI adjustments hurt SO (gain -0.58 over 8 games, bias +0.91), BB (gain -0.17 over 8 games, bias -0.25)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Shay Whitcomb (SF): KPI adjustments hurt SO (gain -0.57 over 6 games, bias +0.19)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jonny DeLuca (TB): KPI adjustments hurt SO (gain -0.55 over 6 games, bias +0.06)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Coby Mayo (BAL): KPI adjustments hurt H (gain -0.13 over 5 games, bias -0.20), SO (gain -0.55 over 5 games, bias +0.20), BB (gain -0.05 over 5 games, bias -0.27)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Manny Machado (SD): KPI adjustments hurt SO (gain -0.53 over 11 games, bias +0.14), H (gain -0.13 over 10 games, bias -0.10)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ty France (SD): KPI adjustments hurt SO (gain -0.52 over 11 games, bias +0.14)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
George Lombard Jr. (NYY): KPI adjustments hurt SO (gain -0.48 over 8 games, bias -0.02), H (gain -0.17 over 7 games, bias -0.40)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jordan Walker (STL): KPI adjustments hurt SO (gain -0.48 over 7 games, bias -0.20)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Christian Walker (HOU): KPI adjustments hurt SO (gain -0.42 over 8 games, bias +0.52)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Tim Tawa (ARI): KPI adjustments hurt H (gain -0.42 over 7 games, bias +0.10)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Mike Trout (LAA): KPI adjustments hurt SO (gain -0.42 over 7 games, bias +0.34), BB (gain -0.25 over 7 games, bias -0.55), HR (gain -0.01 over 7 games, bias +0.03)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jac Caglianone (KC): KPI adjustments hurt SO (gain -0.41 over 6 games, bias +0.40), H (gain -0.13 over 5 games, bias -0.91)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ketel Marte (ARI): KPI adjustments hurt H (gain -0.41 over 7 games, bias +0.05)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Colson Montgomery (CHW): KPI adjustments hurt H (gain -0.28 over 6 games, bias +0.64), SO (gain -0.40 over 6 games, bias -0.66)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Yandy Díaz (TB): KPI adjustments hurt H (gain -0.21 over 5 games, bias -0.52), SO (gain -0.38 over 5 games, bias +0.88)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ozzie Albies (ATL): KPI adjustments hurt SO (gain -0.36 over 11 games, bias -0.34), BB (gain -0.19 over 11 games, bias -0.06)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Cam Smith (HOU): KPI adjustments hurt SO (gain -0.35 over 8 games, bias -0.16), HR (gain -0.04 over 8 games, bias +0.06), H (gain -0.08 over 7 games, bias +0.05)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Anthony Volpe (NYY): KPI adjustments hurt SO (gain -0.34 over 6 games, bias +0.14), H (gain -0.02 over 5 games, bias -0.17)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Bryce Harper (PHI): KPI adjustments hurt SO (gain -0.33 over 8 games, bias +0.05)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Kyle Isbel (KC): KPI adjustments hurt SO (gain -0.33 over 5 games, bias -0.24)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Adrian Morejon (SD): KPI adjustments hurt SO (gain -0.32 over 5 games, bias +0.83), HR (gain -0.03 over 5 games, bias -0.09)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Luke Keaschall (MIN): KPI adjustments hurt SO (gain -0.31 over 7 games, bias +0.31)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
pitcher_home_run_rate_allowed on HR (pitcher): harm survives leaving 1 worst players out, gain -0.0130, interval (-0.01672, -0.011)	KPI recipe	medium	the KPI itself misleads; Luna replacement candidates are scored in shadow