KPI learning cycle: 20261005

Settled pregame forecasts against official finals, a rolling self-assessment, the calibration it produced and the actions it took · 2026-10-06 15:04 UTC

Yesterday: where the forecast went wrong

Games settled

0 excluded · finals data/games.csv

Club-run MAE by stage

– → –

baseline → blend · KPI alone – · published 1.75

Layer moved toward result

0/0

club totals the layer moved

Player lines from

statsapi.mlb.com box scores

Game	Club	Actual	Baseline	KPI alone	Blend	Published	Card right	SP outs err	Pen share err	Feature that pushed hardest the wrong way	Dominant cause
CHW@CLE	CHW	4	–	–	–	4	–	–	–	–	KPI layer did not apply
CHW@CLE	CLE	3	–	–	–	5	–	–	–	–	KPI layer did not apply
NYY@TB	NYY	2	–	–	–	4	–	–	–	–	KPI layer did not apply
NYY@TB	TB	5	–	–	–	2	–	–	–	–	KPI layer did not apply

Causes are checked in a fixed order and the first that applies is named: layer refused, wrong card (3+ of nine), starter length (±6 outs), bullpen share (±0.20), KPI moved away by 0.3+ runs, outlier (5+ runs from the baseline), then variance. Every cause that applied is kept in the settled document.

Bullpen: who the forecast expected against who pitched

Relievers who pitched, in the pregame bullpen

–

0 relievers over 0 club-games

Closer named = arm who finished

–

Probability it pitches

– / –

arms who pitched / stayed idle · Brier –

Batters faced error per arm

–

Game	Club	Pitched and in pool	Pool	Pitched but not in pool	Closer right

Bullpen usage model: kept current (the next slate has started generating)

Luna KPI review

Luna skipped_no_pregame_evidence. KPI-level faults 4, player-level faults 2, players flagged 16. New KPIs admitted 0, active candidates 10, promoted 0. Full review: 20261005_kpi_luna_review.html.

Per-player, per-club and per-KPI failures: 20261005_kpi_failures.html, every hitter and pitcher on the card scored against his real line, with the reason and size of each miss and the rolling ledger.

Self-assessment: 20 settled slates in the last 21 days

Blend gain over baseline

+0.025

90% [-0.086, +0.139] inconclusive

KPI alone gain over baseline

-0.052

90% [-0.240, +0.137] inconclusive

Club totals scored

224

moved toward 112/224

Blend weights: what the settled games imply

Target		Club totals	Model weight	Implied by evidence	Clamped 0–1
away_runs		112	0.69	0.35	0.35
home_runs		112	0.44	0.68	0.68

The implied weight is the least-squares fit of (actual − baseline) on (KPI − baseline) over the window's settled club totals. It is evidence for the next walk-forward refit, never applied directly.

Why club totals missed

Cause	Owner	Share	Count	Mean published error
KPI layer did not apply	operator: fix the missing input named in the reason	22%	56	2.50
Card named hitters who did not start	data feed: posted lineups arrive after the pregame run	10%	26	2.46
Starter length missed by two innings or more	model: starter workload projection	14%	36	2.14
Bullpen share missed by a fifth of the game or more	model: bullpen exposure method	3%	8	2.00
KPI layer moved the total away from the result	KPI model: weights, features or recipe	15%	40	2.65
Outlier game, five or more runs from any mean forecast	none: not forecastable	7%	17	5.24
Ordinary one-game variance	none	30%	77	1.83

Lineups, starters and bullpen exposure

Measure	n	Bias	Mean absolute error
Card hitters who did not start (of nine)	256	–	1.02
Starter outs (projected − actual)	239	+1.94	3.75
Bullpen share, published staff (used by the KPI model)	181	-0.055	0.137
Bullpen share, full relief corps (effects document)	243	-0.054	0.133

Per-hitter KPI matchups

Metric	Hitter-games	MAE published	MAE adjusted	Direction right	Rate	Chance	Lift	Tough per PA	Favourable per PA	Separation	SE	Learned β	β status
H	1662	0.6837	0.6835	729/1427	0.511	0.493	+0.018	0.199	0.233	+0.035	0.012	1.000	measured scale confirmed: the interval includes 1.0
SO	2044	0.6781	0.6871	1021/2008	0.508	0.498	+0.010	0.188	0.259	+0.071	0.011	0.427	learned: the interval excludes 1.0
BB	2044	0.4662	0.4620	1008/1835	0.549	0.537	+0.013	0.075	0.091	+0.016	0.007	0.250	learned: the interval excludes 1.0
HR	2044	0.2216	0.2191	858/1319	0.650	0.621	+0.030	0.028	0.039	+0.011	0.005	1.750	learned: the interval excludes 1.0

Chance is the direction rate expected from the sign mix alone. Separation is the favourable third's rate per plate appearance minus the tough third's, bucketed by the whole-staff multiplier. Learned β scales the matchup adjustment in the effects document from the next slate on.

KPI features ranked by net runs pushed toward the result (worst first)

Feature	Seen	Scored	Pushed away rate	Net runs	Drift ±3σ
home_off_power_spread	76	76	0.59	-12.21	2
home_platoon_barrel	62	62	0.68	-8.34	0
away_off_discipline	57	57	0.68	-7.34	2
home_platoon_whiff	79	79	0.58	-7.21	0
home_off_ground_spread	69	69	0.58	-4.82	0
home_sp_suppress	58	58	0.59	-4.30	0
away_x_fastball	24	24	0.67	-3.95	7
away_sp_ground	62	62	0.61	-3.83	2
away_off_contact_spread	97	97	0.54	-3.72	0
home_x_fastball	36	36	0.61	-2.98	6
away_x_suppress	32	32	0.56	-2.86	8
home_off_top3_contact	66	66	0.58	-2.75	0

Winner decision. Over 130 settled games the published winner was right 0.561 and the trained classifier 0.623. They disagreed 30 times; the classifier was right 0.633 of those (advantage +0.267, interval -0.029 to +0.583). Verdict: no_signal; the policy changes only on 40+ disagreements with an interval clear of zero.

What the system learned and did

Calibration snapshot for 20261006, built from 20 settled slates: H β 1.000 (measured scale confirmed: the interval includes 1.0), SO β 0.427 (learned: the interval excludes 1.0), BB β 0.250 (learned: the interval excludes 1.0), HR β 1.750 (learned: the interval excludes 1.0). Bullpen exposure method preferred: undecided.

KPI model: decision retrain (weekly cadence; away_runs: settled evidence implies weight 0.35 against the model's 0.69) — not executed: today's slate has already started generating. Rollback check: kept.

Needs an operator

Finding	Owner	Priority	Action
starter length biased by +1.9 outs over 239 starts	model	medium	recalibrate the starter workload projection toward real outs per start
home_platoon_barrel pushed the KPI forecast away from the result in 68% of 62 scored club totals, net -8.3 runs	KPI model recipe	low	candidate for removal or rework in the next kpi_model version
away_off_discipline pushed the KPI forecast away from the result in 68% of 57 scored club totals, net -7.3 runs	KPI model recipe	low	candidate for removal or rework in the next kpi_model version
away_x_fastball was beyond ±3 standard deviations of its training range in 7 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
home_x_fastball was beyond ±3 standard deviations of its training range in 6 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_x_suppress was beyond ±3 standard deviations of its training range in 8 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_off_power_spread was beyond ±3 standard deviations of its training range in 8 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_x_ground was beyond ±3 standard deviations of its training range in 6 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_x_breaking was beyond ±3 standard deviations of its training range in 6 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
pitcher_strikeout_rate (elite) on SO: adjustments made SO worse over 226 hitter-games in 12 days, mean gain -0.0401, interval (-0.06725, -0.01307)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_home_run_rate_allowed (poor) on HR: adjustments made HR worse over 259 hitter-games in 12 days, mean gain -0.0142, interval (-0.01776, -0.01148)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_walk_rate (weak) on BB: adjustments made BB worse over 162 hitter-games in 7 days, mean gain -0.0106, interval (-0.01953, -0.00226)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
batter_strikeout_rate (average) on SO: adjustments made SO worse over 1032 hitter-games in 14 days, mean gain -0.0101, interval (-0.01746, -0.00353)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_home_run_rate_allowed (weak) on HR: adjustments made HR worse over 331 hitter-games in 9 days, mean gain -0.0049, interval (-0.00669, -0.00313)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_strikeout_rate (average) on SO: adjustments made SO worse over 910 hitter-games in 14 days, mean gain -0.0038, interval (-0.00645, -0.00097)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_walk_rate (average) on BB: adjustments made BB worse over 957 hitter-games in 14 days, mean gain -0.0009, interval (-0.00174, -0.0002)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
ATL: KPI blend worse than the baseline over 12 games (MAE 2.462 vs 2.455, bias +0.299)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
BAL: KPI blend worse than the baseline over 6 games (MAE 2.227 vs 2.2, bias -0.397)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
BOS: KPI blend worse than the baseline over 8 games (MAE 2.185 vs 1.714, bias +2.185)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
DET: KPI blend worse than the baseline over 6 games (MAE 2.099 vs 2.0, bias +0.464)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
HOU: KPI blend worse than the baseline over 9 games (MAE 2.902 vs 1.875, bias -2.073)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
LAD: KPI blend worse than the baseline over 11 games (MAE 2.776 vs 2.3, bias +1.513)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
MIA: KPI blend worse than the baseline over 9 games (MAE 1.74 vs 1.5, bias +0.134)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
ARI: KPI blend worse than the baseline over 9 games (MAE 3.175 vs 3.125, bias -1.975)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
WAS: KPI blend worse than the baseline over 7 games (MAE 2.081 vs 2.0, bias -0.588)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
COL: KPI blend worse than the baseline over 8 games (MAE 3.165 vs 2.857, bias +0.775)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
SF: KPI blend worse than the baseline over 9 games (MAE 1.861 vs 1.75, bias +1.054)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
STL: KPI blend worse than the baseline over 8 games (MAE 1.834 vs 1.571, bias +0.569)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
Rafael Flores Jr. (PIT): KPI adjustments hurt SO (gain -0.79 over 6 games, bias +0.57), BB (gain -0.01 over 6 games, bias -0.02)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Yuki Matsui (SD): KPI adjustments hurt BB (gain -0.72 over 7 games, bias +0.57), HR (gain -0.01 over 7 games, bias +0.16), H (gain -0.12 over 6 games, bias +0.44)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jeremiah Jackson (BAL): KPI adjustments hurt SO (gain -0.69 over 6 games, bias -0.42)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Mike Trout (LAA): KPI adjustments hurt SO (gain -0.60 over 8 games, bias +0.35), BB (gain -0.31 over 8 games, bias -0.56), HR (gain -0.03 over 8 games, bias +0.04)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Shay Whitcomb (SF): KPI adjustments hurt SO (gain -0.57 over 6 games, bias +0.19)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Trea Turner (PHI): KPI adjustments hurt SO (gain -0.56 over 9 games, bias +0.38)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jonny DeLuca (TB): KPI adjustments hurt SO (gain -0.55 over 6 games, bias +0.06)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Coby Mayo (BAL): KPI adjustments hurt SO (gain -0.55 over 6 games, bias +0.26), BB (gain -0.02 over 6 games, bias -0.16), H (gain -0.13 over 5 games, bias -0.20)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Calvin Faucher (MIA): KPI adjustments hurt BB (gain -0.54 over 5 games, bias +0.48), HR (gain -0.01 over 5 games, bias +0.02)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Cade Smith (CLE): KPI adjustments hurt SO (gain -0.53 over 5 games, bias +0.82), HR (gain -0.03 over 5 games, bias +0.08)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Kyle Schwarber (PHI): KPI adjustments hurt SO (gain -0.52 over 9 games, bias +0.93), BB (gain -0.05 over 9 games, bias -0.26)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ty France (SD): KPI adjustments hurt SO (gain -0.52 over 11 games, bias +0.14)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jordan Walker (STL): KPI adjustments hurt SO (gain -0.51 over 8 games, bias -0.22)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Christian Walker (HOU): KPI adjustments hurt SO (gain -0.46 over 9 games, bias +0.32)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Erik Sabrowski (CLE): KPI adjustments hurt SO (gain -0.45 over 5 games, bias -0.04), HR (gain -0.05 over 5 games, bias +0.12)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Christian Moore (LAA): KPI adjustments hurt SO (gain -0.45 over 8 games, bias +0.09), HR (gain -0.03 over 8 games, bias -0.02)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ryan O'Hearn (PIT): KPI adjustments hurt SO (gain -0.44 over 7 games, bias -0.23), BB (gain -0.02 over 7 games, bias -0.02), H (gain -0.01 over 5 games, bias -0.23)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Drake Baldwin (ATL): KPI adjustments hurt SO (gain -0.43 over 11 games, bias +0.44)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Tim Tawa (ARI): KPI adjustments hurt H (gain -0.42 over 7 games, bias +0.10)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Vaughn Grissom (LAA): KPI adjustments hurt SO (gain -0.41 over 9 games, bias +0.52), HR (gain -0.04 over 9 games, bias -0.07)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ketel Marte (ARI): KPI adjustments hurt H (gain -0.41 over 7 games, bias +0.05)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ozzie Albies (ATL): KPI adjustments hurt SO (gain -0.40 over 12 games, bias -0.26), BB (gain -0.15 over 12 games, bias -0.03)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Colson Montgomery (CHW): KPI adjustments hurt H (gain -0.28 over 6 games, bias +0.64), SO (gain -0.40 over 6 games, bias -0.66)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Trevor Megill (MIL): KPI adjustments hurt SO (gain -0.40 over 5 games, bias +1.21), BB (gain -0.04 over 5 games, bias +0.46)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Trevor Story (BOS): KPI adjustments hurt SO (gain -0.39 over 5 games, bias +0.13)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
pitcher_home_run_rate_allowed on HR (pitcher): harm survives leaving 1 worst players out, gain -0.0124, interval (-0.01547, -0.01038)	KPI recipe	medium	the KPI itself misleads; Luna replacement candidates are scored in shadow
batter_strikeout_rate on SO (batter): harm survives leaving 3 worst players out, gain -0.0117, interval (-0.02087, -0.00364)	KPI recipe	medium	the KPI itself misleads; Luna replacement candidates are scored in shadow
batter_strikeout_rate on SO (batter): harm survives leaving 3 worst players out, gain -0.0074, interval (-0.01356, -0.00136)	KPI recipe	medium	the KPI itself misleads; Luna replacement candidates are scored in shadow
pitcher_home_run_rate_allowed on HR (pitcher): harm survives leaving 2 worst players out, gain -0.0037, interval (-0.00539, -0.00188)	KPI recipe	medium	the KPI itself misleads; Luna replacement candidates are scored in shadow