KPI learning cycle: 20261010

Settled pregame forecasts against official finals, a rolling self-assessment, the calibration it produced and the actions it took · 2026-10-11 09:12 UTC

Yesterday: where the forecast went wrong

Games settled

0 excluded · finals data/games.csv

Club-run MAE by stage

4.50 → 3.98

baseline → blend · KPI alone 3.02 · published 4.50

Layer moved toward result

2/2

club totals the layer moved

Player lines from

statsapi.mlb.com box scores

Game	Club	Actual	Baseline	KPI alone	Blend	Published	Card right	SP outs err	Pen share err	Feature that pushed hardest the wrong way	Dominant cause
CHW@CLE	CHW	0	5.00	3.22	4.39	5	–	–	–	–	KPI layer did not apply
CHW@CLE	CLE	2	6.00	4.83	5.56	6	–	–	–	–	KPI layer did not apply

Causes are checked in a fixed order and the first that applies is named: layer refused, wrong card (3+ of nine), starter length (±6 outs), bullpen share (±0.20), KPI moved away by 0.3+ runs, outlier (5+ runs from the baseline), then variance. Every cause that applied is kept in the settled document.

Bullpen: who the forecast expected against who pitched

Relievers who pitched, in the pregame bullpen

0.800

5 relievers over 2 club-games

Closer named = arm who finished

0.500

Probability it pitches

0.44 / 0.40

arms who pitched / stayed idle · Brier 0.214

Batters faced error per arm

2.51

Game	Club	Pitched and in pool	Pool	Pitched but not in pool	Closer right
CHW@CLE	CHW	2/2	7	-	yes
CHW@CLE	CLE	2/3	7	Cade Smith (5 BF)	no

Bullpen usage model: kept current · challenger holdout log loss 0.4580 vs current 0.4566 vs usage-only baseline 0.5093

Luna KPI review

Luna skipped_no_pregame_evidence. KPI-level faults 0, player-level faults 0, players flagged 3. New KPIs admitted 0, active candidates 10, promoted 0. Full review: 20261010_kpi_luna_review.html.

Per-player, per-club and per-KPI failures: 20261010_kpi_failures.html, every hitter and pitcher on the card scored against his real line, with the reason and size of each miss and the rolling ledger.

Self-assessment: 20 settled slates in the last 21 days

Blend gain over baseline

+0.019

90% [-0.091, +0.135] inconclusive

KPI alone gain over baseline

-0.029

90% [-0.218, +0.157] inconclusive

Club totals scored

220

moved toward 110/220

Blend weights: what the settled games imply

Target		Club totals	Model weight	Implied by evidence	Clamped 0–1
away_runs		110	0.68	0.34	0.34
home_runs		110	0.44	0.73	0.73

The implied weight is the least-squares fit of (actual − baseline) on (KPI − baseline) over the window's settled club totals. It is evidence for the next walk-forward refit, never applied directly.

Why club totals missed

Cause	Owner	Share	Count	Mean published error
KPI layer did not apply	operator: fix the missing input named in the reason	12%	28	2.32
Card named hitters who did not start	data feed: posted lineups arrive after the pregame run	12%	26	2.46
Starter length missed by two innings or more	model: starter workload projection	15%	33	2.21
Bullpen share missed by a fifth of the game or more	model: bullpen exposure method	4%	8	2.00
KPI layer moved the total away from the result	KPI model: weights, features or recipe	17%	38	2.76
Outlier game, five or more runs from any mean forecast	none: not forecastable	7%	16	5.25
Ordinary one-game variance	none	34%	77	1.92

Lineups, starters and bullpen exposure

Measure	n	Bias	Mean absolute error
Card hitters who did not start (of nine)	206	–	0.93
Starter outs (projected − actual)	191	+2.14	3.96
Bullpen share, published staff (used by the KPI model)	181	-0.055	0.137
Bullpen share, full relief corps (effects document)	194	-0.063	0.135

Per-hitter KPI matchups

Metric	Hitter-games	MAE published	MAE adjusted	Direction right	Rate	Chance	Lift	Tough per PA	Favourable per PA	Separation	SE	Learned β	β status
H	1662	0.6837	0.6835	729/1427	0.511	0.493	+0.018	0.199	0.233	+0.035	0.012	1.000	measured scale confirmed: the interval includes 1.0
SO	1662	0.6794	0.6882	821/1634	0.502	0.498	+0.005	0.189	0.258	+0.069	0.013	0.447	learned: the interval excludes 1.0
BB	1662	0.4648	0.4605	808/1474	0.548	0.535	+0.013	0.077	0.092	+0.015	0.008	0.304	learned: the interval excludes 1.0
HR	1662	0.2184	0.2159	699/1075	0.650	0.619	+0.032	0.027	0.038	+0.011	0.005	1.750	learned: the interval excludes 1.0

Chance is the direction rate expected from the sign mix alone. Separation is the favourable third's rate per plate appearance minus the tough third's, bucketed by the whole-staff multiplier. Learned β scales the matchup adjustment in the effects document from the next slate on.

KPI features ranked by net runs pushed toward the result (worst first)

Feature	Seen	Scored	Pushed away rate	Net runs	Drift ±3σ
home_off_power_spread	73	73	0.56	-9.35	2
away_off_discipline	56	56	0.68	-6.71	2
home_platoon_barrel	64	64	0.62	-4.65	0
home_sp_suppress	55	55	0.58	-3.90	0
home_off_ground_spread	61	61	0.56	-3.55	0
home_platoon_whiff	75	75	0.53	-3.36	0
away_x_fastball	22	22	0.64	-3.30	7
away_off_worst3_discipline	31	31	0.58	-2.98	1
away_sp_ground	61	61	0.59	-2.92	2
away_sp_whiff	35	35	0.66	-2.86	2
away_pen_share	36	36	0.56	-2.73	4
away_x_suppress	33	33	0.55	-2.54	8

Winner decision. Over 113 settled games the published winner was right 0.584 and the trained classifier 0.655. They disagreed 28 times; the classifier was right 0.643 of those (advantage +0.286, interval +0.040 to +0.545). Verdict: no_signal; the policy changes only on 40+ disagreements with an interval clear of zero.

What the system learned and did

Calibration snapshot for 20261011, built from 20 settled slates: H β 1.000 (measured scale confirmed: the interval includes 1.0), SO β 0.447 (learned: the interval excludes 1.0), BB β 0.304 (learned: the interval excludes 1.0), HR β 1.750 (learned: the interval excludes 1.0). Bullpen exposure method preferred: undecided.

KPI model: decision retrain (away_runs: settled evidence implies weight 0.34 against the model's 0.68; home_runs: settled evidence implies weight 0.73 against the model's 0.44) — retrained, new candidate 20261011T091221Z_bb8e6dd2. Rollback check: kept.

Needs an operator

Finding	Owner	Priority	Action
starter length biased by +2.1 outs over 191 starts	model	medium	recalibrate the starter workload projection toward real outs per start
away_off_discipline pushed the KPI forecast away from the result in 68% of 56 scored club totals, net -6.7 runs	KPI model recipe	low	candidate for removal or rework in the next kpi_model version
home_platoon_barrel pushed the KPI forecast away from the result in 62% of 64 scored club totals, net -4.6 runs	KPI model recipe	low	candidate for removal or rework in the next kpi_model version
away_x_fastball was beyond ±3 standard deviations of its training range in 7 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_x_suppress was beyond ±3 standard deviations of its training range in 8 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_off_power_spread was beyond ±3 standard deviations of its training range in 8 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_x_ground was beyond ±3 standard deviations of its training range in 6 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_x_breaking was beyond ±3 standard deviations of its training range in 6 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
pitcher_strikeout_rate (elite) on SO: adjustments made SO worse over 188 hitter-games in 10 days, mean gain -0.0327, interval (-0.05982, -0.00164)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_home_run_rate_allowed (poor) on HR: adjustments made HR worse over 218 hitter-games in 10 days, mean gain -0.0148, interval (-0.01888, -0.01168)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
batter_strikeout_rate (average) on SO: adjustments made SO worse over 846 hitter-games in 12 days, mean gain -0.0089, interval (-0.01766, -0.00174)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_home_run_rate_allowed (weak) on HR: adjustments made HR worse over 248 hitter-games in 7 days, mean gain -0.0058, interval (-0.00747, -0.00414)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_strikeout_rate (average) on SO: adjustments made SO worse over 767 hitter-games in 12 days, mean gain -0.0037, interval (-0.00669, -0.00083)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
MIA: KPI blend worse than the baseline over 7 games (MAE 1.697 vs 1.571, bias +0.444)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
ARI: KPI blend worse than the baseline over 7 games (MAE 3.219 vs 3.143, bias -1.847)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
SF: KPI blend worse than the baseline over 7 games (MAE 1.861 vs 1.714, bias +1.47)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
LAD: KPI blend worse than the baseline over 9 games (MAE 2.631 vs 2.111, bias +2.135)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
COL: KPI blend worse than the baseline over 6 games (MAE 3.673 vs 3.333, bias +0.924)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
BOS: KPI blend worse than the baseline over 7 games (MAE 2.185 vs 1.714, bias +2.185)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
HOU: KPI blend worse than the baseline over 7 games (MAE 3.176 vs 2.143, bias -2.509)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
SEA: KPI blend worse than the baseline over 6 games (MAE 2.39 vs 2.333, bias -0.345)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
STL: KPI blend worse than the baseline over 6 games (MAE 1.884 vs 1.667, bias +0.919)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
Jeremiah Jackson (BAL): KPI adjustments hurt SO (gain -0.69 over 5 games, bias -0.29)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Yuki Matsui (SD): KPI adjustments hurt H (gain -0.12 over 6 games, bias +0.44), BB (gain -0.69 over 6 games, bias +0.63), HR (gain -0.02 over 6 games, bias +0.15)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Trea Turner (PHI): KPI adjustments hurt SO (gain -0.61 over 8 games, bias +0.34)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Kyle Schwarber (PHI): KPI adjustments hurt SO (gain -0.58 over 8 games, bias +0.91), BB (gain -0.17 over 8 games, bias -0.25)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jonny DeLuca (TB): KPI adjustments hurt SO (gain -0.55 over 6 games, bias +0.06)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Coby Mayo (BAL): KPI adjustments hurt H (gain -0.13 over 5 games, bias -0.20), SO (gain -0.55 over 5 games, bias +0.20), BB (gain -0.05 over 5 games, bias -0.27)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jordan Walker (STL): KPI adjustments hurt SO (gain -0.53 over 6 games, bias +0.18)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Christian Walker (HOU): KPI adjustments hurt SO (gain -0.45 over 7 games, bias +0.47)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Tim Tawa (ARI): KPI adjustments hurt H (gain -0.42 over 7 games, bias +0.10)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ty France (SD): KPI adjustments hurt SO (gain -0.42 over 10 games, bias +0.03)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ketel Marte (ARI): KPI adjustments hurt H (gain -0.41 over 7 games, bias +0.05)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Colson Montgomery (CHW): KPI adjustments hurt H (gain -0.28 over 6 games, bias +0.64), SO (gain -0.40 over 6 games, bias -0.66)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jac Caglianone (KC): KPI adjustments hurt H (gain -0.13 over 5 games, bias -0.91), SO (gain -0.39 over 5 games, bias +0.20)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Drake Baldwin (ATL): KPI adjustments hurt SO (gain -0.38 over 9 games, bias +0.51), HR (gain -0.01 over 9 games, bias +0.05)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Yandy Díaz (TB): KPI adjustments hurt H (gain -0.21 over 5 games, bias -0.52), SO (gain -0.38 over 5 games, bias +0.88)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
George Lombard Jr. (NYY): KPI adjustments hurt H (gain -0.17 over 7 games, bias -0.40), SO (gain -0.36 over 7 games, bias +0.02)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Heriberto Hernández (MIA): KPI adjustments hurt H (gain -0.10 over 7 games, bias +0.19), SO (gain -0.36 over 7 games, bias +0.43), BB (gain -0.12 over 7 games, bias -0.03)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ozzie Albies (ATL): KPI adjustments hurt SO (gain -0.35 over 10 games, bias -0.34), BB (gain -0.19 over 10 games, bias -0.10)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Bryce Harper (PHI): KPI adjustments hurt SO (gain -0.33 over 8 games, bias +0.05)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Cam Smith (HOU): KPI adjustments hurt H (gain -0.08 over 7 games, bias +0.05), SO (gain -0.32 over 7 games, bias -0.05), HR (gain -0.04 over 7 games, bias +0.05)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Corbin Carroll (ARI): KPI adjustments hurt H (gain -0.30 over 7 games, bias +0.19)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Danny Jansen (TEX): KPI adjustments hurt SO (gain -0.29 over 6 games, bias +0.40)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Christian Moore (LAA): KPI adjustments hurt SO (gain -0.28 over 7 games, bias +0.06), HR (gain -0.02 over 7 games, bias -0.03)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Carson Benge (NYM): KPI adjustments hurt H (gain -0.13 over 6 games, bias -0.64), SO (gain -0.28 over 6 games, bias +0.04)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Taylor Trammell (HOU): KPI adjustments hurt H (gain -0.10 over 5 games, bias +0.71), SO (gain -0.28 over 5 games, bias -0.42), HR (gain -0.03 over 5 games, bias +0.12)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change