KPI learning cycle: 20261011

Settled pregame forecasts against official finals, a rolling self-assessment, the calibration it produced and the actions it took · 2026-10-12 09:15 UTC

Yesterday: where the forecast went wrong

Games settled

0 excluded · finals data/games.csv

Club-run MAE by stage

2.00 → 2.19

baseline → blend · KPI alone 2.41 · published 1.50

Layer moved toward result

1/2

club totals the layer moved

Player lines from

statsapi.mlb.com box scores

Game	Club	Actual	Baseline	KPI alone	Blend	Published	Card right	SP outs err	Pen share err	Feature that pushed hardest the wrong way	Dominant cause
LAD@MIL	LAD	1	3.00	1.15	2.37	2	–	–	–	–	KPI layer did not apply
LAD@MIL	MIL	2	4.00	6.66	5.01	4	–	–	–	–	KPI layer did not apply

Causes are checked in a fixed order and the first that applies is named: layer refused, wrong card (3+ of nine), starter length (±6 outs), bullpen share (±0.20), KPI moved away by 0.3+ runs, outlier (5+ runs from the baseline), then variance. Every cause that applied is kept in the settled document.

Bullpen: who the forecast expected against who pitched

Relievers who pitched, in the pregame bullpen

1.000

7 relievers over 2 club-games

Closer named = arm who finished

1.000

Probability it pitches

0.48 / 0.39

arms who pitched / stayed idle · Brier 0.227

Batters faced error per arm

2.22

Game	Club	Pitched and in pool	Pool	Pitched but not in pool	Closer right
LAD@MIL	LAD	3/3	7	-	yes
LAD@MIL	MIL	4/4	7	-	yes

Bullpen usage model: promoted 20261012T091057Z_861ccdee · challenger holdout log loss 0.4576 vs current 0.4576 vs usage-only baseline 0.5071

Luna KPI review

Luna skipped_no_pregame_evidence. KPI-level faults 0, player-level faults 0, players flagged 0. New KPIs admitted 0, active candidates 10, promoted 0. Full review: 20261011_kpi_luna_review.html.

Per-player, per-club and per-KPI failures: 20261011_kpi_failures.html, every hitter and pitcher on the card scored against his real line, with the reason and size of each miss and the rolling ledger.

Self-assessment: 20 settled slates in the last 21 days

Blend gain over baseline

+0.004

90% [-0.105, +0.113] inconclusive

KPI alone gain over baseline

-0.037

90% [-0.230, +0.159] inconclusive

Club totals scored

212

moved toward 106/212

Blend weights: what the settled games imply

Target		Club totals	Model weight	Implied by evidence	Clamped 0–1
away_runs		106	0.68	0.28	0.28
home_runs		106	0.44	0.73	0.73

The implied weight is the least-squares fit of (actual − baseline) on (KPI − baseline) over the window's settled club totals. It is evidence for the next walk-forward refit, never applied directly.

Why club totals missed

Cause	Owner	Share	Count	Mean published error
KPI layer did not apply	operator: fix the missing input named in the reason	13%	28	2.21
Card named hitters who did not start	data feed: posted lineups arrive after the pregame run	12%	26	2.46
Starter length missed by two innings or more	model: starter workload projection	15%	32	2.25
Bullpen share missed by a fifth of the game or more	model: bullpen exposure method	4%	8	2.00
KPI layer moved the total away from the result	KPI model: weights, features or recipe	16%	35	2.80
Outlier game, five or more runs from any mean forecast	none: not forecastable	7%	15	5.33
Ordinary one-game variance	none	34%	74	1.91

Lineups, starters and bullpen exposure

Measure	n	Bias	Mean absolute error
Card hitters who did not start (of nine)	196	–	0.96
Starter outs (projected − actual)	182	+2.18	3.94
Bullpen share, published staff (used by the KPI model)	181	-0.055	0.137
Bullpen share, full relief corps (effects document)	185	-0.065	0.135

Per-hitter KPI matchups

Metric	Hitter-games	MAE published	MAE adjusted	Direction right	Rate	Chance	Lift	Tough per PA	Favourable per PA	Separation	SE	Learned β	β status
H	1575	0.6837	0.6840	690/1358	0.508	0.493	+0.015	0.201	0.232	+0.031	0.013	1.000	measured scale confirmed: the interval includes 1.0
SO	1575	0.6810	0.6896	781/1547	0.505	0.497	+0.008	0.185	0.258	+0.072	0.013	0.490	learned: the interval excludes 1.0
BB	1575	0.4673	0.4630	781/1421	0.550	0.534	+0.016	0.078	0.094	+0.016	0.009	0.315	learned: the interval excludes 1.0
HR	1575	0.2183	0.2157	674/1029	0.655	0.620	+0.035	0.027	0.039	+0.012	0.006	1.750	learned: the interval excludes 1.0

Chance is the direction rate expected from the sign mix alone. Separation is the favourable third's rate per plate appearance minus the tough third's, bucketed by the whole-staff multiplier. Learned β scales the matchup adjustment in the effects document from the next slate on.

KPI features ranked by net runs pushed toward the result (worst first)

Feature	Seen	Scored	Pushed away rate	Net runs	Drift ±3σ
home_off_power_spread	71	71	0.55	-8.52	2
away_off_discipline	54	54	0.67	-6.13	2
home_sp_suppress	49	49	0.59	-3.67	0
home_platoon_barrel	61	61	0.61	-3.61	0
home_platoon_whiff	72	72	0.54	-3.37	0
away_x_fastball	22	22	0.64	-3.30	7
away_sp_ground	58	58	0.60	-3.29	2
away_off_worst3_discipline	31	31	0.58	-2.98	1
away_sp_whiff	35	35	0.66	-2.86	2
away_pen_share	36	36	0.56	-2.73	4
home_off_ground_spread	58	58	0.53	-2.57	0
away_x_suppress	33	33	0.55	-2.54	8

Winner decision. Over 109 settled games the published winner was right 0.587 and the trained classifier 0.642. They disagreed 26 times; the classifier was right 0.615 of those (advantage +0.231, interval -0.037 to +0.500). Verdict: no_signal; the policy changes only on 40+ disagreements with an interval clear of zero.

What the system learned and did

Calibration snapshot for 20261012, built from 20 settled slates: H β 1.000 (measured scale confirmed: the interval includes 1.0), SO β 0.490 (learned: the interval excludes 1.0), BB β 0.315 (learned: the interval excludes 1.0), HR β 1.750 (learned: the interval excludes 1.0). Bullpen exposure method preferred: undecided.

KPI model: decision retrain (away_runs: settled evidence implies weight 0.28 against the model's 0.68; home_runs: settled evidence implies weight 0.73 against the model's 0.44) — retrained, new candidate 20261012T091527Z_e6f3fdba. Rollback check: kept.

Needs an operator

Finding	Owner	Priority	Action
starter length biased by +2.2 outs over 182 starts	model	medium	recalibrate the starter workload projection toward real outs per start
away_off_discipline pushed the KPI forecast away from the result in 67% of 54 scored club totals, net -6.1 runs	KPI model recipe	low	candidate for removal or rework in the next kpi_model version
home_platoon_barrel pushed the KPI forecast away from the result in 61% of 61 scored club totals, net -3.6 runs	KPI model recipe	low	candidate for removal or rework in the next kpi_model version
away_x_fastball was beyond ±3 standard deviations of its training range in 7 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_x_suppress was beyond ±3 standard deviations of its training range in 8 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_off_power_spread was beyond ±3 standard deviations of its training range in 8 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_x_ground was beyond ±3 standard deviations of its training range in 6 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
away_x_breaking was beyond ±3 standard deviations of its training range in 6 club totals	KPI model recipe	medium	the live feature is built differently from the training rows; align the two constructions and retrain
pitcher_home_run_rate_allowed (poor) on HR: adjustments made HR worse over 200 hitter-games in 9 days, mean gain -0.0151, interval (-0.01922, -0.01174)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
batter_strikeout_rate (average) on SO: adjustments made SO worse over 798 hitter-games in 11 days, mean gain -0.0089, interval (-0.01814, -0.00122)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_home_run_rate_allowed (weak) on HR: adjustments made HR worse over 248 hitter-games in 7 days, mean gain -0.0058, interval (-0.00747, -0.00414)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
pitcher_strikeout_rate (average) on SO: adjustments made SO worse over 723 hitter-games in 11 days, mean gain -0.0041, interval (-0.00728, -0.00157)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
derived_pitcher_hit_rate_allowed (weak) on H: adjustments made H worse over 313 hitter-games in 7 days, mean gain -0.0033, interval (-0.00856, -0.00018)	KPI recipe	medium	review this KPI's weight in the matchup multiplier; it is an operator change to the recipe
SF: KPI blend worse than the baseline over 6 games (MAE 1.813 vs 1.667, bias +1.357)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
ARI: KPI blend worse than the baseline over 6 games (MAE 3.1 vs 3.0, bias -1.5)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
COL: KPI blend worse than the baseline over 6 games (MAE 3.673 vs 3.333, bias +0.924)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
BOS: KPI blend worse than the baseline over 7 games (MAE 2.185 vs 1.714, bias +2.185)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
HOU: KPI blend worse than the baseline over 7 games (MAE 3.176 vs 2.143, bias -2.509)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
SEA: KPI blend worse than the baseline over 6 games (MAE 2.39 vs 2.333, bias -0.345)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
MIA: KPI blend worse than the baseline over 6 games (MAE 1.892 vs 1.833, bias +0.606)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
MIL: KPI blend worse than the baseline over 8 games (MAE 1.645 vs 1.375, bias +0.201)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
LAD: KPI blend worse than the baseline over 8 games (MAE 2.562 vs 1.875, bias +2.004)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
STL: KPI blend worse than the baseline over 6 games (MAE 1.884 vs 1.667, bias +0.919)	KPI model	medium	check the wrong-way features listed for this club and its data coverage
Yuki Matsui (SD): KPI adjustments hurt H (gain -0.12 over 6 games, bias +0.44), BB (gain -0.69 over 6 games, bias +0.63), HR (gain -0.02 over 6 games, bias +0.15)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Trea Turner (PHI): KPI adjustments hurt SO (gain -0.61 over 8 games, bias +0.34)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Kyle Schwarber (PHI): KPI adjustments hurt SO (gain -0.58 over 8 games, bias +0.91), BB (gain -0.17 over 8 games, bias -0.25)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jonny DeLuca (TB): KPI adjustments hurt SO (gain -0.55 over 6 games, bias +0.06)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jordan Walker (STL): KPI adjustments hurt SO (gain -0.53 over 6 games, bias +0.18)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Gabriel Moreno (ARI): KPI adjustments hurt H (gain -0.47 over 5 games, bias -0.20)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Christian Walker (HOU): KPI adjustments hurt SO (gain -0.45 over 7 games, bias +0.47)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ketel Marte (ARI): KPI adjustments hurt H (gain -0.42 over 6 games, bias +0.23)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Tim Tawa (ARI): KPI adjustments hurt H (gain -0.40 over 6 games, bias -0.04)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Colson Montgomery (CHW): KPI adjustments hurt H (gain -0.28 over 6 games, bias +0.64), SO (gain -0.40 over 6 games, bias -0.66)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Jac Caglianone (KC): KPI adjustments hurt H (gain -0.13 over 5 games, bias -0.91), SO (gain -0.39 over 5 games, bias +0.20)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
George Lombard Jr. (NYY): KPI adjustments hurt H (gain -0.15 over 6 games, bias -0.28), SO (gain -0.38 over 6 games, bias -0.12)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Drake Baldwin (ATL): KPI adjustments hurt SO (gain -0.38 over 9 games, bias +0.51), HR (gain -0.01 over 9 games, bias +0.05)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Yandy Díaz (TB): KPI adjustments hurt H (gain -0.21 over 5 games, bias -0.52), SO (gain -0.38 over 5 games, bias +0.88)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ozzie Albies (ATL): KPI adjustments hurt SO (gain -0.35 over 10 games, bias -0.34), BB (gain -0.19 over 10 games, bias -0.10)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Christian Moore (LAA): KPI adjustments hurt SO (gain -0.34 over 6 games, bias +0.22)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Heriberto Hernández (MIA): KPI adjustments hurt H (gain -0.09 over 6 games, bias +0.09), SO (gain -0.34 over 6 games, bias +0.49)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Bryce Harper (PHI): KPI adjustments hurt SO (gain -0.33 over 8 games, bias +0.05)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Cam Smith (HOU): KPI adjustments hurt H (gain -0.08 over 7 games, bias +0.05), SO (gain -0.32 over 7 games, bias -0.05), HR (gain -0.04 over 7 games, bias +0.05)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Corbin Carroll (ARI): KPI adjustments hurt H (gain -0.32 over 6 games, bias +0.24)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Ty France (SD): KPI adjustments hurt SO (gain -0.31 over 9 games, bias +0.07)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Kody Clemens (MIN): KPI adjustments hurt SO (gain -0.31 over 5 games, bias +0.35)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Danny Jansen (TEX): KPI adjustments hurt SO (gain -0.29 over 6 games, bias +0.40)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Carson Benge (NYM): KPI adjustments hurt H (gain -0.13 over 6 games, bias -0.64), SO (gain -0.28 over 6 games, bias +0.04)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change
Taylor Trammell (HOU): KPI adjustments hurt H (gain -0.10 over 5 games, bias +0.71), SO (gain -0.28 over 5 games, bias -0.42), HR (gain -0.03 over 5 games, bias +0.12)	KPI readings	low	his KPI reading or its opponent pairing misleads; watch whether it persists before any change