# Luna deep analysis — slate 20260930 > Coverage receipt: **COMPLETE** · 9/8 sections; 4/4 games; 7/7 causes; 5/6 sources # 1. Evidence receipt and gaps ## Source manifest All seven manifest entries were reviewed: | Source | Status | Purpose | Assessment | |---|---:|---|---| | `prediction_review_20260930.json` | Loaded | Official outcomes and scoring errors | Received | | `PREDICTION_SYSTEM_DEEP_AUDIT_20260930.json` | Loaded | Probability, provenance, starter, and decision contracts | Received | | `continuous_learning_daily_20260930.json` | Loaded | Stage quality, calibration, probability lineage, score distribution, starter comparisons | Received | | `detailed_20260930.html` | Loaded | Full game and simulation rendering | Received | | `20260930_ultra_detailed_player_audit.html` | Loaded | Player-level prediction trace | Received | | `audit_report_20260930.html` | Loaded | Invariant findings | Received | | `audit_report_20260930_integrity_revision.html` | **Missing** | Post-Luna remediation proof for immutable revision artifacts | **Gap** | The missing integrity-revision audit is an artifact-verification gap, not evidence that the prediction artifacts themselves were corrupted. The game-level forecast files were reported clean, and all four structural audits passed. ## Games reviewed All four expected and settled games were reviewed: 1. BOS @ NYY 2. CHC @ SD 3. CHW @ HOU 4. PHI @ ATL Coverage reconciles as follows: - Expected games: 4 - Canonical artifacts loaded: 4 - Continuous-learning games: 4 - Forensic deep-audit games: 4 - Structural audits completed: 4 - Games with starter comparisons: 4 - All games structurally audited: true Overall result: **3 winner hits, 1 winner miss, 75% winner accuracy**. ## Internal contradictions and data gaps 1. **Missing integrity revision audit.** The manifest explicitly marks the post-remediation integrity artifact as missing. 2. **CHW @ HOU player-level feed gap.** The feed contained 63 rows, but none matched this game ID. Consequently, no player-level actuals were available for that game. 3. **Starter-feed gaps in game-level player forensics.** - BOS @ NYY: Sonny Gray had no settled per-player pitching line in the forensic feed, although the separate game-level `starter_lines` record contained a comparison. - CHC @ SD: Kevin Gausman likewise had no settled per-player pitching line in the forensic feed. 4. **Probability-reversion provenance is incomplete across representations.** The lineage table records raw, calibrated, governed, score-coherence, recency, and published values. A separate probability-reversion record reports a CHW @ HOU blend-stage probability of `0.544092`, but the complete named transformation from that value to the published `0.495` is not shown in the lineage table. This is a provenance coverage issue, but not automatically a bypass: the raw-to-published path itself is recorded. 5. **Some derived labels are semantically awkward.** For example, `starter_lines` reports extremely high actual ERA values where the pitcher recorded very few outs. Those values are mathematically consistent with the recorded earned runs and outs, but they should not be interpreted as ordinary full-start ERA performance. 6. No critical, error, or warning structural findings were reported. The structural summary was `0 critical`, `0 error`, `0 warning`, and `30 info`. --- # 2. Executive diagnosis ## Winner accuracy Winner performance was good on this four-game slate but not decisive: - Winner accuracy: **3/4 = 75%** - Walk-forward model accuracy: **60.04%** - Walk-forward baseline accuracy: **57.01%** - Brier scores by game: `0.2116`, `0.2350`, `0.2450`, `0.2630` - Slate mean Brier: approximately **0.2387** The slate beat the historical walk-forward baseline, but the sample was only four games. The three hits included two toss-up selections, and the single miss was also a toss-up. Correct winner selection therefore does not establish strong directional confidence. The top-confidence-pick streak was **0 losses** over the listed valid slates, with the unknown-date entry excluded from interpretation. The top pick hit on 2026-09-30. ## Calibration quality Calibration was poor on this slate: - Published confidence bucket: `0.50–0.55` - Average confidence: `0.5183` - Observed hit rate: `0.75` - Calibration error as defined by the source: `-0.2317` - Weighted absolute calibration error: **0.2317** - Bucket sample: 4 games, 3 hits The system was **underconfident by the supplied definition** on this specific bucket: average published confidence was 51.83%, while observed hit rate was 75%. This does not mean the system is reliably underconfident; four games are insufficient to establish a calibration curve. Across the cross-slate trend, weighted calibration error is marked **regressing**, with a mean of `0.2087`. The self-assessment headline calls the system “confidence_overconfident,” but that label conflicts with this slate’s bucket definition, which shows underconfidence. The safer conclusion is: **calibration is unstable and materially misaligned, but the direction of correction should not be inferred from this slate alone.** ## Score accuracy Score accuracy was materially weak: - All four predicted margins were exactly 1 run. - Actual margins were 7, 3, 4, and 1. - Predicted-margin MAE: **3.25** - Signed margin bias: **-0.75**, meaning margins were too narrow on average. - Two of four actual margins were at least four runs. - Mean predicted margin: `1.00` - Mean actual margin: `3.75` - Team-run MAE: **1.875** - Total-run MAE: **1.75** at the `variance_applied` stage and **2.25** at final published stages. The high-severity `score_margin_compression` finding is supported: every published prediction had a margin of two runs or less, while two actual margins were at least four runs. This is not a reason to widen every forecast indiscriminately. It supports a bounded, rolling-window test of the registered variance controls. The `weak_run_model_signal` finding is also supported by holdout validation: - Home-runs R²: `0.002143` - Away-runs R²: `0.030936` - Total-runs R²: `0.002415` - Validation rows: 2,415 These values explain almost none of the holdout run variance. Exact score estimates should therefore be treated as low-confidence outputs. ## Starter accuracy Starter accuracy was not the main slate failure: - Mean starter ERA error in the matchup-pattern summary: `0.78` - The lever-tension audit says starter projections were accurate enough that changing pitcher form weights would have **hurt all four games**. - Each game’s listed worst starter earned-run error was only `1.0`. - However, starter outs and derived ERA were sometimes materially wrong: - Hunter Brown: projected 17 outs, actual 2 outs. - Nick Pivetta: projected 15 outs, actual 10 outs. - Kevin Gausman and Sonny Gray had player-forensic feed gaps, although game-level comparisons existed. The appropriate diagnosis is that the starter earned-run projections were not the principal cause of the winner miss, but the extreme Hunter Brown workload/outcome mismatch remains a run-model risk. The data do not justify broadly reweighting pitcher form from this slate. ## Artifact integrity Artifact integrity was strong within the available audit set: - All four game structural audits: `PASS` - Each game: zero critical, zero error, zero warning findings - Forecast-file integrity: clean for all four games - Learning eligibility: true because all game artifacts passed critical/error gates - All four games received structural QA The missing integrity-revision audit remains a documentation and artifact-receipt gap. It does not overturn the clean game-level audit results. ## Causal issue register reconciliation ### High: intermediate pipeline winner instability - **ID:** `pipeline_winner_instability` - **Affected game:** PHI @ ATL - **Evidence:** `historical_ml → variance_applied` changed winner accuracy by `-0.2500`, or one hit lost. - **Observed transition:** `PHI → TIE` - **Severity:** High - **Action:** Record the exact pre-stage and post-stage pick for every game. - **Guardrail:** A later recovery does not make the unstable intermediate step harmless. This is a real process/model risk. Historical ML was correct on PHI @ ATL in winner direction; variance application moved it to a tie; enhanced prediction later moved it from tie to ATL, which was still wrong. The later recovery restored the published winner count but did not erase the regression. ### High: score-margin compression - **ID:** `score_margin_compression` - **Affected games:** BOS @ NYY and CHW @ HOU specifically, with all four games exhibiting one-run predicted margins. - **Evidence:** 4/4 predictions had margins ≤2; 2/4 actual margins were ≥4; mean predicted margin `1.00` vs actual `3.75`. - **Severity:** High - **Action:** Check several recent out-of-sample slates for tail coverage and margin MAE before widening score spreads. - **Guardrail:** Do not widen every score prediction because of one blowout. The cross-slate trend now supplies more than one-slate evidence: margin bias is marked regressing, with mean `-2.1615`, and the finding has a two-slate consecutive streak. ### High: single-slate calibration instability - **ID:** `single_slate_calibration_instability` - **Affected games:** All four - **Evidence:** Weighted absolute calibration error `0.2317` - **Severity:** High - **Action:** Monitor the same confidence buckets over the rolling window before changing calibration weights. - **Guardrail:** A four-game bucket does not establish a stable calibration curve. The cross-slate finding has a two-slate consecutive streak and incidence in 7 of 11 loaded slates, so this is no longer merely a one-slate observation. However, the exact direction of a calibration-cap change still requires controlled shadow testing. ### High: weak run-model signal - **ID:** `weak_run_model_signal` - **Affected games:** All four - **Evidence:** Home/away/total R² of `0.0021/0.0309/0.0024` - **Severity:** High - **Action:** Treat exact scores as low-confidence; seek stronger features or a better target. - **Guardrail:** Positive R² does not make exact scores reliable. This is a persistent structural problem: incidence 10 of 11 slates. It is primarily a feature/target and training issue, not a nightly variance-tuning issue. ### Informational findings - Complete slate audit coverage: true, but coverage does not cancel findings inside audited games. - Probability publication delta explained: all material calibrated-to-published movements in the listed lineage have named score-coherence/recency transformations. - Simulation inert on this slate: no winner flips, but zero flips does not prove simulation performed useful score reconciliation. --- # 3. Every game ## BOS @ NYY — correct winner, wrong score - **Predicted score/winner:** BOS 3, NYY 4; NYY - **Actual score/winner:** BOS 2, NYY 9; NYY - **Published home probability:** `0.540` - **System confidence edge:** `0.08` home-minus-away - **Conviction:** Lean - **Brier score:** `0.2116` - **Team-run error:** `6` - **Total-run bias:** `-4`; total predicted 7 versus actual 11 - **Margin:** predicted 1, actual 7; margin error 6 This is explicitly a **correct-winner/wrong-score** game. ### Evidence and edges Pregame evidence favored NYY: - Season edge: `+0.340` home - Recent edge: `+0.230` home - Pitcher edge: `-0.054` home - Composite edge: `+0.306` - Season quality edge: `+0.3398` - Recent quality edge: `+0.16` Run factors were modestly suppressive: - BOS runs versus home pitching: `0.9788` - NYY runs versus away pitching: `0.9251` ### Starter projected versus actual | Starter | Projected | Actual | Error | |---|---:|---:|---:| | Sonny Gray, BOS | 3 ER, 18 outs, ERA 4.50 | 2 ER, 14 outs, ERA 3.86 | ER -1; outs -4; ERA -0.64 | | Max Fried, NYY | 0 ER, 18 outs, ERA 0.00 | 1 ER, 18 outs, ERA 1.50 | ER +1; outs 0; ERA +1.50 | The player-level forensic feed lacked a settled pitching line for Sonny Gray, despite the separate game-level starter comparison. The audit therefore supports the game-level comparison but not a complete player-level starter reconstruction. ### Score path and player adjustments The supplied score path was not expanded for this game, but the final prediction was 3–4. The main observed run miss was NYY offense: - NYY offense was under-projected by 5 runs. - Cody Bellinger: 2-for-5, HR, 4 RBI versus projected 1 hit, 0 HR; run impact `+1.07`. - Luis García Jr.: 3-for-5, HR, RBI versus projected 1 hit, 1 HR; run impact `+0.94`. - Jazz Chisholm Jr.: run impact `+0.80`. - Ben Rice: run impact `-0.93`. The pitcher attribution did not identify a material break: the forensic verdict characterized both sides as closer to variance. ### Audit findings - Structural audit: PASS - Critical/error/warning findings: none - Informational findings: 7 - Forecast file: clean - Diagnostic flag: `large_savant_run_environment_swing` - Starter experience control: not applied; both starter histories met the minimum - Recency guardrail: applied - Score-volatility action: `no_change` ### Causes - **Primary:** Offensive/run-environment underprojection, especially NYY scoring. - **Secondary:** Irreducible game-level variance and narrow score distribution. - **Not primary:** Winner-direction error, starter earned-run projection, or artifact integrity. --- ## CHC @ SD — correct winner, wrong score - **Predicted score/winner:** CHC 4, SD 5; SD - **Actual score/winner:** CHC 1, SD 4; SD - **Published home probability:** `0.515245` - **System confidence edge:** `0.0305` home-minus-away - **Conviction:** Toss-up - **Brier score:** `0.234987` - **Team-run error:** `4` - **Total-run bias:** `+4`; total predicted 9 versus actual 5 - **Margin:** predicted 1, actual 3; margin error 2 This is also a **correct-winner/wrong-score** game. ### Evidence and edges The evidence conflicted: - Season edge: `-0.2355` home - Recent edge: `+0.158` home - Pitcher edge: `+0.0243` home - Composite edge: `+0.1434` - Season quality edge: `-0.2355` - Recent quality edge: `+0.508` The starter-experience control was applied: - Pivetta had 8 prior feed appearances. - Probability was shrunk toward 0.5 from `0.526938` to `0.521101`. - Winner direction did not change. ### Starter projected versus actual | Starter | Projected | Actual | Error | |---|---:|---:|---:| | Kevin Gausman, CHC | 4 ER, 17 outs, ERA 6.35 | 3 ER, 10 outs, ERA 8.10 | ER -1; outs -7; ERA +1.75 | | Nick Pivetta, SD | 2 ER, 15 outs, ERA 3.60 | 1 ER, 10 outs, ERA 2.70 | ER -1; outs -5; ERA -0.90 | The player-level forensic feed had no settled pitching line for Kevin Gausman. Nick Pivetta’s line was available and showed eight prior feed appearances. ### Score path and player adjustments - CHC offense was over-projected by 3 runs. - SD offense was under the projected total according to the matchup breakdown, with home lineup run delta `-2`. - Manny Machado: `-1.26` run impact versus projection. - Fernando Tatis Jr.: `-1.11`. - Jackson Merrill: `+0.80`. - Ty France: `+0.78`. The forensic verdict identified SD offense as the main side that deviated, but characterized the overall result as closer to variance than a decisive model failure. ### Audit findings - Structural audit: PASS - Critical/error/warning findings: none - Informational findings: 8 - Forecast file: clean - Starter-experience shrinkage: applied - Recency guardrail: applied - Score-volatility action: `no_change` ### Causes - **Primary:** Run-total and lineup-output miss, particularly overprojection of CHC and under-realization of SD scoring. - **Secondary:** Narrow one-run score distribution. - **Not primary:** Winner direction or artifact integrity. - The thin-history starter control was appropriate and did not cause a winner error. --- ## CHW @ HOU — correct winner, wrong score - **Predicted score/winner:** CHW 6, HOU 5; CHW - **Actual score/winner:** CHW 7, HOU 3; CHW - **Published home probability:** `0.495`; away pick probability `0.505` - **System confidence edge:** `-0.01` home-minus-away, equivalent to a 0.01 away edge - **Conviction:** Toss-up - **Brier score:** `0.245025` - **Team-run error:** `3` - **Total-run bias:** `+1`; total predicted 11 versus actual 10 - **Margin:** predicted CHW by 1, actual CHW by 4; margin error 3 This is a **correct-winner/wrong-score** game. ### Evidence and edges Evidence strongly favored CHW in the underlying edges, although the final probability was nearly even: - Season edge: `-0.4495` home - Recent edge: `-0.154` home - Pitcher edge: `-0.0304` home - Composite edge: `-0.5326` - Season quality edge: `-0.4495` - Recent quality edge: `-0.224` Run factors: - CHW runs versus home pitching: `1.0225` - HOU runs versus away pitching: `0.9921` ### Starter projected versus actual | Starter | Projected | Actual | Error | |---|---:|---:|---:| | Sean Burke, CHW | 2 ER, 15 outs, ERA 3.60 | 3 ER, 12 outs, ERA 6.75 | ER +1; outs -3; ERA +3.15 | | Hunter Brown, HOU | 3 ER, 17 outs, ERA 4.76 | 3 ER, 2 outs, ERA 40.50 | ER 0; outs -15; derived ERA +35.74 | Hunter Brown’s actual two-out appearance is the largest starter workload anomaly on the slate. Earned runs were projected correctly, but expected duration was not. ### Score path and player adjustments No player-level actuals were joined to this game: - The player forensic feed had 63 rows for the slate but zero matching this game ID. - Therefore, no defensible player-level attribution should be invented. The model nevertheless got the winner correct and the total close, while missing the final margin. ### Audit findings - Structural audit: PASS - Critical/error/warning findings: none - Informational findings: 7 - Forecast file: clean - Player-level forensic coverage: unavailable - Recency guardrail: applied - Score-volatility action: `no_change` ### Causes - **Primary:** Margin compression and starter workload/run-distribution uncertainty. - **Secondary:** Hunter Brown’s two-out actual appearance versus 17 projected outs. - **Not primary:** Winner direction; the underlying season, recent, pitcher, and composite edges all favored the selected away side. - Player-level offense/bullpen causality is unresolvable from the supplied feed. --- ## PHI @ ATL — winner miss - **Predicted score/winner:** PHI 3, ATL 4; ATL - **Actual score/winner:** PHI 4, ATL 3; PHI - **Published home probability:** `0.512824` - **System confidence edge:** `0.0256` home-minus-away - **Conviction:** Toss-up - **Brier score:** `0.262988` - **Team-run error:** `2` - **Total-run bias:** `0`; predicted and actual total both 7 - **Margin:** predicted ATL by 1, actual PHI by 1; signed margin error 2 This was the only winner miss. It was **not** a score-total failure: the total was exact, but the one-run direction was reversed. ### Evidence and edges The pregame evidence favored ATL: - Season edge: `+0.559` home - Recent edge: `+0.406` - Pitcher edge: `+0.099` - Composite edge: `+0.806` - Season quality edge: `+0.559` - Recent quality edge: `+0.336` - Recent PHI form: 4–6, run differential `-16` - Recent ATL form: 5–5, run differential `+1` The five supplied head-to-head results were mixed, including both PHI and ATL blowouts. That history supported uncertainty rather than a strong directional conclusion. ### Starter projected versus actual | Starter | Projected | Actual | Error | |---|---:|---:|---:| | Cristopher Sánchez, PHI | 2 ER, 19 outs, ERA 2.84 | 3 ER, 20 outs, ERA 4.05 | ER +1; outs +1; ERA +1.21 | | Tyler Mahle, ATL | 0 ER, 18 outs, ERA 0.00 | 1 ER, 21 outs, ERA 1.29 | ER +1; outs +3; ERA +1.29 | Both starters were one earned run away from projection. Starter error did not clearly explain the loss. ### Score path The score path shows the crucial direction regression: 1. Historical ML: PHI 4.364, ATL 4.121 2. Variance applied: PHI 4, ATL 4 — PHI direction became a tie 3. Enhanced prediction: PHI 3, ATL 4 — direction moved to ATL 4. Savant run environment: PHI 3, ATL 4 5. Fundamental edge review: PHI 3, ATL 4 6. Capped target: PHI 3, ATL 4 7. Simulation result: PHI 3, ATL 4 The Savant step itself changed the target numerically by `+0.2203` PHI and `-0.1362` ATL, but the listed winner direction remained ATL after the earlier enhanced stage. Final publication did not change the winner. ### Player adjustments There were 23 material player-level changes. Important listed adjustments included: - Alec Bohm: 2-for-4 with HR and RBI versus projection; run impact `+1.73`; classified as a learnable under-rated strong hitter, xwOBA `0.327`. - Bryce Harper: run impact `+0.76`; elite xwOBA `0.379`. - Matt Olson: run impact `-0.93`. - Sean Murphy: run impact `+0.64`. - Cristopher Sánchez: largest changed metric was predicted strikeouts, but dominant effect neutral. - Ozzie Albies: largest change predicted hits `+2.21`. - Trea Turner and Edmundo Sosa had adverse projection adjustments. The player evidence indicates some offense was under-rated, but only half of the 12 material player deviations were supported by the players’ own contact-quality profiles. This limits how much should be learned from the individual outcomes. ### Audit findings - Structural audit: PASS - Critical/error/warning findings: none - Informational findings: 8 - Forecast file: clean - Player-level forensics: available - Score-volatility action: `no_change` - Primary driver label: `toss_up_variance` ### Causes - **Earliest wrong-direction stage:** `variance_applied`, where PHI moved from a historical-ML win direction to a tie. - **Further wrong-direction stage:** `enhanced_prediction`, where the tie became an ATL pick. - **Primary final cause:** Toss-up variance, as recorded by the audit. - **Secondary contributors:** Directional weighting of season/recent/composite edges and the model’s tendency to turn a close latent score into a categorical side. - **Not primary:** Total-run miss, starter earned-run miss, or artifact corruption. --- # 4. Miss reconstruction ## PHI @ ATL: earliest movement toward the wrong result The miss reconstruction must distinguish intermediate model regression from final outcome variance. ### Stage-by-stage direction | Stage | Direction | Assessment | |---|---|---| | Historical ML | PHI | Correct direction relative to actual | | Variance applied | Tie | First deterioration; PHI win signal was removed | | Enhanced prediction | ATL | Explicit winner-direction regression | | Savant run environment | ATL | No reported winner change | | Fundamental edge review | ATL | No reported winner change | | Capped target | ATL | No reported winner change | | Simulation result | ATL | No reported winner change | | Published final | ATL | Miss preserved | The causal register is therefore correct: `historical_ml → variance_applied` degraded winner accuracy by one hit. The later transition from variance-applied to enhanced prediction improved slate winner accuracy from 0.50 to 0.75 because CHW @ HOU moved from HOU to CHW, but it also moved PHI @ ATL from a tie to ATL. A later numerical recovery does not erase the intermediate regression. ## Classification of the loss - **Bad input:** No confirmed bad input. There were feed gaps for some other games, but PHI @ ATL had player-level data and passed structural QA. - **Bad weighting:** Plausible risk. Season/recent/composite edges strongly favored ATL, yet the latent historical ML score favored PHI. The enhanced stage resolved this tension toward ATL. - **Calibration:** The published edge was only `0.0256`; calibration quality is globally unstable, but no game-specific calibration defect is proven. - **Starter error:** Limited. Both starters were one earned run away from projection; outs were somewhat high but not a decisive miss. - **Offense/bullpen miss:** Mixed. PHI’s lineup produced runs from several players, while ATL’s bullpen allowed three runs in the forensic breakdown. However, the matchup verdict did not identify a single material side break. - **Run-environment miss:** No. Total predicted and actual runs were both 7. - **Irreducible variance:** Strongly supported. The official primary driver was `toss_up_variance`, and the published edge was only 0.0256. A home-away confidence edge of **0.12**, if encountered, should be treated as moderate—not as a near coin flip. The current slate did not contain a 0.12 edge; its largest edge was 0.08 in BOS @ NYY. The near-zero edges in CHW @ HOU and PHI @ ATL were genuine toss-up territory. ## Pipeline-stage quality across all stages | Stage | Winner accuracy | Team-run MAE | Total-run MAE | Margin MAE | |---|---:|---:|---:|---:| | historical_ml | 0.75, 3/4 | 1.7834 | 1.8527 | 3.1126 | | variance_applied | 0.50, 2/4 | 1.8750 | 1.7500 | 3.2500 | | enhanced_prediction | 0.75, 3/4 | 1.8750 | 2.2500 | 3.2500 | | embedded_savant_run_environment | 0.75 | 1.8750 | 2.2500 | 3.2500 | | fundamental_edge_target_review | 0.75 | 1.8750 | 2.2500 | 3.2500 | | capped_target | 0.75 | 1.8750 | 2.2500 | 3.2500 | | simulation_result | 0.75 | 1.8750 | 2.2500 | 3.2500 | | published_final | 0.75 | 1.8750 | 2.2500 | 3.2500 | Transitions: - Historical ML → variance applied: - PHI → tie - Winner accuracy down 0.25 - Team-run MAE up 0.0916 - Margin MAE up 0.1374 - Variance applied → enhanced prediction: - HOU → CHW, an improvement - PHI tie → ATL, a regression in direction - Winner accuracy up 0.25 overall - Every later transition: - No further winner changes - No further margin changes - Total-run MAE remained 2.25 The critical interpretation is that **the later recovery of the slate’s aggregate winner accuracy does not erase the intermediate winner-direction regression on PHI @ ATL.** --- # 5. Cross-game counterfactuals ## Lever tension entries ### Score variance multiplier / variance floor - **Lever:** `score.variance_multiplier / score.variance_floor` - **Purpose:** Widen run distributions - **Helps:** None documented on this slate - **Hurts:** None documented on this slate - **Net evidence:** 0 helps, 0 hurts The slate supports investigating margin compression, but it does not show that a variance change would have improved any particular game. BOS @ NYY and CHW @ HOU had under-wide margins, but widening distributions would not necessarily have produced their exact scores or changed winner probabilities. ### Pitcher form weights / prior games - **Lever:** `player.pitcher.v4 form weights / player.pitcher_prior_games` - **Targets:** Starter ER MAE and starter outs MAE - **Helps:** None - **Hurts:** All four games according to the tension audit: - BOS @ NYY: starter projection accurate to one ER - CHC @ SD: starter projection accurate to one ER - CHW @ HOU: earned-run projection for Hunter Brown was accurate despite workload miss - PHI @ ATL: both starters were one ER from projection - **Net evidence:** 0 helps, 4 hurts Do not reweight pitcher form based on this slate. ### Recent/season run-mean weighting - **Lever:** `recent.* / season.* run-mean weighting` - **Targets:** Team-run MAE, total-run MAE, accuracy - **Helps:** - BOS @ NYY: team runs off by 6 - CHC @ SD: team runs off by 4 - **Hurts:** - PHI @ ATL: team runs off by only 2; further reweighting risks damaging an already close total - **CHW @ HOU:** no explicit help/hurt designation in the tension record - **Net evidence:** 2 helps versus 1 hurts This is suggestive but not sufficient for an unconstrained global change. A small shadow experiment is defensible. ## Tempting one-game fixes that should not be made 1. Do not widen every score prediction solely because BOS @ NYY ended 9–2. 2. Do not change pitcher form weights; all four games are recorded as potential casualties. 3. Do not force PHI @ ATL through a categorical PHI pick merely because historical ML favored PHI. The edge was small and the outcome remained a one-run toss-up. 4. Do not tune individual hitters from every deviation. Only 6 of 12 material deviations were supported by the hitter’s own contact quality. 5. Do not infer that simulation was useful merely because it did not flip a winner. The zero-flip pattern has persisted for 11 slates. 6. Do not treat the CHW @ HOU blend-stage home preference as a proven defect without connecting its probability transformation to the published lineage. --- # 6. Data and contract defects ## Critical, error, warning, and informational findings Across structural audits: - Critical: **0** - Error: **0** - Warning: **0** - Informational: **30** - Structural audit status: PASS for all four games No critical or error-level model-contract failure was reported. ## Specific defects and gaps ### Prediction-model defects 1. **Run model near-zero holdout signal** - Total R² `0.002415` - Home R² `0.002143` - Away R² `0.030936` - This is a substantive modeling defect, not a presentation issue. 2. **Margin compression** - Every prediction had a one-run margin. - Actual margins included 7, 3, 4, and 1. 3. **Simulation inertness** - Zero simulation flip candidates and zero published flips on this slate. - Persistence: 11 consecutive slates. - The self-assessment identifies a likely design issue: the simulation uses target-seeking draws and exits early, producing an ineffective win distribution. 4. **Intermediate winner instability** - Variance application moved PHI from a correct historical-ML direction to a tie. 5. **Player-projection spread weakness** - Hitter spread ratios were low: - AB `0.433` - H `0.368` - SO `0.301` - RBI `0.574` - This indicates under-dispersed individual projections and weak separation, not merely calibration. ### Data-contract defects 1. CHW @ HOU had no player-level actuals joined. 2. Sonny Gray and Kevin Gausman lacked settled per-player pitching lines in the forensic feed. 3. The missing integrity-revision audit prevents complete post-remediation artifact verification. 4. Probability-reversion data exposes a blend-stage value for CHW @ HOU that is not directly connected to the primary probability lineage. ### Presentation/artifact issues - The individual game forecast files were clean, so presentation artifacts did not cause the miss. - The missing integrity-revision report is an artifact-receipt defect, not proof of a bad prediction. - High derived ERA values from very short appearances should be clearly labeled as short-outcome rate conversions to avoid overinterpretation. ## Probability lineage audit ### BOS @ NYY - Raw model: `0.6103` - Calibrated: `0.5622` - Governor input: `0.51045` - Governor output: `0.5622` - Score coherence: `0.562176` - Published: `0.540` - Transformation: score coherence followed by matchup-recency guardrail - Material delta explained: yes - Governor bypass: not established ### CHC @ SD - Raw model: `0.5437` - Calibrated: `0.5269` - Governor input: `0.530` - Governor output: `0.5269` - Score coherence: `0.526938` - Published: `0.515245` - Transformation: score coherence followed by matchup-recency guardrail - Starter-experience shrinkage was also recorded. - Material delta explained: yes ### CHW @ HOU - Raw model: `0.500` - Calibrated: `0.505` - Governor input: `0.525995` - Governor output: `0.505` - Score coherence: `0.505` - Published: `0.505` away / `0.495` home - Transformation: published probability matches calibrated/governed value - Separate blend record: trained home probability `0.544092`, blend-favored home probability `0.519185`, but publication ultimately favored away. - No bypass is called because the primary raw-to-published lineage is recorded, but the blend-stage transformation should be added to the formal lineage. ### PHI @ ATL - Raw model: `0.5268` - Calibrated: `0.5177` - Governor input: `0.530` - Governor output: `0.5177` - Score coherence: `0.51775` - Published: `0.512824` - Transformation: published probability matches calibrated/governed value - Material delta explained: yes The lineage is substantially complete, but CHW @ HOU needs explicit blend-stage provenance to eliminate ambiguity. --- # 7. Cross-slate persistence A single slate is noisy, but the supplied cross-slate evidence contains several persistent patterns. ## Finding streaks | Finding | Consecutive-slate count | Present now? | Interpretation | |---|---:|---:|---| | `high_raw_probability_cluster` | 0 | No | Not active now; incidence 1/11 does not support action | | `pipeline_winner_instability` | 2 | Yes | Persistent short streak; material winner-direction risk | | `score_margin_compression` | 2 | Yes | Persistent short streak; margin tails too narrow | | `simulation_inert_on_slate` | 11 | Yes | Established failure, not single-slate noise | | `single_slate_calibration_instability` | 2 | Yes | Persistent short streak; calibration misalignment requires monitoring | | `weak_run_model_signal` | 2 | Yes | Incidence 10/11; persistent model-capability problem | The source reports an escalation streak of 3 and separately marks simulation inertness as an established 11-slate failure. ## Escalated finding ### Simulation inertness - **Streak:** 11 consecutive slates - **Incidence:** 11 of 11 - **Target lever:** `simulation.allow_score` - **Metric group:** Run model - **Metrics harmed:** `team_run_mae`, `total_run_mae` - **Direction:** Confirm the simulation is actually running and not silently no-oping. - **Action type:** Code/diagnostic, not a blind parameter move. The self-assessment identifies the likely root cause: the same target-seeking draws are used to create the score distribution, and the loop can exit after approximately two selected draws. The resulting win rate is not a valid probability distribution. ## Metric trends The required directions are: - **Margin bias:** Regressing. Mean `-2.1615`; predicted margins are increasingly too narrow. - **Mean team-run error on losses:** Improving. Mean `4.808`, although still large. - **Weighted calibration error:** Regressing. Mean `0.2087`. - **Winner accuracy:** Improving. Mean `0.6079`; current slate `0.75`. - **Top-confidence-pick loss streak:** `0`. The latest top pick hit. - **Run-model signal:** Persistent weakness; R² remains near zero. - **Simulation contribution:** Inert for 11 consecutive slates. The improving winner-accuracy trend must not be used to excuse the regressing calibration, score-margin, and simulation patterns. --- # 8. Adjustments for today Recommendations are ranked by persistence, scope, expected metric effect, and regression risk. ## 1. Highest priority: repair and instrument simulation inertness - **Pattern:** 11-slate established failure - **Registered lever:** `simulation.allow_score` - **Recommended change:** Do not simply toggle the lever blindly. Run a bounded diagnostic/code change that confirms the module executes and separates: 1. target-realization search for box-score output, and 2. an independent, unsteered win-probability distribution. - **Concrete implementation direction:** Use a registered independent draw count, such as 200–500 draws, or a closed-form Poisson/Skellam difference, without early exit. - **Expected metric:** Improve simulation contribution to `team_run_mae` and `total_run_mae`; winner-flip diagnostics should become nonzero when justified. - **Helps:** Potentially all four games’ score distributions; especially BOS @ NYY, CHC @ SD, and CHW @ HOU, where margins were under-dispersed. - **Hurts:** Unknown until shadow-tested; an unsteered simulation could introduce noisy winner changes. - **Rollback monitor:** Revert if rolling team-run MAE, total-run MAE, or Brier score worsens beyond the registered gate. - **Reason this is first:** It is persistent for 11 slates and has an identified code-level root cause. Do not force a parameter move merely because `simulation.allow_score` is listed as a lever. ## 2. Persistent run-model weakness: initiate an isolated feature-contract backtest - **Pattern:** Weak run-model signal, incidence 10/11; R² near zero - **Registered target levers:** `training.xgb.n_estimators`, `training.xgb.max_depth`, `training.xgb.learning_rate` - **Recommended change:** No nightly hyperparameter tweak. Run an isolated same-fingerprint backtest after adding registered features: no-vig market probability where source-labelled, park/weather environment, bullpen rest/fatigue, and platoon-adjusted lineup quality. - **Direction:** Feature and target-contract improvement first; hyperparameters only after the feature experiment. - **Expected metrics:** Increase home, away, and total run holdout R²; reduce team-run MAE and total-run MAE. - **Helps:** All four games in score estimation; especially BOS @ NYY, CHC @ SD, and CHW @ HOU. - **Hurts:** Scope risk is broad; potentially increases overfit and could disturb PHI @ ATL’s accurate total. Require same-fingerprint holdout and rollback gates. - **Reason:** The evidence says the current features carry almost no run signal. Nightly variance changes cannot solve this. ## 3. Bounded shadow experiment for persistent margin compression - **Pattern:** `score_margin_compression`, two consecutive slates and regressing margin trend - **Registered parameter:** `score.variance_multiplier` - **Recommended change:** Increase by a small bounded amount, for example **+0.05 relative to its current registered value**, in shadow only. Do not widen the production distribution without the gate. - **Expected metrics:** Improve margin tail coverage and reduce margin MAE; monitor team-run MAE and total-run MAE. - **Helps:** BOS @ NYY, CHC @ SD, and CHW @ HOU, where actual margins exceeded predicted margins by 6, 2, and 3 runs. - **Hurts:** PHI @ ATL, where the total and one-run scale were close; wider variance could add unnecessary noise. - **Rollback:** Revert if rolling total-run MAE, team-run MAE, or winner Brier deteriorates. - **Caution:** The lever-tension record showed no direct slate-level help, so this should remain a shadow experiment, not a production promotion. ## 4. Calibration control shadow test - **Pattern:** Calibration instability, two consecutive slates and weighted calibration error marked regressing - **Registered parameters:** `confidence.base_cap` and `confidence.moderate_cap` - **Recommended change:** Test a small cap reduction, approximately **0.02**, in shadow only; do not infer the direction solely from this slate’s underconfidence reading. - **Expected metrics:** Reduce rolling weighted absolute calibration error and improve Brier score. - **Helps:** Potentially BOS @ NYY, CHC @ SD, CHW @ HOU, and PHI @ ATL by preventing unstable confidence escalation. - **Hurts:** Could over-dampen already modest probabilities and harm winner accuracy if the system is genuinely underconfident in the 0.50–0.55 bucket. - **Rollback:** Require improvement across the rolling bucket, not just this four-game slate. - **Reason:** The trend is persistent, but the current slate’s observed hit rate exceeds published confidence, so a cap reduction needs a formal gate. ## 5. Single-slate winner-direction issue: improve traceability, not a blind pick override - **Pattern:** PHI @ ATL historical-ML direction regressed at variance application - **Registered target lever:** None; the causal action is diagnostic/code trace - **Recommended change:** Record pre-stage and post-stage winner, probability, and score at every transition, especially `historical_ml → variance_applied → enhanced_prediction`. - **Expected metric:** Improve diagnostic attribution and prevent unobserved winner-direction regressions; no immediate accuracy gain is assumed. - **Helps:** PHI @ ATL analysis directly; potentially all future games affected by variance application. - **Hurts:** No direct predictive downside; added trace volume is the main cost. - **Reason:** This is a process-contract issue, not evidence for forcing PHI or suppressing the enhanced stage. ## Abstentions - Abstain from changing `player.pitcher.v4` form weights: the slate-level lever tension is 0 helps versus 4 hurts. - Abstain from tuning individual hitters based on isolated game outcomes: only half of material deviations were supported by contact-quality evidence. - Abstain from changing `score.variance_floor` simultaneously with `score.variance_multiplier`; use one bounded shadow lever at a time for attribution. --- # 9. Bottom line ## Known - The system selected **3 of 4 winners**, including all three games whose selected side won. - The PHI @ ATL miss was a genuine low-edge toss-up, not a large-confidence failure. - The earliest documented wrong-direction movement was at `historical_ml → variance_applied` for PHI @ ATL. - Score predictions were systematically compressed: all four predicted margins were one run, while two actual margins were at least four. - Starter earned-run projections were not the principal slate failure. - Run-model holdout signal is near zero and is a persistent structural concern. - Simulation has been inert for 11 consecutive slates and requires operator/code diagnostics. - Game artifacts passed all critical/error structural gates. - The integrity-revision audit is missing, and CHW @ HOU lacks player-level forensic coverage. ## Uncertain - Whether a wider run distribution would improve future score or winner performance rather than merely add noise. - Whether calibration should be capped downward, because this slate’s 51.83% average confidence produced a 75% hit rate even though the rolling calibration trend is regressing. - Whether the CHW @ HOU blend-stage probability represents an unrecorded bypass or simply a separately logged transformation; the formal lineage should be extended before deciding. - How much of the BOS @ NYY and CHC @ SD run miss reflects feature weakness versus irreducible offensive variance. - The player-level offensive and bullpen causes in CHW @ HOU cannot be established from the missing game join. ## Single safest next action **Run the simulation-inertness diagnostic/code fix in shadow mode before changing model weights: independently generate an unsteered win-probability distribution with a registered draw count, preserve the target-realization search separately, and monitor rolling team-run MAE, total-run MAE, winner Brier score, and actual winner-flip contribution.** That action addresses the longest established failure, does not force an unsupported one-game correction, and creates the evidence needed for safer score and confidence adjustments on the next slate.