Production forensic audit · immutable source review

MLB prediction system deep audit

Slate 20260914. This report separates the canonical final decision from the stochastic inning simulation, audits signed probability confidence, reconciles starters against the official schedule, and records the exact readiness of the adaptive learning and training loop.

Executive conclusion

Official slate completeness

All games at a glance

Final vs simulation contradictions

A score difference is diagnostic. A winner difference is a hard public-surface contradiction unless the display is synchronized to the final decision.

Fixes and verification

Readiness checks

Artifact parsing and integrity

Model provenance

The provenance for every game is retained inside its detail card. This exposes schema, training timestamp, row count, feature count, and publication gate rather than treating the current model as an unexplained black box.

Adaptive learning, review, and training context

Deep findings

Connected execution flow

1. Hourly upstream files change; revision-aware loaders invalidate same-process schedule, player, pitcher, recent-form, and matchup caches.
2. Pregame orchestration reconciles each starter against the current official MLB game row and records source/provenance.
3. Starter-only historical quality and Savant quality are blended by expected starter innings; the bullpen owns the remaining game share.
4. The stable final-decision policy chooses one winner/score, then the public contract synchronizes every public field while retaining the simulation as diagnostic.
5. After official settlement, review_prediction_results joins actual scores to the immutable prediction and preserves signed edge plus pick_probability.
6. The daily and rolling postgame pipeline rebuilds starter comparisons for the selected dates; a separate 120-game adaptive evidence window prevents daily sample limits from disabling proposals.
7. The adaptive cycle ingests settled evidence, evaluates one bounded candidate, asks OpenAI then Claude for strict JSON only when eligible, and never lets LLM text directly mutate production.
8. Starter candidates replay frozen pregame form evidence and settle against separate append-only official starter events; XGBoost candidates use isolated chronological retraining.
9. A deterministic, mode/hash-bound evaluation must pass before manual promotion; approved changes retain exact rollback and starter-aware or game-aware post-promotion monitoring.

Known limits and safety holds

• Games already past first pitch remain immutable. Corrections apply to the public audit layer and to the next safe pregame generation.
• The full-pipeline evaluator is intentionally starter-only: global player-form, hitter, and unscoped pitcher parameters remain non-promotable until they receive their own outcome evaluator.
• The repository has no resident scheduler. Deployment must invoke prediction refresh before games and settlement/learning after games become final.
• Automatic promotion is intentionally unsupported. OpenAI/Claude propose bounded JSON; deterministic tests and an operator approve production activation.
• An official schedule with no announced probable pitcher remains unresolved rather than being labeled verified.

{
  "data_directory": {
    "status": "untouched_by_this_audit",
    "policy": "The report reads output/games and the external official schedule only. It does not rewrite or enumerate data/."
  },
  "adaptive_readiness": {
    "artifact_replay": "operational",
    "enhanced_shadow": "operational",
    "model_training_backtest": "operational_with_manual_promotion",
    "full_pipeline_backtest": "operational_for_allowlisted_starter_projection_parameters_only",
    "provider_order": [
      "openai",
      "anthropic"
    ],
    "provider_output_authority": "proposal_only"
  },
  "confidence_semantics": {
    "display": "home_win_probability - away_win_probability",
    "calibration": "pick_probability / calibration_probability"
  }
}

Evidence sources

• Official MLB Stats API schedule — game identity, current status, and probable-pitcher IDs/names
• FOX Sports STL–CIN game page — Matthew Liberatore versus Chase Burns and the pregame market context
• Baseball Savant — Matthew Liberatore — official pitcher identity and Statcast profile context
• FOX Sports WAS–TEX game page — Cade Cavalli versus Kumar Rocker and the pregame market context