MODEL
REGRESSION
FORENSICS.
A working research direction for tracing behavioral regressions through training lineage and verifying likely causes through controlled intervention.
Observe the regression. Find the cause.
Model teams can observe that a checkpoint regressed, yet still struggle to identify which training change caused the behavioral failure. This project asks whether behavioral differencing, training-lineage search, and targeted interventions can turn regression debugging into a reproducible engineering workflow.
Can an automated debugger localize the training change responsible for a model regression, then verify that diagnosis through intervention rather than correlation alone?
Measure a behavioral delta between baseline and candidate checkpoints.
Rank changed data, configuration, or training phases as candidate causes.
Remove, replace, or replay the suspected change under controlled conditions.
Test recovery on held-out cases and unrelated capabilities.
Adjacent work exists. The gap must be earned.
This project does not currently claim novelty. The working wedge is end-to-end regression forensics that joins behavioral differencing, training lineage, candidate-cause ranking, and intervention-backed verification. The related-work review must establish whether that combination is sufficiently distinct and useful.
Start with one planted regression.
Create a small controlled post-training experiment with a known root cause. Detect the behavioral change, search the recorded training lineage, rank candidate causes, execute the minimal intervention, and test whether the behavior recovers on held-out examples without damaging unrelated capabilities.
Defined the initial question and began reviewing adjacent work.
Plant one reproducible post-training regression and test whether a controlled intervention recovers the held-out behavior.
Reproduction starts with Experiment 000.
Commands, configs, eval definitions, and summarized results will be added after the first experiment has been run and reproduced from a clean checkout.
This page separates the current research question from completed evidence. Experimental results will be added only after they are reproduced.