mirror of
https://github.com/bdbrown4/peltier.io.git
synced 2026-11-07 10:58:28 +00:00
bench: automated A/A + regression-injection calibration (Phase 1)
Adds 'bench-runner calibrate': N A/A sessions counted against a <5% false-positive bar, and N sessions with a synthetic slowdown injected as a real busy-wait inside the candidate's timed window, counted against a >=95% detection bar (detected = ratio CI entirely below 1.0). Bootstrap seed varies per session so resampling noise is not correlated across the very statistic being validated. JSON evidence includes per-session CIs, injected wait, acceptance verdict, and env fingerprint. First run against the real comrak workload (20+20 sessions, pinned core): false-positive rate 0.000, detection rate 0.950 -> PASS. Evidence: results/calibration/2026-07-06-comrak-full-calibration.json. Closes the SPEC 3.1 acceptance-criteria gap; remaining Phase 1 gaps: no-net containers, perf profiling, diff-test orchestration, agent IPC. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
This commit is contained in:
@@ -233,3 +233,4 @@ targets/comrak/candidate-*/
|
||||
targets/comrak/asan*/
|
||||
targets/tokei/candidate-*/
|
||||
targets/tokei/asan/
|
||||
!results/calibration/*.json
|
||||
|
||||
Reference in New Issue
Block a user