Bootstrap hotpath from engineering spec v0.1

Commit the spec and scaffold the system it describes:

- SPEC.md: the full v0.1 engineering spec
- crates/: trust-layer Rust workspace, unit-tested
  - bench-runner: interleaved ABAB scheduling with discarded warm-ups,
    deterministic bootstrap CIs over the ratio of medians,
    CI-lower-bound accept rule, env fingerprinting, and a CLI with
    compare + A/A self-test modes (A/A verified passing end-to-end)
  - diff-test: gate outcome model, sha256 corpus pin verifier that
    refuses on mismatch, per-target equivalence policy parser
    (byte-identical default, explicit fp-tolerance opt-in)
  - ledger: append-only SQLite attempt schema with UPDATE/DELETE
    refused by triggers, anti-double-attempt query
  - report: cores-saved and dollars/yr ROI math over speedup CIs,
    pricing.toml inputs
- playbook/: classes 1-7 with preconditions, procedure, verification
  notes, and known failure modes per class
- agent/: Python package skeleton — six-tool contract and prompting
  spine, harness IPC deferred to Phase 1
- config/: accept.toml (2% threshold, 30 runs/side), pricing.toml
- targets/, corpora/, results/calibration/: layout and policy docs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
This commit is contained in:
Claude
2026-07-06 18:34:51 +00:00
parent 836d4c9208
commit 328aafae77
40 changed files with 2468 additions and 0 deletions
+43
View File
@@ -0,0 +1,43 @@
//! bench-runner core (SPEC §3.1, §7).
//!
//! The product is this crate: interleaved A/B measurement, bootstrap CIs
//! over the ratio of medians, threshold-gated accept decisions, and A/A
//! self-calibration. Pure statistics live here and are unit-tested;
//! process execution and `perf stat` integration live in `exec`.
pub mod config;
pub mod exec;
pub mod fingerprint;
pub mod schedule;
pub mod stats;
use ledger::Verdict;
/// Decide a verdict from a speedup CI and the accept threshold.
///
/// Speedup is baseline/candidate (>1 means the candidate is faster).
/// Accepted only if the CI *lower bound* clears `1 + threshold` — the
/// point estimate is never enough (SPEC §3.1).
pub fn decide(speedup_ci: stats::RatioCi, threshold: f64) -> Verdict {
if speedup_ci.lo >= 1.0 + threshold {
Verdict::Accepted
} else {
Verdict::RejectedBench
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::stats::RatioCi;
#[test]
fn accept_requires_ci_lower_bound_not_point_estimate() {
// 10% median speedup but the CI dips below threshold: rejected.
let noisy = RatioCi { median: 1.10, lo: 1.005, hi: 1.21 };
assert_eq!(decide(noisy, 0.02), Verdict::RejectedBench);
// 4% speedup with a tight CI: accepted.
let tight = RatioCi { median: 1.04, lo: 1.025, hi: 1.055 };
assert_eq!(decide(tight, 0.02), Verdict::Accepted);
}
}