mirror of
https://github.com/bdbrown4/peltier.io.git
synced 2026-11-07 10:58:28 +00:00
Bootstrap hotpath from engineering spec v0.1
Commit the spec and scaffold the system it describes:
- SPEC.md: the full v0.1 engineering spec
- crates/: trust-layer Rust workspace, unit-tested
- bench-runner: interleaved ABAB scheduling with discarded warm-ups,
deterministic bootstrap CIs over the ratio of medians,
CI-lower-bound accept rule, env fingerprinting, and a CLI with
compare + A/A self-test modes (A/A verified passing end-to-end)
- diff-test: gate outcome model, sha256 corpus pin verifier that
refuses on mismatch, per-target equivalence policy parser
(byte-identical default, explicit fp-tolerance opt-in)
- ledger: append-only SQLite attempt schema with UPDATE/DELETE
refused by triggers, anti-double-attempt query
- report: cores-saved and dollars/yr ROI math over speedup CIs,
pricing.toml inputs
- playbook/: classes 1-7 with preconditions, procedure, verification
notes, and known failure modes per class
- agent/: Python package skeleton — six-tool contract and prompting
spine, harness IPC deferred to Phase 1
- config/: accept.toml (2% threshold, 30 runs/side), pricing.toml
- targets/, corpora/, results/calibration/: layout and policy docs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
This commit is contained in:
@@ -0,0 +1,43 @@
|
||||
//! bench-runner core (SPEC §3.1, §7).
|
||||
//!
|
||||
//! The product is this crate: interleaved A/B measurement, bootstrap CIs
|
||||
//! over the ratio of medians, threshold-gated accept decisions, and A/A
|
||||
//! self-calibration. Pure statistics live here and are unit-tested;
|
||||
//! process execution and `perf stat` integration live in `exec`.
|
||||
|
||||
pub mod config;
|
||||
pub mod exec;
|
||||
pub mod fingerprint;
|
||||
pub mod schedule;
|
||||
pub mod stats;
|
||||
|
||||
use ledger::Verdict;
|
||||
|
||||
/// Decide a verdict from a speedup CI and the accept threshold.
|
||||
///
|
||||
/// Speedup is baseline/candidate (>1 means the candidate is faster).
|
||||
/// Accepted only if the CI *lower bound* clears `1 + threshold` — the
|
||||
/// point estimate is never enough (SPEC §3.1).
|
||||
pub fn decide(speedup_ci: stats::RatioCi, threshold: f64) -> Verdict {
|
||||
if speedup_ci.lo >= 1.0 + threshold {
|
||||
Verdict::Accepted
|
||||
} else {
|
||||
Verdict::RejectedBench
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::stats::RatioCi;
|
||||
|
||||
#[test]
|
||||
fn accept_requires_ci_lower_bound_not_point_estimate() {
|
||||
// 10% median speedup but the CI dips below threshold: rejected.
|
||||
let noisy = RatioCi { median: 1.10, lo: 1.005, hi: 1.21 };
|
||||
assert_eq!(decide(noisy, 0.02), Verdict::RejectedBench);
|
||||
// 4% speedup with a tight CI: accepted.
|
||||
let tight = RatioCi { median: 1.04, lo: 1.025, hi: 1.055 };
|
||||
assert_eq!(decide(tight, 0.02), Verdict::Accepted);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user