mirror of
https://github.com/bdbrown4/peltier.io.git
synced 2026-11-07 10:58:28 +00:00
Claude
baa6c4e967
Adds crates/verdict ('just verdict'): runs the core gates on the
candidate binary, then an interleaved A/B bench vs the baseline with
the CI-lower-bound rule, and writes the append-only ledger row with
per-side median CIs, workload statement, and env fingerprint. Gate
failure short-circuits to rejected-gate; --needs-human-review caps an
accepted verdict (never upgrades). CPU pinning moves into accept.toml
(pin_prefix, machine-specific, applied to both sides identically).
The smoke test caught a real trust gap: pointing --baseline-bin at the
workspace build (phase1-verdict-smoke-001) measured patched-vs-patched
and returned null, because gate runs rebuild the workspace binary from
the patched tree. Fix: --rebuild-baseline stashes the patch, builds
the baseline into isolated targets/<name>/baseline/, and restores the
patch — SPEC 3.1's pristine-checkout rule, now enforced by the
harness. Re-run (phase1-verdict-smoke-002) reproduces the Phase 0
result mechanically: +10.4% median, 95% CI [1.0932, 1.1137], accepted.
Both smoke rows are in the ledger, including the null one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
26 lines
767 B
TOML
26 lines
767 B
TOML
# Accept thresholds for bench-runner (SPEC §3.1).
|
|
# This file is trust-layer: read-only to the agent (SPEC §10).
|
|
|
|
# CI lower bound of the speedup ratio must clear 1 + threshold.
|
|
threshold = 0.02
|
|
|
|
# Measured runs per side (SPEC minimum: 30). Interleaved ABAB.
|
|
runs_per_side = 30
|
|
|
|
# Warm-up runs per side, discarded.
|
|
warmup_runs = 3
|
|
|
|
# Bootstrap resamples for the ratio CI.
|
|
bootstrap_iters = 10000
|
|
|
|
# Confidence level for the ratio CI.
|
|
confidence = 0.95
|
|
|
|
# Fixed PRNG seed so calibration runs are reproducible.
|
|
bootstrap_seed = 1887343589
|
|
|
|
# Machine-specific CPU pinning applied identically to both bench sides.
|
|
# Calibration evidence for this machine shows pinning tightens the A/A
|
|
# CI ~5x (results/calibration/). Empty string disables.
|
|
pin_prefix = "taskset -c 2"
|