crates/policy: reads the append-only ledger and ranks playbook classes by
the Wilson score lower bound of their machine-verified win rate — the same
'quote the lower bound' discipline the rest of the project uses. A class
that won 1/1 does not outrank one that won 8/10 (wider interval); untried
classes fall back to the cheapest-first prior; tried-and-failed classes
rank last. Advisory only — the gates still decide every verdict.
On the current 33-row ledger it ranks class 5 (algorithmic) first, then
class 3, then class 1 — the proven winners — with class 7 (untried) above
the evidence-against classes. Wins count only machine-sanitizer-verified
accepts (conservatively excluding pre-gate accepts and the overturned
comrak-010): the policy trusts the ledger's machine record, not narrative.
4 unit tests (Wilson monotonicity, small-sample penalty, ranking order).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
- bench-runner service mode (SPEC §3.1 mode c): a coordinated-omission-
correct, open-loop, fixed-rate load generator; latency measured from
the intended send time so queueing is counted, not hidden. Interleaved
A/B of two server binaries, exact percentiles (bounded sample, no HDR
bucketing error), bootstrap p50/p99 CIs via the existing ratio-CI
machine. service-calibrate: A/A false-positive + injected latency-
regression detection (calibration in progress).
- targets/cjson/service.c: minimal single-threaded HTTP server wrapping
the patched cJSON.c (trust-layer, outside the workspace allowlist);
env-toggled per-request busy-wait for the injection calibration only.
- report generator (crates/report bin): reads a ledger row +
optional service-latency JSON, renders throughput→cores→dollars and/or
latency percentiles, every figure with its CI, pricing source, and the
methodology printed inline (SPEC §9); flags any non-accepted row so a
rejected change never reads as a committed saving.
- ledger::report_row; justfile service/service-calibrate/report.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
The agent's only door to the trust layer (SPEC 3.5, 10): harnessd
serves read_profile, read_ledger, read_playbook, read_target_source,
propose_patch, run_verdict over stdio JSON lines. propose_patch
enforces the path allowlist (relative paths only, no parent
components, no .git, target name validated) before git apply --check
+ apply rooted in the target workspace; run_verdict builds the
candidate into an isolated dir and hands off to the verdict pipeline
(pristine-rebuilt baseline included). Python client
(agent/hotpath_agent/harness.py) + tools.py rewired from stubs to
live calls.
E2E evidence: escape attempts refused (bad target name, ../ paths,
trust-layer diff touching config/accept.toml rejected before any
apply); benign doc patch applied, candidate built, full verdict ran
mechanically end to end -> +8.9% [1.079, 1.104] accepted (ship-tree
patches ride along as the candidate), ledger row phase1-ipc-smoke-001.
Phase 1 caveat recorded in the daemon header: same-uid filesystem
read-only enforcement still needs the separate-user/container setup.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Adds crates/verdict ('just verdict'): runs the core gates on the
candidate binary, then an interleaved A/B bench vs the baseline with
the CI-lower-bound rule, and writes the append-only ledger row with
per-side median CIs, workload statement, and env fingerprint. Gate
failure short-circuits to rejected-gate; --needs-human-review caps an
accepted verdict (never upgrades). CPU pinning moves into accept.toml
(pin_prefix, machine-specific, applied to both sides identically).
The smoke test caught a real trust gap: pointing --baseline-bin at the
workspace build (phase1-verdict-smoke-001) measured patched-vs-patched
and returned null, because gate runs rebuild the workspace binary from
the patched tree. Fix: --rebuild-baseline stashes the patch, builds
the baseline into isolated targets/<name>/baseline/, and restores the
patch — SPEC 3.1's pristine-checkout rule, now enforced by the
harness. Re-run (phase1-verdict-smoke-002) reproduces the Phase 0
result mechanically: +10.4% median, 95% CI [1.0932, 1.1137], accepted.
Both smoke rows are in the ledger, including the null one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Adds the diff-test CLI ('just gates <target>'): loads
targets/<name>/target.toml (new [corpus] section: manifest, root,
golden_sha256; commands now repo-root-relative), verifies the corpus
manifest FIRST and refuses to run any gate on mismatch, then runs the
upstream test suite and golden replay (stdout sha256 vs pinned hash).
Differential fuzz and sanitizers remain per-attempt manual steps in
Phase 1 and are reported as skipped-with-reason; all-skipped is not a
pass.
Verified end-to-end on both vendored targets (comrak: 1 pinned file,
848 tests, golden PASS; tokei: 545 pinned files, 250 tests, golden
PASS) and the refusal path (tampered corpus -> 'HASH MISMATCH ...
refusing to run' before any gate executes).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Commit the spec and scaffold the system it describes:
- SPEC.md: the full v0.1 engineering spec
- crates/: trust-layer Rust workspace, unit-tested
- bench-runner: interleaved ABAB scheduling with discarded warm-ups,
deterministic bootstrap CIs over the ratio of medians,
CI-lower-bound accept rule, env fingerprinting, and a CLI with
compare + A/A self-test modes (A/A verified passing end-to-end)
- diff-test: gate outcome model, sha256 corpus pin verifier that
refuses on mismatch, per-target equivalence policy parser
(byte-identical default, explicit fp-tolerance opt-in)
- ledger: append-only SQLite attempt schema with UPDATE/DELETE
refused by triggers, anti-double-attempt query
- report: cores-saved and dollars/yr ROI math over speedup CIs,
pricing.toml inputs
- playbook/: classes 1-7 with preconditions, procedure, verification
notes, and known failure modes per class
- agent/: Python package skeleton — six-tool contract and prompting
spine, harness IPC deferred to Phase 1
- config/: accept.toml (2% threshold, 30 runs/side), pricing.toml
- targets/, corpora/, results/calibration/: layout and policy docs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Claude