3 Commits

Include renames

Author	SHA1	Message	Date
benbrownandClaude Fable 5	444e5294fa	explain: post-verdict diagnostics — SPEC §3.7, crate, docs ... Verdicts say what; nothing said why. A rejection that reads only rejected-bench teaches nothing, and the loop's compounding value is that the next hypothesis is better than the last. This ships the teaching layer, built so it can never contaminate the thing it explains. New crate crates/explain (`just explain <run-id>`): - Advisory, strictly off the accept path: runs after the verdict is recorded; nothing it emits feeds gates, bench, verdict, or ledger. Not agent-facing: harnessd stays at seven ops, by design (SPEC §3.7 says why - better hypotheses must never become a feedback channel into measurement). - Ledger-only inputs, deterministic: consumes exactly one row via the new ledger::attempt_row; no config, no clock, no re-benching. Identical row, byte-identical text - proven live (cmp on two runs of a real row). - Record vs inference is typographic: restatements print plain; anything beyond is prefixed `inference:` so an explanation can never be quoted as a measurement. - The narrative must agree with the machine verdict. The first live run caught this class of bug: an accepted row without a recorded threshold was narrated as "a policy rejection" two lines under "verdict: accepted". Accepted rows are now always described by their CI lower bound (the defensible claim); rejected-bench rows classify against 1.0 (measured regression / null result / real-but-below-bar); NHR rows name the recorded cap and state the numbers were not the deciding factor; rejected-gate rows name the failing gate. Regression-tested. - Historical rows are flagged, not laundered: accepted with sanitizers_clean=false prints the doctrine caution (comrak-010 shows it); fuzz_iters=0 accepts are named pre-integration rows. verdict now records accept_threshold (the bar in force for that run) into the bench env fingerprint, so rows are self-contained going forward - explain never substitutes today's config for a historical bar, and says so on older rows instead. SPEC §3.7 (Added 2026-07-20) carries the invariants; book chapter docs/src/explain.md shows real output from real rows (the accepted cJSON win, the caught false-accept, the kernel-lane NHR); CLAUDE.md, README, justfile, and the architecture layout updated (the layout also gains the previously-omitted crates/policy line). Verified: 21 workspace test suites green (6 new explain tests incl. determinism and verdict-narrative agreement), clippy -D warnings clean, fmt clean, mdbook builds, live runs against phase3-cjson-002, phase5-matmul-opt, phase2-comrak-010, and phase4-cjson-service. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-20 09:53:14 -04:00
benbrownandClaude Opus 4.8	899d55f4b6	gates: close the audit findings — enforce the gates the docs claimed ... An adversarial audit found the enforcement story ahead of the code: gates that were documented as hard gates were advisory, manual, or absent, and several human-facing claims outran the machine record. This makes the claims true. High findings - Differential fuzz was documentation, not enforcement: verdict hardcoded fuzz_iters=0 and diff-test hardcoded the gate to Skipped, while the case studies claimed 10k-iteration fuzz on the same wins. Fuzz is now a real pipeline gate (strict FUZZ-RESULT contract, executed count recorded), and it is inherently baseline-vs-candidate, so it runs on the accept path and reports Skipped in the standalone `just gates` flow. New hard rule: a machine `accepted` verdict is impossible unless the fuzz gate actually passed — otherwise it caps at needs-human-review. - The agent's system prompt claimed FP/concurrency/unsafe patches were "routed to human review automatically". No such mechanism existed. Added a conservative lexical risk classifier (crates/verdict/src/risk.rs) that forces needs-human-review on any would-be accept carrying those signals; fp-tolerance equivalence always signals. - The local verdict path ran agent-influenced code unsandboxed. It now runs under scripts/no-net.sh (network namespace) and fails closed. Upstream test suites are hash-verified against corpora/<t>/TESTSUITE.sha256 when present (scripts/pin-testsuite.sh generates them). Also fixed, found while fixing the above - The pipeline script's `set -e` was inert (brace group as the left operand of `&&`), so a failing candidate build did not abort the run and verdict could bench a stale binary. Now a scoped subshell whose status is branched on at top level. - A wrapper that exits before exec'ing (no-net.sh refusing, bad wrapper path) stranded RUNNING.lock and left read_verdict polling forever. Failure accounting now lives outside the wrapper. - harnessd's patch allowlist ignored git rename/copy headers and --no-prefix diffs; the rename-escape case is now a committed regression test. - The workspace did not compile on Windows (ungated Unix APIs). cfg-gated; 71 portable tests now run on both platforms. - pin-check regenerated the corpus manifest instead of verifying it; it is now verify-only, with a deliberate `just pin-corpus` to re-pin. - Max RSS is captured per run on Unix; PMU counters are opt-in diagnostics (compare --perf-stat), never part of the accept decision. Docs now match the code, including what is NOT yet enforced: no target ships a TESTSUITE.sha256 pin, no target declares [build].tsan, and matmul is not wired to the fp-tolerance gate — each is stated as mechanized-but-inert rather than done. Corrected the stale "33 ledger rows" (34), the playbook's class-3 cJSON win (rejected), adversarial-review.md's sanitizer-separation claim (five accepted rows carry sanitizers_clean=false), the unsupported "40,000 requests" figure, and the blanket "historical rows record fuzz_iters=0" (8 of 10; phase0-tokei-002/-003 record 4,332). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-14 13:10:14 -04:00
Claude	328aafae77	Bootstrap hotpath from engineering spec v0.1 ... Commit the spec and scaffold the system it describes: - SPEC.md: the full v0.1 engineering spec - crates/: trust-layer Rust workspace, unit-tested - bench-runner: interleaved ABAB scheduling with discarded warm-ups, deterministic bootstrap CIs over the ratio of medians, CI-lower-bound accept rule, env fingerprinting, and a CLI with compare + A/A self-test modes (A/A verified passing end-to-end) - diff-test: gate outcome model, sha256 corpus pin verifier that refuses on mismatch, per-target equivalence policy parser (byte-identical default, explicit fp-tolerance opt-in) - ledger: append-only SQLite attempt schema with UPDATE/DELETE refused by triggers, anti-double-attempt query - report: cores-saved and dollars/yr ROI math over speedup CIs, pricing.toml inputs - playbook/: classes 1-7 with preconditions, procedure, verification notes, and known failure modes per class - agent/: Python package skeleton — six-tool contract and prompting spine, harness IPC deferred to Phase 1 - config/: accept.toml (2% threshold, 30 runs/side), pricing.toml - targets/, corpora/, results/calibration/: layout and policy docs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2	2026-07-06 18:34:51 +00:00