The project ships as peltier.io; "hotpath" was the placeholder working name
(CLAUDE.md: "Rename freely; nothing depends on it"). Replaces the product
name in the README front page and every mdBook source chapter — book title,
nav header, and prose ("Peltier profiles…", "Peltier sells…") — plus the
theme CSS comment.
The metaphor is unchanged (a Peltier device is a thermoelectric cooler:
"spot cooling for hot paths"), and the literal noun phrase "hot paths"
stays. Two code identifiers are intentionally left as-is because they are
not prose: the `agent/hotpath_agent/` Python module path and the
`HOTPATH_ISOLATION` env var — renaming those is a code change, not a docs
one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Phase 5 write-up for the two SPEC §13 forks now that both are built:
- results/phase5/README.md — the full argument for each fork, with the
kernel-lane demo and learned-policy ranking tables and reproduce commands.
- results/phase5/learned-policy.txt — captured `cargo run -p policy` output
over the 34-row ledger.
- docs/src/research-forks.md — new mdBook chapter, wired into SUMMARY.md.
- docs/src/roadmap.md — Phase 5 row added; the Frontier section rewritten
from "out of scope" to "built, both routed to human review", with the
honest note that the actual GPU run needs GPU hardware absent here.
- CLAUDE.md — Phase 5 COMPLETE status paragraph.
No trust-layer or gate change; this is documentation of work already
committed (learned policy b7297ad, kernel lane 5abeb60).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
The parser/service targets only ever needed byte-identical golden replay.
The kernel lane is the case that breaks it: a matmul optimization
(transpose-B for cache locality + eight-accumulator ILP) that REORDERS the
floating-point reduction, so its low bits differ by construction —
byte-identical is the wrong gate.
Adds:
- EquivalencePolicy::compare + Divergence (diff-test): byte-identical is
exact; fp-tolerance tokenizes and compares numeric tokens within
abs + rel*|baseline|, with explicit NaN handling. Non-numeric tokens and
token-count mismatches still fail exactly.
- fp-compare bin: applies a target's equivalence.toml to two output files.
- targets/matmul/kernel.c: matmul_ref (single-accumulator oracle) vs
matmul_opt (reordered, faster), with emit/check/run/bench modes.
- targets/matmul/equivalence.toml: fp-tolerance abs 1e-4 + rel 1e-3.
- scripts/kernel-lane-demo.sh + results/phase5/kernel-lane.txt: byte-identical
FAILs, fp-tolerance is EQUIVALENT, a +0.5 perturbation is still REJECTED,
speedup 3.23× measured with the same interleaved A/B + bootstrap CI.
Ledger row phase5-matmul-opt = needs-human-review: using the FP-tolerance
tier at all is a human-review signal (SPEC §8), same posture as mimalloc.
The trust machinery is unchanged; only the timer and the gate differ, which
is exactly the shape a future GPU kernel would slot into.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
crates/policy: reads the append-only ledger and ranks playbook classes by
the Wilson score lower bound of their machine-verified win rate — the same
'quote the lower bound' discipline the rest of the project uses. A class
that won 1/1 does not outrank one that won 8/10 (wider interval); untried
classes fall back to the cheapest-first prior; tried-and-failed classes
rank last. Advisory only — the gates still decide every verdict.
On the current 33-row ledger it ranks class 5 (algorithmic) first, then
class 3, then class 1 — the proven winners — with class 7 (untried) above
the evidence-against classes. Wins count only machine-sanitizer-verified
accepts (conservatively excluding pre-gate accepts and the overturned
comrak-010): the policy trusts the ledger's machine record, not narrative.
4 unit tests (Wilson monotonicity, small-sample penalty, ranking order).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
- docs/: an mdBook documentation site — thesis, architecture, measurement
discipline, equivalence gates, agent loop, isolation/anti-reward-hacking,
playbook, ROI, reproduce, roadmap, and the five case studies (pulled from
their single source in results/ via {{#include}}). Builds clean (21
pages).
- .github/workflows/docs.yml: builds mdBook and deploys to GitHub Pages on
push to main (one-time setup: Settings → Pages → Source = GitHub Actions).
- README.md: rewritten as the project front page — the thesis, the verified
results table, layout, quick start, how it stays honest, links to the
book. Replaces the stale Phase-0-era text.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Human ruling on the needs-human-review allocator swap (+4.6%, CI
[+3.3%,+5.8%]): accepted as a real win for the measured comrak CLI
single-stream markdown-render workload, explicitly not a blanket
mimalloc mandate. Ledger row stays needs-human-review (append-only);
the ruling is documented in results/rulings/phase0-comrak-002.md and
reflected in the case study.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Two defects found and fixed in a hostile review of the whole package:
1. The ROI report generator trusted the ledger verdict field and would
mint a clean $16,886/year report for phase2-comrak-010 — the
memory-leaking patch the audit overturned (immutably 'accepted' in the
append-only ledger, sanitizers_clean:false). report_row now exposes
sanitizers_clean and the report treats accepted-but-not-sanitizer-clean
as NOT shippable, printing a warning before any dollar figure. Legit
wins (sanitizers_clean:true) still render clean.
2. bench-runner service --rate 0 / --count 0 could panic via
from_secs_f64(1/0); run_session now guards finite/positive rate and
non-zero count/workers.
Attacks correctly refused (verified, no change): ledger UPDATE/DELETE,
propose_patch ../ + .git escapes, read_target_source traversal. All 10
accepted rows satisfy the green-gates + CI>=1.02 invariant. Findings
recorded in results/adversarial-review.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
phase4-cjson-service (ledger): the accepted batch win phase3-cjson-002
measured under a 150 rps coordinated-omission-correct replay (20 rounds,
40k requests, 0 drops). p50 latency +6.2%, 95% CI [+5.8%, +7.2%] —
accepted; p99 CI [0.07, 4.97] too noisy (single-worker loopback tail),
harness correctly rejected. ROI (500-core fleet, $0.04/core-hr): 27.5
cores / $9,621/year at the CI lower bound, report generated mechanically
(results/phase4/cjson-service-roi.md).
- results/phase4/: case study + README + the generated ROI report.
- CLAUDE.md: Phase 4 complete; Phases 0-4 all done.
All SPEC §5 exit criteria met across every phase; zero shipped false
accepts across the whole project.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
- bench-runner service mode (SPEC §3.1 mode c): a coordinated-omission-
correct, open-loop, fixed-rate load generator; latency measured from
the intended send time so queueing is counted, not hidden. Interleaved
A/B of two server binaries, exact percentiles (bounded sample, no HDR
bucketing error), bootstrap p50/p99 CIs via the existing ratio-CI
machine. service-calibrate: A/A false-positive + injected latency-
regression detection (calibration in progress).
- targets/cjson/service.c: minimal single-threaded HTTP server wrapping
the patched cJSON.c (trust-layer, outside the workspace allowlist);
env-toggled per-request busy-wait for the injection calibration only.
- report generator (crates/report bin): reads a ledger row +
optional service-latency JSON, renders throughput→cores→dollars and/or
latency percentiles, every figure with its CI, pricing source, and the
methodology printed inline (SPEC §9); flags any non-accepted row so a
rejected change never reads as a committed saving.
- ledger::report_row; justfile service/service-calibrate/report.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
phase3-cjson-002 (class 5+3): stack-buffer number parsing + strtod
round-trip replacing sscanf. +8.85% median, 95% CI [+7.5%, +10.0%] on
the C target, banked. 100% audit: byte-identical golden replay,
independent ASan+UBSan clean, 10k-input differential fuzz 0 divergences,
patch bounds/free-gating verified by eye.
- verdict: an infrastructure sanitizer-BUILD failure (missing runtime,
broken toolchain) now caps the accept at needs-human-review and logs
it, instead of crashing the verdict and losing a measured bench.
- results/phase3/: four public case studies (tokei class-5 wins; build
flags + allocators; the comrak caught false-accept; the cJSON
cross-language proof) + README. Compound-speedup claim stated as an
approximation, not a measured figure (CIs don't multiply).
- .gitignore: cjson build-artifact dirs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
- coz wired (harness -DHOTPATH_COZ hook, scripts, just coz) + installed,
but the apt coz runtime aborts in its own init_coz/pthread_create
interposition on this glibc (known coz bug, reproduces on the pristine
binary). Documented in results/cjson/coz/README.md; callgrind cache-sim
is the working profiler.
- cjson no-op verdict smoke: full C pipeline flows (clang {out} baseline
rebuild, Unity/CTest + golden gates, interleaved bench) to a correct
null result (median 0.9995).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
~22% of Ir in glibc float serialization (__printf_fp, cJSON print_number)
and parsing (strtod/str_to_mpn, cJSON parse_number), plus parse_value/
parse_string and malloc/free churn. Ample headroom (SPEC §6).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
20/20 sessions: A/A false-positive rate 0.000 (<0.05 required),
injected-5%-slowdown detection 1.000 (>=0.95 required). The C workload
is a trusted measurement anchor; verdict benches on cjson are now valid.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
diff-test standalone (just gates / CI target-gates) passed the now-
templated build.binary into golden replay verbatim — literal {out}.
It now builds the current tree to targets/<t>/gates-build and tests
that, with a --binary override. CI's target-gates-no-net job gains
clang/cmake, fetches+gates the cjson C target (clang build -> Unity/
CTest -> golden replay) alongside tokei.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
20 audited attempts, 0 shipped false accepts, 3 verified auto-accepted
wins (tokei class-5: +10.7%, +3.7%, +9.9%; ~1.25x vs upstream with
Phase-0 wins). Independent 20-way adversarial re-audit (phase2-final-audit
workflow) confirmed every verdict follows its evidence, correct
equivalence tier, no harness-gaming, ledger append-only intact.
Closeout changes:
- case study: full 20-row attempt table, scoreboard, corrected
comrak-010 CI upper bound (+12.1%, was +12.2%), exit-criteria section
flipped to ALL MET, OS-isolation gap marked closed.
- CLAUDE.md Current-phase line: Phase 2 complete, next Phase 3.
- verdict: derive the ledger 'phase' column from the run_id namespace
(phaseN-*) instead of the hardcoded 1 the re-audit flagged; historical
rows are immutable and stay as written (run_id prefix is authoritative).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
The pipeline auto-accepted an LSan-flagged teardown patch (+10.7%
median) because ASan/LSan were per-attempt manual; the 100% audit
overturned it (black_box liveness ends at scope close, process::exit
runs after — 384-byte arena chunk unreachable at exit). verdict now
builds the patched tree with ASan+LSan and runs the pinned workload on
every would-be accept, capping flagged wins at needs-human-review per
SPEC §8. Shipped false accepts remain 0; the win was not banked; case
study carries the corrective record.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
comrak-010's patch added an unresolvable dependency version; the
candidate build failed before the verdict binary ran, no ledger row was
ever written, and the agent polled read_verdict into its turn cap. The
detached pipeline now traps any failure into a log marker, and
read_verdict reports status=failed with the log tail instead of
'running' forever. Attempt relaunched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Class 5 on tokei: forward-only all-whitespace break test replaces
trim().is_empty() (drops the redundant rposition scan per processed
byte). Median 1.0367, 95% CI [1.027, 1.052]. Equivalence eyeball done
(same AsciiExt::is_whitespace predicate, empty-window case identical);
diff-fuzz + ASan running before the win is counted.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
- propose_patch allows the /dev/null headers git emits for new/deleted
files (previously every new-file diff was refused as an absolute path).
- read_target_source serves HEAD (pristine) content instead of the
working tree, matching propose_patch's standalone-vs-pristine base so
post-proposal reads can't drift a later diff's context lines.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
A stale socket file survived a container idle window while the daemon
died; the [ -S ] check skipped the restart and the nested agent's MCP
server crashed at connect. The agent refused cleanly (no ledger row, no
fabrication). Liveness now means answering read_playbook, not existing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
The agent's parse_context skip-scan passed upstream tests but broke
byte-identical golden replay on the pinned corpus; bench correctly never
ran. First rejected-gate of the grind: the equivalence stack blocking a
wrong-output patch that a green test suite would have shipped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
- Standalone-proposal semantics: workspace resets to pristine before
every apply, so probe patches can't accumulate into a tested tree the
ledger's patch field doesn't describe.
- Friendly errors: explicit trailing-newline check, and git apply's
stderr relayed on refusal instead of a generic message.
- Tool docs + task prompt updated to match.
- phase2-comrak-004 audited: honest rejected-bench (median 1.0124,
CI [1.0011, 1.0267] — real but under the 2% bar), Mutex pre-flagged
for human review, cumulative-probe state disclosed in the hypothesis.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Ledger row: class 5 (algorithmic) on tokei, first-byte gate table for
the per-byte token matchers in perform_multi_line_analysis.
Speedup median 1.107, 95% CI [1.084, 1.131] vs the banked-wins baseline,
workload: 221 MB / 4360 files, single rayon thread, pinned core.
Manual audit gates (pipeline covers tests+golden+bench only):
- Differential fuzz, 10k mutated inputs across 200 batched runs:
0 divergences after canonicalizing tokei's benign parallel-walker
output ordering (present in pristine baseline; counts identical).
Fuzzer added as scripts/diff-fuzz-tokei.py.
- ASan run recorded in the case study (next commit).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Mechanically re-checks a ledger row against the accept rule (CI lower
bound vs threshold, gate greenness, patch path allowlist) and prints
the full evidence for the human part of the 100% Phase 2 audit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
- Profiles regenerated with callgrind --cache-sim --branch-sim for both
targets: per-function D1/LL miss and branch-mispredict columns unlock
playbook class-4 preconditions (comrak: syntect write_highlighted owns
31% of D1 read misses; tokei: parse_lines owns 56% of Ir).
- tokei workspace: accepted Phase 0 wins banked as a local commit so
stash-based baseline rebuilds attribute only the marginal patch to new
attempts (false-accept trap closed before grinding).
- read_ledger now returns per-attempt history (class, hotspot,
hypothesis, verdict): a class may be re-entered with a materially new
hypothesis, never a duplicate one. Task prompt updated to match.
- read_verdict long-polls the ledger up to 45s server-side (under the
60s MCP cap), cutting the agent's polling turn burn ~15x.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
harnessd gains --socket mode (Unix socket, trusted uid, mutating ops
serialized); the harness client speaks socket when
HOTPATH_HARNESS_SOCKET is set. scripts/agent-isolated.sh launches an
attempt with the agent tree confined — mountns mode (repo bind-mounted
read-only, CAP_SYS_ADMIN/CAP_SYS_PTRACE dropped; no credential copying)
or user mode (unprivileged hpagent, bench-metal shape).
scripts/isolation-check.sh + just isolation-check verify the boundary
from the agent's side: 19 checks, both modes, all passing — repo
writes, forged ledger INSERTs, trust-binary replacement, rw remounts,
and userns re-entry all refused; the harnessd socket door works.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
The Claude Agent SDK loop (agent/hotpath_agent/loop.py, model
claude-fable-5) drives the seven harness tools over a standalone stdio
MCP server (mcp_server.py) and runs the full profile -> hypothesize ->
patch -> gated-verdict cycle on comrak with no human in the loop.
Three audited attempts, all honest rejections, ZERO false accepts:
- phase2-comrak-001 class 3: syntect buffer with_capacity, [0.971,1.007]
- phase2-comrak-002 class 5: gate dead VecDeque build, [0.982,1.026]
- phase2-comrak-003 class 6: hoist find_special_char check, [0.970,1.007]
The agent picked the cheapest untried class each time (3->5->6) and
TWICE refused to fabricate a verdict it couldn't read back, verifying
ledger integrity instead — the anti-reward-hacking behavior SPEC §10
targets.
Harness changes surfaced by the loop:
- run_verdict is now async: launches the multi-minute build+bench
detached (past the 60s MCP transport cap) and returns immediately;
new read_verdict op polls the append-only ledger row (ledger
verdict_summary). Fixes attempt 001's unreadable-verdict gap.
- read_target_source takes offset/limit line windows so 100KB+ files
page instead of overflowing — removes the agent's reason to reach for
a shell (attempt 003 paged a 2400-line file cleanly, no shell).
Honest boundary note (case study + code comments): in THIS dev
container the SDK tool allow/deny layer is defense-in-depth, not the
trust boundary — a nested CLI as root under a parent session doesn't
reliably gate built-ins. SPEC §10's boundary is OS process/user
isolation; that hardening is the remaining Phase 2 exit item
(docs/infra.md). What holds regardless: propose_patch is the only
mediated write path (path-allowlist + git apply) and the ledger is
append-only.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Full-protocol calibration on a GitHub-hosted runner: A/A false-positive
0/20, injected-5%-slowdown detection 20/20, per-session CIs transcribed
with run provenance (artifacts expire; the md is the durable record).
With the local-container anchor and gates running end-to-end on both
targets in CI, all SPEC §5 Phase 1 exit criteria are met on free
infrastructure. CLAUDE.md phase line moved to Phase 2 (agent);
dedicated pinned hardware remains listed in docs/infra.md as the bar
for Phase 3/4 customer-facing case studies.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
The 8-session probe passed cleanly (0 false positives, 8/8 detection,
A/A CIs ~±0.6%); run the same 20-session protocol used for the local
trust anchor so GH-runner evidence is directly comparable.
Closes the CI-reachable slice of the Phase 1 infra gaps:
- trust-layer job: cargo test + clippy -D warnings + fmt
- target-gates-no-net: fetches pinned targets (new targets/fetch.sh),
regenerates + verifies corpora, runs 'just gates tokei'
mechanically, and executes the bench workload inside a
--network=none container (SPEC 3.1/10 no-net requirement)
- runner-calibration (informational): A/A + injection harness measures
the shared runner's noise floor per run, evidence uploaded as an
artifact — environments are judged empirically, not assumed
- perf-probe (informational): reports which perf event sources work
docs/infra.md maps each remaining hardware requirement (pinned cores,
governor/SMT control, separate-uid trust layer) to the bench-metal
self-hosted runner setup a human can complete in ~15 min.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
The agent's only door to the trust layer (SPEC 3.5, 10): harnessd
serves read_profile, read_ledger, read_playbook, read_target_source,
propose_patch, run_verdict over stdio JSON lines. propose_patch
enforces the path allowlist (relative paths only, no parent
components, no .git, target name validated) before git apply --check
+ apply rooted in the target workspace; run_verdict builds the
candidate into an isolated dir and hands off to the verdict pipeline
(pristine-rebuilt baseline included). Python client
(agent/hotpath_agent/harness.py) + tools.py rewired from stubs to
live calls.
E2E evidence: escape attempts refused (bad target name, ../ paths,
trust-layer diff touching config/accept.toml rejected before any
apply); benign doc patch applied, candidate built, full verdict ran
mechanically end to end -> +8.9% [1.079, 1.104] accepted (ship-tree
patches ride along as the candidate), ledger row phase1-ipc-smoke-001.
Phase 1 caveat recorded in the daemon header: same-uid filesystem
read-only enforcement still needs the separate-user/container setup.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Adds crates/verdict ('just verdict'): runs the core gates on the
candidate binary, then an interleaved A/B bench vs the baseline with
the CI-lower-bound rule, and writes the append-only ledger row with
per-side median CIs, workload statement, and env fingerprint. Gate
failure short-circuits to rejected-gate; --needs-human-review caps an
accepted verdict (never upgrades). CPU pinning moves into accept.toml
(pin_prefix, machine-specific, applied to both sides identically).
The smoke test caught a real trust gap: pointing --baseline-bin at the
workspace build (phase1-verdict-smoke-001) measured patched-vs-patched
and returned null, because gate runs rebuild the workspace binary from
the patched tree. Fix: --rebuild-baseline stashes the patch, builds
the baseline into isolated targets/<name>/baseline/, and restores the
patch — SPEC 3.1's pristine-checkout rule, now enforced by the
harness. Re-run (phase1-verdict-smoke-002) reproduces the Phase 0
result mechanically: +10.4% median, 95% CI [1.0932, 1.1137], accepted.
Both smoke rows are in the ledger, including the null one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Adds the diff-test CLI ('just gates <target>'): loads
targets/<name>/target.toml (new [corpus] section: manifest, root,
golden_sha256; commands now repo-root-relative), verifies the corpus
manifest FIRST and refuses to run any gate on mismatch, then runs the
upstream test suite and golden replay (stdout sha256 vs pinned hash).
Differential fuzz and sanitizers remain per-attempt manual steps in
Phase 1 and are reported as skipped-with-reason; all-skipped is not a
pass.
Verified end-to-end on both vendored targets (comrak: 1 pinned file,
848 tests, golden PASS; tokei: 545 pinned files, 250 tests, golden
PASS) and the refusal path (tampered corpus -> 'HASH MISMATCH ...
refusing to run' before any gate executes).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Adds 'bench-runner calibrate': N A/A sessions counted against a <5%
false-positive bar, and N sessions with a synthetic slowdown injected
as a real busy-wait inside the candidate's timed window, counted
against a >=95% detection bar (detected = ratio CI entirely below 1.0).
Bootstrap seed varies per session so resampling noise is not
correlated across the very statistic being validated. JSON evidence
includes per-session CIs, injected wait, acceptance verdict, and env
fingerprint.
First run against the real comrak workload (20+20 sessions, pinned
core): false-positive rate 0.000, detection rate 0.950 -> PASS.
Evidence: results/calibration/2026-07-06-comrak-full-calibration.json.
Closes the SPEC 3.1 acceptance-criteria gap; remaining Phase 1 gaps:
no-net containers, perf profiling, diff-test orchestration, agent IPC.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Attempt 4 (phase0-tokei-002, class 3, ACCEPTED — first auto-accept):
every file was read through encoding_rs_io's DecodeReaderBytes via
generic read_to_end, which zero-fills every grown buffer chunk (~221MB
memset/run, 8.5% of Ir) and grows without a size hint. Patch: fs::read
with a BOM-gated fallback to the identical decode path (a UTF-8-BOM
file transcodes invalid sequences, so the fallback is required for
equivalence). Gates: golden byte-identical; differential fuzz old-vs-
new binaries 4,332 iterations (BOM grid, invalid UTF-8, binary junk,
randomized) zero divergence; 250 upstream tests green; ASan+LSan clean
with ASan-run output hashing to golden. Bench: +9.9% [8.0%, 12.7%].
Attempt 5 (phase0-tokei-003, ship tree, ACCEPTED): stacks fat LTO+cgu=1
(real +1.6% standalone but under threshold, phase0-tokei-001) on the
accepted patch; benched as a whole vs pristine: +10.4% [8.5%, 12.0%].
Median clears the Phase 0 exit bar; CI lower bound 8.5%.
Case study written (results/phase0/case-study-tokei.md), ledger at 5
attempts, CLAUDE.md phase line moved to Phase 1, next hypotheses
logged (tokei class-5 byte-skip table; comrak class-3 arena).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Session-level non-negotiables, architecture map, phase tracker,
command surface, and definition of done for one optimization attempt.
Commands section notes the currently built command-level harness
entries alongside the planned per-target ones.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Commit the spec and scaffold the system it describes:
- SPEC.md: the full v0.1 engineering spec
- crates/: trust-layer Rust workspace, unit-tested
- bench-runner: interleaved ABAB scheduling with discarded warm-ups,
deterministic bootstrap CIs over the ratio of medians,
CI-lower-bound accept rule, env fingerprinting, and a CLI with
compare + A/A self-test modes (A/A verified passing end-to-end)
- diff-test: gate outcome model, sha256 corpus pin verifier that
refuses on mismatch, per-target equivalence policy parser
(byte-identical default, explicit fp-tolerance opt-in)
- ledger: append-only SQLite attempt schema with UPDATE/DELETE
refused by triggers, anti-double-attempt query
- report: cores-saved and dollars/yr ROI math over speedup CIs,
pricing.toml inputs
- playbook/: classes 1-7 with preconditions, procedure, verification
notes, and known failure modes per class
- agent/: Python package skeleton — six-tool contract and prompting
spine, harness IPC deferred to Phase 1
- config/: accept.toml (2% threshold, 30 runs/side), pricing.toml
- targets/, corpora/, results/calibration/: layout and policy docs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Claude