Claude
5abeb608e0
The parser/service targets only ever needed byte-identical golden replay.
The kernel lane is the case that breaks it: a matmul optimization
(transpose-B for cache locality + eight-accumulator ILP) that REORDERS the
floating-point reduction, so its low bits differ by construction —
byte-identical is the wrong gate.
Adds:
- EquivalencePolicy::compare + Divergence (diff-test): byte-identical is
exact; fp-tolerance tokenizes and compares numeric tokens within
abs + rel*|baseline|, with explicit NaN handling. Non-numeric tokens and
token-count mismatches still fail exactly.
- fp-compare bin: applies a target's equivalence.toml to two output files.
- targets/matmul/kernel.c: matmul_ref (single-accumulator oracle) vs
matmul_opt (reordered, faster), with emit/check/run/bench modes.
- targets/matmul/equivalence.toml: fp-tolerance abs 1e-4 + rel 1e-3.
- scripts/kernel-lane-demo.sh + results/phase5/kernel-lane.txt: byte-identical
FAILs, fp-tolerance is EQUIVALENT, a +0.5 perturbation is still REJECTED,
speedup 3.23× measured with the same interleaved A/B + bootstrap CI.
Ledger row phase5-matmul-opt = needs-human-review: using the FP-tolerance
tier at all is a human-review signal (SPEC §8), same posture as mimalloc.
The trust machinery is unchanged; only the timer and the gate differ, which
is exactly the shape a future GPU kernel would slot into.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
hotpath
Spot cooling for hot paths. A profile-guided optimization agent that only ships wins it can prove.
hotpath profiles real binaries and services, isolates hot paths, proposes optimizations, and accepts a change only when two independent bars are cleared: behavioral equivalence (upstream tests + byte-identical golden replay + differential fuzzing + sanitizers) and statistical significance (the speedup's bootstrap 95% CI lower bound clears a threshold on calibrated hardware). A change without both is discarded and logged.
Consultants sell surveys; compilers sell flags. hotpath sells verified deltas with the methodology attached — a stopwatch on pinned hardware, a proof of equivalence, and an ROI figure whose confidence interval survives hostile review.
📖 Full documentation: the hotpath book · Design spec: SPEC.md · Charter: CLAUDE.md
Verified results
Every number carries its 95% CI and its workload, and every one survived the gates above. Across five phases and 33 ledger rows, zero shipped false accepts — including two pipeline over-accepts the audit caught before anything shipped, each becoming a permanent new gate.
| Target | Win | Verified |
|---|---|---|
| tokei (Rust) — LTO (class 1) | +10.4% | CI [+8.5%, +12.0%] |
| tokei — three class-5 algorithmic wins | +10.7% / +3.7% / +9.9% | each CI-significant |
| cJSON (C) — number-handling rewrite (class 5) | +8.85% | CI [+7.5%, +10.0%] |
| cJSON HTTP service — p50 latency under replay | +6.2% | CI [+5.8%, +7.2%] |
| comrak (Rust) — mimalloc (class 2) | +4.6% | CI [+3.3%, +5.8%], human-ruled |
Layout
crates/ Trust layer — agent has NO write access (SPEC §10)
bench-runner/ Interleaved A/B, bootstrap CIs, A/A calibration, service mode
diff-test/ Equivalence gates, corpus hash-pinning, per-target spec
ledger/ Append-only SQLite attempt ledger (mutation-refusing triggers)
report/ ROI: speedup CI → cores → dollars, methodology inline
verdict/ The pipeline in one command: gates → bench → ledger row
harnessd/ The one door the agent talks through (7 JSON ops)
agent/ Untrusted proposer (Claude Agent SDK, Python)
playbook/ Optimization classes 1–7, tried strictly cheapest-first
config/ accept.toml (thresholds), pricing.toml (ROI inputs)
targets/ Vendored OSS targets — the only agent-writable path
corpora/ Hash-pinned golden-replay inputs (read-only to agent)
results/ Calibration evidence, case studies, generated ROI reports
docs/ The mdBook site (deployed to GitHub Pages)
Quick start
just build / test / lint # trust-layer workspace
just aa # A/A self-test — must yield a null verdict
just gates <target> # corpus pin + upstream tests + golden replay
just verdict <t> <bin> ... # gates + bench vs pristine baseline + ledger row
just report <run-id> # ROI report from a ledger row
just isolation-check # 19 OS-boundary checks (both modes)
Full command reference and reproduction steps: Reproduce it yourself.
How it stays honest
- The agent speaks to the trust layer through seven read-only-plus-one- write JSON operations; it has no shell, cannot write outside a target workspace, and cannot touch the ledger or thresholds.
- The baseline is rebuilt from a pristine checkout every session — a patch never becomes its own comparison.
- The ledger is append-only (SQLite triggers refuse UPDATE/DELETE);
accepted rows are written only by the
verdictbinary after the gates. - The agent process runs under OS-level isolation (read-only mount namespace or an unprivileged uid) — the boundary is filesystem permissions, not a prompt.
See the case studies — especially the caught false-accept, where the pipeline over-accepted a leaking patch and the audit overturned it before it shipped.
License
GPL-3.0-or-later — see LICENSE.