Claude b7297ad31f phase5: learned class-selection policy (SPEC §13) — rank classes by ledger evidence
crates/policy: reads the append-only ledger and ranks playbook classes by
the Wilson score lower bound of their machine-verified win rate — the same
'quote the lower bound' discipline the rest of the project uses. A class
that won 1/1 does not outrank one that won 8/10 (wider interval); untried
classes fall back to the cheapest-first prior; tried-and-failed classes
rank last. Advisory only — the gates still decide every verdict.

On the current 33-row ledger it ranks class 5 (algorithmic) first, then
class 3, then class 1 — the proven winners — with class 7 (untried) above
the evidence-against classes. Wins count only machine-sanitizer-verified
accepts (conservatively excluding pre-gate accepts and the overturned
comrak-010): the policy trusts the ledger's machine record, not narrative.
4 unit tests (Wilson monotonicity, small-sample penalty, ranking order).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
2026-07-12 00:24:00 +00:00
2026-07-06 14:20:25 -04:00

hotpath

Spot cooling for hot paths. A profile-guided optimization agent that only ships wins it can prove.

hotpath profiles real binaries and services, isolates hot paths, proposes optimizations, and accepts a change only when two independent bars are cleared: behavioral equivalence (upstream tests + byte-identical golden replay + differential fuzzing + sanitizers) and statistical significance (the speedup's bootstrap 95% CI lower bound clears a threshold on calibrated hardware). A change without both is discarded and logged.

Consultants sell surveys; compilers sell flags. hotpath sells verified deltas with the methodology attached — a stopwatch on pinned hardware, a proof of equivalence, and an ROI figure whose confidence interval survives hostile review.

📖 Full documentation: the hotpath book · Design spec: SPEC.md · Charter: CLAUDE.md

Verified results

Every number carries its 95% CI and its workload, and every one survived the gates above. Across five phases and 33 ledger rows, zero shipped false accepts — including two pipeline over-accepts the audit caught before anything shipped, each becoming a permanent new gate.

Target Win Verified
tokei (Rust) — LTO (class 1) +10.4% CI [+8.5%, +12.0%]
tokei — three class-5 algorithmic wins +10.7% / +3.7% / +9.9% each CI-significant
cJSON (C) — number-handling rewrite (class 5) +8.85% CI [+7.5%, +10.0%]
cJSON HTTP service — p50 latency under replay +6.2% CI [+5.8%, +7.2%]
comrak (Rust) — mimalloc (class 2) +4.6% CI [+3.3%, +5.8%], human-ruled

Layout

crates/            Trust layer — agent has NO write access (SPEC §10)
  bench-runner/    Interleaved A/B, bootstrap CIs, A/A calibration, service mode
  diff-test/       Equivalence gates, corpus hash-pinning, per-target spec
  ledger/          Append-only SQLite attempt ledger (mutation-refusing triggers)
  report/          ROI: speedup CI → cores → dollars, methodology inline
  verdict/         The pipeline in one command: gates → bench → ledger row
  harnessd/        The one door the agent talks through (7 JSON ops)
agent/             Untrusted proposer (Claude Agent SDK, Python)
playbook/          Optimization classes 1–7, tried strictly cheapest-first
config/            accept.toml (thresholds), pricing.toml (ROI inputs)
targets/           Vendored OSS targets — the only agent-writable path
corpora/           Hash-pinned golden-replay inputs (read-only to agent)
results/           Calibration evidence, case studies, generated ROI reports
docs/              The mdBook site (deployed to GitHub Pages)

Quick start

just build / test / lint       # trust-layer workspace
just aa                        # A/A self-test — must yield a null verdict
just gates <target>            # corpus pin + upstream tests + golden replay
just verdict <t> <bin> ...     # gates + bench vs pristine baseline + ledger row
just report <run-id>           # ROI report from a ledger row
just isolation-check           # 19 OS-boundary checks (both modes)

Full command reference and reproduction steps: Reproduce it yourself.

How it stays honest

  • The agent speaks to the trust layer through seven read-only-plus-one- write JSON operations; it has no shell, cannot write outside a target workspace, and cannot touch the ledger or thresholds.
  • The baseline is rebuilt from a pristine checkout every session — a patch never becomes its own comparison.
  • The ledger is append-only (SQLite triggers refuse UPDATE/DELETE); accepted rows are written only by the verdict binary after the gates.
  • The agent process runs under OS-level isolation (read-only mount namespace or an unprivileged uid) — the boundary is filesystem permissions, not a prompt.

See the case studies — especially the caught false-accept, where the pipeline over-accepted a leaking patch and the audit overturned it before it shipped.

License

GPL-3.0-or-later — see LICENSE.

S
Description
Spot cooling for hot paths. A profile-guided optimization agent that only ships wins it can prove.
Readme MIT
1.3 MiB
Languages
Rust 66.4%
Python 14.4%
Shell 10.9%
C 4.1%
PowerShell 2.9%
Other 1.3%