diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..d4c406a --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,79 @@ +# hotpath — profile-guided optimization agent + +> Working name. Rename freely; nothing depends on it. + +## Mission + +An agent that profiles real binaries and services, isolates hot paths, proposes optimizations, and accepts a change **only** when (a) behavioral-equivalence gates pass and (b) the benchmark improvement is statistically significant on trusted infrastructure. Output: verified performance deltas translated into ROI reports nobody can dispute. + +Full engineering spec: `SPEC.md`. Read it before starting any phase. + +## Non-negotiables (re-read every session) + +1. **No unverified performance claims.** A change without passing equivalence gates AND a significant bench delta does not exist. Discard it. Log it. +2. **Measure before optimizing.** No patch is proposed until profile data identifies the hot path. Never trust a single benchmark run. +3. **Cheap wins first, always.** Build flags → LTO → PGO → allocator swap → *then* code changes. Check the ledger before re-attempting any class of change on a target. +4. **Equivalence gates are hard gates.** Changes touching floating-point ordering, concurrency primitives, or anything UB-adjacent are NEVER auto-accepted — verdict is `needs-human-review`. +5. **Every attempt goes in the ledger**, including failures and rejections. Failures are future training data. +6. **The agent never modifies the trust layer.** `crates/`, `config/`, `corpora/`, and upstream test suites are read-only to the agent. Patches may touch only allowlisted paths under `targets//`. See SPEC.md §10. +7. **Sandbox everything.** Target code runs only inside no-network containers. Never on the host. +8. **Statistical bar:** accept only if the bootstrap 95% CI lower bound of speedup ≥ threshold in `config/accept.toml` (default 2%), from interleaved A/B runs on the same pinned hardware. + +## Architecture map + +- `crates/bench-runner` — trust layer: containerized, CPU-pinned, interleaved A/B benchmarking with bootstrap confidence intervals and A/A self-tests +- `crates/diff-test` — behavioral equivalence: upstream test suite + golden I/O replay + differential fuzzing of changed functions + sanitizers +- `crates/ledger` — append-only SQLite record of every attempt: hypothesis, patch, gate results, bench deltas, verdict, cost +- `crates/report` — ROI generator: cores saved × $/core-hr, latency percentile deltas, CIs, workload caveats printed on every number +- `agent/` — Claude Agent SDK (Python) loop, prompts, tool definitions +- `targets/` — vendored OSS targets for case studies (permissive licenses only) +- `playbook/` — optimization classes, ordered, with preconditions and known risks +- `results/` — generated reports and flamegraphs + +## Current phase + +**Phase 0 — manual dry run.** (Update this line as phases complete; exit criteria in SPEC.md §5.) + +## Commands + +Fill these in as the harness is built; keep them current: + +``` +just profile # perf record + flamegraph for a target's benchmark workload +just bench # interleaved A/B: pristine baseline vs working tree +just gates # test suite + golden replay + diff-fuzz + sanitizers +just verdict # runs gates + bench, writes ledger row, prints verdict +just report # ROI report from ledger +just aa # A/A calibration run (must show null result) +``` + +Built so far (command-level, pre-target-integration): + +``` +just build / test / lint # trust-layer workspace: cargo build/test/clippy+fmt +just aa [cmd] # A/A self-test of bench-runner on a shell command +just compare # interleaved A/B of two shell commands, bootstrap CI +just pin-check # verify corpus MANIFEST.sha256 +``` + +## Definition of done — one optimization attempt + +- [ ] Hypothesis logged in ledger *before* patching +- [ ] Patch touches only allowlisted paths under `targets//` +- [ ] Upstream test suite green +- [ ] Golden replay byte-identical (or within the target's explicit FP tolerance policy) +- [ ] Differential fuzz on changed functions: no divergence (60s or 10k iterations minimum) +- [ ] ASan + UBSan clean; TSan if the patch touches anything threaded +- [ ] Interleaved A/B bench passes the CI-lower-bound threshold +- [ ] Ledger row written with verdict and full evidence +- [ ] If FP-ordering / concurrency / UB-adjacent: verdict `needs-human-review`, never auto-accept + +An attempt that fails any gate is a **valid, complete outcome** — write the ledger row and move on. Do not iterate on a rejected patch more than twice without a new hypothesis. + +## Conventions + +- Rust workspace (edition 2021+) for all harness crates; `just` for task running +- Python 3.11+ for `agent/`; Claude Agent SDK — verify current package name and API at https://docs.claude.com/en/docs/claude-code/overview before scaffolding +- Every number shown to a human carries its confidence interval and workload description. No naked percentages, ever. +- Commit style: `phase0:`, `bench:`, `gates:`, `agent:`, `playbook:` prefixes +- When in doubt between shipping a feature and hardening measurement, harden measurement. The product is trust.