# hotpath — profile-guided optimization agent

> Working name. Rename freely; nothing depends on it.

## Mission

An agent that profiles real binaries and services, isolates hot paths, proposes optimizations, and accepts a change **only** when (a) behavioral-equivalence gates pass and (b) the benchmark improvement is statistically significant on trusted infrastructure. Output: verified performance deltas translated into ROI reports nobody can dispute.

Full engineering spec: `SPEC.md`. Read it before starting any phase.

## Non-negotiables (re-read every session)

1. **No unverified performance claims.** A change without passing equivalence gates AND a significant bench delta does not exist. Discard it. Log it.
2. **Measure before optimizing.** No patch is proposed until profile data identifies the hot path. Never trust a single benchmark run.
3. **Cheap wins first, always.** Build flags → LTO → PGO → allocator swap → *then* code changes. Check the ledger before re-attempting any class of change on a target.
4. **Equivalence gates are hard gates.** Changes touching floating-point ordering, concurrency primitives, or anything UB-adjacent are NEVER auto-accepted — verdict is `needs-human-review`.
5. **Every attempt goes in the ledger**, including failures and rejections. Failures are future training data.
6. **The agent never modifies the trust layer.** `crates/`, `config/`, `corpora/`, and upstream test suites are read-only to the agent. Patches may touch only allowlisted paths under `targets/<name>/`. See SPEC.md §10.
7. **Sandbox everything.** Target code runs only inside no-network containers. Never on the host.
8. **Statistical bar:** accept only if the bootstrap 95% CI lower bound of speedup ≥ threshold in `config/accept.toml` (default 2%), from interleaved A/B runs on the same pinned hardware.

## Architecture map

- `crates/bench-runner` — trust layer: containerized, CPU-pinned, interleaved A/B benchmarking with bootstrap confidence intervals and A/A self-tests
- `crates/diff-test` — behavioral equivalence: upstream test suite + golden I/O replay + differential fuzzing of changed functions + sanitizers
- `crates/ledger` — append-only SQLite record of every attempt: hypothesis, patch, gate results, bench deltas, verdict, cost
- `crates/report` — ROI generator: cores saved × $/core-hr, latency percentile deltas, CIs, workload caveats printed on every number
- `agent/` — Claude Agent SDK (Python) loop, prompts, tool definitions
- `targets/` — vendored OSS targets for case studies (permissive licenses only)
- `playbook/` — optimization classes, ordered, with preconditions and known risks
- `results/` — generated reports and flamegraphs

## Current phase

**Phase 3 COMPLETE — all SPEC §5 exit criteria met.** The trust layer went language-agnostic: build isolation moved from a cargo-specific `CARGO_TARGET_DIR` injection to an explicit `{out}` placeholder + optional `sanitizer`/`sanitizer_binary` templates (`crates/diff-test/src/target.rs`), proven zero-regression on tokei before any C target. **A C target joined the roster** — cJSON v1.7.19 (MIT): trust-layer harness driver compiled with the patched `cJSON.c`, hermetic Unity/CTest gate (19 tests), ASan+UBSan sanitizers, deterministic pinned corpus, own A/A calibration (0/20 FP, 20/20 injection), CI-gated. **First verified cross-language win:** phase3-cjson-002 (class 5+3, stack-buffer number parse + `strtod` round-trip) +8.85%, 95% CI [+7.5%, +10.0%] — banked, byte-identical golden replay, independent ASan+UBSan clean, 10k-input diff-fuzz (`scripts/diff-fuzz-cjson.py`) 0 divergences. **Four public case studies** in `results/phase3/` (tokei class-5 wins; build flags + allocators; the comrak caught false-accept; the cJSON cross-language proof). **Six playbook classes exercised (1–6)**; wins on classes 1/2(review)/3/5 across Rust *and* C. coz wired (`-DHOTPATH_COZ`, `just coz`, `scripts/coz-*`); its apt runtime aborts in init on this container's glibc, so callgrind cache-sim is the working profiler (documented). Zero regressions shipped. **Next: Phase 4** (load replay, latency mode under replayed traffic, mechanical ROI reports — SPEC.md §5; `crates/report` has the ROI math, needs the generator).

## Commands

Fill these in as the harness is built; keep them current:

```
just profile <target>       # perf record + flamegraph for a target's benchmark workload
just bench <target> <ref>   # interleaved A/B: pristine baseline vs working tree
just gates <target>         # test suite + golden replay + diff-fuzz + sanitizers
just verdict <target>       # runs gates + bench, writes ledger row, prints verdict
just report <run-id>        # ROI report from ledger
just aa <target>            # A/A calibration run (must show null result)
```

Built so far (command-level, pre-target-integration):

```
just build / test / lint    # trust-layer workspace: cargo build/test/clippy+fmt
just aa [cmd]               # A/A self-test of bench-runner on a shell command
just compare <a> <b>        # interleaved A/B of two shell commands, bootstrap CI
just gates <target>         # corpus pin + upstream tests + golden replay
just pin-check <target>     # verify corpus MANIFEST.sha256
just calibrate <cmd> <out>  # automated A/A + regression-injection calibration
just verdict <t> <bin> ...  # gates + bench vs pristine-rebuilt baseline + ledger row
cargo run -p harnessd       # agent IPC daemon: six-tool JSON surface (SPEC §3.5)
```

## Definition of done — one optimization attempt

- [ ] Hypothesis logged in ledger *before* patching
- [ ] Patch touches only allowlisted paths under `targets/<name>/`
- [ ] Upstream test suite green
- [ ] Golden replay byte-identical (or within the target's explicit FP tolerance policy)
- [ ] Differential fuzz on changed functions: no divergence (60s or 10k iterations minimum)
- [ ] ASan + UBSan clean; TSan if the patch touches anything threaded
- [ ] Interleaved A/B bench passes the CI-lower-bound threshold
- [ ] Ledger row written with verdict and full evidence
- [ ] If FP-ordering / concurrency / UB-adjacent: verdict `needs-human-review`, never auto-accept

An attempt that fails any gate is a **valid, complete outcome** — write the ledger row and move on. Do not iterate on a rejected patch more than twice without a new hypothesis.

## Conventions

- Rust workspace (edition 2021+) for all harness crates; `just` for task running
- Python 3.11+ for `agent/`; Claude Agent SDK — verify current package name and API at https://docs.claude.com/en/docs/claude-code/overview before scaffolding
- Every number shown to a human carries its confidence interval and workload description. No naked percentages, ever.
- Commit style: `phase0:`, `bench:`, `gates:`, `agent:`, `playbook:` prefixes
- When in doubt between shipping a feature and hardening measurement, harden measurement. The product is trust.