mirror of
https://github.com/bdbrown4/peltier.io.git
synced 2026-11-07 10:58:28 +00:00
Claude
535821cd48
The Claude Agent SDK loop (agent/hotpath_agent/loop.py, model claude-fable-5) drives the seven harness tools over a standalone stdio MCP server (mcp_server.py) and runs the full profile -> hypothesize -> patch -> gated-verdict cycle on comrak with no human in the loop. Three audited attempts, all honest rejections, ZERO false accepts: - phase2-comrak-001 class 3: syntect buffer with_capacity, [0.971,1.007] - phase2-comrak-002 class 5: gate dead VecDeque build, [0.982,1.026] - phase2-comrak-003 class 6: hoist find_special_char check, [0.970,1.007] The agent picked the cheapest untried class each time (3->5->6) and TWICE refused to fabricate a verdict it couldn't read back, verifying ledger integrity instead — the anti-reward-hacking behavior SPEC §10 targets. Harness changes surfaced by the loop: - run_verdict is now async: launches the multi-minute build+bench detached (past the 60s MCP transport cap) and returns immediately; new read_verdict op polls the append-only ledger row (ledger verdict_summary). Fixes attempt 001's unreadable-verdict gap. - read_target_source takes offset/limit line windows so 100KB+ files page instead of overflowing — removes the agent's reason to reach for a shell (attempt 003 paged a 2400-line file cleanly, no shell). Honest boundary note (case study + code comments): in THIS dev container the SDK tool allow/deny layer is defense-in-depth, not the trust boundary — a nested CLI as root under a parent session doesn't reliably gate built-ins. SPEC §10's boundary is OS process/user isolation; that hardening is the remaining Phase 2 exit item (docs/infra.md). What holds regardless: propose_patch is the only mediated write path (path-allowlist + git apply) and the ledger is append-only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
peltier.io
Spot cooling for hot paths. A profile-guided optimization agent that only ships wins it can prove.
hotpath finds the 10–90% left on the floor between "the compiler" and "hire a perf consultant," with two properties no consultant offers: verified equivalence (a change provably preserves behavior or is discarded) and self-verifying ROI (the value claim is a stopwatch on trusted infrastructure, not a survey).
Full design: SPEC.md.
Layout
crates/ Trust layer (agent has NO write access — SPEC §10)
bench-runner/ Interleaved A/B timing, bootstrap CIs, A/A calibration
diff-test/ Equivalence gates, corpus hash-pinning, per-target policy
ledger/ Append-only SQLite attempt ledger (enforced by triggers)
report/ ROI math: speedup CI → cores → dollars, caveats included
agent/ Untrusted proposer (Claude Agent SDK, Python)
playbook/ Optimization classes 1–7, tried strictly cheapest-first
config/ accept.toml (thresholds), pricing.toml (ROI inputs)
targets/ Per-target checkouts — the only agent-writable path
corpora/ Hash-pinned golden-replay inputs (read-only to agent)
results/ Calibration evidence, per-engagement outputs
Quick start
cargo test --workspace # trust-layer unit tests
just aa # A/A self-test: must yield a null verdict
just compare "cmd-a" "cmd-b" # interleaved A/B with bootstrap CI
Status
Phase 0 complete (see SPEC §5). Five attempts in the ledger across two pinned targets (comrak 0.53.0, tokei 14.0.0), A/A-calibrated, every number with CI and workload:
- tokei +10.4% median [95% CI +8.5%, +12.0%] — read-path buffer
zeroing eliminated + fat LTO, byte-identical, 4,332-iteration
diff-fuzz clean, sanitizers clean, auto-accepted
(
results/phase0/case-study-tokei.md). Meets the Phase 0 exit bar. - comrak +4.6% [+3.3%, +5.8%] — mimalloc swap,
needs-human-reviewon an LSan reachability flag (results/phase0/case-study-comrak.md). - Two null/sub-threshold results correctly rejected and logged.
Next: Phase 1 trust-layer hardening (containers, perf, automated calibration, gate orchestration, agent IPC).
Done:
- Workspace with the four trust-layer crates; core statistics (interleaved scheduling, bootstrap ratio CIs, CI-lower-bound accept rule), append-only ledger schema with mutation-refusing triggers, corpus hash-pin verifier, equivalence-policy parser, ROI formulas — all unit-tested.
- bench-runner CLI: whole-program
compareandaamodes with env fingerprinting. - Playbook classes 1–7; agent tool contract and prompting spine.
Next (Phase 1 exit criteria):
perf statcounters + max RSS + RAPL capture per run.- Automated A/A calibration sessions and regression-injection self-test
recorded to
results/calibration/(<5% FP, ≥95% detection). - diff-test orchestration end-to-end on two real targets; harness IPC for the agent tool surface.
License
GPL-3.0-or-later — see LICENSE.