Relicensed to make sharing and consumption frictionless — the skill is
meant to be dropped into arbitrary client repos, and MIT removes every
question a legal department would ask about that.
Clean to do unilaterally: git shortlog shows no third-party human
contributors (Ben Brown, Claude co-authoring Ben's sessions, and two
copilot-swe-agent[bot] workflow commits made at Ben's direction).
All living license surfaces swapped in lockstep: LICENSE (MIT text,
copyright 2026 Ben Brown), Cargo.toml workspace license, README, and the
skill frontmatter in both copies (.claude + .agents mirror, regenerated
byte-identical). Historical documents in results/ are records and keep
whatever they said at the time. Prior releases/clones under GPL-3.0
remain validly GPL-licensed at those commits, as always.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The two halves of the system run on opposite statistical philosophies, on
purpose, and until now the docs never said so. The optimizer is a Bayesian
detective: playbook prior, ledger evidence, policy ranking, explain
feedback — belief-updating that decides what to try next. The trust layer
is a frequentist court: equivalence gates, per-machine calibrated error
rates (A/A false-positive bound, injected-regression power bound), and a
CI-lower-bound accept rule with no prior anywhere on the accept path —
because its audience is a hostile reviewer who shares none of your priors.
The load-bearing edge is the return path: evidence flows from court to
detective (ledger rows + explain); authority never flows the other way.
Invariant, now stated in four places: priors steer where we look; they
never touch what we conclude. A wrong prior costs compute — each wasted
attempt honestly ledgered. A prior in the verdict would cost a false
claim, the one unrecoverable failure for a project whose product is trust.
- thesis.md: full section with the two-box diagram incl. the return edge,
in docket order (equivalence before measurement); notes that estimated
payoff is a belief and dollar figures are minted only by the mechanical
report from accepted rows, and that even the learning layer bets on
Wilson LOWER bounds — the detective is a pessimist too.
- README: the one-breath version, linking the full argument.
- SPEC §1: dated note mapping each half to its sections.
- CLAUDE.md: the mental model, one paragraph under Mission.
Framing credit: emerged from a design conversation on whether peltier is
"Bayesian engineering" — it is exactly half of one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verdicts say what; nothing said why. A rejection that reads only
rejected-bench teaches nothing, and the loop's compounding value is that
the next hypothesis is better than the last. This ships the teaching layer,
built so it can never contaminate the thing it explains.
New crate crates/explain (`just explain <run-id>`):
- Advisory, strictly off the accept path: runs after the verdict is
recorded; nothing it emits feeds gates, bench, verdict, or ledger. Not
agent-facing: harnessd stays at seven ops, by design (SPEC §3.7 says why
- better hypotheses must never become a feedback channel into
measurement).
- Ledger-only inputs, deterministic: consumes exactly one row via the new
ledger::attempt_row; no config, no clock, no re-benching. Identical row,
byte-identical text - proven live (cmp on two runs of a real row).
- Record vs inference is typographic: restatements print plain; anything
beyond is prefixed `inference:` so an explanation can never be quoted as
a measurement.
- The narrative must agree with the machine verdict. The first live run
caught this class of bug: an accepted row without a recorded threshold
was narrated as "a policy rejection" two lines under "verdict: accepted".
Accepted rows are now always described by their CI lower bound (the
defensible claim); rejected-bench rows classify against 1.0 (measured
regression / null result / real-but-below-bar); NHR rows name the
recorded cap and state the numbers were not the deciding factor;
rejected-gate rows name the failing gate. Regression-tested.
- Historical rows are flagged, not laundered: accepted with
sanitizers_clean=false prints the doctrine caution (comrak-010 shows it);
fuzz_iters=0 accepts are named pre-integration rows.
verdict now records accept_threshold (the bar in force for that run) into
the bench env fingerprint, so rows are self-contained going forward -
explain never substitutes today's config for a historical bar, and says so
on older rows instead.
SPEC §3.7 (Added 2026-07-20) carries the invariants; book chapter
docs/src/explain.md shows real output from real rows (the accepted cJSON
win, the caught false-accept, the kernel-lane NHR); CLAUDE.md, README,
justfile, and the architecture layout updated (the layout also gains the
previously-omitted crates/policy line).
Verified: 21 workspace test suites green (6 new explain tests incl.
determinism and verdict-narrative agreement), clippy -D warnings clean,
fmt clean, mdbook builds, live runs against phase3-cjson-002,
phase5-matmul-opt, phase2-comrak-010, and phase4-cjson-service.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The SKILL.md Agent Skills standard (agentskills.io) is now read natively by
Codex, Copilot, VS Code, Cursor, Gemini CLI, opencode, Goose, Amp, Zed,
Factory, and ZeroClaw — so the portable move is not per-harness adapter
files (forked content that drifts) but one canonical skill placed where
each harness actually looks.
- Spec-compliant frontmatter: license, compatibility (the Linux-only
measurement story, stated up front), version/author/tags top-level
(ZeroClaw's runtime reads those; the spec ignores unknown fields).
reference/ renamed to references/ per the spec's conventional layout.
- .agents/skills/peltier/ mirror: the cross-tool convergence path (Codex,
Zed, Cursor, Gemini CLI, Copilot, opencode, Goose, Amp). Byte-identical
to the .claude canonical, enforced by CI — drift between the two copies
is a red build, not a footnote.
- AGENTS.md at the repo root for the ~20 harnesses that read it (Claude
Code does not; CLAUDE.md remains its file). Deliberately thin: it points
at CLAUDE.md and the skill instead of restating rules.
- scripts/install-skill.sh: stamps a consumer repo at both discovery paths
(--layout claude|agents|both), or emits a script-less --zeroclaw-variant.
ZeroClaw's skill security policy blocks script files by default
(skills.allow_scripts off) — its audit rejected the full skill and passes
the variant, verified against the real zeroclaw binary. The variant loses
nothing: preflight.sh still exists exactly once, in the peltier checkout
the skill already requires, and SKILL.md now documents running it from
there. Re-implementing preflight inline is named as a refusal condition.
- CI (skill-preflight): frontmatter upgraded to a full Agent Skills spec
conformance check on BOTH copies (name rules, dir match, description and
body limits, metadata string-map), plus mirror equality and a functional
installer test (both layouts identical to canonical, variant carries no
scripts/, nonexistent target refused).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The reusable artifact here was never the vendored targets — it is the
discipline: prove equivalence, prove significance, or claim nothing. This
packages it as a Claude Code skill usable from any repo.
Two modes. `verify` works anywhere, on any two shell commands in any
language: equivalence before timing -> A/A calibration -> interleaved A/B
with a bootstrap CI -> a verdict decided by the CI lower bound, never the
median. `attempt` drives the full in-repo loop (gates, pristine-baseline
verdict, ledger row, ROI report).
It drives the real bench-runner binary and refuses rather than degrade: on
an unsupported host, or with no trust layer reachable, preflight stops and
says so instead of falling back to a hand-rolled timing loop. A second copy
of the statistics is a second thing that can lie. A refusal ends the claim,
not the work.
An adversarial review of the skill caught six high-severity defects, all
fixed here — several of them the very errors the skill exists to prevent:
- verify mode had no risk classification at all. bench-runner emits only
accepted/rejected-bench; needs-human-review is minted by the in-repo
verdict path, which verify does not run. A concurrency change with
byte-identical output would have sailed through as "accepted". The skill
now makes the agent the classifier and says so explicitly.
- the accept template exposed `pin_prefix`, which is inert for
compare/aa/calibrate (only `verdict` reads it) — an agent would have set
it and reported pinning that never happened. Now documents wrapping both
sides in taskset instead.
- the equivalence gate compared stdout only, so two commands that write to
stderr and a dist/ tree would "GOLDEN PASS" on two empty files. Now hashes
stdout + stderr + exit code + artifact tree, and an empty diff over empty
output is a failed gate, not a passed one.
- nothing told the agent to build two coexisting immutable artifacts, so a
`cargo run` baseline could rebuild from the patched tree mid-bench — the
exact trap phase1-verdict-smoke-001 recorded.
- service mode was documented as generic HTTP. It is not: the server must
take <port> <doc> <iters>, print READY, and close the connection. A stock
Express app hangs the harness forever. Contract now stated.
- the CI refusal test could not pass: preflight also walks up from its own
location, so running it in-place found this very repo. The test now copies
the skill out of the tree first.
CI job `skill-preflight` asserts the contract that matters — resolve when a
trust layer is reachable, REFUSE when it is not, and work when consumed from
another repo via PELTIER_HOME. It deliberately does not assert a statistical
outcome: a shared runner is too noisy for that, which is precisely why the
skill calibrates before it claims.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An adversarial audit found the enforcement story ahead of the code: gates
that were documented as hard gates were advisory, manual, or absent, and
several human-facing claims outran the machine record. This makes the
claims true.
High findings
- Differential fuzz was documentation, not enforcement: verdict hardcoded
fuzz_iters=0 and diff-test hardcoded the gate to Skipped, while the case
studies claimed 10k-iteration fuzz on the same wins. Fuzz is now a real
pipeline gate (strict FUZZ-RESULT contract, executed count recorded), and
it is inherently baseline-vs-candidate, so it runs on the accept path and
reports Skipped in the standalone `just gates` flow. New hard rule: a
machine `accepted` verdict is impossible unless the fuzz gate actually
passed — otherwise it caps at needs-human-review.
- The agent's system prompt claimed FP/concurrency/unsafe patches were
"routed to human review automatically". No such mechanism existed. Added
a conservative lexical risk classifier (crates/verdict/src/risk.rs) that
forces needs-human-review on any would-be accept carrying those signals;
fp-tolerance equivalence always signals.
- The local verdict path ran agent-influenced code unsandboxed. It now runs
under scripts/no-net.sh (network namespace) and fails closed. Upstream
test suites are hash-verified against corpora/<t>/TESTSUITE.sha256 when
present (scripts/pin-testsuite.sh generates them).
Also fixed, found while fixing the above
- The pipeline script's `set -e` was inert (brace group as the left operand
of `&&`), so a failing candidate build did not abort the run and verdict
could bench a stale binary. Now a scoped subshell whose status is branched
on at top level.
- A wrapper that exits before exec'ing (no-net.sh refusing, bad wrapper path)
stranded RUNNING.lock and left read_verdict polling forever. Failure
accounting now lives outside the wrapper.
- harnessd's patch allowlist ignored git rename/copy headers and --no-prefix
diffs; the rename-escape case is now a committed regression test.
- The workspace did not compile on Windows (ungated Unix APIs). cfg-gated;
71 portable tests now run on both platforms.
- pin-check regenerated the corpus manifest instead of verifying it; it is
now verify-only, with a deliberate `just pin-corpus` to re-pin.
- Max RSS is captured per run on Unix; PMU counters are opt-in diagnostics
(compare --perf-stat), never part of the accept decision.
Docs now match the code, including what is NOT yet enforced: no target ships
a TESTSUITE.sha256 pin, no target declares [build].tsan, and matmul is not
wired to the fp-tolerance gate — each is stated as mechanized-but-inert
rather than done. Corrected the stale "33 ledger rows" (34), the playbook's
class-3 cJSON win (rejected), adversarial-review.md's sanitizer-separation
claim (five accepted rows carry sanitizers_clean=false), the unsupported
"40,000 requests" figure, and the blanket "historical rows record
fuzz_iters=0" (8 of 10; phase0-tokei-002/-003 record 4,332).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The project ships as peltier.io; "hotpath" was the placeholder working name
(CLAUDE.md: "Rename freely; nothing depends on it"). Replaces the product
name in the README front page and every mdBook source chapter — book title,
nav header, and prose ("Peltier profiles…", "Peltier sells…") — plus the
theme CSS comment.
The metaphor is unchanged (a Peltier device is a thermoelectric cooler:
"spot cooling for hot paths"), and the literal noun phrase "hot paths"
stays. Two code identifiers are intentionally left as-is because they are
not prose: the `agent/hotpath_agent/` Python module path and the
`HOTPATH_ISOLATION` env var — renaming those is a code change, not a docs
one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
- docs/: an mdBook documentation site — thesis, architecture, measurement
discipline, equivalence gates, agent loop, isolation/anti-reward-hacking,
playbook, ROI, reproduce, roadmap, and the five case studies (pulled from
their single source in results/ via {{#include}}). Builds clean (21
pages).
- .github/workflows/docs.yml: builds mdBook and deploys to GitHub Pages on
push to main (one-time setup: Settings → Pages → Source = GitHub Actions).
- README.md: rewritten as the project front page — the thesis, the verified
results table, layout, quick start, how it stays honest, links to the
book. Replaces the stale Phase-0-era text.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Attempt 4 (phase0-tokei-002, class 3, ACCEPTED — first auto-accept):
every file was read through encoding_rs_io's DecodeReaderBytes via
generic read_to_end, which zero-fills every grown buffer chunk (~221MB
memset/run, 8.5% of Ir) and grows without a size hint. Patch: fs::read
with a BOM-gated fallback to the identical decode path (a UTF-8-BOM
file transcodes invalid sequences, so the fallback is required for
equivalence). Gates: golden byte-identical; differential fuzz old-vs-
new binaries 4,332 iterations (BOM grid, invalid UTF-8, binary junk,
randomized) zero divergence; 250 upstream tests green; ASan+LSan clean
with ASan-run output hashing to golden. Bench: +9.9% [8.0%, 12.7%].
Attempt 5 (phase0-tokei-003, ship tree, ACCEPTED): stacks fat LTO+cgu=1
(real +1.6% standalone but under threshold, phase0-tokei-001) on the
accepted patch; benched as a whole vs pristine: +10.4% [8.5%, 12.0%].
Median clears the Phase 0 exit bar; CI lower bound 8.5%.
Case study written (results/phase0/case-study-tokei.md), ledger at 5
attempts, CLAUDE.md phase line moved to Phase 1, next hypotheses
logged (tokei class-5 byte-skip table; comrak class-3 arena).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
Commit the spec and scaffold the system it describes:
- SPEC.md: the full v0.1 engineering spec
- crates/: trust-layer Rust workspace, unit-tested
- bench-runner: interleaved ABAB scheduling with discarded warm-ups,
deterministic bootstrap CIs over the ratio of medians,
CI-lower-bound accept rule, env fingerprinting, and a CLI with
compare + A/A self-test modes (A/A verified passing end-to-end)
- diff-test: gate outcome model, sha256 corpus pin verifier that
refuses on mismatch, per-target equivalence policy parser
(byte-identical default, explicit fp-tolerance opt-in)
- ledger: append-only SQLite attempt schema with UPDATE/DELETE
refused by triggers, anti-double-attempt query
- report: cores-saved and dollars/yr ROI math over speedup CIs,
pricing.toml inputs
- playbook/: classes 1-7 with preconditions, procedure, verification
notes, and known failure modes per class
- agent/: Python package skeleton — six-tool contract and prompting
spine, harness IPC deferred to Phase 1
- config/: accept.toml (2% threshold, 30 runs/side), pricing.toml
- targets/, corpora/, results/calibration/: layout and policy docs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
benbrownandClaude Fable 5
