From 6e169ab819167020ba3eee4075fe0f76e3121b5f Mon Sep 17 00:00:00 2001 From: Ben Brown Date: Wed, 15 Jul 2026 12:19:49 -0400 Subject: [PATCH] docs: bring the book to parity with the skill; PR-gate the book build MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The cross-harness skill shipped (PR #17/#19) with README/CLAUDE.md/AGENTS.md coverage, but the mdBook — the public disclosure surface — had zero mention of it. By this project's own standard that is a parity defect: the flagship consumable feature was undocumented where it is disclosed. - New chapter "Consume it: the peltier skill" (docs/src/skill.md): install paths and the harness matrix, the ZeroClaw script-blocking variant, the gate order verify mode enforces, the refusal doctrine, and exactly which of these claims CI asserts. Numbers carry their CI and workload. - SUMMARY.md, architecture.md (layout now lists the skill + mirror), reproduce.md, introduction.md: cross-links to the chapter. - docs.yml: pull requests now BUILD the book (never deploy) — a broken chapter or dangling include is a red PR check, not a post-merge surprise. Also closes an audit finding: the book rebuilds when results/** changes, since case-study chapters {{#include}} evidence files from there. Book build verified locally with mdbook 0.4.x (skill.html renders). Co-Authored-By: Claude Fable 5 --- .github/workflows/docs.yml | 18 +++++++- docs/src/SUMMARY.md | 1 + docs/src/architecture.md | 3 ++ docs/src/introduction.md | 3 +- docs/src/reproduce.md | 6 +++ docs/src/skill.md | 94 ++++++++++++++++++++++++++++++++++++++ 6 files changed, 122 insertions(+), 3 deletions(-) create mode 100644 docs/src/skill.md diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index b3710e5..7b05a1f 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -2,12 +2,23 @@ name: docs # Builds the mdBook site in docs/ and deploys it to GitHub Pages. # One-time setup: repo Settings → Pages → Source = "GitHub Actions". +# +# Pull requests build the book but never deploy — so a broken book (or a +# chapter that {{#include}}s a file that moved) is a red PR check, not a +# post-merge surprise. results/** is a trigger because case-study chapters +# include evidence files from there. on: push: branches: [main] paths: - "docs/**" + - "results/**" + - ".github/workflows/docs.yml" + pull_request: + paths: + - "docs/**" + - "results/**" - ".github/workflows/docs.yml" workflow_dispatch: @@ -18,8 +29,8 @@ permissions: # Allow one concurrent deployment; don't cancel an in-progress prod deploy. concurrency: - group: pages - cancel-in-progress: false + group: pages-${{ github.event_name == 'pull_request' && github.ref || 'prod' }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} jobs: build: @@ -29,6 +40,7 @@ jobs: steps: - uses: actions/checkout@v4 - name: Configure Pages + if: github.event_name != 'pull_request' uses: actions/configure-pages@v5 - name: Install mdBook run: | @@ -39,11 +51,13 @@ jobs: - name: Build run: mdbook build docs - name: Upload artifact + if: github.event_name != 'pull_request' uses: actions/upload-pages-artifact@v3 with: path: docs/book deploy: + if: github.event_name != 'pull_request' needs: build runs-on: ubuntu-latest environment: diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md index 7da2aee..5061cca 100644 --- a/docs/src/SUMMARY.md +++ b/docs/src/SUMMARY.md @@ -23,6 +23,7 @@ # Reference +- [Consume it: the peltier skill](./skill.md) - [Reproduce it yourself](./reproduce.md) - [The trust boundary (infra)](./infra.md) - [Roadmap & scope](./roadmap.md) diff --git a/docs/src/architecture.md b/docs/src/architecture.md index 9e04370..4076cd0 100644 --- a/docs/src/architecture.md +++ b/docs/src/architecture.md @@ -19,6 +19,9 @@ config/ accept.toml (thresholds), pricing.toml (ROI inputs) targets/ Vendored OSS targets — the only agent-writable path corpora/ Hash-pinned golden-replay inputs (read-only to agent) results/ Calibration evidence, case studies, generated reports +.claude/skills/ The trust layer as a consumable Agent Skill — mirrored + byte-identically at .agents/skills/ (CI-enforced); see + "Consume it: the peltier skill" ``` ## The seven-tool boundary diff --git a/docs/src/introduction.md b/docs/src/introduction.md index dd13b17..0e80c4c 100644 --- a/docs/src/introduction.md +++ b/docs/src/introduction.md @@ -43,4 +43,5 @@ can quote to a hostile reviewer. That is the entire product, and this site is how it works. Start with [the thesis](./thesis.md), or jump to a -[case study](./case-studies/overview.md). +[case study](./case-studies/overview.md). To run this discipline on your own +code from your own agent, see [the peltier skill](./skill.md). diff --git a/docs/src/reproduce.md b/docs/src/reproduce.md index b470b81..526213f 100644 --- a/docs/src/reproduce.md +++ b/docs/src/reproduce.md @@ -60,3 +60,9 @@ python3 scripts/diff-fuzz-.py 10000 The ledger itself is the record: `results/ledger.sqlite`, append-only, every attempt (win or loss) with its hypothesis, gates, bench evidence, and verdict. + +## Use it on your own code + +The whole discipline is installable into any repo as an Agent Skill, driven +from any SKILL.md-reading harness — see +[Consume it: the peltier skill](./skill.md). diff --git a/docs/src/skill.md b/docs/src/skill.md new file mode 100644 index 0000000..7c90dfc --- /dev/null +++ b/docs/src/skill.md @@ -0,0 +1,94 @@ +# Consume it: the peltier skill + +Everything this site describes — equivalence before timing, A/A calibration, +interleaved A/B, the CI-lower-bound rule — is packaged as an +[Agent Skill](https://agentskills.io) you can drop into **any repo** and drive +from **any agent harness that reads the SKILL.md standard**: Claude Code, +OpenAI Codex, GitHub Copilot, VS Code, Cursor, Gemini CLI, opencode, Goose, +Amp, Zed, Factory, ZeroClaw, and others. + +The skill does not reimplement anything. It drives the same `bench-runner` +binary the case studies used, from a peltier checkout you point it at. That is +deliberate: a second copy of the statistics would be a second thing that can +lie. The corollary is stated in the skill itself — if it cannot find the trust +layer, or the host cannot run it, **it refuses to produce a number** rather +than fall back to a hand-rolled timing loop. A refusal ends the claim, not the +work. + +## Install + +```sh +# from your repo, with a peltier checkout at /path/to/peltier.io +sh /path/to/peltier.io/scripts/install-skill.sh . +export PELTIER_HOME=/path/to/peltier.io +``` + +One command stamps both discovery paths, because harnesses look in different +places: + +| Path installed | Read natively by | +|---|---| +| `.claude/skills/peltier/` | Claude Code, GitHub Copilot, VS Code, opencode, Amp (and Cursor in compat mode) | +| `.agents/skills/peltier/` | Codex, Zed, Cursor, Gemini CLI, Goose, Copilot, opencode, Amp | + +Two copies is a drift risk, so this repo treats it the way it treats every +other claim: **CI enforces the copies stay byte-identical** (`diff -r` in the +`skill-preflight` job), and the same job checks both against the Agent Skills +spec rules (name/directory match, description and body limits) on every pull +request. + +## ZeroClaw: the script-blocking case + +ZeroClaw's skill security policy blocks script files inside skills by default +(`skills.allow_scripts` is off) — its auditor rejects the full skill on sight. +The installer emits a variant for it: + +```sh +sh /path/to/peltier.io/scripts/install-skill.sh --zeroclaw-variant /tmp/peltier-skill +zeroclaw skills install /tmp/peltier-skill +``` + +The variant simply omits `scripts/`. Nothing is lost and nothing is forked: +`preflight.sh` still exists exactly once — in the peltier checkout the skill +already requires — and the skill's instructions run it from there. Transcribing +or re-implementing the preflight logic inline is explicitly named a refusal +condition in the skill, so a harness's restrictions cannot pressure an agent +into forking the trust layer. + +## What the skill enforces + +Two modes. `verify` works anywhere, on any two shell commands, in any +language; `attempt` drives the full in-repo loop this site's case studies +document. In `verify` mode, the gate order is the content: + +1. **Preflight** — find the trust layer, build the real `bench-runner`, refuse + on unsupported hosts. `STATUS=ok` means the harness can *run*, not that the + host can *measure* — only calibration decides that. +2. **Equivalence before timing** — a faster program that computes something + different is a bug with good benchmarks, not an optimization. Comparing two + empty outputs is a failed gate, not a passed one. +3. **A/A calibration** — the same command on both sides must yield a null + verdict. A harness that "finds" a speedup between a binary and itself is + measuring noise, and every number it produces afterwards is fiction. +4. **Interleaved A/B, bootstrap CI** — and the claim is the **CI lower bound**, + never the median. A median of 1.088 with 95% CI [1.074, 1.100] (the shipped + cJSON batch win, on its 5.6 MB / 20k-record parse+serialize workload) + supports "≥7.4% faster" — not "8.8% faster". +5. **Risk routing** — concurrency, `unsafe`, and floating-point-reordering + changes are `needs-human-review` regardless of the numbers. In `verify` + mode the *agent* is instructed to be the classifier, because the standalone + harness only ever emits `accepted`/`rejected-bench`. + +## The platform truth + +Measurement requires a Linux/POSIX host — the same requirement as the rest of +the pipeline. On Windows (or with no checkout reachable) the skill refuses and +says why. That is asserted, not assumed: on every pull request, CI runs the +preflight's refusal path (skill copied outside any checkout) and its resolve +paths (in-checkout and via `PELTIER_HOME` from a foreign repo), alongside a +live plumbing run of the harness it drives. + +The skill's own files are the reference: +[`SKILL.md`](https://github.com/bdbrown4/peltier.io/blob/main/.claude/skills/peltier/SKILL.md), +[`references/verify.md`](https://github.com/bdbrown4/peltier.io/blob/main/.claude/skills/peltier/references/verify.md), +[`references/attempt.md`](https://github.com/bdbrown4/peltier.io/blob/main/.claude/skills/peltier/references/attempt.md).