Merge pull request #20 from bdbrown4/docs/skill-chapter

docs: bring the book to parity with the skill; PR-gate the book build
This commit is contained in:
2026-07-15 12:24:46 -04:00
committed by GitHub
6 changed files with 122 additions and 3 deletions
+16 -2
View File
@@ -2,12 +2,23 @@ name: docs
# Builds the mdBook site in docs/ and deploys it to GitHub Pages. # Builds the mdBook site in docs/ and deploys it to GitHub Pages.
# One-time setup: repo Settings → Pages → Source = "GitHub Actions". # One-time setup: repo Settings → Pages → Source = "GitHub Actions".
#
# Pull requests build the book but never deploy — so a broken book (or a
# chapter that {{#include}}s a file that moved) is a red PR check, not a
# post-merge surprise. results/** is a trigger because case-study chapters
# include evidence files from there.
on: on:
push: push:
branches: [main] branches: [main]
paths: paths:
- "docs/**" - "docs/**"
- "results/**"
- ".github/workflows/docs.yml"
pull_request:
paths:
- "docs/**"
- "results/**"
- ".github/workflows/docs.yml" - ".github/workflows/docs.yml"
workflow_dispatch: workflow_dispatch:
@@ -18,8 +29,8 @@ permissions:
# Allow one concurrent deployment; don't cancel an in-progress prod deploy. # Allow one concurrent deployment; don't cancel an in-progress prod deploy.
concurrency: concurrency:
group: pages group: pages-${{ github.event_name == 'pull_request' && github.ref || 'prod' }}
cancel-in-progress: false cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs: jobs:
build: build:
@@ -29,6 +40,7 @@ jobs:
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Configure Pages - name: Configure Pages
if: github.event_name != 'pull_request'
uses: actions/configure-pages@v5 uses: actions/configure-pages@v5
- name: Install mdBook - name: Install mdBook
run: | run: |
@@ -39,11 +51,13 @@ jobs:
- name: Build - name: Build
run: mdbook build docs run: mdbook build docs
- name: Upload artifact - name: Upload artifact
if: github.event_name != 'pull_request'
uses: actions/upload-pages-artifact@v3 uses: actions/upload-pages-artifact@v3
with: with:
path: docs/book path: docs/book
deploy: deploy:
if: github.event_name != 'pull_request'
needs: build needs: build
runs-on: ubuntu-latest runs-on: ubuntu-latest
environment: environment:
+1
View File
@@ -23,6 +23,7 @@
# Reference # Reference
- [Consume it: the peltier skill](./skill.md)
- [Reproduce it yourself](./reproduce.md) - [Reproduce it yourself](./reproduce.md)
- [The trust boundary (infra)](./infra.md) - [The trust boundary (infra)](./infra.md)
- [Roadmap & scope](./roadmap.md) - [Roadmap & scope](./roadmap.md)
+3
View File
@@ -19,6 +19,9 @@ config/ accept.toml (thresholds), pricing.toml (ROI inputs)
targets/ Vendored OSS targets — the only agent-writable path targets/ Vendored OSS targets — the only agent-writable path
corpora/ Hash-pinned golden-replay inputs (read-only to agent) corpora/ Hash-pinned golden-replay inputs (read-only to agent)
results/ Calibration evidence, case studies, generated reports results/ Calibration evidence, case studies, generated reports
.claude/skills/ The trust layer as a consumable Agent Skill — mirrored
byte-identically at .agents/skills/ (CI-enforced); see
"Consume it: the peltier skill"
``` ```
## The seven-tool boundary ## The seven-tool boundary
+2 -1
View File
@@ -43,4 +43,5 @@ can quote to a hostile reviewer. That is the entire product, and this site
is how it works. is how it works.
Start with [the thesis](./thesis.md), or jump to a Start with [the thesis](./thesis.md), or jump to a
[case study](./case-studies/overview.md). [case study](./case-studies/overview.md). To run this discipline on your own
code from your own agent, see [the peltier skill](./skill.md).
+6
View File
@@ -60,3 +60,9 @@ python3 scripts/diff-fuzz-<target>.py <dir> 10000 <baseline-bin> <candidate-bin>
The ledger itself is the record: `results/ledger.sqlite`, append-only, The ledger itself is the record: `results/ledger.sqlite`, append-only,
every attempt (win or loss) with its hypothesis, gates, bench evidence, and every attempt (win or loss) with its hypothesis, gates, bench evidence, and
verdict. verdict.
## Use it on your own code
The whole discipline is installable into any repo as an Agent Skill, driven
from any SKILL.md-reading harness — see
[Consume it: the peltier skill](./skill.md).
+94
View File
@@ -0,0 +1,94 @@
# Consume it: the peltier skill
Everything this site describes — equivalence before timing, A/A calibration,
interleaved A/B, the CI-lower-bound rule — is packaged as an
[Agent Skill](https://agentskills.io) you can drop into **any repo** and drive
from **any agent harness that reads the SKILL.md standard**: Claude Code,
OpenAI Codex, GitHub Copilot, VS Code, Cursor, Gemini CLI, opencode, Goose,
Amp, Zed, Factory, ZeroClaw, and others.
The skill does not reimplement anything. It drives the same `bench-runner`
binary the case studies used, from a peltier checkout you point it at. That is
deliberate: a second copy of the statistics would be a second thing that can
lie. The corollary is stated in the skill itself — if it cannot find the trust
layer, or the host cannot run it, **it refuses to produce a number** rather
than fall back to a hand-rolled timing loop. A refusal ends the claim, not the
work.
## Install
```sh
# from your repo, with a peltier checkout at /path/to/peltier.io
sh /path/to/peltier.io/scripts/install-skill.sh .
export PELTIER_HOME=/path/to/peltier.io
```
One command stamps both discovery paths, because harnesses look in different
places:
| Path installed | Read natively by |
|---|---|
| `.claude/skills/peltier/` | Claude Code, GitHub Copilot, VS Code, opencode, Amp (and Cursor in compat mode) |
| `.agents/skills/peltier/` | Codex, Zed, Cursor, Gemini CLI, Goose, Copilot, opencode, Amp |
Two copies is a drift risk, so this repo treats it the way it treats every
other claim: **CI enforces the copies stay byte-identical** (`diff -r` in the
`skill-preflight` job), and the same job checks both against the Agent Skills
spec rules (name/directory match, description and body limits) on every pull
request.
## ZeroClaw: the script-blocking case
ZeroClaw's skill security policy blocks script files inside skills by default
(`skills.allow_scripts` is off) — its auditor rejects the full skill on sight.
The installer emits a variant for it:
```sh
sh /path/to/peltier.io/scripts/install-skill.sh --zeroclaw-variant /tmp/peltier-skill
zeroclaw skills install /tmp/peltier-skill
```
The variant simply omits `scripts/`. Nothing is lost and nothing is forked:
`preflight.sh` still exists exactly once — in the peltier checkout the skill
already requires — and the skill's instructions run it from there. Transcribing
or re-implementing the preflight logic inline is explicitly named a refusal
condition in the skill, so a harness's restrictions cannot pressure an agent
into forking the trust layer.
## What the skill enforces
Two modes. `verify` works anywhere, on any two shell commands, in any
language; `attempt` drives the full in-repo loop this site's case studies
document. In `verify` mode, the gate order is the content:
1. **Preflight** — find the trust layer, build the real `bench-runner`, refuse
on unsupported hosts. `STATUS=ok` means the harness can *run*, not that the
host can *measure* — only calibration decides that.
2. **Equivalence before timing** — a faster program that computes something
different is a bug with good benchmarks, not an optimization. Comparing two
empty outputs is a failed gate, not a passed one.
3. **A/A calibration** — the same command on both sides must yield a null
verdict. A harness that "finds" a speedup between a binary and itself is
measuring noise, and every number it produces afterwards is fiction.
4. **Interleaved A/B, bootstrap CI** — and the claim is the **CI lower bound**,
never the median. A median of 1.088 with 95% CI [1.074, 1.100] (the shipped
cJSON batch win, on its 5.6 MB / 20k-record parse+serialize workload)
supports "≥7.4% faster" — not "8.8% faster".
5. **Risk routing** — concurrency, `unsafe`, and floating-point-reordering
changes are `needs-human-review` regardless of the numbers. In `verify`
mode the *agent* is instructed to be the classifier, because the standalone
harness only ever emits `accepted`/`rejected-bench`.
## The platform truth
Measurement requires a Linux/POSIX host — the same requirement as the rest of
the pipeline. On Windows (or with no checkout reachable) the skill refuses and
says why. That is asserted, not assumed: on every pull request, CI runs the
preflight's refusal path (skill copied outside any checkout) and its resolve
paths (in-checkout and via `PELTIER_HOME` from a foreign repo), alongside a
live plumbing run of the harness it drives.
The skill's own files are the reference:
[`SKILL.md`](https://github.com/bdbrown4/peltier.io/blob/main/.claude/skills/peltier/SKILL.md),
[`references/verify.md`](https://github.com/bdbrown4/peltier.io/blob/main/.claude/skills/peltier/references/verify.md),
[`references/attempt.md`](https://github.com/bdbrown4/peltier.io/blob/main/.claude/skills/peltier/references/attempt.md).