mirror of
https://github.com/bdbrown4/peltier.io.git
synced 2026-11-07 10:58:28 +00:00
Merge pull request #20 from bdbrown4/docs/skill-chapter
docs: bring the book to parity with the skill; PR-gate the book build
This commit is contained in:
@@ -2,12 +2,23 @@ name: docs
|
|||||||
|
|
||||||
# Builds the mdBook site in docs/ and deploys it to GitHub Pages.
|
# Builds the mdBook site in docs/ and deploys it to GitHub Pages.
|
||||||
# One-time setup: repo Settings → Pages → Source = "GitHub Actions".
|
# One-time setup: repo Settings → Pages → Source = "GitHub Actions".
|
||||||
|
#
|
||||||
|
# Pull requests build the book but never deploy — so a broken book (or a
|
||||||
|
# chapter that {{#include}}s a file that moved) is a red PR check, not a
|
||||||
|
# post-merge surprise. results/** is a trigger because case-study chapters
|
||||||
|
# include evidence files from there.
|
||||||
|
|
||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
branches: [main]
|
branches: [main]
|
||||||
paths:
|
paths:
|
||||||
- "docs/**"
|
- "docs/**"
|
||||||
|
- "results/**"
|
||||||
|
- ".github/workflows/docs.yml"
|
||||||
|
pull_request:
|
||||||
|
paths:
|
||||||
|
- "docs/**"
|
||||||
|
- "results/**"
|
||||||
- ".github/workflows/docs.yml"
|
- ".github/workflows/docs.yml"
|
||||||
workflow_dispatch:
|
workflow_dispatch:
|
||||||
|
|
||||||
@@ -18,8 +29,8 @@ permissions:
|
|||||||
|
|
||||||
# Allow one concurrent deployment; don't cancel an in-progress prod deploy.
|
# Allow one concurrent deployment; don't cancel an in-progress prod deploy.
|
||||||
concurrency:
|
concurrency:
|
||||||
group: pages
|
group: pages-${{ github.event_name == 'pull_request' && github.ref || 'prod' }}
|
||||||
cancel-in-progress: false
|
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
build:
|
build:
|
||||||
@@ -29,6 +40,7 @@ jobs:
|
|||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@v4
|
||||||
- name: Configure Pages
|
- name: Configure Pages
|
||||||
|
if: github.event_name != 'pull_request'
|
||||||
uses: actions/configure-pages@v5
|
uses: actions/configure-pages@v5
|
||||||
- name: Install mdBook
|
- name: Install mdBook
|
||||||
run: |
|
run: |
|
||||||
@@ -39,11 +51,13 @@ jobs:
|
|||||||
- name: Build
|
- name: Build
|
||||||
run: mdbook build docs
|
run: mdbook build docs
|
||||||
- name: Upload artifact
|
- name: Upload artifact
|
||||||
|
if: github.event_name != 'pull_request'
|
||||||
uses: actions/upload-pages-artifact@v3
|
uses: actions/upload-pages-artifact@v3
|
||||||
with:
|
with:
|
||||||
path: docs/book
|
path: docs/book
|
||||||
|
|
||||||
deploy:
|
deploy:
|
||||||
|
if: github.event_name != 'pull_request'
|
||||||
needs: build
|
needs: build
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
environment:
|
environment:
|
||||||
|
|||||||
@@ -23,6 +23,7 @@
|
|||||||
|
|
||||||
# Reference
|
# Reference
|
||||||
|
|
||||||
|
- [Consume it: the peltier skill](./skill.md)
|
||||||
- [Reproduce it yourself](./reproduce.md)
|
- [Reproduce it yourself](./reproduce.md)
|
||||||
- [The trust boundary (infra)](./infra.md)
|
- [The trust boundary (infra)](./infra.md)
|
||||||
- [Roadmap & scope](./roadmap.md)
|
- [Roadmap & scope](./roadmap.md)
|
||||||
|
|||||||
@@ -19,6 +19,9 @@ config/ accept.toml (thresholds), pricing.toml (ROI inputs)
|
|||||||
targets/ Vendored OSS targets — the only agent-writable path
|
targets/ Vendored OSS targets — the only agent-writable path
|
||||||
corpora/ Hash-pinned golden-replay inputs (read-only to agent)
|
corpora/ Hash-pinned golden-replay inputs (read-only to agent)
|
||||||
results/ Calibration evidence, case studies, generated reports
|
results/ Calibration evidence, case studies, generated reports
|
||||||
|
.claude/skills/ The trust layer as a consumable Agent Skill — mirrored
|
||||||
|
byte-identically at .agents/skills/ (CI-enforced); see
|
||||||
|
"Consume it: the peltier skill"
|
||||||
```
|
```
|
||||||
|
|
||||||
## The seven-tool boundary
|
## The seven-tool boundary
|
||||||
|
|||||||
@@ -43,4 +43,5 @@ can quote to a hostile reviewer. That is the entire product, and this site
|
|||||||
is how it works.
|
is how it works.
|
||||||
|
|
||||||
Start with [the thesis](./thesis.md), or jump to a
|
Start with [the thesis](./thesis.md), or jump to a
|
||||||
[case study](./case-studies/overview.md).
|
[case study](./case-studies/overview.md). To run this discipline on your own
|
||||||
|
code from your own agent, see [the peltier skill](./skill.md).
|
||||||
|
|||||||
@@ -60,3 +60,9 @@ python3 scripts/diff-fuzz-<target>.py <dir> 10000 <baseline-bin> <candidate-bin>
|
|||||||
The ledger itself is the record: `results/ledger.sqlite`, append-only,
|
The ledger itself is the record: `results/ledger.sqlite`, append-only,
|
||||||
every attempt (win or loss) with its hypothesis, gates, bench evidence, and
|
every attempt (win or loss) with its hypothesis, gates, bench evidence, and
|
||||||
verdict.
|
verdict.
|
||||||
|
|
||||||
|
## Use it on your own code
|
||||||
|
|
||||||
|
The whole discipline is installable into any repo as an Agent Skill, driven
|
||||||
|
from any SKILL.md-reading harness — see
|
||||||
|
[Consume it: the peltier skill](./skill.md).
|
||||||
|
|||||||
@@ -0,0 +1,94 @@
|
|||||||
|
# Consume it: the peltier skill
|
||||||
|
|
||||||
|
Everything this site describes — equivalence before timing, A/A calibration,
|
||||||
|
interleaved A/B, the CI-lower-bound rule — is packaged as an
|
||||||
|
[Agent Skill](https://agentskills.io) you can drop into **any repo** and drive
|
||||||
|
from **any agent harness that reads the SKILL.md standard**: Claude Code,
|
||||||
|
OpenAI Codex, GitHub Copilot, VS Code, Cursor, Gemini CLI, opencode, Goose,
|
||||||
|
Amp, Zed, Factory, ZeroClaw, and others.
|
||||||
|
|
||||||
|
The skill does not reimplement anything. It drives the same `bench-runner`
|
||||||
|
binary the case studies used, from a peltier checkout you point it at. That is
|
||||||
|
deliberate: a second copy of the statistics would be a second thing that can
|
||||||
|
lie. The corollary is stated in the skill itself — if it cannot find the trust
|
||||||
|
layer, or the host cannot run it, **it refuses to produce a number** rather
|
||||||
|
than fall back to a hand-rolled timing loop. A refusal ends the claim, not the
|
||||||
|
work.
|
||||||
|
|
||||||
|
## Install
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# from your repo, with a peltier checkout at /path/to/peltier.io
|
||||||
|
sh /path/to/peltier.io/scripts/install-skill.sh .
|
||||||
|
export PELTIER_HOME=/path/to/peltier.io
|
||||||
|
```
|
||||||
|
|
||||||
|
One command stamps both discovery paths, because harnesses look in different
|
||||||
|
places:
|
||||||
|
|
||||||
|
| Path installed | Read natively by |
|
||||||
|
|---|---|
|
||||||
|
| `.claude/skills/peltier/` | Claude Code, GitHub Copilot, VS Code, opencode, Amp (and Cursor in compat mode) |
|
||||||
|
| `.agents/skills/peltier/` | Codex, Zed, Cursor, Gemini CLI, Goose, Copilot, opencode, Amp |
|
||||||
|
|
||||||
|
Two copies is a drift risk, so this repo treats it the way it treats every
|
||||||
|
other claim: **CI enforces the copies stay byte-identical** (`diff -r` in the
|
||||||
|
`skill-preflight` job), and the same job checks both against the Agent Skills
|
||||||
|
spec rules (name/directory match, description and body limits) on every pull
|
||||||
|
request.
|
||||||
|
|
||||||
|
## ZeroClaw: the script-blocking case
|
||||||
|
|
||||||
|
ZeroClaw's skill security policy blocks script files inside skills by default
|
||||||
|
(`skills.allow_scripts` is off) — its auditor rejects the full skill on sight.
|
||||||
|
The installer emits a variant for it:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
sh /path/to/peltier.io/scripts/install-skill.sh --zeroclaw-variant /tmp/peltier-skill
|
||||||
|
zeroclaw skills install /tmp/peltier-skill
|
||||||
|
```
|
||||||
|
|
||||||
|
The variant simply omits `scripts/`. Nothing is lost and nothing is forked:
|
||||||
|
`preflight.sh` still exists exactly once — in the peltier checkout the skill
|
||||||
|
already requires — and the skill's instructions run it from there. Transcribing
|
||||||
|
or re-implementing the preflight logic inline is explicitly named a refusal
|
||||||
|
condition in the skill, so a harness's restrictions cannot pressure an agent
|
||||||
|
into forking the trust layer.
|
||||||
|
|
||||||
|
## What the skill enforces
|
||||||
|
|
||||||
|
Two modes. `verify` works anywhere, on any two shell commands, in any
|
||||||
|
language; `attempt` drives the full in-repo loop this site's case studies
|
||||||
|
document. In `verify` mode, the gate order is the content:
|
||||||
|
|
||||||
|
1. **Preflight** — find the trust layer, build the real `bench-runner`, refuse
|
||||||
|
on unsupported hosts. `STATUS=ok` means the harness can *run*, not that the
|
||||||
|
host can *measure* — only calibration decides that.
|
||||||
|
2. **Equivalence before timing** — a faster program that computes something
|
||||||
|
different is a bug with good benchmarks, not an optimization. Comparing two
|
||||||
|
empty outputs is a failed gate, not a passed one.
|
||||||
|
3. **A/A calibration** — the same command on both sides must yield a null
|
||||||
|
verdict. A harness that "finds" a speedup between a binary and itself is
|
||||||
|
measuring noise, and every number it produces afterwards is fiction.
|
||||||
|
4. **Interleaved A/B, bootstrap CI** — and the claim is the **CI lower bound**,
|
||||||
|
never the median. A median of 1.088 with 95% CI [1.074, 1.100] (the shipped
|
||||||
|
cJSON batch win, on its 5.6 MB / 20k-record parse+serialize workload)
|
||||||
|
supports "≥7.4% faster" — not "8.8% faster".
|
||||||
|
5. **Risk routing** — concurrency, `unsafe`, and floating-point-reordering
|
||||||
|
changes are `needs-human-review` regardless of the numbers. In `verify`
|
||||||
|
mode the *agent* is instructed to be the classifier, because the standalone
|
||||||
|
harness only ever emits `accepted`/`rejected-bench`.
|
||||||
|
|
||||||
|
## The platform truth
|
||||||
|
|
||||||
|
Measurement requires a Linux/POSIX host — the same requirement as the rest of
|
||||||
|
the pipeline. On Windows (or with no checkout reachable) the skill refuses and
|
||||||
|
says why. That is asserted, not assumed: on every pull request, CI runs the
|
||||||
|
preflight's refusal path (skill copied outside any checkout) and its resolve
|
||||||
|
paths (in-checkout and via `PELTIER_HOME` from a foreign repo), alongside a
|
||||||
|
live plumbing run of the harness it drives.
|
||||||
|
|
||||||
|
The skill's own files are the reference:
|
||||||
|
[`SKILL.md`](https://github.com/bdbrown4/peltier.io/blob/main/.claude/skills/peltier/SKILL.md),
|
||||||
|
[`references/verify.md`](https://github.com/bdbrown4/peltier.io/blob/main/.claude/skills/peltier/references/verify.md),
|
||||||
|
[`references/attempt.md`](https://github.com/bdbrown4/peltier.io/blob/main/.claude/skills/peltier/references/attempt.md).
|
||||||
Reference in New Issue
Block a user