mirror of
https://github.com/bdbrown4/peltier.io.git
synced 2026-11-07 10:58:28 +00:00
benbrownandClaude Fable 5
477c5472dd
The SKILL.md Agent Skills standard (agentskills.io) is now read natively by Codex, Copilot, VS Code, Cursor, Gemini CLI, opencode, Goose, Amp, Zed, Factory, and ZeroClaw — so the portable move is not per-harness adapter files (forked content that drifts) but one canonical skill placed where each harness actually looks. - Spec-compliant frontmatter: license, compatibility (the Linux-only measurement story, stated up front), version/author/tags top-level (ZeroClaw's runtime reads those; the spec ignores unknown fields). reference/ renamed to references/ per the spec's conventional layout. - .agents/skills/peltier/ mirror: the cross-tool convergence path (Codex, Zed, Cursor, Gemini CLI, Copilot, opencode, Goose, Amp). Byte-identical to the .claude canonical, enforced by CI — drift between the two copies is a red build, not a footnote. - AGENTS.md at the repo root for the ~20 harnesses that read it (Claude Code does not; CLAUDE.md remains its file). Deliberately thin: it points at CLAUDE.md and the skill instead of restating rules. - scripts/install-skill.sh: stamps a consumer repo at both discovery paths (--layout claude|agents|both), or emits a script-less --zeroclaw-variant. ZeroClaw's skill security policy blocks script files by default (skills.allow_scripts off) — its audit rejected the full skill and passes the variant, verified against the real zeroclaw binary. The variant loses nothing: preflight.sh still exists exactly once, in the peltier checkout the skill already requires, and SKILL.md now documents running it from there. Re-implementing preflight inline is named as a refusal condition. - CI (skill-preflight): frontmatter upgraded to a full Agent Skills spec conformance check on BOTH copies (name rules, dir match, description and body limits, metadata string-map), plus mirror equality and a functional installer test (both layouts identical to canonical, variant carries no scripts/, nonexistent target refused). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
105 lines
4.6 KiB
Markdown
105 lines
4.6 KiB
Markdown
# attempt — the full loop, inside a peltier checkout
|
|
|
|
Profile → hypothesize → patch → gated verdict → ledger row. This mode only
|
|
applies when you are working **inside a peltier checkout** on a target that has
|
|
a `targets/<name>/target.toml`. Everywhere else, use
|
|
[verify.md](verify.md).
|
|
|
|
Requires Linux (harnessd uses Unix sockets and `setsid`; every gate shells
|
|
through `sh`).
|
|
|
|
## Before you touch anything
|
|
|
|
**Read the ledger first.** It records every attempt ever made on this target,
|
|
including the failures — which is exactly what stops you re-running a dead
|
|
end:
|
|
|
|
cargo run -p policy # ranks playbook classes by their evidence
|
|
|
|
`policy` reads the ledger and ranks optimization classes by the Wilson lower
|
|
bound of their shippable-win rate. It is **advisory** — a learned prior over
|
|
the fixed cheapest-first order. The gates still decide every verdict.
|
|
|
|
**Then respect the order.** Cheapest wins first: build flags → LTO → PGO →
|
|
allocator swap → *then* code changes. Do not hand-write SIMD before trying
|
|
`-O3`. The playbook (`playbook/`) has the preconditions and known risks for
|
|
each class.
|
|
|
|
## The boundary you may not cross
|
|
|
|
The agent may modify **only** allowlisted paths under `targets/<name>/`.
|
|
`crates/`, `config/`, `corpora/`, and the upstream test suites are read-only:
|
|
the thing being measured does not get to grade itself. `harnessd` enforces
|
|
this on every proposed diff (path allowlist over every diff header, including
|
|
`rename`/`copy` — a patch that tries to escape is rejected, not sanitized).
|
|
|
|
## Log the hypothesis *before* patching
|
|
|
|
A hypothesis written after seeing the numbers is not a hypothesis. State what
|
|
you think is slow, why, and what you expect the change to buy — then patch.
|
|
|
|
## Run the gates
|
|
|
|
just gates <target>
|
|
|
|
Corpus pin → test-suite pin (when `corpora/<t>/TESTSUITE.sha256` exists) →
|
|
upstream tests → golden replay. Note what this **does not** do: differential
|
|
fuzz needs a pristine baseline to differ against, so it reports `Skipped` here
|
|
and runs for real on the accept path below. `just gates` is a fast equivalence
|
|
check, not a fuzz run — do not report it as one.
|
|
|
|
## The verdict — the only thing that can accept a change
|
|
|
|
just verdict <target> <candidate-bin> <run-id> <class> "<hypothesis>" "<hotspot>" \
|
|
--patch-file <diff>
|
|
|
|
This rebuilds a **pristine baseline** from the pinned upstream commit (not
|
|
whatever is lying around in your tree), runs the full gate set against the
|
|
candidate, benches interleaved A/B against that baseline, and writes one
|
|
append-only ledger row with the verdict and the complete evidence.
|
|
|
|
It will refuse to hand you an `accepted` unless:
|
|
|
|
- the upstream test suite is green **and** golden replay is byte-identical (or
|
|
within the target's declared FP tolerance);
|
|
- the **differential-fuzz gate actually passed** — a real run reporting zero
|
|
divergences. No fuzz run, no machine accept. Ever;
|
|
- ASan/UBSan are clean (and TSan, when the target configures it);
|
|
- the bootstrap 95% CI lower bound of the speedup clears the threshold in
|
|
`config/accept.toml` (default 2%);
|
|
- and a conservative lexical risk classifier finds **no** concurrency, unsafe,
|
|
or floating-point signals in the patch. If it does, the verdict is capped at
|
|
`needs-human-review` no matter how good the numbers are.
|
|
|
|
Any of these failing produces a rejected row. **That is a complete, valid
|
|
outcome** — the row is the deliverable. Write it and move on. Do not iterate on
|
|
a rejected patch more than twice without a genuinely new hypothesis.
|
|
|
|
## Report
|
|
|
|
just report <run-id>
|
|
|
|
Mechanically generates the ROI from the ledger row: throughput → cores →
|
|
dollars, and/or latency percentiles — every figure with its CI, workload, and
|
|
methodology printed inline. It flags any row that was not accepted, so a
|
|
rejected attempt cannot be laundered into a dollar figure.
|
|
|
|
## Unattended
|
|
|
|
just agent-attempt <target> <run-id>
|
|
|
|
Runs one full attempt behind the OS boundary: `harnessd` as the trusted uid on
|
|
a Unix socket, the agent loop as an unprivileged user, the verdict pipeline
|
|
wrapped in a network namespace (`scripts/no-net.sh`) that **fails closed** — if
|
|
namespaces are unavailable it refuses to run rather than quietly running with
|
|
the network up. `just isolation-check` verifies the boundary from the agent's
|
|
side.
|
|
|
|
## When the numbers are good and the change is scary
|
|
|
|
FP-ordering, concurrency primitives, anything UB-adjacent: `needs-human-review`,
|
|
always, regardless of the speedup. This is not a formality. The one false
|
|
accept in peltier's history (`phase2-comrak-010`) passed its bench and was
|
|
caught by a human audit afterwards — which is precisely why the sanitizer lane
|
|
is now machine-enforced and why this rule does not bend.
|