On a Windows host the preflight's job is to deliver a structured refusal
(measurement is Linux/POSIX-only). But a Windows-native harness with no
POSIX shell never got that far: preflight.sh fails to LAUNCH, and an agent
staring at "sh: not found" is exactly the agent that hand-rolls a timing
loop. The refusal doctrine has to be as portable as the skill.
- scripts/preflight.ps1: faithful port of the preflight contract, runs on
Windows PowerShell 5.1 and pwsh 7+ on any OS. Host check first (so
Windows refuses before touching anything), same walk-up resolution, same
always-build rule, same key=value output, same exit codes. The source is
pure ASCII with the shared em dashes composed at runtime — Windows
PowerShell 5.1 misdecodes BOM-less UTF-8, and that exact bug class
already bit this machine once (the francis-watchdog parse failure).
- Two implementations of one contract is a drift risk, so it is gated like
every other claim in this repo: the skill-preflight CI job now runs BOTH
scripts on the same ubuntu host and byte-diffs their stdout on all three
paths (in-checkout resolve, no-checkout refusal, PELTIER_HOME from a
foreign repo). Disagreement is a red build, not a footnote.
- New skill-preflight-windows job (windows-latest): asserts the native
refusal end-to-end under Windows PowerShell 5.1 AND pwsh (STATUS=refuse,
nonzero exit, the 'unsupported host' reason), plus Git Bash sh for good
measure. Verified live on a real Windows 11 box before shipping: 5.1
parses the pure-ASCII source clean and refuses with exit 1.
- SKILL.md / references/verify.md: document the second entry point ("use
whichever your host can launch; never both, never neither"); book chapter
and CLAUDE.md updated to match. .agents mirror regenerated (identical).
The ZeroClaw variant is unaffected: rm -rf scripts/ drops the ps1 too,
and its audit still passes (7 files scanned).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The reusable artifact here was never the vendored targets — it is the
discipline: prove equivalence, prove significance, or claim nothing. This
packages it as a Claude Code skill usable from any repo.
Two modes. `verify` works anywhere, on any two shell commands in any
language: equivalence before timing -> A/A calibration -> interleaved A/B
with a bootstrap CI -> a verdict decided by the CI lower bound, never the
median. `attempt` drives the full in-repo loop (gates, pristine-baseline
verdict, ledger row, ROI report).
It drives the real bench-runner binary and refuses rather than degrade: on
an unsupported host, or with no trust layer reachable, preflight stops and
says so instead of falling back to a hand-rolled timing loop. A second copy
of the statistics is a second thing that can lie. A refusal ends the claim,
not the work.
An adversarial review of the skill caught six high-severity defects, all
fixed here — several of them the very errors the skill exists to prevent:
- verify mode had no risk classification at all. bench-runner emits only
accepted/rejected-bench; needs-human-review is minted by the in-repo
verdict path, which verify does not run. A concurrency change with
byte-identical output would have sailed through as "accepted". The skill
now makes the agent the classifier and says so explicitly.
- the accept template exposed `pin_prefix`, which is inert for
compare/aa/calibrate (only `verdict` reads it) — an agent would have set
it and reported pinning that never happened. Now documents wrapping both
sides in taskset instead.
- the equivalence gate compared stdout only, so two commands that write to
stderr and a dist/ tree would "GOLDEN PASS" on two empty files. Now hashes
stdout + stderr + exit code + artifact tree, and an empty diff over empty
output is a failed gate, not a passed one.
- nothing told the agent to build two coexisting immutable artifacts, so a
`cargo run` baseline could rebuild from the patched tree mid-bench — the
exact trap phase1-verdict-smoke-001 recorded.
- service mode was documented as generic HTTP. It is not: the server must
take <port> <doc> <iters>, print READY, and close the connection. A stock
Express app hangs the harness forever. Contract now stated.
- the CI refusal test could not pass: preflight also walks up from its own
location, so running it in-place found this very repo. The test now copies
the skill out of the tree first.
CI job `skill-preflight` asserts the contract that matters — resolve when a
trust layer is reachable, REFUSE when it is not, and work when consumed from
another repo via PELTIER_HOME. It deliberately does not assert a statistical
outcome: a shared runner is too noisy for that, which is precisely why the
skill calibrates before it claims.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
benbrownandClaude Fable 5