mirror of
https://github.com/bdbrown4/peltier.io.git
synced 2026-11-07 10:58:28 +00:00
skill: cross-OS preflight — ps1 entry point, one CI-enforced contract
On a Windows host the preflight's job is to deliver a structured refusal
(measurement is Linux/POSIX-only). But a Windows-native harness with no
POSIX shell never got that far: preflight.sh fails to LAUNCH, and an agent
staring at "sh: not found" is exactly the agent that hand-rolls a timing
loop. The refusal doctrine has to be as portable as the skill.
- scripts/preflight.ps1: faithful port of the preflight contract, runs on
Windows PowerShell 5.1 and pwsh 7+ on any OS. Host check first (so
Windows refuses before touching anything), same walk-up resolution, same
always-build rule, same key=value output, same exit codes. The source is
pure ASCII with the shared em dashes composed at runtime — Windows
PowerShell 5.1 misdecodes BOM-less UTF-8, and that exact bug class
already bit this machine once (the francis-watchdog parse failure).
- Two implementations of one contract is a drift risk, so it is gated like
every other claim in this repo: the skill-preflight CI job now runs BOTH
scripts on the same ubuntu host and byte-diffs their stdout on all three
paths (in-checkout resolve, no-checkout refusal, PELTIER_HOME from a
foreign repo). Disagreement is a red build, not a footnote.
- New skill-preflight-windows job (windows-latest): asserts the native
refusal end-to-end under Windows PowerShell 5.1 AND pwsh (STATUS=refuse,
nonzero exit, the 'unsupported host' reason), plus Git Bash sh for good
measure. Verified live on a real Windows 11 box before shipping: 5.1
parses the pure-ASCII source clean and refuses with exit 1.
- SKILL.md / references/verify.md: document the second entry point ("use
whichever your host can launch; never both, never neither"); book chapter
and CLAUDE.md updated to match. .agents mirror regenerated (identical).
The ZeroClaw variant is unaffected: rm -rf scripts/ drops the ps1 too,
and its audit still passes (7 files scanned).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Claude Fable 5
@@ -30,6 +30,13 @@ plugin, or copied into another project):
|
||||
|
||||
sh <skill-dir>/scripts/preflight.sh
|
||||
|
||||
# no POSIX shell on this host (Windows-native harness)? same contract:
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File <skill-dir>/scripts/preflight.ps1
|
||||
|
||||
The two entry points implement one contract and are byte-diffed against each
|
||||
other in CI — same checks, same output, same refusals. Use whichever your
|
||||
host can launch; never both, never neither.
|
||||
|
||||
Some harnesses block script files inside skills, so your installation may
|
||||
carry no `scripts/` directory. The script still exists exactly once — in the
|
||||
peltier checkout this skill drives (which you need anyway; it is where
|
||||
|
||||
@@ -6,7 +6,9 @@ faster. Find out. Works on any pair of shell commands, in any language.
|
||||
## 0. Preflight
|
||||
|
||||
sh <skill-dir>/scripts/preflight.sh
|
||||
# or, if this installation carries no scripts/ (script-blocking harness):
|
||||
# no POSIX shell (Windows-native harness)? same contract, CI-diffed against the sh version:
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File <skill-dir>/scripts/preflight.ps1
|
||||
# installation carries no scripts/ (script-blocking harness)? run from the checkout:
|
||||
sh "$PELTIER_HOME/.claude/skills/peltier/scripts/preflight.sh"
|
||||
|
||||
Read the `key=value` output; you need `BENCH_RUNNER` and `PIN_SUPPORTED`. A
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
# peltier preflight - PowerShell entry point. Same contract as preflight.sh,
|
||||
# for hosts where no POSIX shell exists (Windows-native harnesses). Works on
|
||||
# Windows PowerShell 5.1 and pwsh 7+ on any OS.
|
||||
#
|
||||
# THE CONTRACT IS DEFINED ONCE: this script and preflight.sh must emit
|
||||
# byte-identical output for the same environment. CI enforces that by running
|
||||
# both on the same host and diffing stdout (skill-preflight job). If you
|
||||
# change one, change the other, or the build goes red.
|
||||
#
|
||||
# STATUS=ok means the harness is present and runnable here. It does NOT mean
|
||||
# this host can measure your change: only the A/A self-test decides that.
|
||||
# A refusal (STATUS=refuse, exit 1) ends the *claim*, not the work.
|
||||
#
|
||||
# Source is deliberately pure ASCII: Windows PowerShell 5.1 misdecodes
|
||||
# BOM-less UTF-8, so the em dashes in the shared refusal strings are composed
|
||||
# at runtime instead of written literally.
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$EM = [string][char]0x2014 # em dash, byte-identical to preflight.sh output
|
||||
|
||||
function Say([string]$s) { Write-Output $s }
|
||||
function Refuse([string]$reason) {
|
||||
Say 'STATUS=refuse'
|
||||
Say "REASON=$reason"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# --- host -------------------------------------------------------------
|
||||
# bench-runner shells every timed run through `sh -c`, and the in-repo
|
||||
# pipeline is POSIX-only. Windows compiles the workspace but cannot run it.
|
||||
# Host check comes FIRST, exactly as in preflight.sh.
|
||||
$onLinux = (Test-Path variable:IsLinux) -and $IsLinux
|
||||
$onMac = (Test-Path variable:IsMacOS) -and $IsMacOS
|
||||
if ($onLinux) { $hostKind = 'linux' }
|
||||
elseif ($onMac) { $hostKind = 'darwin' }
|
||||
else {
|
||||
Refuse ("unsupported host 'Windows' $EM peltier's harness is Linux/POSIX-only at runtime. " +
|
||||
"Run it on Linux (or macOS for verify mode). Do not substitute another benchmark.")
|
||||
}
|
||||
|
||||
# --- trust layer ------------------------------------------------------
|
||||
function Test-Peltier([string]$dir) {
|
||||
(Test-Path (Join-Path $dir 'crates/bench-runner/Cargo.toml') -PathType Leaf) -and
|
||||
(Test-Path (Join-Path $dir 'config/accept.toml') -PathType Leaf)
|
||||
}
|
||||
|
||||
# Walk up from a starting dir looking for a checkout; parents terminate at
|
||||
# the filesystem root, so this cannot spin.
|
||||
function Find-Peltier([string]$start) {
|
||||
$d = $start
|
||||
while (-not [string]::IsNullOrEmpty($d)) {
|
||||
if (Test-Peltier $d) { return $d }
|
||||
$parent = Split-Path -Parent $d
|
||||
if ($parent -eq $d) { break }
|
||||
$d = $parent
|
||||
}
|
||||
return $null
|
||||
}
|
||||
|
||||
$peltierHome = $null
|
||||
if ($env:PELTIER_HOME) {
|
||||
$canon = $null
|
||||
try { $canon = (Resolve-Path -LiteralPath $env:PELTIER_HOME).Path } catch {}
|
||||
if (-not $canon) { Refuse "PELTIER_HOME='$($env:PELTIER_HOME)' does not exist" }
|
||||
if (-not (Test-Peltier $canon)) {
|
||||
Refuse "PELTIER_HOME='$($env:PELTIER_HOME)' is not a peltier checkout (no crates/bench-runner/Cargo.toml + config/accept.toml)"
|
||||
}
|
||||
$peltierHome = $canon
|
||||
} else {
|
||||
# cwd first (working inside a checkout), then this script's own location
|
||||
# (the skill still living inside one). A skill copied into another project
|
||||
# finds neither - which is why PELTIER_HOME exists.
|
||||
$peltierHome = Find-Peltier (Get-Location).Path
|
||||
if (-not $peltierHome) { $peltierHome = Find-Peltier $PSScriptRoot }
|
||||
}
|
||||
if (-not $peltierHome) {
|
||||
Refuse ("no peltier checkout found $EM clone https://github.com/bdbrown4/peltier.io and set " +
|
||||
"PELTIER_HOME=/path/to/peltier.io. The statistics live in bench-runner; do not reimplement them.")
|
||||
}
|
||||
|
||||
if (-not (Get-Command cargo -ErrorAction SilentlyContinue)) {
|
||||
Refuse "cargo not on PATH $EM needed to build bench-runner from $peltierHome"
|
||||
}
|
||||
|
||||
# --- build the real harness -------------------------------------------
|
||||
# Always build. cargo is a no-op when current, and skipping this because a
|
||||
# binary happens to exist would silently measure with a stale copy of the
|
||||
# statistics after a `git pull`.
|
||||
$br = Join-Path $peltierHome 'target/release/bench-runner'
|
||||
Push-Location $peltierHome
|
||||
try { & cargo build --release -q -p bench-runner; $buildOk = ($LASTEXITCODE -eq 0) }
|
||||
finally { Pop-Location }
|
||||
if (-not $buildOk) {
|
||||
Refuse "bench-runner failed to build in $peltierHome $EM fix the build before claiming any number"
|
||||
}
|
||||
if (-not (Test-Path $br -PathType Leaf)) {
|
||||
Refuse "bench-runner did not appear at $br after a successful build"
|
||||
}
|
||||
|
||||
# --- pinning ----------------------------------------------------------
|
||||
# NOTE: config/accept.toml's `pin_prefix` is read ONLY by the in-repo verdict
|
||||
# pipeline. For compare/aa/calibrate you must wrap the commands yourself
|
||||
# (`taskset -c N <cmd>` on both sides). PIN_SUPPORTED just says whether that
|
||||
# tool is available here.
|
||||
if ($onLinux -and (Get-Command taskset -ErrorAction SilentlyContinue)) { $pin = 'yes' } else { $pin = 'no' }
|
||||
|
||||
Say 'STATUS=ok'
|
||||
Say "PELTIER_HOME=$peltierHome"
|
||||
Say "BENCH_RUNNER=$br"
|
||||
Say "HOST=$hostKind"
|
||||
Say "PIN_SUPPORTED=$pin"
|
||||
Say 'NOTE=harness runnable here; the A/A self-test decides whether it can measure your change'
|
||||
@@ -30,6 +30,13 @@ plugin, or copied into another project):
|
||||
|
||||
sh <skill-dir>/scripts/preflight.sh
|
||||
|
||||
# no POSIX shell on this host (Windows-native harness)? same contract:
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File <skill-dir>/scripts/preflight.ps1
|
||||
|
||||
The two entry points implement one contract and are byte-diffed against each
|
||||
other in CI — same checks, same output, same refusals. Use whichever your
|
||||
host can launch; never both, never neither.
|
||||
|
||||
Some harnesses block script files inside skills, so your installation may
|
||||
carry no `scripts/` directory. The script still exists exactly once — in the
|
||||
peltier checkout this skill drives (which you need anyway; it is where
|
||||
|
||||
@@ -6,7 +6,9 @@ faster. Find out. Works on any pair of shell commands, in any language.
|
||||
## 0. Preflight
|
||||
|
||||
sh <skill-dir>/scripts/preflight.sh
|
||||
# or, if this installation carries no scripts/ (script-blocking harness):
|
||||
# no POSIX shell (Windows-native harness)? same contract, CI-diffed against the sh version:
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File <skill-dir>/scripts/preflight.ps1
|
||||
# installation carries no scripts/ (script-blocking harness)? run from the checkout:
|
||||
sh "$PELTIER_HOME/.claude/skills/peltier/scripts/preflight.sh"
|
||||
|
||||
Read the `key=value` output; you need `BENCH_RUNNER` and `PIN_SUPPORTED`. A
|
||||
|
||||
@@ -0,0 +1,112 @@
|
||||
# peltier preflight - PowerShell entry point. Same contract as preflight.sh,
|
||||
# for hosts where no POSIX shell exists (Windows-native harnesses). Works on
|
||||
# Windows PowerShell 5.1 and pwsh 7+ on any OS.
|
||||
#
|
||||
# THE CONTRACT IS DEFINED ONCE: this script and preflight.sh must emit
|
||||
# byte-identical output for the same environment. CI enforces that by running
|
||||
# both on the same host and diffing stdout (skill-preflight job). If you
|
||||
# change one, change the other, or the build goes red.
|
||||
#
|
||||
# STATUS=ok means the harness is present and runnable here. It does NOT mean
|
||||
# this host can measure your change: only the A/A self-test decides that.
|
||||
# A refusal (STATUS=refuse, exit 1) ends the *claim*, not the work.
|
||||
#
|
||||
# Source is deliberately pure ASCII: Windows PowerShell 5.1 misdecodes
|
||||
# BOM-less UTF-8, so the em dashes in the shared refusal strings are composed
|
||||
# at runtime instead of written literally.
|
||||
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$EM = [string][char]0x2014 # em dash, byte-identical to preflight.sh output
|
||||
|
||||
function Say([string]$s) { Write-Output $s }
|
||||
function Refuse([string]$reason) {
|
||||
Say 'STATUS=refuse'
|
||||
Say "REASON=$reason"
|
||||
exit 1
|
||||
}
|
||||
|
||||
# --- host -------------------------------------------------------------
|
||||
# bench-runner shells every timed run through `sh -c`, and the in-repo
|
||||
# pipeline is POSIX-only. Windows compiles the workspace but cannot run it.
|
||||
# Host check comes FIRST, exactly as in preflight.sh.
|
||||
$onLinux = (Test-Path variable:IsLinux) -and $IsLinux
|
||||
$onMac = (Test-Path variable:IsMacOS) -and $IsMacOS
|
||||
if ($onLinux) { $hostKind = 'linux' }
|
||||
elseif ($onMac) { $hostKind = 'darwin' }
|
||||
else {
|
||||
Refuse ("unsupported host 'Windows' $EM peltier's harness is Linux/POSIX-only at runtime. " +
|
||||
"Run it on Linux (or macOS for verify mode). Do not substitute another benchmark.")
|
||||
}
|
||||
|
||||
# --- trust layer ------------------------------------------------------
|
||||
function Test-Peltier([string]$dir) {
|
||||
(Test-Path (Join-Path $dir 'crates/bench-runner/Cargo.toml') -PathType Leaf) -and
|
||||
(Test-Path (Join-Path $dir 'config/accept.toml') -PathType Leaf)
|
||||
}
|
||||
|
||||
# Walk up from a starting dir looking for a checkout; parents terminate at
|
||||
# the filesystem root, so this cannot spin.
|
||||
function Find-Peltier([string]$start) {
|
||||
$d = $start
|
||||
while (-not [string]::IsNullOrEmpty($d)) {
|
||||
if (Test-Peltier $d) { return $d }
|
||||
$parent = Split-Path -Parent $d
|
||||
if ($parent -eq $d) { break }
|
||||
$d = $parent
|
||||
}
|
||||
return $null
|
||||
}
|
||||
|
||||
$peltierHome = $null
|
||||
if ($env:PELTIER_HOME) {
|
||||
$canon = $null
|
||||
try { $canon = (Resolve-Path -LiteralPath $env:PELTIER_HOME).Path } catch {}
|
||||
if (-not $canon) { Refuse "PELTIER_HOME='$($env:PELTIER_HOME)' does not exist" }
|
||||
if (-not (Test-Peltier $canon)) {
|
||||
Refuse "PELTIER_HOME='$($env:PELTIER_HOME)' is not a peltier checkout (no crates/bench-runner/Cargo.toml + config/accept.toml)"
|
||||
}
|
||||
$peltierHome = $canon
|
||||
} else {
|
||||
# cwd first (working inside a checkout), then this script's own location
|
||||
# (the skill still living inside one). A skill copied into another project
|
||||
# finds neither - which is why PELTIER_HOME exists.
|
||||
$peltierHome = Find-Peltier (Get-Location).Path
|
||||
if (-not $peltierHome) { $peltierHome = Find-Peltier $PSScriptRoot }
|
||||
}
|
||||
if (-not $peltierHome) {
|
||||
Refuse ("no peltier checkout found $EM clone https://github.com/bdbrown4/peltier.io and set " +
|
||||
"PELTIER_HOME=/path/to/peltier.io. The statistics live in bench-runner; do not reimplement them.")
|
||||
}
|
||||
|
||||
if (-not (Get-Command cargo -ErrorAction SilentlyContinue)) {
|
||||
Refuse "cargo not on PATH $EM needed to build bench-runner from $peltierHome"
|
||||
}
|
||||
|
||||
# --- build the real harness -------------------------------------------
|
||||
# Always build. cargo is a no-op when current, and skipping this because a
|
||||
# binary happens to exist would silently measure with a stale copy of the
|
||||
# statistics after a `git pull`.
|
||||
$br = Join-Path $peltierHome 'target/release/bench-runner'
|
||||
Push-Location $peltierHome
|
||||
try { & cargo build --release -q -p bench-runner; $buildOk = ($LASTEXITCODE -eq 0) }
|
||||
finally { Pop-Location }
|
||||
if (-not $buildOk) {
|
||||
Refuse "bench-runner failed to build in $peltierHome $EM fix the build before claiming any number"
|
||||
}
|
||||
if (-not (Test-Path $br -PathType Leaf)) {
|
||||
Refuse "bench-runner did not appear at $br after a successful build"
|
||||
}
|
||||
|
||||
# --- pinning ----------------------------------------------------------
|
||||
# NOTE: config/accept.toml's `pin_prefix` is read ONLY by the in-repo verdict
|
||||
# pipeline. For compare/aa/calibrate you must wrap the commands yourself
|
||||
# (`taskset -c N <cmd>` on both sides). PIN_SUPPORTED just says whether that
|
||||
# tool is available here.
|
||||
if ($onLinux -and (Get-Command taskset -ErrorAction SilentlyContinue)) { $pin = 'yes' } else { $pin = 'no' }
|
||||
|
||||
Say 'STATUS=ok'
|
||||
Say "PELTIER_HOME=$peltierHome"
|
||||
Say "BENCH_RUNNER=$br"
|
||||
Say "HOST=$hostKind"
|
||||
Say "PIN_SUPPORTED=$pin"
|
||||
Say 'NOTE=harness runnable here; the A/A self-test decides whether it can measure your change'
|
||||
@@ -186,6 +186,31 @@ jobs:
|
||||
sh "$GITHUB_WORKSPACE/.claude/skills/peltier/scripts/preflight.sh" | tee pf.txt
|
||||
grep -q '^STATUS=ok$' pf.txt
|
||||
|
||||
- name: sh and ps1 preflights are byte-equivalent (two entry points, one contract)
|
||||
run: |
|
||||
# preflight.ps1 exists for hosts with no POSIX shell. Two
|
||||
# implementations of the same contract can drift, so both run here
|
||||
# on the same host and their stdout is byte-diffed on every path.
|
||||
# 1. in-checkout resolve
|
||||
sh .claude/skills/peltier/scripts/preflight.sh > /tmp/eq-sh.txt
|
||||
pwsh -NoProfile -File .claude/skills/peltier/scripts/preflight.ps1 > /tmp/eq-ps.txt
|
||||
diff /tmp/eq-sh.txt /tmp/eq-ps.txt
|
||||
# 2. refusal with no checkout reachable (copied out of the repo)
|
||||
cp -r .claude/skills/peltier /tmp/eqskill
|
||||
mkdir -p /tmp/eq-not-a-repo && cd /tmp/eq-not-a-repo
|
||||
set +e
|
||||
env -u PELTIER_HOME sh /tmp/eqskill/scripts/preflight.sh > sh.txt 2>&1; shrc=$?
|
||||
env -u PELTIER_HOME pwsh -NoProfile -File /tmp/eqskill/scripts/preflight.ps1 > ps.txt 2>&1; psrc=$?
|
||||
set -e
|
||||
test "$shrc" -ne 0 && test "$psrc" -ne 0
|
||||
diff sh.txt ps.txt
|
||||
# 3. resolve via PELTIER_HOME from a foreign repo
|
||||
mkdir -p /tmp/eq-foreign && cd /tmp/eq-foreign
|
||||
PELTIER_HOME="$GITHUB_WORKSPACE" sh /tmp/eqskill/scripts/preflight.sh > sh2.txt
|
||||
PELTIER_HOME="$GITHUB_WORKSPACE" pwsh -NoProfile -File /tmp/eqskill/scripts/preflight.ps1 > ps2.txt
|
||||
diff sh2.txt ps2.txt
|
||||
echo "equivalence holds on all three paths"
|
||||
|
||||
- name: The harness the skill drives actually runs (plumbing, not verdict)
|
||||
run: |
|
||||
BR=$(sed -n 's/^BENCH_RUNNER=//p' /tmp/pf.txt)
|
||||
@@ -200,3 +225,40 @@ jobs:
|
||||
run: |
|
||||
BR=$(sed -n 's/^BENCH_RUNNER=//p' /tmp/pf.txt)
|
||||
"$BR" --config /tmp/accept.toml aa --cmd 'sleep 0.02'
|
||||
|
||||
# The refusal doctrine must survive hosts with no POSIX shell: on a real
|
||||
# Windows runner, BOTH preflight entry points must refuse with a structured
|
||||
# STATUS=refuse — never a launch failure an agent could misread as license
|
||||
# to hand-roll a timing loop. Windows PowerShell 5.1 is tested separately
|
||||
# from pwsh because 5.1 misdecodes BOM-less UTF-8 (the ps1 source is pure
|
||||
# ASCII for exactly that reason).
|
||||
skill-preflight-windows:
|
||||
runs-on: windows-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: ps1 refuses natively (Windows PowerShell 5.1)
|
||||
shell: powershell
|
||||
run: |
|
||||
$out = & powershell -NoProfile -ExecutionPolicy Bypass -File .claude\skills\peltier\scripts\preflight.ps1
|
||||
$out | Write-Output
|
||||
if ($LASTEXITCODE -eq 0) { throw "preflight did not refuse on Windows" }
|
||||
$joined = $out -join "`n"
|
||||
if ($joined -notlike "*STATUS=refuse*") { throw "no STATUS=refuse in output" }
|
||||
if ($joined -notlike "*unsupported host 'Windows'*") { throw "missing refusal reason" }
|
||||
- name: ps1 refuses natively (pwsh)
|
||||
shell: pwsh
|
||||
run: |
|
||||
$out = & pwsh -NoProfile -File .claude\skills\peltier\scripts\preflight.ps1
|
||||
$out | Write-Output
|
||||
if ($LASTEXITCODE -eq 0) { throw "preflight did not refuse under pwsh" }
|
||||
if (($out -join "`n") -notlike "*STATUS=refuse*") { throw "no STATUS=refuse in output" }
|
||||
- name: sh entry point also refuses here (Git Bash)
|
||||
shell: bash
|
||||
run: |
|
||||
set +e
|
||||
sh .claude/skills/peltier/scripts/preflight.sh > out.txt 2>&1
|
||||
rc=$?
|
||||
set -e
|
||||
cat out.txt
|
||||
test "$rc" -ne 0
|
||||
grep -q '^STATUS=refuse$' out.txt
|
||||
|
||||
@@ -29,7 +29,7 @@ Full engineering spec: `SPEC.md`. Read it before starting any phase.
|
||||
- `targets/` — vendored OSS targets for case studies (permissive licenses only)
|
||||
- `playbook/` — optimization classes, ordered, with preconditions and known risks
|
||||
- `results/` — generated reports and flamegraphs
|
||||
- `.claude/skills/peltier/` — the trust layer packaged as a consumable **Agent Skill** (agentskills.io SKILL.md standard, spec-compliant frontmatter): `verify` mode (equivalence → A/A calibration → interleaved A/B + bootstrap CI) works in any repo against any two shell commands; `attempt` mode drives the full in-repo loop. Drives the real `bench-runner` binary and refuses on unsupported hosts rather than falling back to a second, forkable copy of the statistics. Cross-harness: mirrored byte-identically at `.agents/skills/peltier/` (the Codex/Zed/Cursor/Gemini/Copilot convergence path — CI enforces the mirror), `AGENTS.md` at the repo root points non-Claude harnesses here, and `scripts/install-skill.sh` stamps consumer repos (plus a script-less `--zeroclaw-variant`, since ZeroClaw blocks script files in skills by default — its preflight runs from the checkout). Preflight refusal, spec conformance, mirror equality, and the installer are all asserted in CI (`skill-preflight`).
|
||||
- `.claude/skills/peltier/` — the trust layer packaged as a consumable **Agent Skill** (agentskills.io SKILL.md standard, spec-compliant frontmatter): `verify` mode (equivalence → A/A calibration → interleaved A/B + bootstrap CI) works in any repo against any two shell commands; `attempt` mode drives the full in-repo loop. Drives the real `bench-runner` binary and refuses on unsupported hosts rather than falling back to a second, forkable copy of the statistics. Cross-harness: mirrored byte-identically at `.agents/skills/peltier/` (the Codex/Zed/Cursor/Gemini/Copilot convergence path — CI enforces the mirror), `AGENTS.md` at the repo root points non-Claude harnesses here, and `scripts/install-skill.sh` stamps consumer repos (plus a script-less `--zeroclaw-variant`, since ZeroClaw blocks script files in skills by default — its preflight runs from the checkout). Preflight ships as two entry points implementing one contract — `preflight.sh` (POSIX) and `preflight.ps1` (Windows PowerShell 5.1/pwsh; pure-ASCII source, em dashes composed at runtime) — CI byte-diffs their output on all three paths and a `windows-latest` job asserts the native refusal. Preflight refusal, sh↔ps1 equivalence, spec conformance, mirror equality, and the installer are all asserted in CI (`skill-preflight`, `skill-preflight-windows`).
|
||||
|
||||
**Phase 4 COMPLETE — all SPEC §5 exit criteria met. Phases 0–4 are all done.** Service-mode latency benchmarking shipped: a coordinated-omission-correct, open-loop, fixed-rate load generator (`crates/bench-runner/src/service.rs`) measuring latency from each request's intended send time (queueing counted, not hidden), interleaved A/B, exact percentiles, bootstrap p50/p99 CIs. A real service target — `targets/cjson/service.c`, a minimal HTTP server wrapping the patched `cJSON.c` (trust-layer, outside the workspace allowlist). Service calibration PASS (0/10 A/A false-positive, 10/10 injected-5%-latency-regression detection; `results/calibration/cjson-service-aa.json`). **The verified batch win phase3-cjson-002 measured under a 150 rps-target replay (20 interleaved rounds, 0 drops — the `phase4-cjson-service` ledger row records sessions=20, rate_rps=150, drop_rate=0.0; total request count is not machine-recorded): p50 latency +6.2%, 95% CI [+5.8%, +7.2%] — accepted; p99 NOT claimed (CI [0.07, 4.97], single-worker loopback tail jitter) — harness correctly rejected it.** Mechanical ROI report generator (`crates/report` bin, `just report`, SPEC §9): reads a ledger row + service-latency JSON → throughput→cores→dollars (27.5 cores / $9,621-per-year CI lower bound on a 500-core fleet) *and* latency percentiles, every figure with CI + workload + methodology inline; flags any non-accepted row. Case study: `results/phase4/`. **The roadmap is complete — the profile→hypothesize→patch→gated-verdict→ROI loop runs end to end across Rust and C, batch and service, with zero shipped false accepts across all phases.**
|
||||
|
||||
|
||||
+7
-1
@@ -83,7 +83,13 @@ document. In `verify` mode, the gate order is the content:
|
||||
|
||||
Measurement requires a Linux/POSIX host — the same requirement as the rest of
|
||||
the pipeline. On Windows (or with no checkout reachable) the skill refuses and
|
||||
says why. That is asserted, not assumed: on every pull request, CI runs the
|
||||
says why. The refusal itself is cross-OS: preflight ships as both `preflight.sh`
|
||||
(POSIX) and `preflight.ps1` (Windows PowerShell 5.1 / pwsh), so a Windows-native
|
||||
harness with no POSIX shell still gets a structured `STATUS=refuse` instead of a
|
||||
launch error it might misread as license to hand-roll a timing loop. Two
|
||||
implementations of one contract is a drift risk, so CI byte-diffs their output
|
||||
on every path (resolve, refuse, `PELTIER_HOME`) on the same host, and a Windows
|
||||
runner asserts the native refusal on every pull request. That is asserted, not assumed: on every pull request, CI runs the
|
||||
preflight's refusal path (skill copied outside any checkout) and its resolve
|
||||
paths (in-checkout and via `PELTIER_HOME` from a foreign repo), alongside a
|
||||
live plumbing run of the harness it drives.
|
||||
|
||||
Reference in New Issue
Block a user