skill: cross-OS preflight — ps1 entry point, one CI-enforced contract

On a Windows host the preflight's job is to deliver a structured refusal
(measurement is Linux/POSIX-only). But a Windows-native harness with no
POSIX shell never got that far: preflight.sh fails to LAUNCH, and an agent
staring at "sh: not found" is exactly the agent that hand-rolls a timing
loop. The refusal doctrine has to be as portable as the skill.

- scripts/preflight.ps1: faithful port of the preflight contract, runs on
  Windows PowerShell 5.1 and pwsh 7+ on any OS. Host check first (so
  Windows refuses before touching anything), same walk-up resolution, same
  always-build rule, same key=value output, same exit codes. The source is
  pure ASCII with the shared em dashes composed at runtime — Windows
  PowerShell 5.1 misdecodes BOM-less UTF-8, and that exact bug class
  already bit this machine once (the francis-watchdog parse failure).
- Two implementations of one contract is a drift risk, so it is gated like
  every other claim in this repo: the skill-preflight CI job now runs BOTH
  scripts on the same ubuntu host and byte-diffs their stdout on all three
  paths (in-checkout resolve, no-checkout refusal, PELTIER_HOME from a
  foreign repo). Disagreement is a red build, not a footnote.
- New skill-preflight-windows job (windows-latest): asserts the native
  refusal end-to-end under Windows PowerShell 5.1 AND pwsh (STATUS=refuse,
  nonzero exit, the 'unsupported host' reason), plus Git Bash sh for good
  measure. Verified live on a real Windows 11 box before shipping: 5.1
  parses the pure-ASCII source clean and refuses with exit 1.
- SKILL.md / references/verify.md: document the second entry point ("use
  whichever your host can launch; never both, never neither"); book chapter
  and CLAUDE.md updated to match. .agents mirror regenerated (identical).
  The ZeroClaw variant is unaffected: rm -rf scripts/ drops the ps1 too,
  and its audit still passes (7 files scanned).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-16 08:15:43 -04:00
co-authored by Claude Fable 5
parent c5531ed30c
commit 0f2020779e
9 changed files with 314 additions and 4 deletions
+7
View File
@@ -30,6 +30,13 @@ plugin, or copied into another project):
sh <skill-dir>/scripts/preflight.sh sh <skill-dir>/scripts/preflight.sh
# no POSIX shell on this host (Windows-native harness)? same contract:
powershell -NoProfile -ExecutionPolicy Bypass -File <skill-dir>/scripts/preflight.ps1
The two entry points implement one contract and are byte-diffed against each
other in CI — same checks, same output, same refusals. Use whichever your
host can launch; never both, never neither.
Some harnesses block script files inside skills, so your installation may Some harnesses block script files inside skills, so your installation may
carry no `scripts/` directory. The script still exists exactly once — in the carry no `scripts/` directory. The script still exists exactly once — in the
peltier checkout this skill drives (which you need anyway; it is where peltier checkout this skill drives (which you need anyway; it is where
+3 -1
View File
@@ -6,7 +6,9 @@ faster. Find out. Works on any pair of shell commands, in any language.
## 0. Preflight ## 0. Preflight
sh <skill-dir>/scripts/preflight.sh sh <skill-dir>/scripts/preflight.sh
# or, if this installation carries no scripts/ (script-blocking harness): # no POSIX shell (Windows-native harness)? same contract, CI-diffed against the sh version:
powershell -NoProfile -ExecutionPolicy Bypass -File <skill-dir>/scripts/preflight.ps1
# installation carries no scripts/ (script-blocking harness)? run from the checkout:
sh "$PELTIER_HOME/.claude/skills/peltier/scripts/preflight.sh" sh "$PELTIER_HOME/.claude/skills/peltier/scripts/preflight.sh"
Read the `key=value` output; you need `BENCH_RUNNER` and `PIN_SUPPORTED`. A Read the `key=value` output; you need `BENCH_RUNNER` and `PIN_SUPPORTED`. A
@@ -0,0 +1,112 @@
# peltier preflight - PowerShell entry point. Same contract as preflight.sh,
# for hosts where no POSIX shell exists (Windows-native harnesses). Works on
# Windows PowerShell 5.1 and pwsh 7+ on any OS.
#
# THE CONTRACT IS DEFINED ONCE: this script and preflight.sh must emit
# byte-identical output for the same environment. CI enforces that by running
# both on the same host and diffing stdout (skill-preflight job). If you
# change one, change the other, or the build goes red.
#
# STATUS=ok means the harness is present and runnable here. It does NOT mean
# this host can measure your change: only the A/A self-test decides that.
# A refusal (STATUS=refuse, exit 1) ends the *claim*, not the work.
#
# Source is deliberately pure ASCII: Windows PowerShell 5.1 misdecodes
# BOM-less UTF-8, so the em dashes in the shared refusal strings are composed
# at runtime instead of written literally.
$ErrorActionPreference = 'Stop'
$EM = [string][char]0x2014 # em dash, byte-identical to preflight.sh output
function Say([string]$s) { Write-Output $s }
function Refuse([string]$reason) {
Say 'STATUS=refuse'
Say "REASON=$reason"
exit 1
}
# --- host -------------------------------------------------------------
# bench-runner shells every timed run through `sh -c`, and the in-repo
# pipeline is POSIX-only. Windows compiles the workspace but cannot run it.
# Host check comes FIRST, exactly as in preflight.sh.
$onLinux = (Test-Path variable:IsLinux) -and $IsLinux
$onMac = (Test-Path variable:IsMacOS) -and $IsMacOS
if ($onLinux) { $hostKind = 'linux' }
elseif ($onMac) { $hostKind = 'darwin' }
else {
Refuse ("unsupported host 'Windows' $EM peltier's harness is Linux/POSIX-only at runtime. " +
"Run it on Linux (or macOS for verify mode). Do not substitute another benchmark.")
}
# --- trust layer ------------------------------------------------------
function Test-Peltier([string]$dir) {
(Test-Path (Join-Path $dir 'crates/bench-runner/Cargo.toml') -PathType Leaf) -and
(Test-Path (Join-Path $dir 'config/accept.toml') -PathType Leaf)
}
# Walk up from a starting dir looking for a checkout; parents terminate at
# the filesystem root, so this cannot spin.
function Find-Peltier([string]$start) {
$d = $start
while (-not [string]::IsNullOrEmpty($d)) {
if (Test-Peltier $d) { return $d }
$parent = Split-Path -Parent $d
if ($parent -eq $d) { break }
$d = $parent
}
return $null
}
$peltierHome = $null
if ($env:PELTIER_HOME) {
$canon = $null
try { $canon = (Resolve-Path -LiteralPath $env:PELTIER_HOME).Path } catch {}
if (-not $canon) { Refuse "PELTIER_HOME='$($env:PELTIER_HOME)' does not exist" }
if (-not (Test-Peltier $canon)) {
Refuse "PELTIER_HOME='$($env:PELTIER_HOME)' is not a peltier checkout (no crates/bench-runner/Cargo.toml + config/accept.toml)"
}
$peltierHome = $canon
} else {
# cwd first (working inside a checkout), then this script's own location
# (the skill still living inside one). A skill copied into another project
# finds neither - which is why PELTIER_HOME exists.
$peltierHome = Find-Peltier (Get-Location).Path
if (-not $peltierHome) { $peltierHome = Find-Peltier $PSScriptRoot }
}
if (-not $peltierHome) {
Refuse ("no peltier checkout found $EM clone https://github.com/bdbrown4/peltier.io and set " +
"PELTIER_HOME=/path/to/peltier.io. The statistics live in bench-runner; do not reimplement them.")
}
if (-not (Get-Command cargo -ErrorAction SilentlyContinue)) {
Refuse "cargo not on PATH $EM needed to build bench-runner from $peltierHome"
}
# --- build the real harness -------------------------------------------
# Always build. cargo is a no-op when current, and skipping this because a
# binary happens to exist would silently measure with a stale copy of the
# statistics after a `git pull`.
$br = Join-Path $peltierHome 'target/release/bench-runner'
Push-Location $peltierHome
try { & cargo build --release -q -p bench-runner; $buildOk = ($LASTEXITCODE -eq 0) }
finally { Pop-Location }
if (-not $buildOk) {
Refuse "bench-runner failed to build in $peltierHome $EM fix the build before claiming any number"
}
if (-not (Test-Path $br -PathType Leaf)) {
Refuse "bench-runner did not appear at $br after a successful build"
}
# --- pinning ----------------------------------------------------------
# NOTE: config/accept.toml's `pin_prefix` is read ONLY by the in-repo verdict
# pipeline. For compare/aa/calibrate you must wrap the commands yourself
# (`taskset -c N <cmd>` on both sides). PIN_SUPPORTED just says whether that
# tool is available here.
if ($onLinux -and (Get-Command taskset -ErrorAction SilentlyContinue)) { $pin = 'yes' } else { $pin = 'no' }
Say 'STATUS=ok'
Say "PELTIER_HOME=$peltierHome"
Say "BENCH_RUNNER=$br"
Say "HOST=$hostKind"
Say "PIN_SUPPORTED=$pin"
Say 'NOTE=harness runnable here; the A/A self-test decides whether it can measure your change'
+7
View File
@@ -30,6 +30,13 @@ plugin, or copied into another project):
sh <skill-dir>/scripts/preflight.sh sh <skill-dir>/scripts/preflight.sh
# no POSIX shell on this host (Windows-native harness)? same contract:
powershell -NoProfile -ExecutionPolicy Bypass -File <skill-dir>/scripts/preflight.ps1
The two entry points implement one contract and are byte-diffed against each
other in CI — same checks, same output, same refusals. Use whichever your
host can launch; never both, never neither.
Some harnesses block script files inside skills, so your installation may Some harnesses block script files inside skills, so your installation may
carry no `scripts/` directory. The script still exists exactly once — in the carry no `scripts/` directory. The script still exists exactly once — in the
peltier checkout this skill drives (which you need anyway; it is where peltier checkout this skill drives (which you need anyway; it is where
+3 -1
View File
@@ -6,7 +6,9 @@ faster. Find out. Works on any pair of shell commands, in any language.
## 0. Preflight ## 0. Preflight
sh <skill-dir>/scripts/preflight.sh sh <skill-dir>/scripts/preflight.sh
# or, if this installation carries no scripts/ (script-blocking harness): # no POSIX shell (Windows-native harness)? same contract, CI-diffed against the sh version:
powershell -NoProfile -ExecutionPolicy Bypass -File <skill-dir>/scripts/preflight.ps1
# installation carries no scripts/ (script-blocking harness)? run from the checkout:
sh "$PELTIER_HOME/.claude/skills/peltier/scripts/preflight.sh" sh "$PELTIER_HOME/.claude/skills/peltier/scripts/preflight.sh"
Read the `key=value` output; you need `BENCH_RUNNER` and `PIN_SUPPORTED`. A Read the `key=value` output; you need `BENCH_RUNNER` and `PIN_SUPPORTED`. A
@@ -0,0 +1,112 @@
# peltier preflight - PowerShell entry point. Same contract as preflight.sh,
# for hosts where no POSIX shell exists (Windows-native harnesses). Works on
# Windows PowerShell 5.1 and pwsh 7+ on any OS.
#
# THE CONTRACT IS DEFINED ONCE: this script and preflight.sh must emit
# byte-identical output for the same environment. CI enforces that by running
# both on the same host and diffing stdout (skill-preflight job). If you
# change one, change the other, or the build goes red.
#
# STATUS=ok means the harness is present and runnable here. It does NOT mean
# this host can measure your change: only the A/A self-test decides that.
# A refusal (STATUS=refuse, exit 1) ends the *claim*, not the work.
#
# Source is deliberately pure ASCII: Windows PowerShell 5.1 misdecodes
# BOM-less UTF-8, so the em dashes in the shared refusal strings are composed
# at runtime instead of written literally.
$ErrorActionPreference = 'Stop'
$EM = [string][char]0x2014 # em dash, byte-identical to preflight.sh output
function Say([string]$s) { Write-Output $s }
function Refuse([string]$reason) {
Say 'STATUS=refuse'
Say "REASON=$reason"
exit 1
}
# --- host -------------------------------------------------------------
# bench-runner shells every timed run through `sh -c`, and the in-repo
# pipeline is POSIX-only. Windows compiles the workspace but cannot run it.
# Host check comes FIRST, exactly as in preflight.sh.
$onLinux = (Test-Path variable:IsLinux) -and $IsLinux
$onMac = (Test-Path variable:IsMacOS) -and $IsMacOS
if ($onLinux) { $hostKind = 'linux' }
elseif ($onMac) { $hostKind = 'darwin' }
else {
Refuse ("unsupported host 'Windows' $EM peltier's harness is Linux/POSIX-only at runtime. " +
"Run it on Linux (or macOS for verify mode). Do not substitute another benchmark.")
}
# --- trust layer ------------------------------------------------------
function Test-Peltier([string]$dir) {
(Test-Path (Join-Path $dir 'crates/bench-runner/Cargo.toml') -PathType Leaf) -and
(Test-Path (Join-Path $dir 'config/accept.toml') -PathType Leaf)
}
# Walk up from a starting dir looking for a checkout; parents terminate at
# the filesystem root, so this cannot spin.
function Find-Peltier([string]$start) {
$d = $start
while (-not [string]::IsNullOrEmpty($d)) {
if (Test-Peltier $d) { return $d }
$parent = Split-Path -Parent $d
if ($parent -eq $d) { break }
$d = $parent
}
return $null
}
$peltierHome = $null
if ($env:PELTIER_HOME) {
$canon = $null
try { $canon = (Resolve-Path -LiteralPath $env:PELTIER_HOME).Path } catch {}
if (-not $canon) { Refuse "PELTIER_HOME='$($env:PELTIER_HOME)' does not exist" }
if (-not (Test-Peltier $canon)) {
Refuse "PELTIER_HOME='$($env:PELTIER_HOME)' is not a peltier checkout (no crates/bench-runner/Cargo.toml + config/accept.toml)"
}
$peltierHome = $canon
} else {
# cwd first (working inside a checkout), then this script's own location
# (the skill still living inside one). A skill copied into another project
# finds neither - which is why PELTIER_HOME exists.
$peltierHome = Find-Peltier (Get-Location).Path
if (-not $peltierHome) { $peltierHome = Find-Peltier $PSScriptRoot }
}
if (-not $peltierHome) {
Refuse ("no peltier checkout found $EM clone https://github.com/bdbrown4/peltier.io and set " +
"PELTIER_HOME=/path/to/peltier.io. The statistics live in bench-runner; do not reimplement them.")
}
if (-not (Get-Command cargo -ErrorAction SilentlyContinue)) {
Refuse "cargo not on PATH $EM needed to build bench-runner from $peltierHome"
}
# --- build the real harness -------------------------------------------
# Always build. cargo is a no-op when current, and skipping this because a
# binary happens to exist would silently measure with a stale copy of the
# statistics after a `git pull`.
$br = Join-Path $peltierHome 'target/release/bench-runner'
Push-Location $peltierHome
try { & cargo build --release -q -p bench-runner; $buildOk = ($LASTEXITCODE -eq 0) }
finally { Pop-Location }
if (-not $buildOk) {
Refuse "bench-runner failed to build in $peltierHome $EM fix the build before claiming any number"
}
if (-not (Test-Path $br -PathType Leaf)) {
Refuse "bench-runner did not appear at $br after a successful build"
}
# --- pinning ----------------------------------------------------------
# NOTE: config/accept.toml's `pin_prefix` is read ONLY by the in-repo verdict
# pipeline. For compare/aa/calibrate you must wrap the commands yourself
# (`taskset -c N <cmd>` on both sides). PIN_SUPPORTED just says whether that
# tool is available here.
if ($onLinux -and (Get-Command taskset -ErrorAction SilentlyContinue)) { $pin = 'yes' } else { $pin = 'no' }
Say 'STATUS=ok'
Say "PELTIER_HOME=$peltierHome"
Say "BENCH_RUNNER=$br"
Say "HOST=$hostKind"
Say "PIN_SUPPORTED=$pin"
Say 'NOTE=harness runnable here; the A/A self-test decides whether it can measure your change'
+62
View File
@@ -186,6 +186,31 @@ jobs:
sh "$GITHUB_WORKSPACE/.claude/skills/peltier/scripts/preflight.sh" | tee pf.txt sh "$GITHUB_WORKSPACE/.claude/skills/peltier/scripts/preflight.sh" | tee pf.txt
grep -q '^STATUS=ok$' pf.txt grep -q '^STATUS=ok$' pf.txt
- name: sh and ps1 preflights are byte-equivalent (two entry points, one contract)
run: |
# preflight.ps1 exists for hosts with no POSIX shell. Two
# implementations of the same contract can drift, so both run here
# on the same host and their stdout is byte-diffed on every path.
# 1. in-checkout resolve
sh .claude/skills/peltier/scripts/preflight.sh > /tmp/eq-sh.txt
pwsh -NoProfile -File .claude/skills/peltier/scripts/preflight.ps1 > /tmp/eq-ps.txt
diff /tmp/eq-sh.txt /tmp/eq-ps.txt
# 2. refusal with no checkout reachable (copied out of the repo)
cp -r .claude/skills/peltier /tmp/eqskill
mkdir -p /tmp/eq-not-a-repo && cd /tmp/eq-not-a-repo
set +e
env -u PELTIER_HOME sh /tmp/eqskill/scripts/preflight.sh > sh.txt 2>&1; shrc=$?
env -u PELTIER_HOME pwsh -NoProfile -File /tmp/eqskill/scripts/preflight.ps1 > ps.txt 2>&1; psrc=$?
set -e
test "$shrc" -ne 0 && test "$psrc" -ne 0
diff sh.txt ps.txt
# 3. resolve via PELTIER_HOME from a foreign repo
mkdir -p /tmp/eq-foreign && cd /tmp/eq-foreign
PELTIER_HOME="$GITHUB_WORKSPACE" sh /tmp/eqskill/scripts/preflight.sh > sh2.txt
PELTIER_HOME="$GITHUB_WORKSPACE" pwsh -NoProfile -File /tmp/eqskill/scripts/preflight.ps1 > ps2.txt
diff sh2.txt ps2.txt
echo "equivalence holds on all three paths"
- name: The harness the skill drives actually runs (plumbing, not verdict) - name: The harness the skill drives actually runs (plumbing, not verdict)
run: | run: |
BR=$(sed -n 's/^BENCH_RUNNER=//p' /tmp/pf.txt) BR=$(sed -n 's/^BENCH_RUNNER=//p' /tmp/pf.txt)
@@ -200,3 +225,40 @@ jobs:
run: | run: |
BR=$(sed -n 's/^BENCH_RUNNER=//p' /tmp/pf.txt) BR=$(sed -n 's/^BENCH_RUNNER=//p' /tmp/pf.txt)
"$BR" --config /tmp/accept.toml aa --cmd 'sleep 0.02' "$BR" --config /tmp/accept.toml aa --cmd 'sleep 0.02'
# The refusal doctrine must survive hosts with no POSIX shell: on a real
# Windows runner, BOTH preflight entry points must refuse with a structured
# STATUS=refuse — never a launch failure an agent could misread as license
# to hand-roll a timing loop. Windows PowerShell 5.1 is tested separately
# from pwsh because 5.1 misdecodes BOM-less UTF-8 (the ps1 source is pure
# ASCII for exactly that reason).
skill-preflight-windows:
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- name: ps1 refuses natively (Windows PowerShell 5.1)
shell: powershell
run: |
$out = & powershell -NoProfile -ExecutionPolicy Bypass -File .claude\skills\peltier\scripts\preflight.ps1
$out | Write-Output
if ($LASTEXITCODE -eq 0) { throw "preflight did not refuse on Windows" }
$joined = $out -join "`n"
if ($joined -notlike "*STATUS=refuse*") { throw "no STATUS=refuse in output" }
if ($joined -notlike "*unsupported host 'Windows'*") { throw "missing refusal reason" }
- name: ps1 refuses natively (pwsh)
shell: pwsh
run: |
$out = & pwsh -NoProfile -File .claude\skills\peltier\scripts\preflight.ps1
$out | Write-Output
if ($LASTEXITCODE -eq 0) { throw "preflight did not refuse under pwsh" }
if (($out -join "`n") -notlike "*STATUS=refuse*") { throw "no STATUS=refuse in output" }
- name: sh entry point also refuses here (Git Bash)
shell: bash
run: |
set +e
sh .claude/skills/peltier/scripts/preflight.sh > out.txt 2>&1
rc=$?
set -e
cat out.txt
test "$rc" -ne 0
grep -q '^STATUS=refuse$' out.txt
+1 -1
View File
@@ -29,7 +29,7 @@ Full engineering spec: `SPEC.md`. Read it before starting any phase.
- `targets/` — vendored OSS targets for case studies (permissive licenses only) - `targets/` — vendored OSS targets for case studies (permissive licenses only)
- `playbook/` — optimization classes, ordered, with preconditions and known risks - `playbook/` — optimization classes, ordered, with preconditions and known risks
- `results/` — generated reports and flamegraphs - `results/` — generated reports and flamegraphs
- `.claude/skills/peltier/` — the trust layer packaged as a consumable **Agent Skill** (agentskills.io SKILL.md standard, spec-compliant frontmatter): `verify` mode (equivalence → A/A calibration → interleaved A/B + bootstrap CI) works in any repo against any two shell commands; `attempt` mode drives the full in-repo loop. Drives the real `bench-runner` binary and refuses on unsupported hosts rather than falling back to a second, forkable copy of the statistics. Cross-harness: mirrored byte-identically at `.agents/skills/peltier/` (the Codex/Zed/Cursor/Gemini/Copilot convergence path — CI enforces the mirror), `AGENTS.md` at the repo root points non-Claude harnesses here, and `scripts/install-skill.sh` stamps consumer repos (plus a script-less `--zeroclaw-variant`, since ZeroClaw blocks script files in skills by default — its preflight runs from the checkout). Preflight refusal, spec conformance, mirror equality, and the installer are all asserted in CI (`skill-preflight`). - `.claude/skills/peltier/` — the trust layer packaged as a consumable **Agent Skill** (agentskills.io SKILL.md standard, spec-compliant frontmatter): `verify` mode (equivalence → A/A calibration → interleaved A/B + bootstrap CI) works in any repo against any two shell commands; `attempt` mode drives the full in-repo loop. Drives the real `bench-runner` binary and refuses on unsupported hosts rather than falling back to a second, forkable copy of the statistics. Cross-harness: mirrored byte-identically at `.agents/skills/peltier/` (the Codex/Zed/Cursor/Gemini/Copilot convergence path — CI enforces the mirror), `AGENTS.md` at the repo root points non-Claude harnesses here, and `scripts/install-skill.sh` stamps consumer repos (plus a script-less `--zeroclaw-variant`, since ZeroClaw blocks script files in skills by default — its preflight runs from the checkout). Preflight ships as two entry points implementing one contract — `preflight.sh` (POSIX) and `preflight.ps1` (Windows PowerShell 5.1/pwsh; pure-ASCII source, em dashes composed at runtime) — CI byte-diffs their output on all three paths and a `windows-latest` job asserts the native refusal. Preflight refusal, sh↔ps1 equivalence, spec conformance, mirror equality, and the installer are all asserted in CI (`skill-preflight`, `skill-preflight-windows`).
**Phase 4 COMPLETE — all SPEC §5 exit criteria met. Phases 0–4 are all done.** Service-mode latency benchmarking shipped: a coordinated-omission-correct, open-loop, fixed-rate load generator (`crates/bench-runner/src/service.rs`) measuring latency from each request's intended send time (queueing counted, not hidden), interleaved A/B, exact percentiles, bootstrap p50/p99 CIs. A real service target — `targets/cjson/service.c`, a minimal HTTP server wrapping the patched `cJSON.c` (trust-layer, outside the workspace allowlist). Service calibration PASS (0/10 A/A false-positive, 10/10 injected-5%-latency-regression detection; `results/calibration/cjson-service-aa.json`). **The verified batch win phase3-cjson-002 measured under a 150 rps-target replay (20 interleaved rounds, 0 drops — the `phase4-cjson-service` ledger row records sessions=20, rate_rps=150, drop_rate=0.0; total request count is not machine-recorded): p50 latency +6.2%, 95% CI [+5.8%, +7.2%] — accepted; p99 NOT claimed (CI [0.07, 4.97], single-worker loopback tail jitter) — harness correctly rejected it.** Mechanical ROI report generator (`crates/report` bin, `just report`, SPEC §9): reads a ledger row + service-latency JSON → throughput→cores→dollars (27.5 cores / $9,621-per-year CI lower bound on a 500-core fleet) *and* latency percentiles, every figure with CI + workload + methodology inline; flags any non-accepted row. Case study: `results/phase4/`. **The roadmap is complete — the profile→hypothesize→patch→gated-verdict→ROI loop runs end to end across Rust and C, batch and service, with zero shipped false accepts across all phases.** **Phase 4 COMPLETE — all SPEC §5 exit criteria met. Phases 0–4 are all done.** Service-mode latency benchmarking shipped: a coordinated-omission-correct, open-loop, fixed-rate load generator (`crates/bench-runner/src/service.rs`) measuring latency from each request's intended send time (queueing counted, not hidden), interleaved A/B, exact percentiles, bootstrap p50/p99 CIs. A real service target — `targets/cjson/service.c`, a minimal HTTP server wrapping the patched `cJSON.c` (trust-layer, outside the workspace allowlist). Service calibration PASS (0/10 A/A false-positive, 10/10 injected-5%-latency-regression detection; `results/calibration/cjson-service-aa.json`). **The verified batch win phase3-cjson-002 measured under a 150 rps-target replay (20 interleaved rounds, 0 drops — the `phase4-cjson-service` ledger row records sessions=20, rate_rps=150, drop_rate=0.0; total request count is not machine-recorded): p50 latency +6.2%, 95% CI [+5.8%, +7.2%] — accepted; p99 NOT claimed (CI [0.07, 4.97], single-worker loopback tail jitter) — harness correctly rejected it.** Mechanical ROI report generator (`crates/report` bin, `just report`, SPEC §9): reads a ledger row + service-latency JSON → throughput→cores→dollars (27.5 cores / $9,621-per-year CI lower bound on a 500-core fleet) *and* latency percentiles, every figure with CI + workload + methodology inline; flags any non-accepted row. Case study: `results/phase4/`. **The roadmap is complete — the profile→hypothesize→patch→gated-verdict→ROI loop runs end to end across Rust and C, batch and service, with zero shipped false accepts across all phases.**
+7 -1
View File
@@ -83,7 +83,13 @@ document. In `verify` mode, the gate order is the content:
Measurement requires a Linux/POSIX host — the same requirement as the rest of Measurement requires a Linux/POSIX host — the same requirement as the rest of
the pipeline. On Windows (or with no checkout reachable) the skill refuses and the pipeline. On Windows (or with no checkout reachable) the skill refuses and
says why. That is asserted, not assumed: on every pull request, CI runs the says why. The refusal itself is cross-OS: preflight ships as both `preflight.sh`
(POSIX) and `preflight.ps1` (Windows PowerShell 5.1 / pwsh), so a Windows-native
harness with no POSIX shell still gets a structured `STATUS=refuse` instead of a
launch error it might misread as license to hand-roll a timing loop. Two
implementations of one contract is a drift risk, so CI byte-diffs their output
on every path (resolve, refuse, `PELTIER_HOME`) on the same host, and a Windows
runner asserts the native refusal on every pull request. That is asserted, not assumed: on every pull request, CI runs the
preflight's refusal path (skill copied outside any checkout) and its resolve preflight's refusal path (skill copied outside any checkout) and its resolve
paths (in-checkout and via `PELTIER_HOME` from a foreign repo), alongside a paths (in-checkout and via `PELTIER_HOME` from a foreign repo), alongside a
live plumbing run of the harness it drives. live plumbing run of the harness it drives.