phase4: service-mode latency bench + mechanical ROI report generator

- bench-runner service mode (SPEC §3.1 mode c): a coordinated-omission-
  correct, open-loop, fixed-rate load generator; latency measured from
  the intended send time so queueing is counted, not hidden. Interleaved
  A/B of two server binaries, exact percentiles (bounded sample, no HDR
  bucketing error), bootstrap p50/p99 CIs via the existing ratio-CI
  machine. service-calibrate: A/A false-positive + injected latency-
  regression detection (calibration in progress).
- targets/cjson/service.c: minimal single-threaded HTTP server wrapping
  the patched cJSON.c (trust-layer, outside the workspace allowlist);
  env-toggled per-request busy-wait for the injection calibration only.
- report generator (crates/report bin): reads a ledger row +
  optional service-latency JSON, renders throughput→cores→dollars and/or
  latency percentiles, every figure with its CI, pricing source, and the
  methodology printed inline (SPEC §9); flags any non-accepted row so a
  rejected change never reads as a committed saving.
- ledger::report_row; justfile service/service-calibrate/report.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
This commit is contained in:
Claude
2026-07-11 23:31:58 +00:00
parent 3951857cf2
commit a78692bbb7
11 changed files with 981 additions and 0 deletions
+19
View File
@@ -65,3 +65,22 @@ install-coz:
coz target iters="200":
sh scripts/coz-profile.sh {{target}} {{iters}}
python3 scripts/coz-summary.py results/{{target}}/coz/profile.coz
# Service mode (Phase 4): interleaved A/B latency bench of two server
# binaries under coordinated-omission-correct open-loop load. p50/p99 CIs.
service baseline candidate doc *flags:
cargo run -p bench-runner -- --config config/accept.toml service \
--baseline-bin "{{baseline}}" --candidate-bin "{{candidate}}" \
--doc "{{doc}}" --pin "taskset -c 2" {{flags}}
# Service-mode calibration: A/A false-positive + injected latency-regression
# detection (SPEC §3.1); writes JSON evidence.
service-calibrate server doc out *flags:
cargo run -p bench-runner -- --config config/accept.toml service-calibrate \
--server-bin "{{server}}" --doc "{{doc}}" --pin "taskset -c 2" \
--out "{{out}}" {{flags}}
# Mechanical ROI report from a ledger row (SPEC §9): throughput→cores→
# dollars and/or latency percentiles, CIs + methodology printed inline.
report run_id *flags:
cargo run -p report -- --run-id "{{run_id}}" {{flags}}