benbrown/peltier.io
2
1
Fork 0
mirror of https://github.com/bdbrown4/peltier.io.git synced 2026-11-07 10:58:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

phase4: COMPLETE — service latency win + mechanical ROI report; roadmap done

Browse Source 
phase4-cjson-service (ledger): the accepted batch win phase3-cjson-002
measured under a 150 rps coordinated-omission-correct replay (20 rounds,
40k requests, 0 drops). p50 latency +6.2%, 95% CI [+5.8%, +7.2%] —
accepted; p99 CI [0.07, 4.97] too noisy (single-worker loopback tail),
harness correctly rejected. ROI (500-core fleet, $0.04/core-hr): 27.5
cores / $9,621/year at the CI lower bound, report generated mechanically
(results/phase4/cjson-service-roi.md).

- results/phase4/: case study + README + the generated ROI report.
- CLAUDE.md: Phase 4 complete; Phases 0-4 all done.

All SPEC §5 exit criteria met across every phase; zero shipped false
accepts across the whole project.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
This commit is contained in:
Claude
2026-07-11 23:48:22 +00:00
parent 4cf729e8c1
commit 0ec111663b
4 changed files with 132 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
results/phase4/cjson-service-roi.md
+50
View File
@@ -0,0 +1,50 @@
# ROI report — `phase4-cjson-service`
**Target:** cjson · **Playbook class:** 5 · **Verdict:** `accepted`
**Workload:** corpora/cjson/input/service.json, 1 parse+print/req, 150 rps open-loop, CO-correct
## Throughput → cores → dollars
Measured speedup (baseline/candidate): **1.0624**, 95% bootstrap CI **[1.0581, 1.0725]**.
On a **500-core** fleet running this workload:
| metric | median | 95% CI |
|---|---|---|
| cores returned | 29.4 | [27.5, 33.8] |
| annualized saving | $10290 | [$9621, $11838] |
Pricing: **$0.04/core-hour**, **8760 h/year**. Source: _example: AWS m7i on-demand us-east-1, per-vCPU approximation — replace per engagement_
The **CI lower bound is the number to quote** — 27.5 cores / $9621 per year is
the saving that survives the 95% interval; the point estimate is not a promise.
## Latency under replayed load
Workload: corpora/cjson/input/service.json, 1 parse+print/req, 150 rps open-loop, CO-correct. Coordinated-omission-correct open-loop replay.
| percentile | baseline | candidate | speedup | 95% CI |
|---|---|---|---|---|
| p50 | 3.332 ms | 3.136 ms | 1.0624 | [1.0581, 1.0725] |
| p99 | 24.726 ms | 23.947 ms | 1.0325 | [0.0715, 4.9714] |
## Methodology (ships with the number)
- **Interleaved A/B**, baseline rebuilt from a pristine checkout, never the agent's
workspace. Candidate and baseline alternate to control thermal/background drift.
- **Bootstrap 95% CI** of the ratio-of-medians; an effect is accepted only if the CI
lower bound clears the threshold in `config/accept.toml` (2%).
- **Equivalence gates** passed before any number was trusted: upstream test suite,
byte-identical golden replay, differential fuzzing, and ASan/UBSan sanitizers.
- **Calibrated hardware**: the workload passed A/A (false-positive <5%) and injected-
regression (≥95% detection) self-tests, recorded in `results/calibration/`.
- **Coordinated-omission correct** latency (service mode): each request's latency is
measured from its intended send time, so queueing is counted, not hidden.
## Caveats
Every figure is specific to the workload and hardware named above. Throughput→cores
assumes the fleet is CPU-bound on this path and scales linearly; latency figures are
for the stated arrival rate. The saving to commit to is the **CI lower bound**, not
the median.