Files
peltier.io/docs/src/architecture.md
T
benbrownandClaude Fable 5 444e5294fa explain: post-verdict diagnostics — SPEC §3.7, crate, docs
Verdicts say what; nothing said why. A rejection that reads only
rejected-bench teaches nothing, and the loop's compounding value is that
the next hypothesis is better than the last. This ships the teaching layer,
built so it can never contaminate the thing it explains.

New crate crates/explain (`just explain <run-id>`):

- Advisory, strictly off the accept path: runs after the verdict is
  recorded; nothing it emits feeds gates, bench, verdict, or ledger. Not
  agent-facing: harnessd stays at seven ops, by design (SPEC §3.7 says why
  - better hypotheses must never become a feedback channel into
  measurement).
- Ledger-only inputs, deterministic: consumes exactly one row via the new
  ledger::attempt_row; no config, no clock, no re-benching. Identical row,
  byte-identical text - proven live (cmp on two runs of a real row).
- Record vs inference is typographic: restatements print plain; anything
  beyond is prefixed `inference:` so an explanation can never be quoted as
  a measurement.
- The narrative must agree with the machine verdict. The first live run
  caught this class of bug: an accepted row without a recorded threshold
  was narrated as "a policy rejection" two lines under "verdict: accepted".
  Accepted rows are now always described by their CI lower bound (the
  defensible claim); rejected-bench rows classify against 1.0 (measured
  regression / null result / real-but-below-bar); NHR rows name the
  recorded cap and state the numbers were not the deciding factor;
  rejected-gate rows name the failing gate. Regression-tested.
- Historical rows are flagged, not laundered: accepted with
  sanitizers_clean=false prints the doctrine caution (comrak-010 shows it);
  fuzz_iters=0 accepts are named pre-integration rows.

verdict now records accept_threshold (the bar in force for that run) into
the bench env fingerprint, so rows are self-contained going forward -
explain never substitutes today's config for a historical bar, and says so
on older rows instead.

SPEC §3.7 (Added 2026-07-20) carries the invariants; book chapter
docs/src/explain.md shows real output from real rows (the accepted cJSON
win, the caught false-accept, the kernel-lane NHR); CLAUDE.md, README,
justfile, and the architecture layout updated (the layout also gains the
previously-omitted crates/policy line).

Verified: 21 workspace test suites green (6 new explain tests incl.
determinism and verdict-narrative agreement), clippy -D warnings clean,
fmt clean, mdbook builds, live runs against phase3-cjson-002,
phase5-matmul-opt, phase2-comrak-010, and phase4-cjson-service.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-20 09:53:14 -04:00

3.4 KiB
Raw Blame History

Architecture

Peltier is split into a trust layer (Rust, the agent cannot write it) and an untrusted proposer (the agent). The split is structural, not advisory: the guarantees hold even if the proposer is adversarial.

crates/            Trust layer — agent has NO write access (SPEC §10)
  bench-runner/    Interleaved A/B timing, bootstrap CIs, A/A calibration,
                   service-mode latency (coordinated-omission correct)
  diff-test/       Equivalence gates, corpus hash-pinning, per-target spec
  ledger/          Append-only SQLite attempt ledger (enforced by triggers)
  report/          ROI: speedup CI → cores → dollars, methodology inline
  explain/         Advisory post-verdict diagnosis of a ledger row (why it
                   won or lost) — never on the accept path
  policy/          Learned class-selection prior from the ledger (advisory)
  verdict/         The pipeline in one command: gates → bench → ledger row
  harnessd/        The one door the agent talks through (7 JSON ops)
agent/             Untrusted proposer (Claude Agent SDK, Python)
playbook/          Optimization classes 1–7, tried strictly cheapest-first
config/            accept.toml (thresholds), pricing.toml (ROI inputs)
targets/           Vendored OSS targets — the only agent-writable path
corpora/           Hash-pinned golden-replay inputs (read-only to agent)
results/           Calibration evidence, case studies, generated reports
.claude/skills/    The trust layer as a consumable Agent Skill — mirrored
                   byte-identically at .agents/skills/ (CI-enforced); see
                   "Consume it: the peltier skill"

The seven-tool boundary

The agent never runs a shell, never writes a file, never touches the ledger directly. It speaks to harnessd — the only door — through exactly seven JSON operations:

Tool What it can do
read_profile Read a target's ranked hotspots (read-only)
read_ledger Read prior attempts — the anti-double-attempt memory
read_playbook Read an optimization class's preconditions/procedure
read_target_source Read a workspace file at the pinned commit (windowed)
propose_patch Submit a unified diff — path-allowlisted, then git apply
run_verdict Launch the gate+bench pipeline (detached)
read_verdict Poll the append-only ledger for the result

Note what is absent: no write outside targets/<name>/workspace, no shell, no way to run the bench or write the ledger except through the verdict binary, which only records a pass after the full gated pipeline.

The data flow of one attempt

profile ─▶ agent hypothesizes ─▶ propose_patch ─▶ [path allowlist + git apply]
                                                          │
   ledger row ◀─ verdict ◀─ bench (interleaved A/B) ◀─ gates (tests, golden,
   (append-only)             vs pristine-rebuilt         fuzz, sanitizers)
                             baseline

The baseline is rebuilt from a pristine checkout every session — never from the agent's workspace — so a patch cannot poison its own comparison. The verdict is decided by the CI-lower-bound rule and written to an append-only ledger the agent cannot mutate.

The next chapters walk each stage: measurement, equivalence gates, the agent loop, and the isolation that makes the boundary load-bearing.