docs: name the statistical identity — Bayesian detective, frequentist court

The two halves of the system run on opposite statistical philosophies, on
purpose, and until now the docs never said so. The optimizer is a Bayesian
detective: playbook prior, ledger evidence, policy ranking, explain
feedback — belief-updating that decides what to try next. The trust layer
is a frequentist court: equivalence gates, per-machine calibrated error
rates (A/A false-positive bound, injected-regression power bound), and a
CI-lower-bound accept rule with no prior anywhere on the accept path —
because its audience is a hostile reviewer who shares none of your priors.

The load-bearing edge is the return path: evidence flows from court to
detective (ledger rows + explain); authority never flows the other way.
Invariant, now stated in four places: priors steer where we look; they
never touch what we conclude. A wrong prior costs compute — each wasted
attempt honestly ledgered. A prior in the verdict would cost a false
claim, the one unrecoverable failure for a project whose product is trust.

- thesis.md: full section with the two-box diagram incl. the return edge,
  in docket order (equivalence before measurement); notes that estimated
  payoff is a belief and dollar figures are minted only by the mechanical
  report from accepted rows, and that even the learning layer bets on
  Wilson LOWER bounds — the detective is a pessimist too.
- README: the one-breath version, linking the full argument.
- SPEC §1: dated note mapping each half to its sections.
- CLAUDE.md: the mental model, one paragraph under Mission.

Framing credit: emerged from a design conversation on whether peltier is
"Bayesian engineering" — it is exactly half of one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-20 12:52:44 -04:00
co-authored by Claude Fable 5
parent ffc6674e13
commit 24127bbcb4
4 changed files with 71 additions and 0 deletions
+2
View File
@@ -6,6 +6,8 @@
An agent that profiles real binaries and services, isolates hot paths, proposes optimizations, and accepts a change **only** when (a) behavioral-equivalence gates pass and (b) the benchmark improvement is statistically significant on trusted infrastructure. Output: verified performance deltas translated into ROI reports nobody can dispute.
Mental model: **the optimizer is a Bayesian detective; the trust layer is a frequentist court.** The detective (agent + playbook prior + policy ranking + ledger evidence + explain feedback) chooses what to try; the court (gates + calibrated bench + CI-lower-bound rule) decides what is true, with measured error rates and no priors on the accept path. Evidence flows back; authority never does. Priors steer where we look — they never touch what we conclude.
Full engineering spec: `SPEC.md`. Read it before starting any phase.
## Non-negotiables (re-read every session)
+8
View File
@@ -15,6 +15,14 @@ deltas with the methodology attached** — a stopwatch on pinned hardware, a
proof of equivalence, and an ROI figure whose confidence interval survives
hostile review.
The design in one breath: **the optimizer is a Bayesian detective; the
trust layer is a frequentist court.** The detective holds priors, updates
them from the ledger, and picks the next bet; the court runs calibrated
procedures with measured error rates and no priors at all. Evidence flows
back from court to detective — authority never does. Priors steer where we
look; they never touch what we conclude.
[The full argument](https://bdbrown4.github.io/peltier.io/thesis.html#the-detective-and-the-court).
📖 **Full documentation: the [peltier.io book](https://bdbrown4.github.io/peltier.io/)**
· Design spec: [SPEC.md](SPEC.md) · Charter: [CLAUDE.md](CLAUDE.md)
+2
View File
@@ -9,6 +9,8 @@ Why this wins:
- **The metric defends itself.** Same inputs, same outputs, fewer cycles, with confidence intervals. No correlation-causation problem is possible by construction.
- **The ledger compounds.** Every accepted/rejected attempt with full evidence is future RL training data and, eventually, the substrate for learned optimization policies (see §13, research fork).
> **Added 2026-07-20 — the statistical identity, named.** The optimizer is a **Bayesian detective**; the trust layer is a **frequentist court**. The detective (agent loop, playbook prior, §13 policy ranking, ledger evidence, §3.7 explain feedback) decides *what to try next* by belief-updating over accumulated evidence. The court (§3.1–§3.3: gates, calibrated bench, CI-lower-bound accept rule) decides *what is true* via procedures with empirically measured error rates — no prior appears anywhere on the accept path. Evidence flows from court to detective (ledger rows, explanations); authority never flows the other way. Invariant: **priors steer where we look; they never touch what we conclude.** A wrong prior costs compute; a prior in the verdict would cost trust.
## 2. System overview
```
+59
View File
@@ -46,3 +46,62 @@ cannot touch the thresholds. It can only propose a patch and ask the
harness to judge it. The [architecture](./architecture.md) is designed so
that even a compromised or adversarial proposer cannot manufacture a false
win.
## The detective and the court
> The optimizer is a **Bayesian detective**. The trust layer is a
> **frequentist court**.
That sentence is the whole design. The two halves run on opposite
statistical philosophies, deliberately:
```
BAYESIAN DETECTIVE FREQUENTIST COURT
the optimizer: agent loop, the trust layer: gates,
playbook prior, policy ranking calibrated bench, verdict
┌───────────────────────────┐ ┌────────────────────────────┐
│ generate hypotheses │ │ prove equivalence first │
│ prioritize the docket │ proposes │ calibrate the instrument │
│ estimate likely payoff │ ─────────▶ │ measure, interleaved A/B │
└───────────────────────────┘ │ reject noise (CI lower │
▲ │ bound vs. threshold) │
│ └─────────────┬──────────────┘
│ ▼
│ evidence flows back verdict
└───── ledger row + explain ◀──────────────┘
```
The **detective** reasons like a Bayesian: it holds a prior over where wins
live (the [playbook's](./playbook.md) cheapest-first ordering), updates it
from accumulated evidence (the append-only ledger — failures included,
because failures are the most informative updates), ranks its next bets
(`crates/policy`, by the Wilson *lower* bound of each class's proven win
rate — even the learning layer bets pessimistically), and reads the
[post-verdict diagnosis](./explain.md) so the next hypothesis is sharper
than the last. Its "estimate likely payoff" is a belief, and stays one:
dollar figures are minted only by the court's
[mechanical report](./roi.md), only from accepted rows.
The **court** reasons like a frequentist, because its audience is a hostile
reviewer who shares none of your priors. Its verdicts are decision
procedures with *measured* long-run error rates: the A/A calibration bounds
the false-positive rate and the injected-regression test bounds the power —
per machine, empirically, before any real measurement is believed. The
accept rule is a conservative bound, not a posterior; there is no prior
anywhere on the accept path, so there is nothing to argue about except the
evidence.
The return edge is what makes the detective Bayesian at all — and note its
direction. **Evidence flows back; authority never does.** The invariant
underneath the whole system:
> **Priors may steer where you look. They may never touch what you
> conclude.**
A wrong prior in the detective costs compute — a few wasted attempts, each
honestly ledgered. A prior in the court could cost a false claim, the one
unrecoverable failure for a project whose product is trust. The gates are
what make speculative, prior-driven exploration *safe*: the detective can
believe anything it likes, because nothing it believes survives into a
number.