mirror of
https://github.com/bdbrown4/peltier.io.git
synced 2026-11-07 10:58:28 +00:00
docs: name the statistical identity — Bayesian detective, frequentist court
The two halves of the system run on opposite statistical philosophies, on purpose, and until now the docs never said so. The optimizer is a Bayesian detective: playbook prior, ledger evidence, policy ranking, explain feedback — belief-updating that decides what to try next. The trust layer is a frequentist court: equivalence gates, per-machine calibrated error rates (A/A false-positive bound, injected-regression power bound), and a CI-lower-bound accept rule with no prior anywhere on the accept path — because its audience is a hostile reviewer who shares none of your priors. The load-bearing edge is the return path: evidence flows from court to detective (ledger rows + explain); authority never flows the other way. Invariant, now stated in four places: priors steer where we look; they never touch what we conclude. A wrong prior costs compute — each wasted attempt honestly ledgered. A prior in the verdict would cost a false claim, the one unrecoverable failure for a project whose product is trust. - thesis.md: full section with the two-box diagram incl. the return edge, in docket order (equivalence before measurement); notes that estimated payoff is a belief and dollar figures are minted only by the mechanical report from accepted rows, and that even the learning layer bets on Wilson LOWER bounds — the detective is a pessimist too. - README: the one-breath version, linking the full argument. - SPEC §1: dated note mapping each half to its sections. - CLAUDE.md: the mental model, one paragraph under Mission. Framing credit: emerged from a design conversation on whether peltier is "Bayesian engineering" — it is exactly half of one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -46,3 +46,62 @@ cannot touch the thresholds. It can only propose a patch and ask the
|
||||
harness to judge it. The [architecture](./architecture.md) is designed so
|
||||
that even a compromised or adversarial proposer cannot manufacture a false
|
||||
win.
|
||||
|
||||
## The detective and the court
|
||||
|
||||
> The optimizer is a **Bayesian detective**. The trust layer is a
|
||||
> **frequentist court**.
|
||||
|
||||
That sentence is the whole design. The two halves run on opposite
|
||||
statistical philosophies, deliberately:
|
||||
|
||||
```
|
||||
BAYESIAN DETECTIVE FREQUENTIST COURT
|
||||
the optimizer: agent loop, the trust layer: gates,
|
||||
playbook prior, policy ranking calibrated bench, verdict
|
||||
|
||||
┌───────────────────────────┐ ┌────────────────────────────┐
|
||||
│ generate hypotheses │ │ prove equivalence first │
|
||||
│ prioritize the docket │ proposes │ calibrate the instrument │
|
||||
│ estimate likely payoff │ ─────────▶ │ measure, interleaved A/B │
|
||||
└───────────────────────────┘ │ reject noise (CI lower │
|
||||
▲ │ bound vs. threshold) │
|
||||
│ └─────────────┬──────────────┘
|
||||
│ ▼
|
||||
│ evidence flows back verdict
|
||||
└───── ledger row + explain ◀──────────────┘
|
||||
```
|
||||
|
||||
The **detective** reasons like a Bayesian: it holds a prior over where wins
|
||||
live (the [playbook's](./playbook.md) cheapest-first ordering), updates it
|
||||
from accumulated evidence (the append-only ledger — failures included,
|
||||
because failures are the most informative updates), ranks its next bets
|
||||
(`crates/policy`, by the Wilson *lower* bound of each class's proven win
|
||||
rate — even the learning layer bets pessimistically), and reads the
|
||||
[post-verdict diagnosis](./explain.md) so the next hypothesis is sharper
|
||||
than the last. Its "estimate likely payoff" is a belief, and stays one:
|
||||
dollar figures are minted only by the court's
|
||||
[mechanical report](./roi.md), only from accepted rows.
|
||||
|
||||
The **court** reasons like a frequentist, because its audience is a hostile
|
||||
reviewer who shares none of your priors. Its verdicts are decision
|
||||
procedures with *measured* long-run error rates: the A/A calibration bounds
|
||||
the false-positive rate and the injected-regression test bounds the power —
|
||||
per machine, empirically, before any real measurement is believed. The
|
||||
accept rule is a conservative bound, not a posterior; there is no prior
|
||||
anywhere on the accept path, so there is nothing to argue about except the
|
||||
evidence.
|
||||
|
||||
The return edge is what makes the detective Bayesian at all — and note its
|
||||
direction. **Evidence flows back; authority never does.** The invariant
|
||||
underneath the whole system:
|
||||
|
||||
> **Priors may steer where you look. They may never touch what you
|
||||
> conclude.**
|
||||
|
||||
A wrong prior in the detective costs compute — a few wasted attempts, each
|
||||
honestly ledgered. A prior in the court could cost a false claim, the one
|
||||
unrecoverable failure for a project whose product is trust. The gates are
|
||||
what make speculative, prior-driven exploration *safe*: the detective can
|
||||
believe anything it likes, because nothing it believes survives into a
|
||||
number.
|
||||
|
||||
Reference in New Issue
Block a user