Merge pull request #24 from bdbrown4/docs/detective-court

docs: name the statistical identity — Bayesian detective, frequentist court
This commit is contained in:
2026-07-20 13:04:56 -04:00
committed by GitHub
4 changed files with 71 additions and 0 deletions
+2
View File
@@ -6,6 +6,8 @@
An agent that profiles real binaries and services, isolates hot paths, proposes optimizations, and accepts a change **only** when (a) behavioral-equivalence gates pass and (b) the benchmark improvement is statistically significant on trusted infrastructure. Output: verified performance deltas translated into ROI reports nobody can dispute.
Mental model: **the optimizer is a Bayesian detective; the trust layer is a frequentist court.** The detective (agent + playbook prior + policy ranking + ledger evidence + explain feedback) chooses what to try; the court (gates + calibrated bench + CI-lower-bound rule) decides what is true, with measured error rates and no priors on the accept path. Evidence flows back; authority never does. Priors steer where we look — they never touch what we conclude.
Full engineering spec: `SPEC.md`. Read it before starting any phase.
## Non-negotiables (re-read every session)
+8
View File
@@ -15,6 +15,14 @@ deltas with the methodology attached** — a stopwatch on pinned hardware, a
proof of equivalence, and an ROI figure whose confidence interval survives
hostile review.
The design in one breath: **the optimizer is a Bayesian detective; the
trust layer is a frequentist court.** The detective holds priors, updates
them from the ledger, and picks the next bet; the court runs calibrated
procedures with measured error rates and no priors at all. Evidence flows
back from court to detective — authority never does. Priors steer where we
look; they never touch what we conclude.
[The full argument](https://bdbrown4.github.io/peltier.io/thesis.html#the-detective-and-the-court).
📖 **Full documentation: the [peltier.io book](https://bdbrown4.github.io/peltier.io/)**
· Design spec: [SPEC.md](SPEC.md) · Charter: [CLAUDE.md](CLAUDE.md)
+2
View File
@@ -9,6 +9,8 @@ Why this wins:
- **The metric defends itself.** Same inputs, same outputs, fewer cycles, with confidence intervals. No correlation-causation problem is possible by construction.
- **The ledger compounds.** Every accepted/rejected attempt with full evidence is future RL training data and, eventually, the substrate for learned optimization policies (see §13, research fork).
> **Added 2026-07-20 — the statistical identity, named.** The optimizer is a **Bayesian detective**; the trust layer is a **frequentist court**. The detective (agent loop, playbook prior, §13 policy ranking, ledger evidence, §3.7 explain feedback) decides *what to try next* by belief-updating over accumulated evidence. The court (§3.1–§3.3: gates, calibrated bench, CI-lower-bound accept rule) decides *what is true* via procedures with empirically measured error rates — no prior appears anywhere on the accept path. Evidence flows from court to detective (ledger rows, explanations); authority never flows the other way. Invariant: **priors steer where we look; they never touch what we conclude.** A wrong prior costs compute; a prior in the verdict would cost trust.
## 2. System overview
```
+59
View File
@@ -46,3 +46,62 @@ cannot touch the thresholds. It can only propose a patch and ask the
harness to judge it. The [architecture](./architecture.md) is designed so
that even a compromised or adversarial proposer cannot manufacture a false
win.
## The detective and the court
> The optimizer is a **Bayesian detective**. The trust layer is a
> **frequentist court**.
That sentence is the whole design. The two halves run on opposite
statistical philosophies, deliberately:
```
BAYESIAN DETECTIVE FREQUENTIST COURT
the optimizer: agent loop, the trust layer: gates,
playbook prior, policy ranking calibrated bench, verdict
┌───────────────────────────┐ ┌────────────────────────────┐
│ generate hypotheses │ │ prove equivalence first │
│ prioritize the docket │ proposes │ calibrate the instrument │
│ estimate likely payoff │ ─────────▶ │ measure, interleaved A/B │
└───────────────────────────┘ │ reject noise (CI lower │
▲ │ bound vs. threshold) │
│ └─────────────┬──────────────┘
│ ▼
│ evidence flows back verdict
└───── ledger row + explain ◀──────────────┘
```
The **detective** reasons like a Bayesian: it holds a prior over where wins
live (the [playbook's](./playbook.md) cheapest-first ordering), updates it
from accumulated evidence (the append-only ledger — failures included,
because failures are the most informative updates), ranks its next bets
(`crates/policy`, by the Wilson *lower* bound of each class's proven win
rate — even the learning layer bets pessimistically), and reads the
[post-verdict diagnosis](./explain.md) so the next hypothesis is sharper
than the last. Its "estimate likely payoff" is a belief, and stays one:
dollar figures are minted only by the court's
[mechanical report](./roi.md), only from accepted rows.
The **court** reasons like a frequentist, because its audience is a hostile
reviewer who shares none of your priors. Its verdicts are decision
procedures with *measured* long-run error rates: the A/A calibration bounds
the false-positive rate and the injected-regression test bounds the power —
per machine, empirically, before any real measurement is believed. The
accept rule is a conservative bound, not a posterior; there is no prior
anywhere on the accept path, so there is nothing to argue about except the
evidence.
The return edge is what makes the detective Bayesian at all — and note its
direction. **Evidence flows back; authority never does.** The invariant
underneath the whole system:
> **Priors may steer where you look. They may never touch what you
> conclude.**
A wrong prior in the detective costs compute — a few wasted attempts, each
honestly ledgered. A prior in the court could cost a false claim, the one
unrecoverable failure for a project whose product is trust. The gates are
what make speculative, prior-driven exploration *safe*: the detective can
believe anything it likes, because nothing it believes survives into a
number.