diff --git a/CLAUDE.md b/CLAUDE.md index 29241ea..e88354c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,6 +6,8 @@ An agent that profiles real binaries and services, isolates hot paths, proposes optimizations, and accepts a change **only** when (a) behavioral-equivalence gates pass and (b) the benchmark improvement is statistically significant on trusted infrastructure. Output: verified performance deltas translated into ROI reports nobody can dispute. +Mental model: **the optimizer is a Bayesian detective; the trust layer is a frequentist court.** The detective (agent + playbook prior + policy ranking + ledger evidence + explain feedback) chooses what to try; the court (gates + calibrated bench + CI-lower-bound rule) decides what is true, with measured error rates and no priors on the accept path. Evidence flows back; authority never does. Priors steer where we look โ€” they never touch what we conclude. + Full engineering spec: `SPEC.md`. Read it before starting any phase. ## Non-negotiables (re-read every session) diff --git a/README.md b/README.md index 162e3b5..09b0ef6 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,14 @@ deltas with the methodology attached** โ€” a stopwatch on pinned hardware, a proof of equivalence, and an ROI figure whose confidence interval survives hostile review. +The design in one breath: **the optimizer is a Bayesian detective; the +trust layer is a frequentist court.** The detective holds priors, updates +them from the ledger, and picks the next bet; the court runs calibrated +procedures with measured error rates and no priors at all. Evidence flows +back from court to detective โ€” authority never does. Priors steer where we +look; they never touch what we conclude. +[The full argument](https://bdbrown4.github.io/peltier.io/thesis.html#the-detective-and-the-court). + ๐Ÿ“– **Full documentation: the [peltier.io book](https://bdbrown4.github.io/peltier.io/)** ยท Design spec: [SPEC.md](SPEC.md) ยท Charter: [CLAUDE.md](CLAUDE.md) diff --git a/SPEC.md b/SPEC.md index 4a03d7b..b7275bb 100644 --- a/SPEC.md +++ b/SPEC.md @@ -9,6 +9,8 @@ Why this wins: - **The metric defends itself.** Same inputs, same outputs, fewer cycles, with confidence intervals. No correlation-causation problem is possible by construction. - **The ledger compounds.** Every accepted/rejected attempt with full evidence is future RL training data and, eventually, the substrate for learned optimization policies (see ยง13, research fork). +> **Added 2026-07-20 โ€” the statistical identity, named.** The optimizer is a **Bayesian detective**; the trust layer is a **frequentist court**. The detective (agent loop, playbook prior, ยง13 policy ranking, ledger evidence, ยง3.7 explain feedback) decides *what to try next* by belief-updating over accumulated evidence. The court (ยง3.1โ€“ยง3.3: gates, calibrated bench, CI-lower-bound accept rule) decides *what is true* via procedures with empirically measured error rates โ€” no prior appears anywhere on the accept path. Evidence flows from court to detective (ledger rows, explanations); authority never flows the other way. Invariant: **priors steer where we look; they never touch what we conclude.** A wrong prior costs compute; a prior in the verdict would cost trust. + ## 2. System overview ``` diff --git a/docs/src/thesis.md b/docs/src/thesis.md index 923d9b9..a323d32 100644 --- a/docs/src/thesis.md +++ b/docs/src/thesis.md @@ -46,3 +46,62 @@ cannot touch the thresholds. It can only propose a patch and ask the harness to judge it. The [architecture](./architecture.md) is designed so that even a compromised or adversarial proposer cannot manufacture a false win. + +## The detective and the court + +> The optimizer is a **Bayesian detective**. The trust layer is a +> **frequentist court**. + +That sentence is the whole design. The two halves run on opposite +statistical philosophies, deliberately: + +``` + BAYESIAN DETECTIVE FREQUENTIST COURT + the optimizer: agent loop, the trust layer: gates, + playbook prior, policy ranking calibrated bench, verdict + + โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” + โ”‚ generate hypotheses โ”‚ โ”‚ prove equivalence first โ”‚ + โ”‚ prioritize the docket โ”‚ proposes โ”‚ calibrate the instrument โ”‚ + โ”‚ estimate likely payoff โ”‚ โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ–ถ โ”‚ measure, interleaved A/B โ”‚ + โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ reject noise (CI lower โ”‚ + โ–ฒ โ”‚ bound vs. threshold) โ”‚ + โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ฌโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ + โ”‚ โ–ผ + โ”‚ evidence flows back verdict + โ””โ”€โ”€โ”€โ”€โ”€ ledger row + explain โ—€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +The **detective** reasons like a Bayesian: it holds a prior over where wins +live (the [playbook's](./playbook.md) cheapest-first ordering), updates it +from accumulated evidence (the append-only ledger โ€” failures included, +because failures are the most informative updates), ranks its next bets +(`crates/policy`, by the Wilson *lower* bound of each class's proven win +rate โ€” even the learning layer bets pessimistically), and reads the +[post-verdict diagnosis](./explain.md) so the next hypothesis is sharper +than the last. Its "estimate likely payoff" is a belief, and stays one: +dollar figures are minted only by the court's +[mechanical report](./roi.md), only from accepted rows. + +The **court** reasons like a frequentist, because its audience is a hostile +reviewer who shares none of your priors. Its verdicts are decision +procedures with *measured* long-run error rates: the A/A calibration bounds +the false-positive rate and the injected-regression test bounds the power โ€” +per machine, empirically, before any real measurement is believed. The +accept rule is a conservative bound, not a posterior; there is no prior +anywhere on the accept path, so there is nothing to argue about except the +evidence. + +The return edge is what makes the detective Bayesian at all โ€” and note its +direction. **Evidence flows back; authority never does.** The invariant +underneath the whole system: + +> **Priors may steer where you look. They may never touch what you +> conclude.** + +A wrong prior in the detective costs compute โ€” a few wasted attempts, each +honestly ledgered. A prior in the court could cost a false claim, the one +unrecoverable failure for a project whose product is trust. The gates are +what make speculative, prior-driven exploration *safe*: the detective can +believe anything it likes, because nothing it believes survives into a +number.