phase5: learned class-selection policy (SPEC §13) — rank classes by ledger evidence

crates/policy: reads the append-only ledger and ranks playbook classes by
the Wilson score lower bound of their machine-verified win rate — the same
'quote the lower bound' discipline the rest of the project uses. A class
that won 1/1 does not outrank one that won 8/10 (wider interval); untried
classes fall back to the cheapest-first prior; tried-and-failed classes
rank last. Advisory only — the gates still decide every verdict.

On the current 33-row ledger it ranks class 5 (algorithmic) first, then
class 3, then class 1 — the proven winners — with class 7 (untried) above
the evidence-against classes. Wins count only machine-sanitizer-verified
accepts (conservatively excluding pre-gate accepts and the overturned
comrak-010): the policy trusts the ledger's machine record, not narrative.
4 unit tests (Wilson monotonicity, small-sample penalty, ranking order).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
This commit is contained in:
Claude
2026-07-12 00:24:00 +00:00
parent 5d6c259b30
commit b7297ad31f
7 changed files with 314 additions and 0 deletions
+22
View File
@@ -190,6 +190,28 @@ impl Ledger {
/// onto a completed attempt (harnessd `read_verdict`). Returns the
/// verdict plus the bench CIs; never enough to game, since the row
/// is already written and immutable.
/// Every attempt's (playbook_class, target, verdict, sanitizers_clean)
/// — the training signal for the learned class-selection policy
/// (SPEC §13). A "shippable win" is `accepted` AND sanitizers-clean,
/// matching the report generator's definition.
pub fn all_outcomes(&self) -> Result<Vec<(u8, String, String, bool)>, LedgerError> {
let mut stmt = self
.conn
.prepare("SELECT playbook_class, target, verdict, gates FROM attempts")?;
let rows = stmt.query_map([], |r| {
let class: u8 = r.get(0)?;
let target: String = r.get(1)?;
let verdict: String = r.get(2)?;
let gates: Option<String> = r.get(3)?;
let sanitizers_clean = gates
.and_then(|g| serde_json::from_str::<serde_json::Value>(&g).ok())
.and_then(|g| g.get("sanitizers_clean").and_then(|v| v.as_bool()))
.unwrap_or(false);
Ok((class, target, verdict, sanitizers_clean))
})?;
Ok(rows.collect::<Result<Vec<_>, _>>()?)
}
/// Full row data the ROI report generator needs (SPEC §9). Includes
/// the workload string pulled out of the bench evidence.
pub fn report_row(&self, run_id: &str) -> Result<Option<serde_json::Value>, LedgerError> {