mirror of
https://github.com/bdbrown4/peltier.io.git
synced 2026-11-07 10:58:28 +00:00
agent: unattended Phase 2 loop — profile to gated verdict, zero false accepts
The Claude Agent SDK loop (agent/hotpath_agent/loop.py, model claude-fable-5) drives the seven harness tools over a standalone stdio MCP server (mcp_server.py) and runs the full profile -> hypothesize -> patch -> gated-verdict cycle on comrak with no human in the loop. Three audited attempts, all honest rejections, ZERO false accepts: - phase2-comrak-001 class 3: syntect buffer with_capacity, [0.971,1.007] - phase2-comrak-002 class 5: gate dead VecDeque build, [0.982,1.026] - phase2-comrak-003 class 6: hoist find_special_char check, [0.970,1.007] The agent picked the cheapest untried class each time (3->5->6) and TWICE refused to fabricate a verdict it couldn't read back, verifying ledger integrity instead — the anti-reward-hacking behavior SPEC §10 targets. Harness changes surfaced by the loop: - run_verdict is now async: launches the multi-minute build+bench detached (past the 60s MCP transport cap) and returns immediately; new read_verdict op polls the append-only ledger row (ledger verdict_summary). Fixes attempt 001's unreadable-verdict gap. - read_target_source takes offset/limit line windows so 100KB+ files page instead of overflowing — removes the agent's reason to reach for a shell (attempt 003 paged a 2400-line file cleanly, no shell). Honest boundary note (case study + code comments): in THIS dev container the SDK tool allow/deny layer is defense-in-depth, not the trust boundary — a nested CLI as root under a parent session doesn't reliably gate built-ins. SPEC §10's boundary is OS process/user isolation; that hardening is the remaining Phase 2 exit item (docs/infra.md). What holds regardless: propose_patch is the only mediated write path (path-allowlist + git apply) and the ledger is append-only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
This commit is contained in:
@@ -165,6 +165,32 @@ impl Ledger {
|
||||
Ok(rows.collect::<Result<Vec<_>, _>>()?)
|
||||
}
|
||||
|
||||
/// Read-only verdict summary for one run — the agent's only window
|
||||
/// onto a completed attempt (harnessd `read_verdict`). Returns the
|
||||
/// verdict plus the bench CIs; never enough to game, since the row
|
||||
/// is already written and immutable.
|
||||
pub fn verdict_summary(&self, run_id: &str) -> Result<Option<serde_json::Value>, LedgerError> {
|
||||
let row = self.conn.query_row(
|
||||
"SELECT verdict, bench FROM attempts WHERE run_id = ?1",
|
||||
[run_id],
|
||||
|r| Ok((r.get::<_, String>(0)?, r.get::<_, Option<String>>(1)?)),
|
||||
);
|
||||
match row {
|
||||
Ok((verdict, bench)) => {
|
||||
let bench: Option<serde_json::Value> =
|
||||
bench.map(|b| serde_json::from_str(&b)).transpose()?;
|
||||
Ok(Some(serde_json::json!({
|
||||
"run_id": run_id,
|
||||
"verdict": verdict,
|
||||
"speedup_median": bench.as_ref().and_then(|b| b.get("speedup_median").cloned()),
|
||||
"speedup_ci": bench.as_ref().and_then(|b| b.get("speedup_ci").cloned()),
|
||||
})))
|
||||
}
|
||||
Err(rusqlite::Error::QueryReturnedNoRows) => Ok(None),
|
||||
Err(e) => Err(e.into()),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn count(&self) -> Result<u64, LedgerError> {
|
||||
Ok(self
|
||||
.conn
|
||||
|
||||
Reference in New Issue
Block a user