agent: unattended Phase 2 loop — profile to gated verdict, zero false accepts

The Claude Agent SDK loop (agent/hotpath_agent/loop.py, model
claude-fable-5) drives the seven harness tools over a standalone stdio
MCP server (mcp_server.py) and runs the full profile -> hypothesize ->
patch -> gated-verdict cycle on comrak with no human in the loop.

Three audited attempts, all honest rejections, ZERO false accepts:
- phase2-comrak-001 class 3: syntect buffer with_capacity, [0.971,1.007]
- phase2-comrak-002 class 5: gate dead VecDeque build, [0.982,1.026]
- phase2-comrak-003 class 6: hoist find_special_char check, [0.970,1.007]
The agent picked the cheapest untried class each time (3->5->6) and
TWICE refused to fabricate a verdict it couldn't read back, verifying
ledger integrity instead — the anti-reward-hacking behavior SPEC §10
targets.

Harness changes surfaced by the loop:
- run_verdict is now async: launches the multi-minute build+bench
  detached (past the 60s MCP transport cap) and returns immediately;
  new read_verdict op polls the append-only ledger row (ledger
  verdict_summary). Fixes attempt 001's unreadable-verdict gap.
- read_target_source takes offset/limit line windows so 100KB+ files
  page instead of overflowing — removes the agent's reason to reach for
  a shell (attempt 003 paged a 2400-line file cleanly, no shell).

Honest boundary note (case study + code comments): in THIS dev
container the SDK tool allow/deny layer is defense-in-depth, not the
trust boundary — a nested CLI as root under a parent session doesn't
reliably gate built-ins. SPEC §10's boundary is OS process/user
isolation; that hardening is the remaining Phase 2 exit item
(docs/infra.md). What holds regardless: propose_patch is the only
mediated write path (path-allowlist + git apply) and the ledger is
append-only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
This commit is contained in:
Claude
2026-07-09 23:29:52 +00:00
parent 5be5cad0d3
commit 535821cd48
13 changed files with 487 additions and 45 deletions
+10 -5
View File
@@ -46,9 +46,9 @@ def read_playbook(class_number: int) -> str:
return _c().call("read_playbook", **{"class": class_number})["markdown"]
def read_target_source(target: str, path: str) -> str:
"""Read-only view of the target workspace at the pinned commit."""
return _c().call("read_target_source", target=target, path=path)["content"]
def read_target_source(target: str, path: str, offset: int = 0, limit: int = 400) -> dict[str, Any]:
"""Read-only line window of a target workspace file at the pinned commit."""
return _c().call("read_target_source", target=target, path=path, offset=offset, limit=limit)
def propose_patch(target: str, diff: str, hypothesis: str) -> str:
@@ -60,8 +60,8 @@ def propose_patch(target: str, diff: str, hypothesis: str) -> str:
def run_verdict(patch_id: str, run_id: str, playbook_class: int, hotspot: str) -> dict[str, Any]:
"""Gates + bench + ledger row for a proposed patch. This is the only
way the agent learns whether a change 'worked'."""
"""Launch the gate + bench pipeline for a proposed patch (runs detached).
Returns immediately; poll run_verdict's result via read_verdict(run_id)."""
return _c().call(
"run_verdict",
patch_id=patch_id,
@@ -69,3 +69,8 @@ def run_verdict(patch_id: str, run_id: str, playbook_class: int, hotspot: str) -
playbook_class=str(playbook_class),
hotspot=hotspot,
)
def read_verdict(run_id: str) -> dict[str, Any]:
"""Poll for a completed verdict: verdict + speedup CI, or a running status."""
return _c().call("read_verdict", run_id=run_id)