mirror of https://github.com/bdbrown4/peltier.io.git synced 2026-11-07 10:58:28 +00:00

Files

T

Claude 03113312ad phase3: cJSON cross-language win verified + audited; robustness fix; case studies

phase3-cjson-002 (class 5+3): stack-buffer number parsing + strtod
round-trip replacing sscanf. +8.85% median, 95% CI [+7.5%, +10.0%] on
the C target, banked. 100% audit: byte-identical golden replay,
independent ASan+UBSan clean, 10k-input differential fuzz 0 divergences,
patch bounds/free-gating verified by eye.

- verdict: an infrastructure sanitizer-BUILD failure (missing runtime,
  broken toolchain) now caps the accept at needs-human-review and logs
  it, instead of crashing the verdict and losing a measured bench.
- results/phase3/: four public case studies (tokei class-5 wins; build
  flags + allocators; the comrak caught false-accept; the cJSON
  cross-language proof) + README. Compound-speedup claim stated as an
  approximation, not a measured figure (CIs don't multiply).
- .gitignore: cjson build-artifact dirs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2

2026-07-11 23:06:59 +00:00

3.4 KiB

Raw Permalink Blame History

Case study 1 — tokei: compounding algorithmic wins (Rust, class 5)

Target: tokei v14.0.0 — a source-code line counter. Workload: 221 MB / 4360 files (8 replicas of the Pro Git book + comrak's own tree), single rayon thread, CPU pinned. Result: three independent, CI-verified class-5 wins on the same hot function — each proposed by the unattended agent and 100%-audited. Multiplied together they are ~1.26× over the Phase-0 baseline (1.107 × 1.037 × 1.099), but that product is an approximation, not a measured figure: each speedup carries its own bootstrap CI against its own baseline, and confidence intervals do not compose through multiplication. The trustworthy claims are the three individual increments below.

The hot path

tokei spends the overwhelming majority of its instructions in one function — LanguageType::parse_lines and the per-byte loop inside SyntaxCounter::perform_multi_line_analysis. The cache-sim profile put it at 55.9% of instructions, 62% of all branches, 44% of branch mispredicts on the pinned corpus: for every byte of every line, the counter tested a stack of quote/comment token matchers.

The three wins

run	mechanism	speedup (median)	95% CI
`phase0-tokei-002` (class 3)	read path: `fs::read` pre-sized, BOM-only decode	1.099	[1.080, 1.127]
`phase2-tokei-001` (class 5)	256-entry first-byte gate table for the token matchers	1.107	[1.084, 1.131]
`phase2-tokei-003` (class 5)	forward-only whitespace break (drop the `rposition` rescan)	1.037	[1.027, 1.052]
`phase2-tokei-008` (class 5)	streaming leftmost `important_syntax` scan (one vectorized pass, not per-line restart)	1.099	[1.073, 1.112]

Each is a redundant-work elimination: the first-byte gate replaces five per-byte matcher loops with one table load; the whitespace-break change drops a full backward scan of the file tail per byte; the streaming scan replaces a per-line aho-corasick restart (which kept memmem's SIMD prefilter in its short-haystack slow path) with a single leftmost search advanced by integer comparisons.

Why the numbers are trustworthy

Byte-identical golden replay on the pinned corpus for every win — a code counter that miscounts one line fails the gate.
10,000-input differential fuzz (scripts/diff-fuzz-tokei.py), pristine vs candidate, 0 divergences — after canonicalizing tokei's benign parallel-walker output ordering (a real finding: raw-byte differential comparison over-counts when the target is legitimately nondeterministic in presentation).
ASan/LSan clean on the patched tree over the full corpus.
Interleaved A/B vs a pristine-rebuilt baseline, bootstrap 95% CI lower bound ≥ the 2% bar. phase2-tokei-008 was the first accept through the machine-enforced sanitizer gate (see case study 3).

What it demonstrates

The richest wins on a mature Rust binary were not exotic — they were "this loop does work it doesn't need to, on every byte." The agent found each by reading the cache-sim profile, checking the ledger to avoid re-attempting a class on the same hotspot, and proposing the cheapest untried mechanism. Three separate hypotheses, three separate verified increments, all on the same 40%-of-runtime function — and one honest rejection in between (phase2-tokei-009, a real +5.7% median whose noisy CI lower bound of +0.2% could not clear the bar, so it does not count).

3.4 KiB Raw Permalink Blame History Unescape Escape