mirror of
https://github.com/bdbrown4/peltier.io.git
synced 2026-11-07 19:08:27 +00:00
Claude
03113312ad
phase3-cjson-002 (class 5+3): stack-buffer number parsing + strtod round-trip replacing sscanf. +8.85% median, 95% CI [+7.5%, +10.0%] on the C target, banked. 100% audit: byte-identical golden replay, independent ASan+UBSan clean, 10k-input differential fuzz 0 divergences, patch bounds/free-gating verified by eye. - verdict: an infrastructure sanitizer-BUILD failure (missing runtime, broken toolchain) now caps the accept at needs-human-review and logs it, instead of crashing the verdict and losing a measured bench. - results/phase3/: four public case studies (tokei class-5 wins; build flags + allocators; the comrak caught false-accept; the cJSON cross-language proof) + README. Compound-speedup claim stated as an approximation, not a measured figure (CIs don't multiply). - .gitignore: cjson build-artifact dirs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
65 lines
3.4 KiB
Markdown
65 lines
3.4 KiB
Markdown
# Case study 1 — tokei: compounding algorithmic wins (Rust, class 5)
|
||
|
||
**Target:** [tokei](https://github.com/XAMPPRocky/tokei) v14.0.0 — a
|
||
source-code line counter. **Workload:** 221 MB / 4360 files (8 replicas
|
||
of the Pro Git book + comrak's own tree), single rayon thread, CPU
|
||
pinned. **Result:** three independent, CI-verified class-5 wins on the
|
||
same hot function — each proposed *by the unattended agent* and
|
||
100%-audited. Multiplied together they are **~1.26× over the Phase-0
|
||
baseline** (1.107 × 1.037 × 1.099), but that product is an
|
||
approximation, not a measured figure: each speedup carries its own
|
||
bootstrap CI against its own baseline, and confidence intervals do not
|
||
compose through multiplication. The trustworthy claims are the three
|
||
individual increments below.
|
||
|
||
## The hot path
|
||
|
||
tokei spends the overwhelming majority of its instructions in one
|
||
function — `LanguageType::parse_lines` and the per-byte loop inside
|
||
`SyntaxCounter::perform_multi_line_analysis`. The cache-sim profile put
|
||
it at **55.9% of instructions, 62% of all branches, 44% of branch
|
||
mispredicts** on the pinned corpus: for every byte of every line, the
|
||
counter tested a stack of quote/comment token matchers.
|
||
|
||
## The three wins
|
||
|
||
| run | mechanism | speedup (median) | 95% CI |
|
||
|---|---|---|---|
|
||
| `phase0-tokei-002` (class 3) | read path: `fs::read` pre-sized, BOM-only decode | 1.099 | [1.080, 1.127] |
|
||
| `phase2-tokei-001` (class 5) | 256-entry first-byte gate table for the token matchers | 1.107 | [1.084, 1.131] |
|
||
| `phase2-tokei-003` (class 5) | forward-only whitespace break (drop the `rposition` rescan) | 1.037 | [1.027, 1.052] |
|
||
| `phase2-tokei-008` (class 5) | streaming leftmost `important_syntax` scan (one vectorized pass, not per-line restart) | 1.099 | [1.073, 1.112] |
|
||
|
||
Each is a **redundant-work elimination**: the first-byte gate replaces
|
||
five per-byte matcher loops with one table load; the whitespace-break
|
||
change drops a full backward scan of the file tail per byte; the
|
||
streaming scan replaces a per-line aho-corasick restart (which kept
|
||
memmem's SIMD prefilter in its short-haystack slow path) with a single
|
||
leftmost search advanced by integer comparisons.
|
||
|
||
## Why the numbers are trustworthy
|
||
|
||
- **Byte-identical golden replay** on the pinned corpus for every win —
|
||
a code counter that miscounts one line fails the gate.
|
||
- **10,000-input differential fuzz** (`scripts/diff-fuzz-tokei.py`),
|
||
pristine vs candidate, **0 divergences** — after canonicalizing
|
||
tokei's benign parallel-walker output ordering (a real finding:
|
||
raw-byte differential comparison over-counts when the target is
|
||
legitimately nondeterministic in presentation).
|
||
- **ASan/LSan clean** on the patched tree over the full corpus.
|
||
- **Interleaved A/B** vs a pristine-rebuilt baseline, bootstrap 95% CI
|
||
lower bound ≥ the 2% bar. `phase2-tokei-008` was the first accept
|
||
through the *machine-enforced* sanitizer gate (see case study 3).
|
||
|
||
## What it demonstrates
|
||
|
||
The richest wins on a mature Rust binary were not exotic — they were
|
||
"this loop does work it doesn't need to, on every byte." The agent found
|
||
each by reading the cache-sim profile, checking the ledger to avoid
|
||
re-attempting a class on the same hotspot, and proposing the cheapest
|
||
untried mechanism. Three separate hypotheses, three separate verified
|
||
increments, all on the same 40%-of-runtime function — and one honest
|
||
rejection in between (`phase2-tokei-009`, a real +5.7% median whose
|
||
noisy CI lower bound of +0.2% could not clear the bar, so it does not
|
||
count).
|