phase0: dry run — comrak & tokei vendored, 3 attempts in ledger

Target selection (SPEC §6): comrak 0.53.0 (45c1995, BSD-2-Clause) and
tokei 14.0.0 (fa44e51, MIT/Apache-2.0); both CPU-bound with pinned
corpora (progit book 11MB; 8x replicated pinned trees, 221MB) and
committed MANIFEST.sha256 + golden-output pins + regen scripts.

Calibration: A/A null verdicts on both real workloads; taskset -c 2
pinning tightens the A/A CI from [0.938, 1.008] to [0.993, 1.008],
making the 2% accept threshold meaningful in this container.

Profiles (callgrind; perf unavailable): comrak allocator symbols 26.7%
of Ir; tokei perform_multi_line_analysis 31.7%.

Attempts (ledger + results/phase0/):
- comrak class-1 target-cpu=native: null [0.983, 1.011], rejected-bench
- comrak class-2 mimalloc swap: +4.6% [1.033, 1.058], 848 upstream
  tests green, golden byte-identical, ASan-proper clean; LSan flags
  484B in onig (reachability false-positive analysis in case study) ->
  needs-human-review per SPEC §8
- tokei class-1 fat LTO+cgu=1: +1.6% [1.003, 1.026], real but under
  the 2% bar, rejected-bench

Also: ledger-cli for manual rows, targets/ excluded from trust-layer
workspace, corpora/build artifacts gitignored, case study written.
Phase 0 exit criteria (>=10% verified win) not yet met; next
hypotheses logged in results/phase0/attempts.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
This commit is contained in:
Claude
2026-07-06 19:28:50 +00:00
parent 1ef2bb4fdb
commit 7c3c59f5f8
26 changed files with 1001 additions and 2 deletions
@@ -0,0 +1,4 @@
env fingerprint: {"kernel":"6.18.5","cpu_model":"Intel(R) Xeon(R) Processor @ 2.80GHz","governor":"unknown","smt":"0","turbo":"unknown","aslr":"mode=system-default randomize_va_space=2","cpu_pinning":"none"}
plan: 30 measured + 3 warm-up runs per side, interleaved
speedup (baseline/candidate): median 1.0006, 95% CI [0.9932, 1.0080]
A/A self-test passed (null verdict)
@@ -0,0 +1,4 @@
env fingerprint: {"kernel":"6.18.5","cpu_model":"Intel(R) Xeon(R) Processor @ 2.80GHz","governor":"unknown","smt":"0","turbo":"unknown","aslr":"mode=system-default randomize_va_space=2","cpu_pinning":"none"}
plan: 30 measured + 3 warm-up runs per side, interleaved
speedup (baseline/candidate): median 0.9773, 95% CI [0.9376, 1.0080]
A/A self-test passed (null verdict)
@@ -0,0 +1,4 @@
env fingerprint: {"kernel":"6.18.5","cpu_model":"Intel(R) Xeon(R) Processor @ 2.80GHz","governor":"unknown","smt":"0","turbo":"unknown","aslr":"mode=system-default randomize_va_space=2","cpu_pinning":"none"}
plan: 30 measured + 3 warm-up runs per side, interleaved
speedup (baseline/candidate): median 0.9984, 95% CI [0.9891, 1.0118]
A/A self-test passed (null verdict)
+40
View File
@@ -0,0 +1,40 @@
--------------------------------------------------------------------------------
Profile data file 'results/comrak/callgrind.out' (creator: callgrind-3.22.0)
--------------------------------------------------------------------------------
I1 cache:
D1 cache:
LL cache:
Timerange: Basic block 0 - 196578070
Trigger: Program termination
Profiled target: targets/comrak/workspace/target/release/comrak (PID 17601, part 1)
Events recorded: Ir
Events shown: Ir
Event sort order: Ir
Thresholds: 99
Include dirs:
User annotated:
Auto-annotation: on
--------------------------------------------------------------------------------
Ir
--------------------------------------------------------------------------------
883,766,827 (100.0%) PROGRAM TOTALS
--------------------------------------------------------------------------------
Ir file:function
--------------------------------------------------------------------------------
88,282,747 ( 9.99%) ./malloc/./malloc/malloc.c:_int_malloc [/usr/lib/x86_64-linux-gnu/libc.so.6]
60,568,128 ( 6.85%) ./malloc/./malloc/malloc.c:_int_free [/usr/lib/x86_64-linux-gnu/libc.so.6]
58,911,428 ( 6.67%) ???:comrak::parser::inlines::Subject::parse_inline [/home/user/peltier.io/targets/comrak/workspace/target/release/comrak]
57,026,217 ( 6.45%) ???:comrak::parser::Parser::process_line [/home/user/peltier.io/targets/comrak/workspace/target/release/comrak]
45,107,073 ( 5.10%) ???:core::str::converts::from_utf8 [/home/user/peltier.io/targets/comrak/workspace/target/release/comrak]
41,962,238 ( 4.75%) ???:<comrak::plugins::syntect::SyntectAdapter as comrak::adapters::SyntaxHighlighterAdapter>::write_highlighted [/home/user/peltier.io/targets/comrak/workspace/target/release/comrak]
34,662,512 ( 3.92%) ./malloc/./malloc/malloc.c:malloc [/usr/lib/x86_64-linux-gnu/libc.so.6]
33,819,139 ( 3.83%) ???:comrak::parser::Parser::parse [/home/user/peltier.io/targets/comrak/workspace/target/release/comrak]
30,762,786 ( 3.48%) ./malloc/./malloc/malloc.c:malloc_consolidate [/usr/lib/x86_64-linux-gnu/libc.so.6]
27,843,363 ( 3.15%) ???:syntect::parsing::parser::ParseState::parse_line [/home/user/peltier.io/targets/comrak/workspace/target/release/comrak]
27,272,869 ( 3.09%) ./string/../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:__memcpy_avx_unaligned_erms [/usr/lib/x86_64-linux-gnu/libc.so.6]
26,634,752 ( 3.01%) ???:comrak::parser::Parser::open_new_blocks [/home/user/peltier.io/targets/comrak/workspace/target/release/comrak]
22,049,414 ( 2.49%) ./malloc/./malloc/malloc.c:free [/usr/lib/x86_64-linux-gnu/libc.so.6]
21,200,547 ( 2.40%) ???:jetscii::simd::Bytes::find [/home/user/peltier.io/targets/comrak/workspace/target/release/comrak]
20,549,182 ( 2.33%) ???:alloc::raw_vec::RawVecInner<A>::finish_grow [/home/user/peltier.io/targets/comrak/workspace/target/release/comrak]
Binary file not shown.
+4
View File
@@ -0,0 +1,4 @@
env fingerprint: {"kernel":"6.18.5","cpu_model":"Intel(R) Xeon(R) Processor @ 2.80GHz","governor":"unknown","smt":"0","turbo":"unknown","aslr":"mode=system-default randomize_va_space=2","cpu_pinning":"none"}
plan: 30 measured + 3 warm-up runs per side, interleaved
speedup (baseline/candidate): median 0.9965, 95% CI [0.9826, 1.0110]
verdict: rejected-bench
+4
View File
@@ -0,0 +1,4 @@
env fingerprint: {"kernel":"6.18.5","cpu_model":"Intel(R) Xeon(R) Processor @ 2.80GHz","governor":"unknown","smt":"0","turbo":"unknown","aslr":"mode=system-default randomize_va_space=2","cpu_pinning":"none"}
plan: 30 measured + 3 warm-up runs per side, interleaved
speedup (baseline/candidate): median 1.0459, 95% CI [1.0333, 1.0577]
verdict: accepted
+4
View File
@@ -0,0 +1,4 @@
env fingerprint: {"kernel":"6.18.5","cpu_model":"Intel(R) Xeon(R) Processor @ 2.80GHz","governor":"unknown","smt":"0","turbo":"unknown","aslr":"mode=system-default randomize_va_space=2","cpu_pinning":"none"}
plan: 30 measured + 3 warm-up runs per side, interleaved
speedup (baseline/candidate): median 1.0161, 95% CI [1.0026, 1.0262]
verdict: rejected-bench
+84
View File
@@ -0,0 +1,84 @@
# Phase 0 dry-run attempt log
Hypotheses are recorded here *before* any patch is applied; final
evidence and verdicts go to the SQLite ledger and are mirrored here.
Environment for the whole session: remote 4-core container
(Intel Xeon @ 2.10GHz, kernel 6.18.5), governor/turbo not controllable,
benches pinned with `taskset -c 2`. A/A calibration on both target
workloads passed with pinning (see results/calibration/).
## Deviations from SPEC (Phase 0 manual dry run)
- No nested no-network container available (no docker daemon): target
code runs in this session's own isolated ephemeral container, which
serves as the sandbox. Phase 1 must restore the dedicated container.
- `perf` unavailable (container): profiling via valgrind/callgrind
(deterministic instruction counts, Ir), not cycle sampling. Ir% is a
proxy for cycles%; it under-weights cache/branch effects.
- Corpus files >10 MB are pinned by committed MANIFEST.sha256 +
regeneration scripts rather than committed raw (repo bloat).
## Attempt 1 — comrak, class 1 (build configuration)
- **Logged**: 2026-07-06, before patching.
- **Hotspot**: whole-program; upstream already ships lto=true +
codegen-units=1, so classic class-1 flags are exhausted upstream.
- **Hypothesis**: `RUSTFLAGS=-C target-cpu=native` (baseline ISA: the
bench machine's Xeon) enables wider SIMD paths in string scanning
(jetscii, memchr paths visible in profile) and better scheduling;
expect a small win, 2-6% median speedup. No FP semantics change
(integer/string workload; not a -ffast-math-class flag), so
auto-acceptable if gates pass. Deploy caveat: win only valid for
fleets whose ISA baseline matches; recorded in workload statement.
- **Patch**: none (env-level build config); candidate built to a
separate CARGO_TARGET_DIR from the same pinned commit.
## Attempt 2 — comrak, class 2 (allocator swap)
- **Logged**: 2026-07-06, before patching, after Attempt 1's hypothesis.
- **Hotspot**: allocator symbols sum to ~26.7% of Ir (_int_malloc 9.99%,
_int_free 6.85%, malloc 3.92%, malloc_consolidate 3.48%, free 2.49%)
— the class-2 precondition (≥5%) is exceeded 5x.
- **Hypothesis**: glibc malloc is the allocator; comrak's AST/inline
parsing churns small allocations. Swapping the CLI's global allocator
to mimalloc cuts allocator instructions substantially; expect 8-20%
median speedup on the progit workload.
- **Patch plan**: 2-line change in comrak CLI main + Cargo.toml dep,
inside targets/comrak/workspace only.
## Attempt 3 — tokei, class 1 (build configuration)
- **Logged**: 2026-07-06, before patching.
- **Hotspot**: whole-program; upstream ships lto="thin", panic="abort",
default codegen-units (16).
- **Hypothesis**: lto=true (fat) + codegen-units=1 lets the analysis
loop (perform_multi_line_analysis, 31.7% Ir) inline across cgu and
crate boundaries (memchr/aho-corasick calls visible in profile);
expect 2-8% median speedup. No semantics change; auto-acceptable if
golden replay passes.
- **Patch**: [profile.release] lto = true, codegen-units = 1 in
targets/tokei/workspace/Cargo.toml only.
## Outcomes (ledger is authoritative: results/ledger.sqlite)
| run_id | class | speedup median | 95% CI | verdict |
|---|---|---|---|---|
| phase0-comrak-001 | 1 build-config | 0.9965 | [0.9826, 1.0110] | rejected-bench (null) |
| phase0-comrak-002 | 2 allocator | **1.0459** | **[1.0333, 1.0577]** | needs-human-review |
| phase0-tokei-001 | 1 build-config | 1.0161 | [1.0026, 1.0262] | rejected-bench (< 2% threshold) |
- comrak-002 gates: 848 upstream tests green on patched tree; golden
replay byte-identical; pristine ASan+LSan clean; patched ASan-proper
clean; LSan flags 484 B in onig (analysis: LSan × custom-allocator
reachability false positive — see case-study-comrak.md). SPEC §8
routes any sanitizer flag to human review, so that is the verdict.
- tokei-001 is a *real but small* effect — CI excludes 1.0 but its
lower bound (+0.26%) is under the 2% bar. Correctly rejected; logged
so nobody re-grinds it.
- Phase 0 exit criteria (≥1 verified win ≥10%) **not yet met**: best
verified candidate is +4.6% pending human review. Next cheapest
hypotheses per profile: comrak class 3 (allocation churn — arena the
inline parser's Vec growth, RawVec::finish_grow 2.3% + memcpy 3.1%);
tokei class 3 (memset 8.5% suggests per-file buffer zeroing) and
class 5 (perform_multi_line_analysis 31.7% single frame).
+87
View File
@@ -0,0 +1,87 @@
# Phase 0 case study — comrak (CommonMark/GFM renderer)
**Result: +4.6% median wall-time speedup (95% CI [+3.3%, +5.8%]) from a
2-line allocator swap, byte-identical output, upstream tests green,
verdict `needs-human-review` on a LeakSanitizer reachability flag.**
## Target
- comrak v0.53.0, commit `45c1995f`, BSD-2-Clause, built `--release`
(upstream profile already ships `lto = true`, `codegen-units = 1`).
- Workload: the target's own benchmark (benches/bench.sh) — the Pro Git
book, all translations, 11.0 MB concatenated markdown, single stream
through stdin, HTML to /dev/null. Corpus pinned:
`corpora/comrak/MANIFEST.sha256` (progit @ `61833a52`).
## Environment
Remote 4-core container, Intel Xeon @ 2.10GHz, kernel 6.18.5, governor
and turbo state unavailable/uncontrollable, SMT off, ASLR system
default, benches pinned with `taskset -c 2`, 30 measured + 3 warm-up
runs per side, interleaved ABAB, bootstrap 95% CI (10k resamples).
**A/A calibration, same session, pinned** (results/calibration/):
median 1.0006, 95% CI [0.9932, 1.0080] — null, as required. Unpinned
A/A for contrast: CI [0.9376, 1.0080] — 5x noisier; pinning is what
makes a 2% threshold meaningful in this container.
## Loop
1. **Profile** (callgrind; perf unavailable in container — Ir counts,
not cycles): allocator symbols total **26.7% of instructions**
(`_int_malloc` 9.99%, `_int_free` 6.85%, `malloc` 3.92%,
`malloc_consolidate` 3.48%, `free` 2.49%). Parser frames
(`parse_inline` 6.7%, `process_line` 6.5%) come *after* the
allocator. Full ranking: results/comrak/hotspots.txt.
2. **Attempt 1 — class 1, build config** (`target-cpu=native`):
hypothesis logged first; golden replay byte-identical; bench median
0.9965, 95% CI [0.9826, 1.0110] → **rejected-bench** (null). Ledger
`phase0-comrak-001`.
3. **Attempt 2 — class 2, allocator swap** (mimalloc as
`#[global_allocator]` in the CLI binary only; library consumers
unaffected; patch: results/phase0/comrak-mimalloc.patch):
- Golden replay: stdout sha256 identical to pinned golden.
- Upstream test suite on the patched tree: green (see ledger row).
- Bench, interleaved A/B vs pristine-built baseline: **median
speedup 1.0459, 95% CI [1.0333, 1.0577]** — CI lower bound 3.3%
clears the 2% accept threshold.
- Sanitizers (nightly ASan on the full corpus run):
- pristine binary: clean, output hash = golden.
- patched binary: ASan-proper clean with `detect_leaks=0`;
**LeakSanitizer reports 484 B / 3 allocations in
`onig_new`/`onig_compile`** (oniguruma, syntect's C regex
engine). Analysis: these allocations exist identically in the
pristine binary (LSan exit 0 there) — the report appears because
LSan cannot scan mimalloc-owned memory for roots, so a
still-reachable global compiled regex looks unreachable. Almost
certainly a false positive of the LSan × custom-allocator
interaction, **not** a leak introduced by the patch.
- Per SPEC §8 ("anything the sanitizers flag, even as warnings"):
verdict **needs-human-review**, not auto-accept. Ledger
`phase0-comrak-002`.
## ROI (illustrative, config/pricing.toml rates)
For a hypothetical fleet spending 100 cores 24/7 on this workload shape,
at $0.04/core-hr (public-cloud example rate — replace per engagement):
- cores returned: median **4.4**, 95% CI [3.2, 5.5]
(`fleet × (1 − 1/speedup)`)
- dollars/yr: median **$1,540**, 95% CI [$1,130, $1,910]
Caveats printed with the number, per policy: single workload shape
(large-document batch rendering); Ir-based profile, not cycles;
container hardware without governor control; speedup CI is wall-time on
pinned single-core execution; ISA = generic x86-64 (no target-cpu
narrowing — attempt 1 showed it buys nothing here anyway).
## Reproduction
```
just pin-check comrak # corpus manifest verify
cargo run -p bench-runner -- --config config/accept.toml compare \
--baseline "taskset -c 2 sh -c '<pristine comrak> < corpora/comrak/progit.md > /dev/null'" \
--candidate "taskset -c 2 sh -c '<patched comrak> < corpora/comrak/progit.md > /dev/null'"
```
Patch: `results/phase0/comrak-mimalloc.patch` applied at `45c1995f`.
+39
View File
@@ -0,0 +1,39 @@
diff --git a/Cargo.toml b/Cargo.toml
index bd1dfe9..fcc70f3 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -69,7 +69,7 @@ phf_codegen = "0.13"
[features]
default = ["cli", "syntect-onig", "bon"]
-cli = ["clap", "bon", "shell-words", "xdg", "fmt2io", "shortcodes", "phoenix_heex", "attributes"]
+cli = ["clap", "bon", "shell-words", "xdg", "fmt2io", "shortcodes", "phoenix_heex", "attributes", "mimalloc"]
shortcodes = ["emojis"]
phoenix_heex = []
attributes = []
@@ -107,6 +107,7 @@ clap = { version = "4", optional = true, features = [
"string",
"wrap_help",
] }
+mimalloc = { version = "0.1", optional = true }
[[example]]
name = "syntect"
diff --git a/src/main.rs b/src/main.rs
index 6c5b2a6..a439fb7 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -10,6 +10,13 @@ use std::{boxed::Box, io::BufWriter};
use clap::{Parser, ValueEnum};
+// The CLI churns many small AST allocations; mimalloc services them with
+// far fewer instructions than glibc malloc. Library consumers are
+// unaffected: this only applies to the comrak binary.
+#[cfg(not(target_arch = "wasm32"))]
+#[global_allocator]
+static GLOBAL: mimalloc::MiMalloc = mimalloc::MiMalloc;
+
use comrak::options;
use comrak::{Arena, Options};
#[cfg(feature = "syntect")]
+14
View File
@@ -0,0 +1,14 @@
diff --git a/Cargo.toml b/Cargo.toml
index 8b19522..b64e2b4 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -30,7 +30,8 @@ default = ["cli"]
yaml = ["dep:serde_yaml"]
[profile.release]
-lto = "thin"
+lto = true
+codegen-units = 1
panic = "abort"
[[bin]]
+40
View File
@@ -0,0 +1,40 @@
--------------------------------------------------------------------------------
Profile data file 'results/tokei/callgrind.out' (creator: callgrind-3.22.0)
--------------------------------------------------------------------------------
I1 cache:
D1 cache:
LL cache:
Timerange: Basic block 0 - 600400493
Trigger: Program termination
Profiled target: targets/tokei/workspace/target/release/tokei --sort code corpora/tokei/input (PID 17959, part 1)
Events recorded: Ir
Events shown: Ir
Event sort order: Ir
Thresholds: 99
Include dirs:
User annotated:
Auto-annotation: on
--------------------------------------------------------------------------------
Ir
--------------------------------------------------------------------------------
1,808,959,061 (100.0%) PROGRAM TOTALS
--------------------------------------------------------------------------------
Ir file:function
--------------------------------------------------------------------------------
572,758,799 (31.66%) ???:tokei::language::syntax::SyntaxCounter::perform_multi_line_analysis [/home/user/peltier.io/targets/tokei/workspace/target/release/tokei]
172,785,648 ( 9.55%) ???:tokei::language::language_type::LanguageType::parse_lines [/home/user/peltier.io/targets/tokei/workspace/target/release/tokei]
154,178,149 ( 8.52%) ./string/../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:__memset_avx2_unaligned_erms [/usr/lib/x86_64-linux-gnu/libc.so.6]
146,199,746 ( 8.08%) ./string/../sysdeps/x86_64/multiarch/memcmp-avx2-movbe.S:__memcmp_avx2_movbe [/usr/lib/x86_64-linux-gnu/libc.so.6]
104,071,008 ( 5.75%) ???:tokei::language::syntax::SyntaxCounter::try_perform_single_line_analysis [/home/user/peltier.io/targets/tokei/workspace/target/release/tokei]
93,030,608 ( 5.14%) ???:memchr::arch::x86_64::avx2::packedpair::Finder::find_impl [/home/user/peltier.io/targets/tokei/workspace/target/release/tokei]
75,136,592 ( 4.15%) ???:aho_corasick::automaton::try_find_fwd [/home/user/peltier.io/targets/tokei/workspace/target/release/tokei]
71,820,456 ( 3.97%) ./string/../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:__memcpy_avx_unaligned_erms [/usr/lib/x86_64-linux-gnu/libc.so.6]
50,442,206 ( 2.79%) ???:memchr::arch::x86_64::avx2::memchr::One::find_raw_avx2 [/home/user/peltier.io/targets/tokei/workspace/target/release/tokei]
28,664,232 ( 1.58%) ???:memchr::memmem::searcher::searcher_kind_avx2 [/home/user/peltier.io/targets/tokei/workspace/target/release/tokei]
27,626,392 ( 1.53%) ???:<aho_corasick::util::prefilter::Memmem as aho_corasick::util::prefilter::PrefilterI>::find_in [/home/user/peltier.io/targets/tokei/workspace/target/release/tokei]
26,582,656 ( 1.47%) ???:memchr::arch::x86_64::memchr::memchr_raw::find_avx2 [/home/user/peltier.io/targets/tokei/workspace/target/release/tokei]
22,263,784 ( 1.23%) ???:memchr::arch::x86_64::avx2::memchr::Three::find_raw [/home/user/peltier.io/targets/tokei/workspace/target/release/tokei]
20,246,771 ( 1.12%) ???:tokei::language::syntax::SyntaxCounter::perform_multi_line_analysis'2 [/home/user/peltier.io/targets/tokei/workspace/target/release/tokei]
20,005,440 ( 1.11%) ???:memchr::arch::x86_64::avx2::memchr::One::find_raw [/home/user/peltier.io/targets/tokei/workspace/target/release/tokei]