mirror of
https://github.com/bdbrown4/peltier.io.git
synced 2026-11-07 10:58:28 +00:00
Claude
5abeb608e0
The parser/service targets only ever needed byte-identical golden replay.
The kernel lane is the case that breaks it: a matmul optimization
(transpose-B for cache locality + eight-accumulator ILP) that REORDERS the
floating-point reduction, so its low bits differ by construction —
byte-identical is the wrong gate.
Adds:
- EquivalencePolicy::compare + Divergence (diff-test): byte-identical is
exact; fp-tolerance tokenizes and compares numeric tokens within
abs + rel*|baseline|, with explicit NaN handling. Non-numeric tokens and
token-count mismatches still fail exactly.
- fp-compare bin: applies a target's equivalence.toml to two output files.
- targets/matmul/kernel.c: matmul_ref (single-accumulator oracle) vs
matmul_opt (reordered, faster), with emit/check/run/bench modes.
- targets/matmul/equivalence.toml: fp-tolerance abs 1e-4 + rel 1e-3.
- scripts/kernel-lane-demo.sh + results/phase5/kernel-lane.txt: byte-identical
FAILs, fp-tolerance is EQUIVALENT, a +0.5 perturbation is still REJECTED,
speedup 3.23× measured with the same interleaved A/B + bootstrap CI.
Ledger row phase5-matmul-opt = needs-human-review: using the FP-tolerance
tier at all is a human-review signal (SPEC §8), same posture as mimalloc.
The trust machinery is unchanged; only the timer and the gate differ, which
is exactly the shape a future GPU kernel would slot into.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
61 lines
2.6 KiB
Bash
61 lines
2.6 KiB
Bash
#!/bin/sh
|
|
# Kernel-lane demonstration (SPEC §13 research fork). Shows the one thing
|
|
# the parser/service targets never needed: an optimization that reorders
|
|
# floating-point accumulation, so byte-identical golden replay is the WRONG
|
|
# gate and a floating-point TOLERANCE gate is required — while a genuine
|
|
# wrong result is still caught. Measured with the same interleaved A/B +
|
|
# bootstrap CI as every other target.
|
|
#
|
|
# sh scripts/kernel-lane-demo.sh [n]
|
|
set -eu
|
|
ROOT=$(cd "$(dirname "$0")/.." && pwd)
|
|
N=${1:-512}
|
|
OUT="$ROOT/results/phase5"
|
|
mkdir -p "$OUT" /tmp/kl
|
|
|
|
cd "$ROOT"
|
|
clang -O2 -o /tmp/kl/kernel targets/matmul/kernel.c -lm
|
|
cargo build -q -p diff-test -p bench-runner
|
|
|
|
{
|
|
echo "# Kernel-lane demonstration — matmul (n=$N, f32)"
|
|
echo
|
|
echo "## 1. Equivalence: byte-identical is the wrong gate"
|
|
/tmp/kl/kernel "$N" emit ref > /tmp/kl/ref.txt
|
|
/tmp/kl/kernel "$N" emit opt > /tmp/kl/opt.txt
|
|
if diff -q /tmp/kl/ref.txt /tmp/kl/opt.txt >/dev/null; then
|
|
echo "byte-identical: MATCH (unexpected — no reorder)"
|
|
else
|
|
d=$(diff /tmp/kl/ref.txt /tmp/kl/opt.txt | grep -c '^<' || true)
|
|
echo "byte-identical: FAIL — $d of $((N*N)) result values differ (accumulation reordered)"
|
|
fi
|
|
echo
|
|
echo "diff-test FP-tolerance policy (targets/matmul/equivalence.toml):"
|
|
"$ROOT/target/debug/fp-compare" targets/matmul /tmp/kl/ref.txt /tmp/kl/opt.txt | sed 's/^/ /'
|
|
echo
|
|
echo "in-process reference differential test:"
|
|
/tmp/kl/kernel "$N" check 1e-4 1e-3 | sed 's/^/ /'
|
|
echo
|
|
echo "control — a genuine wrong result IS caught by the tolerance:"
|
|
awk 'NR==1{$1=$1+0.5} {print}' /tmp/kl/opt.txt > /tmp/kl/wrong.txt
|
|
if "$ROOT/target/debug/fp-compare" targets/matmul /tmp/kl/ref.txt /tmp/kl/wrong.txt >/dev/null 2>&1; then
|
|
echo " !! tolerance accepted a wrong result (bug)"
|
|
else
|
|
echo " tolerance correctly REJECTS a +0.5 perturbation of one element"
|
|
fi
|
|
echo
|
|
echo "## 2. Speedup: interleaved A/B, bootstrap 95% CI"
|
|
cargo run -q -p bench-runner -- --config config/accept.toml compare \
|
|
--baseline "taskset -c 2 /tmp/kl/kernel $N run ref" \
|
|
--candidate "taskset -c 2 /tmp/kl/kernel $N run opt" 2>/dev/null \
|
|
| grep -E "speedup|verdict" | sed 's/^/ /'
|
|
echo
|
|
echo "## What it demonstrates"
|
|
echo "A real matmul speedup (transpose for cache locality + eight-accumulator"
|
|
echo "ILP) that reorders the reduction. Byte-identical rejects it; the"
|
|
echo "FP-tolerance gate accepts it (abs 1e-4 + rel 1e-3) while still catching"
|
|
echo "a wrong result; the bench measures it with the same CI machinery as"
|
|
echo "every other target. The GPU extension (Triton/CUDA vs a reference"
|
|
echo "kernel) is the same shape — only the timer and the hardware change."
|
|
} | tee "$OUT/kernel-lane.txt"
|