phase5: kernel lane (SPEC §13) — FP-tolerance equivalence gate + reference-diff testing

The parser/service targets only ever needed byte-identical golden replay.
The kernel lane is the case that breaks it: a matmul optimization
(transpose-B for cache locality + eight-accumulator ILP) that REORDERS the
floating-point reduction, so its low bits differ by construction —
byte-identical is the wrong gate.

Adds:
  - EquivalencePolicy::compare + Divergence (diff-test): byte-identical is
    exact; fp-tolerance tokenizes and compares numeric tokens within
    abs + rel*|baseline|, with explicit NaN handling. Non-numeric tokens and
    token-count mismatches still fail exactly.
  - fp-compare bin: applies a target's equivalence.toml to two output files.
  - targets/matmul/kernel.c: matmul_ref (single-accumulator oracle) vs
    matmul_opt (reordered, faster), with emit/check/run/bench modes.
  - targets/matmul/equivalence.toml: fp-tolerance abs 1e-4 + rel 1e-3.
  - scripts/kernel-lane-demo.sh + results/phase5/kernel-lane.txt: byte-identical
    FAILs, fp-tolerance is EQUIVALENT, a +0.5 perturbation is still REJECTED,
    speedup 3.23× measured with the same interleaved A/B + bootstrap CI.

Ledger row phase5-matmul-opt = needs-human-review: using the FP-tolerance
tier at all is a human-review signal (SPEC §8), same posture as mimalloc.
The trust machinery is unchanged; only the timer and the gate differ, which
is exactly the shape a future GPU kernel would slot into.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EcKPWQ3MQD4NdEHwar96u2
This commit is contained in:
Claude
2026-07-12 00:35:31 +00:00
parent b7297ad31f
commit 5abeb608e0
7 changed files with 439 additions and 0 deletions
+39
View File
@@ -0,0 +1,39 @@
//! Apply a target's equivalence policy to two output files (SPEC §13
//! kernel lane). Loads `targets/<target>/equivalence.toml` (byte-identical
//! by default, or fp-tolerance) and reports whether the candidate output
//! is equivalent to the baseline under it.
//!
//! fp-compare <target-dir> <baseline-file> <candidate-file>
use anyhow::{Context, Result};
use diff_test::policy::EquivalencePolicy;
use std::path::PathBuf;
fn main() -> Result<()> {
let mut args = std::env::args().skip(1);
let target_dir = PathBuf::from(
args.next()
.context("usage: fp-compare <target-dir> <baseline> <candidate>")?,
);
let baseline_path = args.next().context("missing baseline file")?;
let candidate_path = args.next().context("missing candidate file")?;
let policy = EquivalencePolicy::load(&target_dir)?;
let baseline = std::fs::read_to_string(&baseline_path)?;
let candidate = std::fs::read_to_string(&candidate_path)?;
println!("policy: {policy:?}");
match policy.compare(&baseline, &candidate) {
Ok(()) => {
println!("EQUIVALENT under policy");
Ok(())
}
Err(d) => {
println!(
"DIVERGENT at token/offset {}: baseline={:?} candidate={:?} — {}",
d.token, d.baseline, d.candidate, d.reason
);
std::process::exit(1);
}
}
}
+114
View File
@@ -40,6 +40,96 @@ impl EquivalencePolicy {
}
Ok(policy)
}
/// Compare two outputs under this policy. `ByteIdentical` requires an
/// exact match. `FpTolerance` tokenizes on whitespace and compares
/// numeric tokens within `abs + rel*|baseline|`; non-numeric tokens
/// (and mismatched token counts) must still match exactly. This is the
/// gate the kernel lane (SPEC §13) needs: an optimization that reorders
/// floating-point accumulation changes the last ULPs, so it fails
/// byte-identical but passes a declared tolerance — while a genuine
/// correctness bug (a wrong result) still exceeds it.
pub fn compare(&self, baseline: &str, candidate: &str) -> Result<(), Divergence> {
match self {
Self::ByteIdentical => {
if baseline == candidate {
Ok(())
} else {
let pos = baseline
.bytes()
.zip(candidate.bytes())
.position(|(a, b)| a != b)
.unwrap_or(baseline.len().min(candidate.len()));
Err(Divergence {
token: pos,
baseline: snippet(baseline, pos),
candidate: snippet(candidate, pos),
reason: "byte-identical mismatch".into(),
})
}
}
Self::FpTolerance { abs, rel } => {
let bt: Vec<&str> = baseline.split_whitespace().collect();
let ct: Vec<&str> = candidate.split_whitespace().collect();
if bt.len() != ct.len() {
return Err(Divergence {
token: bt.len().min(ct.len()),
baseline: format!("{} tokens", bt.len()),
candidate: format!("{} tokens", ct.len()),
reason: "token-count mismatch".into(),
});
}
for (i, (b, c)) in bt.iter().zip(&ct).enumerate() {
match (b.parse::<f64>(), c.parse::<f64>()) {
(Ok(bn), Ok(cn)) => {
if bn.is_nan() && cn.is_nan() {
continue;
}
let d = (bn - cn).abs();
let tol = abs + rel * bn.abs();
// Explicit NaN handling: a lone NaN (the other
// side finite) is a divergence, not a pass.
if d.is_nan() || d > tol {
return Err(Divergence {
token: i,
baseline: (*b).to_string(),
candidate: (*c).to_string(),
reason: format!("|Δ|={d:.3e} exceeds tolerance {tol:.3e}"),
});
}
}
_ => {
if b != c {
return Err(Divergence {
token: i,
baseline: (*b).to_string(),
candidate: (*c).to_string(),
reason: "non-numeric token mismatch".into(),
});
}
}
}
}
Ok(())
}
}
}
}
/// The first place two outputs diverge under a policy.
#[derive(Debug, Clone, PartialEq)]
pub struct Divergence {
/// Token index (FpTolerance) or byte offset (ByteIdentical).
pub token: usize,
pub baseline: String,
pub candidate: String,
pub reason: String,
}
fn snippet(s: &str, pos: usize) -> String {
let start = pos.saturating_sub(8);
let end = (pos + 8).min(s.len());
s.get(start..end).unwrap_or("").replace('\n', "\\n")
}
#[cfg(test)]
@@ -73,4 +163,28 @@ mod tests {
}
);
}
#[test]
fn byte_identical_rejects_reordered_fp() {
// The blocked kernel's low-bit difference: byte-identical says no.
let p = EquivalencePolicy::ByteIdentical;
assert!(p.compare("1.0000000 2.0", "1.0000001 2.0").is_err());
}
#[test]
fn fp_tolerance_accepts_low_bits_rejects_real_error() {
let p = EquivalencePolicy::FpTolerance {
abs: 1e-6,
rel: 1e-5,
};
// Last-ULP reordering: within tolerance.
assert!(p.compare("1.0000000 42.0", "1.0000001 42.0").is_ok());
// A genuine wrong result: exceeds tolerance, caught.
let d = p.compare("1.0 42.0", "1.0 42.5").unwrap_err();
assert_eq!(d.token, 1);
// Non-numeric tokens still must match exactly.
assert!(p.compare("ok 1.0", "bad 1.0").is_err());
// Token-count mismatch is a divergence.
assert!(p.compare("1.0 2.0", "1.0").is_err());
}
}