Abstract
We present a claim–computation binding audit: a deterministic instrument that verifies whether each claim in an AI-generated analysis is bound to the executed tool call that produced its statistic, classifying each binding (errored / variable-mismatch / stat-mismatch / clean), together with the keyed ground-truth benchmark and the pre-registered, provenance-stamped run corpora on which we ablate it. The setting is agentic statistical root-cause analysis (the Analyze phase of Six Sigma DMAIC), where the trust signals practitioners rely on prove gameable: a no-LLM fill-in-the-blanks template scores 0.867 on a completeness rubric with verified-recall 0, so "complete and well-cited" measures form, not correctness. Applying the audit to a pre-hardening corpus of agentic DMAIC runs (n=23 supported claims, 3 small open-weight models), 52.2% of supported claims (12/23) carry a binding defect even though audit-ID provenance resolves 100%; 6/23 (~26%) are the motivating subclass, a claim citing a real, audit-logged statistic that actually concerns different variables. We call such provenance spoofable. In a pre-registered ablation (180 runs; 45 pairs per contrast; hash-frozen scoring and binding code), the full 12-safeguard stack (base-6 artifact-hygiene checks plus our six correctness safeguards D1–D6, anchored by the claim–computation binding check D1) cuts the run-level binding-defect rate from 28.9% to 2.2% (Holm-corrected p=0.0055). The contrasts isolating single components are not significant (D1–D5: directionally consistent, Holm-corrected p=0.1406; D6: uninformative, 1/45 vs 0/45), so the confirmed effect is stack-level. Descriptively, 9 of the 13 unguarded defective runs are MISSING-only, a class an artifact-existence check can already catch, while the semantic classes (wrong variables, mismatched statistics, or unverifiable statistics; this paper's motivating phenomenon) fall from 4/45 to 0/45 under the full stack. The design was not power-analyzed, and discovery shows no detectable improvement (95% CI on the paired difference spans zero): the safeguards reduce fabricated support, not analytical error. Defects are rarer at larger scale (1/15 unguarded runs at each of a 32B and a frontier-family model, both exploratory) but do not vanish, and binding cleanliness holds on a sealed held-out pair. We contribute a measurement that separates grounded from correct, a detector for spoofable provenance, an honestly-bounded safeguard, and an artifact bundle containing the pre-registration with hash-frozen analysis code, per-run score and binding tables for all five corpora, and per-run summaries (provenance-stamped for the four pre-registered corpora).