falsify-eval

The Broken Ruler Problem

Imagine a ruler made of rubber that stretches — but stretches the same way for everyone. You measure a table and get 120 cm. So does everyone else. The number is real. But it means nothing because there's no calibrated baseline.

This is what happens in AI evaluation. A score of 0.770 nDCG sounds impressive. But what if a system that returns completely random documents also scores 0.60? Does your 0.770 prove anything?

ECE analogy: You're testing a band-pass filter. You measure the output signal level and get a big number. But you never checked the noise floor. Your filter might be amplifying everything — signal and noise — equally. The output level alone tells you nothing without a noise baseline.

🧱 Layer 2 — The Null Hypothesis

What is a "null"?

A null is a deliberately dumb, cheating, or random system used as a baseline. If your real system can't beat the dumb system by a meaningful margin, it isn't doing anything useful — it just looks like it is.

In science: "Can we reject the null hypothesis?" = "Can we prove our result isn't just noise or a known pattern?"

⚡ Layer 3 — The Four Nulls

NULL A

Permuted Gold

Shuffle which answer belongs to which question — but keep the set of answers the same. A bijection: every answer still appears exactly once, just in a different slot.

Q1 gets Q3's answer. Q3 gets Q7's answer. The answers aren't random — just rearranged.

NULL B

Uniform Random Gold

For each question, pick a completely random correct answer from the answer pool — independently, with equal probability for each option.

Q1 gets a random answer. Q2 gets a different random answer. No structure, no frequency weighting.

NULL C

Random Retrieval

Replace your system's output entirely — return K random documents from the corpus instead of your actual search results.

Your search results are discarded. Five random documents from the corpus are returned instead.

NULL D NEW CONTRIBUTION

Marginal-Matched Random

Pick random answers — but weighted by how common each answer actually is in the dataset. This catches a cheater that never understands anything but always guesses the most frequent class.

If "Rigveda" appears 30% of the time, this null answers "Rigveda" ~30% of the time. Nulls A and B miss this cheater. Null D catches it.

Why Null D is the key insight

🎯

Imagine a doctor who only ever diagnoses "the flu." In flu season, they're 60% accurate — without understanding a single patient. They learned the frequency, not medicine.

🧠

Nulls A and B test against permuted and uniform-random labels — but don't simulate a system that learned the frequency distribution. They can false-positive on the frequency cheater.

🔑

Null D directly simulates the frequency cheater. If your system can't beat it, it hasn't learned anything beyond "this answer appears often." This is the novel contribution of this work.

📊 Layer 4 — How the Gate Works

Bar = real score · Red line = null score · The gap is Δ. Δ must be ≥ 0.05 to pass.

🔒 Layer 5 — The Integrity Lock

Even if the gate passes, there's a second problem: did the benchmark change between runs? If someone (or something) silently modifies the test data, your score comparisons across commits mean nothing.

📁

Walk all artifact files (.db, .json, .pkl, .npy, etc.) in the benchmark directory

🔐

SHA-256 fingerprint each file — if even one byte changes, the hash changes entirely

🌿

Bind to git commit — record the exact code version that produced this result

📋

Write lock.json — run verify_state() anytime to check nothing drifted silently

ECE analogy: It's a checksum on a firmware binary. Before flashing a device, you verify the SHA-256 of the file matches what the manufacturer published — ensuring you're flashing the right, unmodified version. The lock does the same for your benchmark.

🎓 Layer 6 — How to Explain It

The three-sentence version for academics

"Standard retrieval evaluation reports a bare aggregate metric — without any comparison to null distributions. We show this is insufficient because four distinct null predictors (permuted labels, uniform random labels, random retrieval, and marginal-matched random labels) can produce non-trivial scores without performing genuine retrieval. Our four-null Δ-gate is passed if and only if the real system exceeds all four null distributions by a threshold τ; the fourth null — marginal-matched — is a novel contribution that catches a failure mode not addressed in prior work."

📐

It's falsificationist (Popper). You're not just claiming your system works — you're trying to prove it could fail, and showing it doesn't. That's philosophically rigorous science.

🧪

It's like a clinical trial. Drug trials require control groups. You built four different types of control groups for AI evaluation, each targeting a different exploit.

⚡

Null D is genuinely novel. The marginal-matched null catches the "frequency cheater" that existing evaluation frameworks miss. That's a real gap in the literature.

🗺️ Learning Path — Concepts to Paper

Null Hypothesis Testing

Khan Academy: "Hypothesis testing" (free, 2 hrs). This is the conceptual foundation of all four nulls. No calculus needed.

nDCG & Recall@K

Search "NDCG explained visually". Understand what metric_fn measures before worrying about the gate wrapped around it.

Re-read gate.py

With the concept clear, re-read gate.py top-to-bottom. Narrate each function: "This creates a world where..."

Δ and Bootstrap CIs

Δ = real − null. τ is the threshold. Then: what does "statistically significant" mean? Learn bootstrap confidence intervals.

PREPRINT.md §3

Definition 1 + Proposition 1. Now that you know the code, the math notation will map directly to what you already built.

Story Before LaTeX

Write a 1-page plain English version first. arXiv papers that open with the problem are read. Those that open with formulas are not.

🔍 Layer 1 — The Problem