Measured, not claimed
The false-positive benchmark
A scanner that cries wolf is worse than none, so this page publishes how often xlogs does. Two corpora of pages and repositories a careful reviewer would call clean, each written by reading a detector and attacking a construct in it, are run through the shipped scanner. What came back is below, with the misses named.
The result, 2026-09-04, engine 636f647
Read it as: of 45 clean inputs whose expected outcome is settled, 45 produced no loud finding. The 13 open cases are product decisions not yet made and count for neither side. The controls are real leaks and real exposures planted beside the clean cases; if any stopped firing, a clean score here would mean nothing.
Round 1: secret patterns
- 10 cases, of which 0 are open judgement calls and 10 are decided.
- 10 of the 10 decided cases produced no loud finding; 0 still do.
- 1 of 1 true-positive controls fired.
Round 2: every other detector
- 48 cases, of which 13 are open judgement calls and 35 are decided.
- 35 of the 35 decided cases produced no loud finding; 0 still do.
- 20 of 20 true-positive controls fired.
Open judgement calls, by case id
Each is a product decision, not a bug; the ledger id follows the case where a row exists. When decided, the case becomes a permanent regression guard in whichever direction the decision goes.
env-200-only-public-build-varsXL-206sourcemap-nosourcesXL-203sourcemap-of-vendored-public-libraryXL-204supabase-reference-tables-onlyXL-200cname-to-github-pages-healthy-page-quoting-fingerprintXL-205deploy-script-uses-own-ssh-keyXL-207imds-instance-id-on-ec2XL-207slack-deploy-notificationXL-208dockerfile-rustup-official-installerXL-201terraform-https-ingress-for-public-webXL-202claude-md-warning-that-quotes-an-injectionXL-209readme-token-format-without-placeholder-wordXL-209setup-py-with-build-extXL-222
What this number is, and is not
- It is a floor. Every case was written to make a detector fire on something clean. A random sample of real sites is a different instrument and would give a different figure.
- It is not a score. There is no single percentage here on purpose. The cases, the denominator and the misses are published so you can compute whatever you find meaningful and see exactly what it leaves out.
- It changes when the engine changes. Each run is stamped with the commit. Fixing a detector alters comparability, so earlier runs are kept beside this one rather than overwritten.
- The corpora are in the repository.
test/false-positive-corpus.test.mjsandtest/false-positive-corpus-2.test.mjsrun on every commit; round 2 stays outside the default suite until its open calls are decided, so that a red gate is never quietly allowlisted.