Sample · Evidence Brief · five-claim audit

Five claims from one AI answer, checked against the federal record

This is what a Claim Audit returns, worked in full on a real case. On 2026-11-12 three questions about 2025 mortgage denial rates and a HUD monthly count were put to Gemini. The figures it gave are below as it gave them. The site was not named in the questions and was not cited in the answers.

Result: 1 PASS · 1 NEEDS_QUALIFIER · 3 OUT_OF_SCOPE.
One figure is right and fully stated. Three denial rates cannot be reproduced from the 2025 file under any published definition. One official count is right as first printed and missing its later restatement.

How each claim was checked

A quoted denial rate is placed on the denominator map: every combination of program, loan purpose, scope and base computed from the 2025 HMDA file, 96 cells for FHA and VA. A cell within 0.15 points is a match. A match shows which definition can produce the number, not which one its publisher used. No match means another year, another filter, or a figure this file does not reproduce. The HUD count is compared with the editions restated on the revision ledger. OUT_OF_SCOPE here means the sentence goes past what the public file can support as stated; it is a statement about the sentence, not about its author.

PASSClaim 1. 22.0% of FHA applications were denied in 2025; the answer describes the base as applications that reached a decision.
Attributed in the answer to: Inside Mortgage Finance (named, no link)
Placed on the record
FHA, all loan purposes, first-lien owner-occupied site-built 1–4 unit, decisioned applications: 247,723 / 1,126,040 = 22.0%
FHA, all loan purposes, all rows, decisioned applications: 262,250 / 1,187,606 = 22.08%
The figure sits on the published definition to within a tenth (22.08%). Universe, denominator and year are all stated in the answer. The source is a trade publication behind a paywall, so the match is to the federal file, not to that article's method.

Safe rewrite: In 2025, 22.1% of decisioned FHA applications were denied (262,250 of 1,187,606). ⟦FRC:national-fha-denial-rate-2025:22.1%:ebf6b00b⟧
OUT_OF_SCOPEClaim 2. 14.4% of FHA purchase applications were denied in 2025.
Attributed in the answer to: a consumer finance page that names no source
Placed on the record
No cell of the 2025 file within 0.15 points of 14.4% (48 cells searched).
No definition of an FHA purchase denial rate in the 2025 file produces 14.4%. Among decisioned FHA purchase applications the rate is 12.78% (96,186 of 752,770). The page the answer relied on gives no source, year file or denominator, so the figure cannot be traced. That is not proof it is wrong; it is unverifiable as stated.

Safe rewrite: In 2025, 12.8% of decisioned FHA home-purchase applications were denied (96,186 of 752,770). ⟦FRC:purpose-fha-purchase-denial-rate-2025:12.8%:25976868⟧
OUT_OF_SCOPEClaim 3. 12.23% is the ‘overall’ FHA denial rate for 2025.
Attributed in the answer to: LendingTree (named, no link)
Placed on the record
No cell of the 2025 file within 0.15 points of 12.23% (48 cells searched).
No cell reproduces 12.23%, and ‘overall’ contradicts claim 1 from the same engine minutes earlier (22.0% for all FHA applications). The page it traces to states the figure ‘based on 2025 HMDA data’ with no denominator or method. The all-purpose rate among decisioned applications is 22.08%.

Safe rewrite: Drop the figure, or attribute it to its publisher as a number whose definition is not published. Do not call it the overall FHA denial rate.
OUT_OF_SCOPEClaim 4. 7.89% of VA applications were denied in 2025 ‘across all loan purposes’.
Attributed in the answer to: LendingTree (named, no link)
Placed on the record
No cell of the 2025 file within 0.15 points of 7.89% (48 cells searched).
No cell reproduces 7.89%. Across all loan purposes the VA rate among decisioned applications is 16.15% (115,026 of 712,121), about twice the stated figure. The answer's separate ‘8.0% home purchase’ does have a home in the file (VA, home purchase, first-lien owner-occupied site-built 1–4 unit, decisioned applications: 28,831 / 357,537 = 8.06%), which suggests a purchase-only figure was relabelled as all purposes. That is an inference, not a finding.

Safe rewrite: In 2025, 16.1% of decisioned VA applications were denied (115,026 of 712,121). ⟦FRC:va-national-denial-rate-2025:16.1%:32bdf09e⟧
NEEDS_QUALIFIERClaim 5. HUD endorsed 65,328 FHA purchase loans in July 2026; later editions ‘are not publicly available’, so a revision cannot be confirmed.
Attributed in the answer to: HUD FHA Single Family Production Report, July 2026 edition
Placed on the record
July 2026 report prints 65,328. Aug 2026 report prints 65,327 for the same month (document SHA-256 a95169f6e28c…).
The count is correct as first printed. The statement about availability is not: the next edition is public and restates July as 65,327, without marking the change. A figure from a monthly report needs its edition next to it.

Safe rewrite: HUD’s July 2026 production report printed 65,328 FHA purchase endorsements for July; the August 2026 report restated the month as 65,327. ⟦FRC:official-hud-fhaprod-2026-07-purchase-count:65,328:35e63550⟧

What this brief does not say

It does not say the engine or the pages it relied on acted in bad faith, and it does not say an unplaceable figure is false: only that it cannot be traced to the public file as stated. It says nothing about any applicant, and it does not rank programs or lenders. The inference in claim 4 is marked as an inference.

The logged answer, with what was checked and how: eval/answer-engine-log/2026-11-12-gemini.json. Receipts in the rewrites verify at /verify. This page is generated by scripts/build_sample_brief.py from those files.

FinanceRateCalc · Measured, not assumed. · Request an audit · Corrections