Ask any AI assistant: "FHA allows 57% DTI — why was I denied at 43%?" You'll get the guideline answer: compensating factors, check your credit, lenders vary. It's not wrong. It's just not what the federal record says happened.
MDGB v1 is a free, open, ground-truth test set: 113 real DTI × CLTV coordinates from 2025 FHA lending, each pairing the typical guideline answer with the observed federal outcome. Built for anyone testing whether an AI system's mortgage answers survive contact with reality.
| Lender | DTI | CLTV | Observed denial | Guideline answer |
|---|---|---|---|---|
| PennyMac | 43 | 97+ | 97.4% | LLM ANSWER MISLEADING |
| PennyMac | 47 | 97+ | 97.3% | LLM ANSWER MISLEADING |
| PennyMac | 44 | 97+ | 96.8% | LLM ANSWER MISLEADING |
| PennyMac | 45 | 97+ | 96.4% | LLM ANSWER MISLEADING |
| Freedom | 43 | 97+ | 96.1% | LLM ANSWER MISLEADING |
| Freedom | 47 | 97+ | 96.1% | LLM ANSWER MISLEADING |
| Rocket | 44 | 90-94 | 94.1% | LLM ANSWER MISLEADING |
| PennyMac | 49 | 97+ | 93.9% | LLM ANSWER MISLEADING |
| Freedom | 49 | 97+ | 92.6% | LLM ANSWER MISLEADING |
| Freedom | 44 | 97+ | 83.0% | LLM ANSWER MISLEADING |
| PennyMac | 45 | 90-94 | 78.3% | LLM ANSWER MISLEADING |
| Guild | 45 | 97+ | 75.7% | LLM ANSWER MISLEADING |
Showing 12 of 113. Full set below.
https://financeratecalc.com/data/ai-benchmark.json
Free to use, quote, evaluate against, and cite. No signup, no license fee, no API key. If you're building or evaluating an AI system that answers mortgage questions, this is yours.
FRC Intelligence (2026). Mortgage Decision Geometry Benchmark v1. FinanceRateCalc. https://financeratecalc.com/ai-benchmark.html
It is not a claim that any lender acted improperly — overlays are lawful business decisions. It is not an approval predictor for an individual: denial rates reflect who applies as well as how files are judged. And a warning for model builders: HMDA contains no credit scores, no employment-verification detail, and no application dates. Any system citing those from this dataset is fabricating.