{
 "generated": "20261011T0004",
 "runs": [
  {
   "generated": "2026-10-11T00:02:40Z",
   "summary": {
    "model": "claude-haiku-4-5-20251001",
    "condition": "with_rule",
    "n": 25,
    "correct": 25,
    "errors": 0,
    "by_rule": {
     "fhfa-conforming-loan-limits-2026": "5/5",
     "fha-forward-loan-limits-2026": "5/5",
     "cfpb-hoepa-qm-thresholds-2026": "5/5",
     "va-funding-fee-2023-04-07": "5/5",
     "fha-annual-mip-ml-2023-05": "5/5"
    }
   },
   "rows": [
    {
     "rule": "fhfa-conforming-loan-limits-2026",
     "qid": "CL1",
     "question": "What is the 2026 baseline conforming loan limit for a one-unit property in most of the United States? Answer with the dollar amount first.",
     "truth": 832750,
     "kind": "usd",
     "answer": "$832,750\n\nThis is the baseline conforming loan limit for a one-unit property in most of the US for 2026, as specified in the first cell of the rule data.",
     "extracted": 832750.0,
     "correct": true
    },
    {
     "rule": "fhfa-conforming-loan-limits-2026",
     "qid": "CL2",
     "question": "What is the 2026 conforming loan limit ceiling for a one-unit property in high-cost areas of the contiguous US? Dollar amount first.",
     "truth": 1249125,
     "kind": "usd",
     "answer": "# $1,249,125\n\nThis is the high-cost ceiling for one-unit properties in the contiguous US (excluding Alaska, Hawaii, Guam, and US Virgin Islands) for 2026.",
     "extracted": 1249125.0,
     "correct": true
    },
    {
     "rule": "fhfa-conforming-loan-limits-2026",
     "qid": "CL3",
     "question": "What is the 2026 baseline conforming loan limit for a one-unit property in Alaska? Dollar amount first.",
     "truth": 1249125,
     "kind": "usd",
     "answer": "# $1,249,125\n\nAccording to the rule data, Alaska is listed under \"Alaska, Hawaii, Guam, US Virgin Islands baseline\" with a one-unit limit of $1,249,125 for 2026.\n\nThis baseline for Alaska/Hawaii/Guam/USVI is higher than the general US baseline of $832,750 due to these areas' higher cost of living.",
     "extracted": 1249125.0,
     "correct": true
    },
    {
     "rule": "fhfa-conforming-loan-limits-2026",
     "qid": "CL4",
     "question": "What is the 2026 conforming loan limit ceiling for a one-unit property in Hawaii's high-cost areas? Dollar amount first.",
     "truth": 1873675,
     "kind": "usd",
     "answer": "# $1,873,675\n\nThis is the ceiling limit for Alaska, Hawaii, Guam, and US Virgin Islands according to the rule data provided.",
     "extracted": 1873675.0,
     "correct": true
    },
    {
     "rule": "fhfa-conforming-loan-limits-2026",
     "qid": "CL5",
     "question": "What was the 2025 baseline conforming loan limit for a one-unit property (the prior year's value)? Dollar amount first.",
     "truth": 806500,
     "kind": "usd",
     "answer": "# $806,500\n\nThis is stated in the `supersedes` field: \"2025 values (baseline one-unit $806,500; ceiling $1,209,750)\"",
     "extracted": 806500.0,
     "correct": true
    },
    {
     "rule": "fha-forward-loan-limits-2026",
     "qid": "FL1",
     "question": "For an FHA case number assigned in 2026, what is the one-unit loan limit floor (low-cost areas)? Dollar amount first.",
     "truth": 541287,
     "kind": "usd",
     "answer": "# $541,287\n\nThis is the floor limit for one-unit properties under the 2026 FHA forward mortgage loan limits, applicable to case numbers assigned in 2026.",
     "extracted": 541287.0,
     "correct": true
    },
    {
     "rule": "fha-forward-loan-limits-2026",
     "qid": "FL2",
     "question": "For an FHA case number assigned in 2026, what is the one-unit loan limit ceiling (high-cost areas)? Dollar amount first.",
     "truth": 1249125,
     "kind": "usd",
     "answer": "# $1,249,125\n\nThis is the ceiling for one-unit properties under the 2026 FHA forward mortgage loan limits (effective 2026-01-01 through 2026-12-31), which applies to your October 2026 case number assignment date.",
     "extracted": 1249125.0,
     "correct": true
    },
    {
     "rule": "fha-forward-loan-limits-2026",
     "qid": "FL3",
     "question": "For an FHA case number assigned in 2026, what is the two-unit loan limit floor? Dollar amount first.",
     "truth": 693050,
     "kind": "usd",
     "answer": "# $693,050\n\nThis is the floor for a 2-unit property under the 2026 FHA forward mortgage loan limits, as specified in the \"cells\" array for units=2.",
     "extracted": 693050.0,
     "correct": true
    },
    {
     "rule": "fha-forward-loan-limits-2026",
     "qid": "FL4",
     "question": "For an FHA case number assigned in 2026, what is the four-unit loan limit ceiling? Dollar amount first.",
     "truth": 2402625,
     "kind": "usd",
     "answer": "# $2,402,625\n\nThis is the ceiling for a 4-unit property under the 2026 FHA forward mortgage loan limits, as specified in the rule data you provided.",
     "extracted": 2402625.0,
     "correct": true
    },
    {
     "rule": "fha-forward-loan-limits-2026",
     "qid": "FL5",
     "question": "What is the 2026 HECM (FHA reverse mortgage) maximum claim amount? Dollar amount first.",
     "truth": 1249125,
     "kind": "usd",
     "answer": "# $1,249,125\n\nThis is the 2026 HECM maximum claim amount according to the rule data you provided, which is effective through December 31, 2026.",
     "extracted": 1249125.0,
     "correct": true
    },
    {
     "rule": "cfpb-hoepa-qm-thresholds-2026",
     "qid": "HQ1",
     "question": "For a loan consummated in 2026, what is the HOEPA total loan amount threshold below which the points-and-fees dollar trigger applies? Dollar amount first.",
     "truth": 27592,
     "kind": "usd",
     "answer": "# $27,592\n\nThis is the HOEPA total loan amount threshold for 2026. Loans at or below this amount are subject to the points-and-fees dollar trigger of $1,380 (rather than a percentage-based calculation).",
     "extracted": 27592.0,
     "correct": true
    },
    {
     "rule": "cfpb-hoepa-qm-thresholds-2026",
     "qid": "HQ2",
     "question": "For a loan consummated in 2026, what is the HOEPA points-and-fees dollar trigger? Dollar amount first.",
     "truth": 1380,
     "kind": "usd",
     "answer": "# $1,380\n\nThis is the HOEPA points-and-fees dollar trigger for loans consummated in 2026, as specified in the rule effective from 2026-01-01 to 2026-12-31.",
     "extracted": 1380.0,
     "correct": true
    },
    {
     "rule": "cfpb-hoepa-qm-thresholds-2026",
     "qid": "HQ3",
     "question": "For a qualified mortgage consummated in 2026 with a total loan amount of $100,000, what is the maximum points and fees in dollars? Dollar amount first.",
     "truth": 4139,
     "kind": "usd",
     "answer": "# $4,139\n\nFor a loan amount of $100,000, this falls in the tier \"$82,775 to < $137,958\", which has a fixed cap of $4,139 USD in points and fees.",
     "extracted": 4139.0,
     "correct": true
    },
    {
     "rule": "cfpb-hoepa-qm-thresholds-2026",
     "qid": "HQ4",
     "question": "For a qualified mortgage consummated in 2026 with a total loan amount of $200,000, what is the points-and-fees limit as a percentage of the loan amount? Percentage first.",
     "truth": 3.0,
     "kind": "pct",
     "answer": "# 3%\n\nFor a qualified mortgage with a total loan amount of $200,000 consummated in 2026, the applicable threshold is \"QM points-and-fees cap, loan >= $137,958,\" which specifies **3% of total loan amount**.",
     "extracted": 3.0,
     "correct": true
    },
    {
     "rule": "cfpb-hoepa-qm-thresholds-2026",
     "qid": "HQ5",
     "question": "For a qualified mortgage consummated in 2026 with a total loan amount of $50,000, what is the points-and-fees limit as a percentage of the loan amount? Percentage first.",
     "truth": 5.0,
     "kind": "pct",
     "answer": "# 5%\n\nFor a loan amount of $50,000, this falls in the tier \"$27,592 to < $82,775\", which has a QM points-and-fees cap of **5% of total loan amount**.",
     "extracted": 5.0,
     "correct": true
    },
    {
     "rule": "va-funding-fee-2023-04-07",
     "qid": "VA1",
     "question": "A veteran uses the VA home loan benefit for the first time to buy a home with no down payment, closing in 2026 and not exempt. What is the funding fee percentage? Percentage first.",
     "truth": 2.15,
     "kind": "pct",
     "answer": "# 2.15%\n\nThis is a first-use purchase with less than 5% down payment, which according to the rule effective 2023-04-07 carries a 2.15% funding fee.",
     "extracted": 2.15,
     "correct": true
    },
    {
     "rule": "va-funding-fee-2023-04-07",
     "qid": "VA2",
     "question": "A veteran uses the VA home loan benefit for the second time to buy a home with no down payment, closing in 2026 and not exempt. What is the funding fee percentage? Percentage first.",
     "truth": 3.3,
     "kind": "pct",
     "answer": "# 3.3%\n\nThis is a \"subsequent use\" purchase with less than 5% down payment (no down payment falls into this category), so according to the rule, the funding fee is **3.3%**.",
     "extracted": 3.3,
     "correct": true
    },
    {
     "rule": "va-funding-fee-2023-04-07",
     "qid": "VA3",
     "question": "First-use VA purchase loan with a 10% down payment, closing in 2026, not exempt: what is the funding fee percentage? Percentage first.",
     "truth": 1.25,
     "kind": "pct",
     "answer": "# 1.25%\n\nThis matches the first row of the cells array: \"purchase or construction, first use\" with \"10% or more\" down payment, which has a fee_pct of 1.25. The rule is effective from 2023-04-07 with no end date, so it applies to your October 2026 closing.",
     "extracted": 1.25,
     "correct": true
    },
    {
     "rule": "va-funding-fee-2023-04-07",
     "qid": "VA4",
     "question": "First-use VA purchase loan with a 5% down payment, closing in 2026, not exempt: what is the funding fee percentage? Percentage first.",
     "truth": 1.5,
     "kind": "pct",
     "answer": "# 1.5%\n\nThis falls under \"purchase or construction, first use\" with \"5% to less than 10%\" down payment, which carries a 1.5% funding fee.",
     "extracted": 1.5,
     "correct": true
    },
    {
     "rule": "va-funding-fee-2023-04-07",
     "qid": "VA5",
     "question": "What is the VA funding fee percentage on an Interest Rate Reduction Refinance Loan (IRRRL) closing in 2026 for a non-exempt borrower? Percentage first.",
     "truth": 0.5,
     "kind": "pct",
     "answer": "# 0.5%\n\nThis rate applies to IRRRL (streamline refinance) loans regardless of first/subsequent use or down payment, and remains in effect as of October 2026 (the rule has no end date specified).",
     "extracted": 0.5,
     "correct": true
    },
    {
     "rule": "fha-annual-mip-ml-2023-05",
     "qid": "MP1",
     "question": "For an FHA purchase loan with a base loan amount of $300,000, LTV 96.5%, 30-year term, case number assigned this year, what is the annual MIP rate? Percentage first.",
     "truth": 0.55,
     "kind": "pct",
     "answer": "# 0.55%\n\n**Explanation:**\n\nFor your loan parameters:\n- **Term:** 30 years \u2192 \"gt15\" (over 15 years)\n- **Base loan amount:** $300,000 \u2192 \"le726200\" (at or below $726,200)\n- **LTV:** 96.5% \u2192 \"gt95\" (greater than 95%)\n- **Case number assigned:** 2026 \u2192 on or after 2023-03-20 \u2713\n\nLooking up the matching cell in the rule:\n- term: gt15\n- base_loan: le726200\n- ltv: gt95\n- **annual_mip_rate: 0.0055 = 0.55%**",
     "extracted": 0.55,
     "correct": true
    },
    {
     "rule": "fha-annual-mip-ml-2023-05",
     "qid": "MP2",
     "question": "For an FHA purchase loan with a base loan amount of $800,000, LTV 96.5%, 30-year term, case number assigned this year, what is the annual MIP rate? Percentage first.",
     "truth": 0.75,
     "kind": "pct",
     "answer": "# Annual MIP Rate: 0.75%\n\n**Breakdown:**\n\n- **Base loan amount**: $800,000 (exceeds $726,200 threshold) \u2192 `gt726200`\n- **LTV**: 96.5% (exceeds 95%) \u2192 `ltv: gt95`\n- **Term**: 30 years (over 15 years) \u2192 `gt15`\n- **Case number assigned**: October 2026 (on or after 2023-03-20) \u2713\n\n**Applicable cell**: term=gt15, base_loan=gt726200, ltv=gt95 \u2192 **annual_mip_rate = 0.0075 = 0.75%**\n\nThis rate applies to the base loan amount of $800,000 annually, per HUD Mortgagee Letter 2023-05.",
     "extracted": 0.75,
     "correct": true
    },
    {
     "rule": "fha-annual-mip-ml-2023-05",
     "qid": "MP3",
     "question": "For an FHA purchase loan with a base loan amount of $300,000, LTV 85%, 15-year term, case number assigned this year, what is the annual MIP rate? Percentage first.",
     "truth": 0.15,
     "kind": "pct",
     "answer": "# 0.15%\n\nBased on the rule:\n- **Term**: le15 (15 years or less) \u2713\n- **Base loan**: le726200 ($300,000 is below $726,200) \u2713\n- **LTV**: le90 (85% is \u226490%) \u2713\n- **Case number assigned**: October 2026 is on or after 2023-03-20 \u2713\n\nThis matches the cell with `annual_mip_rate: 0.0015`, which equals **0.15%** or **15 basis points**.",
     "extracted": 0.15,
     "correct": true
    },
    {
     "rule": "fha-annual-mip-ml-2023-05",
     "qid": "MP4",
     "question": "For an FHA purchase loan with a base loan amount of $800,000, LTV 95%, 15-year term, case number assigned this year, what is the annual MIP rate? Percentage first.",
     "truth": 0.65,
     "kind": "pct",
     "answer": "# 0.65%\n\n**Explanation:**\n\nLooking up the applicable cell in the rule:\n- **term**: \"le15\" (15-year term qualifies as \u226415 years)\n- **base_loan**: \"gt726200\" ($800,000 exceeds $726,200)\n- **ltv**: \"gt90\" (95% LTV is >90%)\n\nThe matching cell shows:\n- **annual_mip_rate**: 0.0065 = **0.65%**\n\nThis applies because the case number was assigned in 2026, which is on or after the effective date of 2023-03-20, and all other qualifiers are met.",
     "extracted": 0.65,
     "correct": true
    },
    {
     "rule": "fha-annual-mip-ml-2023-05",
     "qid": "MP5",
     "question": "What is the FHA upfront mortgage insurance premium rate for a forward purchase loan with a case number assigned this year? Percentage first.",
     "truth": 1.75,
     "kind": "pct",
     "answer": "# 1.75%\n\nAccording to the rule, the **upfront_mip_rate is 0.0175** (1.75%), which applies to all FHA forward mortgages with case numbers assigned on or after March 20, 2023\u2014including those assigned in October 2026.\n\nThis upfront premium is separate from the annual MIP, which varies by loan term, base loan amount, and LTV ratio.",
     "extracted": 1.75,
     "correct": true
    }
   ]
  },
  {
   "generated": "2026-10-11T00:04:08Z",
   "summary": {
    "model": "claude-sonnet-4-5-20250929",
    "condition": "with_rule",
    "n": 25,
    "correct": 24,
    "errors": 0,
    "by_rule": {
     "fhfa-conforming-loan-limits-2026": "5/5",
     "fha-forward-loan-limits-2026": "5/5",
     "cfpb-hoepa-qm-thresholds-2026": "5/5",
     "va-funding-fee-2023-04-07": "5/5",
     "fha-annual-mip-ml-2023-05": "4/5"
    }
   },
   "rows": [
    {
     "rule": "fhfa-conforming-loan-limits-2026",
     "qid": "CL1",
     "question": "What is the 2026 baseline conforming loan limit for a one-unit property in most of the United States? Answer with the dollar amount first.",
     "truth": 832750,
     "kind": "usd",
     "answer": "$832,750\n\nThis is the 2026 baseline conforming loan limit for a one-unit property in most of the United States, according to the FHFA rule effective from January 1, 2026 through December 31, 2026.",
     "extracted": 832750.0,
     "correct": true
    },
    {
     "rule": "fhfa-conforming-loan-limits-2026",
     "qid": "CL2",
     "question": "What is the 2026 conforming loan limit ceiling for a one-unit property in high-cost areas of the contiguous US? Dollar amount first.",
     "truth": 1249125,
     "kind": "usd",
     "answer": "$1,249,125\n\nThis is the 2026 conforming loan limit ceiling for one-unit properties in high-cost areas of the contiguous United States (excluding Alaska, Hawaii, Guam, and the US Virgin Islands, which have a higher ceiling of $1,873,675).",
     "extracted": 1249125.0,
     "correct": true
    },
    {
     "rule": "fhfa-conforming-loan-limits-2026",
     "qid": "CL3",
     "question": "What is the 2026 baseline conforming loan limit for a one-unit property in Alaska? Dollar amount first.",
     "truth": 1249125,
     "kind": "usd",
     "answer": "$1,249,125\n\nAccording to the rule data, Alaska has a special baseline conforming loan limit of $1,249,125 for one-unit properties in 2026, which is 150% of the standard baseline limit. This is specified in the cell for \"Alaska, Hawaii, Guam, US Virgin Islands baseline\" with 1 unit.",
     "extracted": 1249125.0,
     "correct": true
    },
    {
     "rule": "fhfa-conforming-loan-limits-2026",
     "qid": "CL4",
     "question": "What is the 2026 conforming loan limit ceiling for a one-unit property in Hawaii's high-cost areas? Dollar amount first.",
     "truth": 1873675,
     "kind": "usd",
     "answer": "$1,873,675\n\nThis is the 2026 conforming loan limit ceiling for one-unit properties in Alaska, Hawaii, Guam, and the US Virgin Islands, as specified in the rule effective from January 1, 2026 through December 31, 2026.",
     "extracted": 1873675.0,
     "correct": true
    },
    {
     "rule": "fhfa-conforming-loan-limits-2026",
     "qid": "CL5",
     "question": "What was the 2025 baseline conforming loan limit for a one-unit property (the prior year's value)? Dollar amount first.",
     "truth": 806500,
     "kind": "usd",
     "answer": "$806,500\n\nThis information is found in the \"supersedes\" field of the rule, which states: \"2025 values (baseline one-unit $806,500; ceiling $1,209,750)\".",
     "extracted": 806500.0,
     "correct": true
    },
    {
     "rule": "fha-forward-loan-limits-2026",
     "qid": "FL1",
     "question": "For an FHA case number assigned in 2026, what is the one-unit loan limit floor (low-cost areas)? Dollar amount first.",
     "truth": 541287,
     "kind": "usd",
     "answer": "**$541,287**\n\nThis is the one-unit floor for FHA forward mortgage loan limits for case numbers assigned in 2026, applicable in low-cost areas.",
     "extracted": 541287.0,
     "correct": true
    },
    {
     "rule": "fha-forward-loan-limits-2026",
     "qid": "FL2",
     "question": "For an FHA case number assigned in 2026, what is the one-unit loan limit ceiling (high-cost areas)? Dollar amount first.",
     "truth": 1249125,
     "kind": "usd",
     "answer": "$1,249,125\n\nThis is the one-unit ceiling for FHA forward mortgage loan limits for case numbers assigned in 2026, applicable to high-cost areas.",
     "extracted": 1249125.0,
     "correct": true
    },
    {
     "rule": "fha-forward-loan-limits-2026",
     "qid": "FL3",
     "question": "For an FHA case number assigned in 2026, what is the two-unit loan limit floor? Dollar amount first.",
     "truth": 693050,
     "kind": "usd",
     "answer": "$693,050\n\nThis is the two-unit floor for FHA forward mortgage loan limits for case numbers assigned in 2026, according to the rule data provided.",
     "extracted": 693050.0,
     "correct": true
    },
    {
     "rule": "fha-forward-loan-limits-2026",
     "qid": "FL4",
     "question": "For an FHA case number assigned in 2026, what is the four-unit loan limit ceiling? Dollar amount first.",
     "truth": 2402625,
     "kind": "usd",
     "answer": "$2,402,625\n\nThis is the four-unit ceiling for FHA forward mortgage loan limits for case numbers assigned in 2026, according to the rule data from HUD/FHA Mortgagee Letter 2025-23.",
     "extracted": 2402625.0,
     "correct": true
    },
    {
     "rule": "fha-forward-loan-limits-2026",
     "qid": "FL5",
     "question": "What is the 2026 HECM (FHA reverse mortgage) maximum claim amount? Dollar amount first.",
     "truth": 1249125,
     "kind": "usd",
     "answer": "$1,249,125\n\nThis is the HECM (Home Equity Conversion Mortgage) maximum claim amount for 2026, as specified in the rule for case numbers assigned on or after January 1, 2026.",
     "extracted": 1249125.0,
     "correct": true
    },
    {
     "rule": "cfpb-hoepa-qm-thresholds-2026",
     "qid": "HQ1",
     "question": "For a loan consummated in 2026, what is the HOEPA total loan amount threshold below which the points-and-fees dollar trigger applies? Dollar amount first.",
     "truth": 27592,
     "kind": "usd",
     "answer": "**$27,592**\n\nThis is the HOEPA total loan amount threshold for loans consummated in 2026. For loans below this amount, the points-and-fees dollar trigger of $1,380 applies when determining whether a mortgage is a high-cost mortgage under HOEPA.",
     "extracted": 27592.0,
     "correct": true
    },
    {
     "rule": "cfpb-hoepa-qm-thresholds-2026",
     "qid": "HQ2",
     "question": "For a loan consummated in 2026, what is the HOEPA points-and-fees dollar trigger? Dollar amount first.",
     "truth": 1380,
     "kind": "usd",
     "answer": "**$1,380**\n\nThis is the HOEPA points-and-fees dollar trigger for loans consummated in 2026, as specified in the CFPB's 2026 annual threshold adjustments (effective January 1, 2026 through December 31, 2026).",
     "extracted": 1380.0,
     "correct": true
    },
    {
     "rule": "cfpb-hoepa-qm-thresholds-2026",
     "qid": "HQ3",
     "question": "For a qualified mortgage consummated in 2026 with a total loan amount of $100,000, what is the maximum points and fees in dollars? Dollar amount first.",
     "truth": 4139,
     "kind": "usd",
     "answer": "**$4,139**\n\nFor a qualified mortgage consummated in 2026 with a total loan amount of $100,000, the loan falls into the tier \"$82,775 to < $137,958\" according to the rule.\n\nFor this tier, the QM points-and-fees cap is a flat dollar amount of **$4,139**.",
     "extracted": 4139.0,
     "correct": true
    },
    {
     "rule": "cfpb-hoepa-qm-thresholds-2026",
     "qid": "HQ4",
     "question": "For a qualified mortgage consummated in 2026 with a total loan amount of $200,000, what is the points-and-fees limit as a percentage of the loan amount? Percentage first.",
     "truth": 3.0,
     "kind": "pct",
     "answer": "**3%**\n\nFor a qualified mortgage consummated in 2026 with a total loan amount of $200,000, the points-and-fees limit is **3% of the total loan amount**.\n\nThis is because $200,000 is greater than or equal to $137,958, which places it in the tier: \"QM points-and-fees cap, loan >= $137,958\" with a cap of \"3% of total loan amount.\"\n\nIn dollar terms, this equals a maximum of $6,000 in points and fees ($200,000 \u00d7 0.03).",
     "extracted": 3.0,
     "correct": true
    },
    {
     "rule": "cfpb-hoepa-qm-thresholds-2026",
     "qid": "HQ5",
     "question": "For a qualified mortgage consummated in 2026 with a total loan amount of $50,000, what is the points-and-fees limit as a percentage of the loan amount? Percentage first.",
     "truth": 5.0,
     "kind": "pct",
     "answer": "**5%**\n\nFor a qualified mortgage consummated in 2026 with a total loan amount of $50,000, the points-and-fees limit is **5% of the total loan amount**.\n\nThis loan falls into the tier \"$27,592 to < $82,775\" according to the 2026 thresholds, which specifies a cap of 5% of the total loan amount.\n\nIn dollar terms, this equals $2,500 (5% \u00d7 $50,000).",
     "extracted": 5.0,
     "correct": true
    },
    {
     "rule": "va-funding-fee-2023-04-07",
     "qid": "VA1",
     "question": "A veteran uses the VA home loan benefit for the first time to buy a home with no down payment, closing in 2026 and not exempt. What is the funding fee percentage? Percentage first.",
     "truth": 2.15,
     "kind": "pct",
     "answer": "**2.15%**\n\nAccording to the rule, for a purchase with first use of the VA home loan benefit and a down payment of less than 5% (which includes no down payment), the funding fee percentage is 2.15%.\n\nThis rate has been effective since April 7, 2023, and applies to the October 2026 closing date specified in your scenario.",
     "extracted": 2.15,
     "correct": true
    },
    {
     "rule": "va-funding-fee-2023-04-07",
     "qid": "VA2",
     "question": "A veteran uses the VA home loan benefit for the second time to buy a home with no down payment, closing in 2026 and not exempt. What is the funding fee percentage? Percentage first.",
     "truth": 3.3,
     "kind": "pct",
     "answer": "**3.3%**\n\nAccording to the rule, for a loan that is:\n- **\"purchase or construction, subsequent use\"** (second time using VA benefit)\n- **\"less than 5%\"** down payment (no down payment = 0%)\n- Closing in 2026 (after the 2023-04-07 effective date)\n- Not exempt\n\nThe funding fee percentage is **3.3%**.",
     "extracted": 3.3,
     "correct": true
    },
    {
     "rule": "va-funding-fee-2023-04-07",
     "qid": "VA3",
     "question": "First-use VA purchase loan with a 10% down payment, closing in 2026, not exempt: what is the funding fee percentage? Percentage first.",
     "truth": 1.25,
     "kind": "pct",
     "answer": "**1.25%**\n\nBased on the rule effective from 2023-04-07 (still in effect as of October 2026, since \"effective_to\" is null), a first-use VA purchase loan with a 10% down payment falls under:\n\n- **use**: \"purchase or construction, first use\"\n- **down_payment**: \"10% or more\"\n- **fee_pct**: 1.25",
     "extracted": 1.25,
     "correct": true
    },
    {
     "rule": "va-funding-fee-2023-04-07",
     "qid": "VA4",
     "question": "First-use VA purchase loan with a 5% down payment, closing in 2026, not exempt: what is the funding fee percentage? Percentage first.",
     "truth": 1.5,
     "kind": "pct",
     "answer": "**1.5%**\n\nAccording to the rule effective from 2023-04-07 (which is still in effect as of October 2026, since \"effective_to\" is null), a first-use VA purchase loan with a down payment of \"5% to less than 10%\" has a funding fee of 1.5%.\n\nThe relevant cell from the rule:\n```json\n{\n  \"use\": \"purchase or construction, first use\",\n  \"down_payment\": \"5% to less than 10%\",\n  \"fee_pct\": 1.5\n}\n```",
     "extracted": 1.5,
     "correct": true
    },
    {
     "rule": "va-funding-fee-2023-04-07",
     "qid": "VA5",
     "question": "What is the VA funding fee percentage on an Interest Rate Reduction Refinance Loan (IRRRL) closing in 2026 for a non-exempt borrower? Percentage first.",
     "truth": 0.5,
     "kind": "pct",
     "answer": "**0.5%**\n\nAccording to the rule, the VA funding fee for an IRRRL (streamline refinance) is 0.5% for non-exempt borrowers. Since this rule became effective on 2023-04-07 with no end date (\"effective_to\": null), it remains in effect in October 2026.",
     "extracted": 0.5,
     "correct": true
    },
    {
     "rule": "fha-annual-mip-ml-2023-05",
     "qid": "MP1",
     "question": "For an FHA purchase loan with a base loan amount of $300,000, LTV 96.5%, 30-year term, case number assigned this year, what is the annual MIP rate? Percentage first.",
     "truth": 0.55,
     "kind": "pct",
     "answer": "Based on the rule data provided:\n\n**0.55%** (55 basis points)\n\n**Explanation:**\n\nFor your FHA purchase loan:\n- **Case number assigned**: October 2026 (on or after 2023-03-20) \u2713\n- **Base loan amount**: $300,000 (\u2264 $726,200) \u2192 \"le726200\"\n- **LTV**: 96.5% (> 95%) \u2192 \"gt95\"\n- **Term**: 30 years (> 15 years) \u2192 \"gt15\"\n\nThis matches the cell:\n```json\n{\n  \"term\": \"gt15\",\n  \"base_loan\": \"le726200\",\n  \"ltv\": \"gt95\",\n  \"annual_mip_bps\": 55,\n  \"annual_mip_rate\": 0.0055\n}\n```\n\n**Annual MIP rate: 0.55%** of the base loan amount, per HUD Mortgagee Letter 2023-05.",
     "extracted": 0.55,
     "correct": true
    },
    {
     "rule": "fha-annual-mip-ml-2023-05",
     "qid": "MP2",
     "question": "For an FHA purchase loan with a base loan amount of $800,000, LTV 96.5%, 30-year term, case number assigned this year, what is the annual MIP rate? Percentage first.",
     "truth": 0.75,
     "kind": "pct",
     "answer": "Based on the rule data provided:\n\n**0.75%**\n\n**Explanation:**\n\nFor your FHA purchase loan:\n- **Term**: 30 years = gt15 (greater than 15 years)\n- **Base loan amount**: $800,000 = gt726200 (above $726,200)\n- **LTV**: 96.5% = gt95 (greater than 95%)\n- **Case number assigned**: October 2026 = on or after 2023-03-20 \u2713\n\nThis matches the cell:\n```json\n{\n  \"term\": \"gt15\",\n  \"base_loan\": \"gt726200\",\n  \"ltv\": \"gt95\",\n  \"annual_mip_bps\": 75,\n  \"annual_mip_rate\": 0.0075\n}\n```\n\n**The annual MIP rate is 0.75% (75 basis points) of the base loan amount.**\n\nFor an FHA forward loan with a case number assigned on or after 20 March 2023, a base loan amount above $726,200 and LTV greater than 95% on a term greater than 15 years, the annual MIP is 0.75% of the base loan amount (HUD Mortgagee Letter 2023-05).",
     "extracted": 0.75,
     "correct": true
    },
    {
     "rule": "fha-annual-mip-ml-2023-05",
     "qid": "MP3",
     "question": "For an FHA purchase loan with a base loan amount of $300,000, LTV 85%, 15-year term, case number assigned this year, what is the annual MIP rate? Percentage first.",
     "truth": 0.15,
     "kind": "pct",
     "answer": "Based on the rule data provided:\n\n**0.15%**\n\n**Explanation:**\n- Case number assigned in October 2026 (after 2023-03-20) \u2713\n- Base loan amount: $300,000 (\u2264 $726,200) \u2192 \"le726200\"\n- LTV: 85% (\u2264 90%) \u2192 \"le90\"\n- Term: 15 years \u2192 \"le15\"\n\nLooking up the matching cell:\n```json\n{\n  \"term\": \"le15\",\n  \"base_loan\": \"le726200\",\n  \"ltv\": \"le90\",\n  \"annual_mip_bps\": 15,\n  \"annual_mip_rate\": 0.0015\n}\n```\n\n**The annual MIP rate is 0.15% (15 basis points) of the base loan amount.**",
     "extracted": 0.15,
     "correct": true
    },
    {
     "rule": "fha-annual-mip-ml-2023-05",
     "qid": "MP4",
     "question": "For an FHA purchase loan with a base loan amount of $800,000, LTV 95%, 15-year term, case number assigned this year, what is the annual MIP rate? Percentage first.",
     "truth": 0.65,
     "kind": "pct",
     "answer": "Based on the rule data provided:\n\n**0.40%** (40 basis points)\n\n**Explanation:**\n- Case number assigned in October 2026 \u2192 on or after 2023-03-20 \u2713\n- Base loan amount: $800,000 \u2192 **gt726200** (greater than $726,200)\n- Term: 15 years \u2192 **le15** (15 years or less)\n- LTV: 95% \u2192 **78to90** (falls in the >78% to \u226490% band for loans >$726,200 with \u226415 year terms)\n\nLooking at the matching cell:\n```json\n{\n  \"term\": \"le15\",\n  \"base_loan\": \"gt726200\",\n  \"ltv\": \"78to90\",\n  \"annual_mip_bps\": 40,\n  \"annual_mip_rate\": 0.004\n}\n```\n\n**The annual MIP rate is 0.40% (0.004) of the base loan amount.**",
     "extracted": 0.4,
     "correct": false
    },
    {
     "rule": "fha-annual-mip-ml-2023-05",
     "qid": "MP5",
     "question": "What is the FHA upfront mortgage insurance premium rate for a forward purchase loan with a case number assigned this year? Percentage first.",
     "truth": 1.75,
     "kind": "pct",
     "answer": "# FHA Upfront MIP Rate\n\n**1.75%**\n\nAccording to the rule data (HUD Mortgagee Letter 2023-05, effective from March 20, 2023, with no end date), the upfront mortgage insurance premium rate is:\n\n```json\n\"upfront_mip_rate\": 0.0175\n```\n\nThis applies to forward mortgages with case numbers assigned on or after March 20, 2023. Since your case number was assigned in October 2026, and this rule remains effective (no `effective_to` date is specified), the upfront MIP rate is **1.75%** of the base loan amount.\n\n**Note:** This rate is separate from the annual MIP, which varies based on loan term, base loan amount, and LTV ratio.",
     "extracted": 1.75,
     "correct": true
    }
   ]
  }
 ],
 "summary": [
  {
   "model": "claude-haiku-4-5-20251001",
   "condition": "with_rule",
   "n": 25,
   "correct": 25,
   "errors": 0,
   "by_rule": {
    "fhfa-conforming-loan-limits-2026": "5/5",
    "fha-forward-loan-limits-2026": "5/5",
    "cfpb-hoepa-qm-thresholds-2026": "5/5",
    "va-funding-fee-2023-04-07": "5/5",
    "fha-annual-mip-ml-2023-05": "5/5"
   }
  },
  {
   "model": "claude-sonnet-4-5-20250929",
   "condition": "with_rule",
   "n": 25,
   "correct": 24,
   "errors": 0,
   "by_rule": {
    "fhfa-conforming-loan-limits-2026": "5/5",
    "fha-forward-loan-limits-2026": "5/5",
    "cfpb-hoepa-qm-thresholds-2026": "5/5",
    "va-funding-fee-2023-04-07": "5/5",
    "fha-annual-mip-ml-2023-05": "4/5"
   }
  }
 ],
 "regraded": "2026-10-11: percentage extractor skips percentages restated from the question; answers unchanged"
}