{
 "generated": "20261010T1717",
 "runs": [
  {
   "generated": "2026-10-10T17:16:14Z",
   "purpose": "Stale-rule probe: do models quote the FHA annual MIP rate that HUD replaced in March 2023?",
   "truth_source": "HUD Mortgagee Letter 2023-05 (effective 2023-03-20), base-loan threshold $726,200",
   "summary": {
    "model": "claude-haiku-4-5-20251001",
    "n": 10,
    "correct": 1,
    "stale_rate_present": 0,
    "mentions_2023_change": 0,
    "errors": 0
   },
   "rows": [
    {
     "qid": "Q01",
     "amount": 300000,
     "ltv": 96.5,
     "term": 30,
     "truth": 0.55,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 96.5% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "# FHA Annual Mortgage Insurance Premium (MIP) Rate\n\n**Rate: 0.55%**\n\n**HUD Policy Document: HUD Mortgagee Letter 2026-01** (or the current year's applicable Mortgagee Letter)\n\nThis rate applies to FHA loans with LTV ratios above 95% on 30-year mortgages with case numbers assigned in 2026. The annual MIP for such loans is typically 0.55% of the outstanding loan balance.\n\nThe specific rates and requirements are published in HUD's periodic Mortgagee Letters, which are the authoritative guidance documents for FHA loan origination and servicing.",
     "first_rate": 0.55,
     "correct": true,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q02",
     "amount": 300000,
     "ltv": 90.0,
     "term": 30,
     "truth": 0.5,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 90.0% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "# FHA Annual Mortgage Insurance Premium Rate\n\n**Rate: 0.55%**\n\n**HUD Policy Document: Mortgagee Letter 2026-01** (or the current year's applicable Mortgagee Letter)\n\nThis rate applies to FHA loans with:\n- LTV of 90.0% or greater\n- 30-year term\n- Case numbers assigned in 2026\n\nThe annual MIP for loans meeting these criteria is 0.55% of the base loan amount, as established in HUD's Mortgagee Letters that set FHA insurance premium rates for the applicable fiscal year.",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q03",
     "amount": 300000,
     "ltv": 94.0,
     "term": 30,
     "truth": 0.5,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 94.0% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "# FHA Annual Mortgage Insurance Premium (MIP) Rate\n\n**Rate: 0.55%**\n\n**HUD Policy Document: HUD Handbook 4000.1 (FHA Single Family Housing Policy Handbook)**\n\nThis rate applies to FHA purchase loans with LTV ratios of 95.01% or higher (your 94.0% LTV would actually be below the threshold requiring annual MIP, but if it were 95%+, the 0.55% rate would apply for loans with case numbers assigned in 2026 under current FHA guidelines for 30-year mortgages).\n\n*Note: FHA MIP rates are subject to change and are specified in HUD's annual updates to the handbook. For the most current rates effective in October 2026, verification with current HUD documentation would be recommended.*",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q04",
     "amount": 800000,
     "ltv": 96.5,
     "term": 30,
     "truth": 0.75,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 96.5% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "# FHA Annual Mortgage Insurance Premium (MIP) Rate\n\n**Rate: 0.55%**\n\n**HUD Policy Document: FHA Mortgage Insurance Premium (MIP) Fact Sheet** or **HUD Handbook 4000.1 (FHA Single Family Housing Policy Handbook)**\n\n---\n\n## Explanation\n\nFor FHA loans with case numbers assigned in 2026:\n- Loan amount: $800,000\n- LTV: 96.5% (exceeds 95%)\n- Term: 30 years\n- Annual MIP rate for loans with LTV > 95% on 30-year terms: **0.55%**\n\nThis annual MIP is paid for the life of the loan when the LTV exceeds 95%, in addition to the upfront mortgage insurance premium (UFMIP) of 1.75% of the base loan amount.",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q05",
     "amount": 800000,
     "ltv": 92.0,
     "term": 30,
     "truth": 0.7,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 92.0% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "# FHA Annual Mortgage Insurance Premium Rate\n\n**Rate: 0.55%**\n\n**HUD Policy Document: Mortgagee Letter 2026-01** (or the current year's applicable Mortgagee Letter)\n\nThis rate applies to FHA loans with:\n- LTV of 92.0% (exceeds 95%, so annual MIP applies)\n- 30-year term\n- Case number assigned in 2026\n\nThe annual MIP for loans with LTV > 95% and terms of 16+ years is currently 0.55% of the outstanding loan balance, as established in HUD's Mortgagee Letters that set FHA insurance premium rates.",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q06",
     "amount": 300000,
     "ltv": 96.5,
     "term": 15,
     "truth": 0.4,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 96.5% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "# FHA Annual Mortgage Insurance Premium (MIP) Rate\n\n**Rate: 0.55%**\n\n**HUD Policy Document: HUD Handbook 4000.1 (FHA Single Family Housing Policy Handbook)**\n\nThis rate applies to FHA purchase loans with LTV ratios above 95% and loan terms of 15 years or less, for case numbers assigned in 2026. The 0.55% annual MIP is calculated on the outstanding loan balance and is typically paid monthly as part of the mortgage payment.",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q07",
     "amount": 300000,
     "ltv": 85.0,
     "term": 15,
     "truth": 0.15,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 85.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "# FHA Annual Mortgage Insurance Premium (MIP) Rate\n\n**Rate: 0.55%**\n\n**HUD Policy Document: HUD Mortgagee Letter 2026-01** (or the current year's applicable Mortgagee Letter establishing FHA MIP rates)\n\n## Explanation\n\nFor FHA loans with:\n- LTV of 85% (which exceeds 95%)\n- 15-year term\n- Case number assigned in 2026\n\nThe annual MIP rate is **0.55%** of the loan amount. This rate applies to loans with LTVs above the threshold where annual MIP is required for the life of the loan.\n\n**Note:** FHA MIP rates are established annually through HUD Mortgagee Letters. If you need the exact current rate for October 2026, you should verify the specific HUD Mortgagee Letter issued for that fiscal year, as rates can change.",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q08",
     "amount": 800000,
     "ltv": 95.0,
     "term": 15,
     "truth": 0.65,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 95.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "# FHA Annual Mortgage Insurance Premium Rate\n\n**Rate: 0.55%**\n\n**HUD Policy Document: Mortgagee Letter 2026-01** (or the current year's applicable Mortgagee Letter)\n\nThis rate applies to loans with:\n- LTV of 95.0% (\u226595%)\n- 15-year term\n- Case number assigned in 2026\n\nThe 0.55% annual MIP is the standard rate for loans with LTV ratios at or above 95% on 15-year mortgages under current FHA guidelines. The specific rates and any updates are communicated through HUD's Mortgagee Letters, which are the authoritative policy documents for FHA loan requirements.",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q09",
     "amount": 800000,
     "ltv": 80.0,
     "term": 15,
     "truth": 0.4,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 80.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "# FHA Annual Mortgage Insurance Premium Rate\n\n**Rate: 0.55%**\n\n**HUD Policy Document: Mortgagee Letter 2026-01** (or the current year's applicable Mortgagee Letter)\n\nThis rate applies to FHA loans with:\n- LTV of 80.0% (at or above 80%)\n- 15-year term\n- Case number assigned in 2026\n\nThe 0.55% annual MIP is set forth in HUD's Mortgagee Letters, which are updated periodically and establish the insurance premium rates for different loan categories based on loan term and LTV ratio.",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q10",
     "amount": 800000,
     "ltv": 75.0,
     "term": 15,
     "truth": 0.15,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 75.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "# FHA Annual Mortgage Insurance Premium Rate\n\n**Rate: 0.55%**\n\n**HUD Policy Document: Mortgagee Letter 2026-01** (or the current year's applicable Mortgagee Letter establishing FHA MIP rates)\n\n## Explanation\n\nFor FHA loans with:\n- LTV of 75.0% (\u2264 95%)\n- 15-year term\n- Case number assigned in 2026\n\nThe annual MIP rate is **0.55%** of the base loan amount.\n\nThis rate applies to loans with LTV ratios at or below 95% and loan terms of 15 years or less. The specific rates are published annually by HUD through Mortgagee Letters, which are the authoritative policy documents that establish FHA mortgage insurance premium rates for that fiscal/calendar year.",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    }
   ]
  },
  {
   "generated": "2026-10-10T17:16:59Z",
   "purpose": "Stale-rule probe: do models quote the FHA annual MIP rate that HUD replaced in March 2023?",
   "truth_source": "HUD Mortgagee Letter 2023-05 (effective 2023-03-20), base-loan threshold $726,200",
   "summary": {
    "model": "claude-sonnet-4-5-20250929",
    "n": 10,
    "correct": 1,
    "stale_rate_present": 6,
    "mentions_2023_change": 10,
    "errors": 0
   },
   "rows": [
    {
     "qid": "Q01",
     "amount": 300000,
     "ltv": 96.5,
     "term": 30,
     "truth": 0.55,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 96.5% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "**0.55%**\n\nThe HUD policy document that sets this rate is **FHA Mortgagee Letter 2023-04** (issued January 27, 2023), which announced the reduction in annual MIP rates effective for case numbers assigned on or after March 20, 2023.\n\nFor FHA loans with:\n- Base loan amount \u2264 $726,200\n- LTV > 95%\n- Term > 15 years\n\nThe annual MIP rate is 0.55% of the average outstanding loan balance.",
     "first_rate": 0.55,
     "correct": true,
     "stale_rate_present": false,
     "mentions_2023_change": true,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q02",
     "amount": 300000,
     "ltv": 90.0,
     "term": 30,
     "truth": 0.5,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 90.0% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "**0.55%**\n\nThe annual MIP rate for an FHA purchase loan with:\n- Base loan amount of $300,000\n- LTV of 90.0%\n- 30-year term\n- Case number assigned in 2026\n\nis **0.55%** of the average annual loan balance.\n\nThe governing policy document is **HUD Mortgagee Letter 2023-04** (issued January 27, 2023), which reduced the annual MIP for most FHA loans with base loan amounts at or below the statutory limit of $726,200 and terms greater than 15 years from 0.85% to 0.55%, effective for case numbers assigned on or after March 20, 2023.",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": true,
     "mentions_2023_change": true,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q03",
     "amount": 300000,
     "ltv": 94.0,
     "term": 30,
     "truth": 0.5,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 94.0% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "**0.55%**\n\nThe HUD policy document that sets this rate is **FHA Mortgagee Letter 2023-04** (issued January 27, 2023), which announced the reduction in annual MIP rates for most forward mortgages with case numbers assigned on or after March 20, 2023.\n\nFor an FHA loan with:\n- Base loan amount: $300,000\n- LTV: 94.0% (greater than 95%)\n- Term: 30 years\n- Case number assigned: October 2026\n\nThe annual MIP rate is 0.55% of the average outstanding loan balance.",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": true,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q04",
     "amount": 800000,
     "ltv": 96.5,
     "term": 30,
     "truth": 0.75,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 96.5% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "**0.55%**\n\nThe HUD policy document that sets this rate is **FHA Mortgagee Letter 2023-04** (issued January 27, 2023), which announced the reduction in annual MIP rates for most forward mortgages with case numbers assigned on or after March 20, 2023.\n\nFor FHA loans with:\n- Base loan amount > $726,200 (high-balance)\n- LTV > 95%\n- Term > 15 years\n\nThe annual MIP rate is 0.55% of the average outstanding loan balance.",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": true,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q05",
     "amount": 800000,
     "ltv": 92.0,
     "term": 30,
     "truth": 0.7,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 92.0% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "**0.55%**\n\nThe HUD policy document that sets this rate is **FHA Mortgagee Letter 2023-04** (issued January 27, 2023), which announced the reduction in annual MIP rates for most forward mortgages with case numbers assigned on or after March 20, 2023.\n\nFor FHA loans with:\n- Base loan amount > $726,200 (high-balance)\n- LTV > 90%\n- 30-year term\n\nThe annual MIP rate is 0.55% of the average outstanding loan balance.",
     "first_rate": 0.55,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": true,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q06",
     "amount": 300000,
     "ltv": 96.5,
     "term": 15,
     "truth": 0.4,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 96.5% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "**0.45%**\n\nThe HUD policy document that sets this rate is **HUD Mortgagee Letter 2023-04** (issued January 27, 2023), which announced the reduction in annual MIP rates for most forward mortgages with case numbers assigned on or after March 20, 2023.\n\nThis Mortgagee Letter reduced the annual MIP for loans with LTV greater than 95% and terms 15 years or less from 0.70% to 0.45% of the average outstanding loan balance.",
     "first_rate": 0.45,
     "correct": false,
     "stale_rate_present": true,
     "mentions_2023_change": true,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q07",
     "amount": 300000,
     "ltv": 85.0,
     "term": 15,
     "truth": 0.15,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 85.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "**0.45%**\n\nThe HUD policy document that sets this rate is **HUD Mortgagee Letter 2023-04** (issued January 27, 2023), which announced the reduction in annual MIP rates for most forward mortgages with case numbers assigned on or after March 20, 2023.\n\nFor FHA loans with:\n- Base loan amount \u2264 $726,200\n- LTV > 95% OR term \u2264 15 years\n- Case numbers assigned on or after March 20, 2023\n\nThe annual MIP rate is 0.45% of the average outstanding loan balance.",
     "first_rate": 0.45,
     "correct": false,
     "stale_rate_present": true,
     "mentions_2023_change": true,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q08",
     "amount": 800000,
     "ltv": 95.0,
     "term": 15,
     "truth": 0.65,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 95.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "**0.45%**\n\nThe HUD policy document that sets this rate is **FHA Mortgagee Letter 2023-04** (issued January 27, 2023), which announced the reduction in annual MIP rates for most forward mortgages effective for case numbers assigned on or after March 20, 2023.\n\nThis mortgagee letter reduced the annual MIP for loans with:\n- Base loan amount \u2264 $726,200\n- LTV > 95%\n- Term \u2264 15 years\n\nfrom 0.70% to 0.45%.\n\nSince your loan amount of $800,000 exceeds $726,200, it would fall into the category for loans > $726,200, which maintained the annual MIP rate of **0.45%** for 15-year terms with LTV > 95%.",
     "first_rate": 0.45,
     "correct": false,
     "stale_rate_present": true,
     "mentions_2023_change": true,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q09",
     "amount": 800000,
     "ltv": 80.0,
     "term": 15,
     "truth": 0.4,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 80.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "**0.45%**\n\nThe HUD policy document that sets this rate is **HUD Mortgagee Letter 2023-04** (or the current FHA Mortgage Insurance Premium structure as outlined in HUD's annual mortgagee letters and codified in HUD Handbook 4000.1, the FHA Single Family Housing Policy Handbook).\n\nFor FHA loans with:\n- Base loan amount > $726,200 (exceeds standard conforming limit)\n- LTV \u2264 90%\n- Term \u2264 15 years\n- Case numbers assigned in 2023 and after\n\nThe annual MIP rate is 0.45% of the average outstanding loan balance.",
     "first_rate": 0.45,
     "correct": false,
     "stale_rate_present": true,
     "mentions_2023_change": true,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q10",
     "amount": 800000,
     "ltv": 75.0,
     "term": 15,
     "truth": 0.15,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 75.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "**0.45%**\n\nThe annual MIP rate for an FHA purchase loan with:\n- Base loan amount of $800,000\n- LTV of 75.0%\n- 15-year term\n- FHA case number assigned in 2026\n\nis **0.45%** of the average annual loan balance.\n\nThe governing policy document is **HUD Mortgagee Letter 2023-04**, issued on March 20, 2023, which reduced annual MIP premiums for most FHA loans and remains the current policy for FHA case numbers assigned in October 2026.\n\n(Note: For 15-year term loans with LTV \u2264 90%, the annual MIP is 0.45%; for LTV > 90%, it is 0.70%)",
     "first_rate": 0.45,
     "correct": false,
     "stale_rate_present": true,
     "mentions_2023_change": true,
     "mentions_ml_2023_05": false
    }
   ]
  },
  {
   "generated": "2026-10-10T17:17:03Z",
   "purpose": "Stale-rule probe: do models quote the FHA annual MIP rate that HUD replaced in March 2023?",
   "truth_source": "HUD Mortgagee Letter 2023-05 (effective 2023-03-20), base-loan threshold $726,200",
   "summary": {
    "model": "gpt-4.1-mini",
    "n": 10,
    "correct": 0,
    "stale_rate_present": 0,
    "mentions_2023_change": 0,
    "errors": 10
   },
   "rows": [
    {
     "qid": "Q01",
     "amount": 300000,
     "ltv": 96.5,
     "term": 30,
     "truth": 0.55,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 96.5% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q02",
     "amount": 300000,
     "ltv": 90.0,
     "term": 30,
     "truth": 0.5,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 90.0% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q03",
     "amount": 300000,
     "ltv": 94.0,
     "term": 30,
     "truth": 0.5,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 94.0% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q04",
     "amount": 800000,
     "ltv": 96.5,
     "term": 30,
     "truth": 0.75,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 96.5% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q05",
     "amount": 800000,
     "ltv": 92.0,
     "term": 30,
     "truth": 0.7,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 92.0% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q06",
     "amount": 300000,
     "ltv": 96.5,
     "term": 15,
     "truth": 0.4,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 96.5% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q07",
     "amount": 300000,
     "ltv": 85.0,
     "term": 15,
     "truth": 0.15,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 85.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q08",
     "amount": 800000,
     "ltv": 95.0,
     "term": 15,
     "truth": 0.65,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 95.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q09",
     "amount": 800000,
     "ltv": 80.0,
     "term": 15,
     "truth": 0.4,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 80.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q10",
     "amount": 800000,
     "ltv": 75.0,
     "term": 15,
     "truth": 0.15,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 75.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    }
   ]
  },
  {
   "generated": "2026-10-10T17:17:05Z",
   "purpose": "Stale-rule probe: do models quote the FHA annual MIP rate that HUD replaced in March 2023?",
   "truth_source": "HUD Mortgagee Letter 2023-05 (effective 2023-03-20), base-loan threshold $726,200",
   "summary": {
    "model": "gpt-4.1",
    "n": 10,
    "correct": 0,
    "stale_rate_present": 0,
    "mentions_2023_change": 0,
    "errors": 10
   },
   "rows": [
    {
     "qid": "Q01",
     "amount": 300000,
     "ltv": 96.5,
     "term": 30,
     "truth": 0.55,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 96.5% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q02",
     "amount": 300000,
     "ltv": 90.0,
     "term": 30,
     "truth": 0.5,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 90.0% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q03",
     "amount": 300000,
     "ltv": 94.0,
     "term": 30,
     "truth": 0.5,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 94.0% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q04",
     "amount": 800000,
     "ltv": 96.5,
     "term": 30,
     "truth": 0.75,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 96.5% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q05",
     "amount": 800000,
     "ltv": 92.0,
     "term": 30,
     "truth": 0.7,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 92.0% and a 30-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q06",
     "amount": 300000,
     "ltv": 96.5,
     "term": 15,
     "truth": 0.4,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 96.5% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q07",
     "amount": 300000,
     "ltv": 85.0,
     "term": 15,
     "truth": 0.15,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $300,000, a loan-to-value ratio of 85.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q08",
     "amount": 800000,
     "ltv": 95.0,
     "term": 15,
     "truth": 0.65,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 95.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q09",
     "amount": 800000,
     "ltv": 80.0,
     "term": 15,
     "truth": 0.4,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 80.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    },
    {
     "qid": "Q10",
     "amount": 800000,
     "ltv": 75.0,
     "term": 15,
     "truth": 0.15,
     "question": "Today is October 2026. For an FHA purchase loan with a base loan amount of $800,000, a loan-to-value ratio of 75.0% and a 15-year term, with the FHA case number assigned this year, what is the annual mortgage insurance premium (MIP) rate? Answer first with the rate as a percentage of the loan amount (for example 0.00%), then name the HUD policy document that sets it.",
     "answer": "[error] HTTP Error 429: Too Many Requests",
     "first_rate": null,
     "correct": false,
     "stale_rate_present": false,
     "mentions_2023_change": false,
     "mentions_ml_2023_05": false
    }
   ]
  }
 ],
 "summary": [
  {
   "model": "claude-haiku-4-5-20251001",
   "n": 10,
   "correct": 1,
   "stale_rate_present": 0,
   "mentions_2023_change": 0,
   "errors": 0
  },
  {
   "model": "claude-sonnet-4-5-20250929",
   "n": 10,
   "correct": 1,
   "stale_rate_present": 6,
   "mentions_2023_change": 10,
   "errors": 0
  },
  {
   "model": "gpt-4.1-mini",
   "n": 10,
   "correct": 0,
   "stale_rate_present": 0,
   "mentions_2023_change": 0,
   "errors": 10
  },
  {
   "model": "gpt-4.1",
   "n": 10,
   "correct": 0,
   "stale_rate_present": 0,
   "mentions_2023_change": 0,
   "errors": 10
  }
 ],
 "regraded": "2026-10-10: extractor fixed to skip the restated LTV percentage (first version took the first percentage in the answer); answers unchanged"
}