The GSEs Are Quietly Taxing Your Refinance — And the Loan Officer Who Said "Just Refi Later" Didn't Tell You That

There is a phrase so embedded in the American real estate transaction that it has achieved the status of folk wisdom. You've heard it at open houses, across kitchen tables, in text messages from agents, and in closing offices:

"Just buy now and refinance when rates come down."

I've been in the mortgage business long enough to remember when that advice was not only reasonable — it was correct. The math worked. When rates dropped, you refinanced, your payment went down, and the temporary pain of a higher rate disappeared. It was a clean, simple calculus.

That calculus is broken. And the reason it's broken isn't a market condition. It isn't a temporary disruption you can wait out. It's deliberate government policy, engineered by Fannie Mae and Freddie Mac under the direction of their federal regulator, the Federal Housing Finance Agency (FHFA). It's called the Loan-Level Price Adjustment system — LLPAs — and what those two agencies have done to refinance pricing over the past three years represents one of the most consequential and least-discussed changes in the American mortgage market in decades.

I'm writing this for borrowers who deserve to know the truth before they buy. I'm writing it for real estate agents who are still repeating the "just refinance later" line to close deals. And yes — I'm writing it for the loan officers who keep saying it too, because at this point, that advice isn't just outdated. It's a disservice.

---

First: What Is an LLPA?

When a lender makes a conventional mortgage and sells it to Fannie Mae or Freddie Mac — which is what happens with the overwhelming majority of conventional loans in the United States — the GSEs charge what are called Loan-Level Price Adjustments. Think of them as risk-based surcharges, expressed as a percentage of the loan amount, that vary based on the characteristics of your loan.

Your credit score, your down payment, your property type, whether you're buying or refinancing, and how large your loan is — all of these factors feed into the LLPA matrix. Each applicable fee is added on top of the others. They stack. And the total is passed directly to you, the borrower, either in the form of a higher interest rate or in upfront points paid at closing.

LLPAs have existed since 2008, when the GSEs introduced them in the aftermath of the financial crisis to better account for credit risk. For most of their history, the system was defensible: riskier loans cost more. Reasonable people could disagree about the calibration, but the principle was sound.

That began to change in January 2022, and again in January 2023, when FHFA directed sweeping revisions to the LLPA framework that went far beyond risk-based pricing. The GSEs added entirely new fee categories. They increased existing ones dramatically. And in doing so, they created a two-tiered pricing system in which the act of refinancing a conventional loan — regardless of your creditworthiness, your equity, or your payment history — costs materially more than the act of purchasing one.

Let me show you exactly what that means in real dollars.

---

The Numbers Don't Lie

All three scenarios below use a 740 FICO score and 20% down payment — a creditworthy, financially responsible borrower by any standard. I've selected rates that price at or very near par (100 cents on the dollar), meaning neither the borrower nor the lender is absorbing points in either direction. This is the cleanest way to illustrate what the GSE fees alone are doing to your pricing.

Scenario One: The Mainstream Conforming Loan

$500,000 Purchase Price | $400,000 Loan | 740 FICO | 20% Down | Primary Residence

This is not an exotic scenario. This is a completely ordinary, creditworthy borrower buying a home in a mid-tier market.

At Purchase:

Par rate at purchase: 6.50% The GSEs collect $3,500 in fees embedded in the pricing of this loan.

On a No-Cash-Out Refinance of the Same Loan:

Par rate on refinance: 6.625% The GSEs collect $4,500 in fees — $1,000 more than on the purchase — on an identical loan to an identical borrower.

The same person. The same house. The same loan balance. The same 20% equity. The same 740 FICO score. The only difference is the word "refinance" on the application — and that costs you an extra $1,000 in GSE fees and forces your par rate an eighth of a point higher.

What is the justification? There isn't one that holds up to scrutiny. A borrower who has been making payments on time for two or three years is less risky than they were at origination, not more. Their payment history has been proven. Their ability to carry the debt is established. And yet the GSEs charge them more to refinance it than to originate it new.

The Mortgage Bankers Association has said it directly: "For a rate-and-term refi where the borrower has a timely payment history — let's say, for the last 12 or 18 months — and they're refinancing to lower their payment, they've already proven their creditworthiness at the higher payment amount." There is no coherent risk argument for the refi surcharge. It exists, in practical effect, as a toll on refinancing — a friction charge that makes it harder, not easier, for borrowers to take advantage of lower rates when they arrive.

If you're weighing a refinance right now, use our refinance savings calculator to see the real break-even math on your specific scenario.

---

Scenario Two: The Agency High Balance Loan

$1,500,000 Purchase Price | $1,200,000 Loan | 740 FICO | 20% Down | Primary Residence

In high-cost markets — Southern California, the Bay Area, New York, Seattle, South Florida — this is not a luxury loan. It's a standard purchase for a professional buyer in a major metro area. The loan falls into what's called the agency high balance category: above the standard conforming limit of $806,500, but within the high-cost area ceiling of approximately $1,209,750 for 2026. Same Fannie and Freddie. Same conventional loan. But a separate, more punishing LLPA grid that didn't exist before April 1, 2022.

That's when FHFA's Lender Letter LL-2022-01 took effect, introducing a standalone high-balance LLPA — a surcharge for loan size alone — where none had previously existed. Prior to that date, if you were creditworthy and had 20% down, your loan size didn't trigger an additional fee. After April 2022, it did.

At Purchase:

Par rate at purchase: 6.875% On a $1,200,000 loan, that's $22,500 in GSE fees embedded in the rate.

On a No-Cash-Out Refinance:

Par rate on refinance: 6.99% GSE fees: $25,500 — $3,000 more than on the purchase.

Consider what this means for any borrower who purchased a high-balance home in 2023 or 2024 at the peak of the rate cycle and is now hoping to refinance into a lower rate. They are not simply waiting for the market to offer them a better rate. They are waiting for the market to fall far enough to absorb both the market pricing and an additional 2.125 points of GSE overhead that a buyer of the same home today doesn't face to the same degree. That is a fundamentally different — and far higher — refinance threshold than the one they were likely sold on when they signed their purchase contract.

Anyone who told that buyer "just refinance when rates come down" owed them this information and didn't provide it.

For California buyers navigating high-balance territory, our jumbo loan programs and current rate sheet can help you compare your real options.

---

Scenario Three: The Vacation Home — Where the GSEs Lost the Plot Entirely

$500,000 Purchase Price | $400,000 Loan | 740 FICO | 20% Down | Second Home

This is the scenario that should make every vacation home owner reading this genuinely angry.

For decades — for the entire history of the modern mortgage system — second homes were financed by Fannie Mae and Freddie Mac at essentially the same pricing as primary residences. If you had a 740 FICO and 20% down on a beach house or a mountain cabin, you paid LLPAs based on your credit and LTV, same as everyone else. There was no surcharge for the property being a vacation home. The historical data supported this: owner-occupied properties, whether primary or secondary, performed similarly in terms of default rates. The risk profile was comparable. The pricing reflected that.

Then, on January 5, 2022, FHFA Acting Director Sandra Thompson issued Lender Letter LL-2022-01, and everything changed overnight. In a single announcement, Fannie and Freddie added standalone second home LLPAs ranging from 1.125% to 3.875% based on LTV — fees that had not existed the business day before. Prior to the announcement, the second home LLPA below 85% LTV was precisely zero. After April 1, 2022, it was as high as 3.875%.

What happened? Did second homes suddenly become dangerous? Did vacation property owners across America wake up one morning and decide to stop paying their mortgages?

Of course not. What happened was that FHFA decided the GSEs were too deeply involved in financing what regulators viewed as "mission-tenuous" lending — loans that didn't serve their affordable housing mandate — and chose to price those loans out of the conventional market. The agency also pointed to concerns about second homes being used as short-term rentals through platforms like Airbnb and VRBO, a practice that some felt represented housing speculation rather than genuine homeownership.

Here is where I have a serious problem with the policy — not with the concern, but with the response to it.

If the GSEs were worried about short-term rental abuse, the answer was simple and surgical: prohibit short-term rentals in the promissory note and deed of trust. Some private lenders already do exactly this. Fannie and Freddie could have required an owner-occupancy covenant that specifically forbids commercial short-term rental activity as a condition of the loan. Borrowers who violated it would be in default. Problem addressed. Second home owners who genuinely use their vacation properties as vacation properties would have been completely unaffected.

Instead, the GSEs chose to punish the entire category — every legitimate vacation home buyer, every family with a mountain cabin they visit three weekends a month, every coastal homeowner who has owned their property for twenty years — with a fee structure so extreme it essentially prices conventional financing out of the second home market entirely.

The proof is in the numbers. Here is what that 740 FICO, 20% down second home borrower faces today, on a $400,000 loan where the identical primary residence loan prices at a 6.50% par rate:

At Purchase:

Par rate at purchase: 7.50% — a full 100 basis points higher than the identical loan on a primary residence, with the same borrower, the same down payment, the same credit score.

On a $400,000 loan, that's $17,000 in GSE fees. For a borrower whose only transgression is buying a vacation home instead of a primary residence.

On a No-Cash-Out Refinance:

The GSE fees on a vacation home refinance are so large — 4.500 points — that no available rate on the conventional rate sheet reaches par pricing. The rate sheet runs out. The math doesn't work. You would need a rate above 7.50% just to reach par on a refinance, and that's before the lender earns any margin at all.

If you own a vacation home that you financed before April 2022 — when these fees didn't exist — and you're hoping to refinance someday, understand what you're dealing with. The conventional GSE system has structurally priced your refinance into near-impossibility. Your options are a jumbo refinance (if your loan qualifies), a portfolio product, or accepting a rate that buries thousands of dollars of GSE fees in the pricing. The "just refi later" promise, which may have been made to you in good faith, has been legislated away by regulatory fiat.

---

The Comparison That Says It All

Here, side by side, is what today's GSE pricing looks like for our 740 FICO, 20% down borrower across all three scenarios:

Now compare that to what a VA loan, an FHA loan, a USDA loan, or a jumbo loan carries in equivalent GSE fees: zero. Not lower. Zero. None of these programs impose transaction-type or property-type LLPAs of this kind. A VA borrower refinancing on an IRRRL pays no LLPAs. An FHA borrower refinancing via streamline pays no LLPAs. A jumbo borrower doesn't interact with the GSE system at all.

The LLPA surcharges I've described above — every dollar of them — exist exclusively on conventional conforming and high-balance loans. They are a tax imposed specifically on the product type that covers the largest share of American home purchases. And the borrowers least able to shop around it, most likely to be told "don't worry, just refinance later," are precisely the ones paying the heaviest price.

For a deeper look at how rates are built from the ground up, read our breakdown of how mortgage-backed securities drive the rates on your rate sheet.

---

What This Means for You Right Now

If you are a buyer being told "just buy now and refinance later": understand that the person saying it may not be accounting for the LLPA spread between purchase and refinance pricing, particularly if you're in a high-cost market or considering a second home. The refinance you're being promised as an easy future option carries structural fees the purchase does not. Ask your loan officer to show you the LLPA grids and price out both scenarios before you sign anything.

If you are a real estate agent: I know the "just refinance later" line is a tool that helps close transactions, and I know you're not a mortgage professional. But when you repeat advice you don't fully understand to buyers who trust you, you're doing them a disservice. The mortgage market has changed materially since 2022. That script is outdated. Encourage your buyers to get a full LLPA breakdown from their lender before assuming refinancing later is a simple proposition.

If you are a current homeowner with a high-balance conventional loan or a second home loan originated before April 2022: you need to understand that your refinance math is fundamentally different from what it would have been under the rules that existed when you bought. Get an honest assessment from a loan officer who will show you the actual LLPA burden on your specific loan and give you a realistic break-even calculation before you chase a rate drop.

The GSEs have every right to price their products as they see fit. And there are legitimate arguments on both sides of the LLPA debate. But none of that changes the reality on the ground for borrowers making decisions today.

The "just refinance later" era is over. The numbers say so. The rate sheet says so. And now you know it too.

---

Don't Navigate This Alone — Get Your LLPA Breakdown

If you're buying, refinancing, or just trying to understand what the GSE fee structure means for your specific loan, I'll walk you through it — no sugarcoating, no "just refi later" hand-waving. Get your personalized rate quote →

I'll show you the actual LLPA grids for your credit score, LTV, property type, and loan purpose so you can make an informed decision — not one based on outdated advice.

---

A Final Word on the Bigger Picture

It is worth stepping back and asking a question that rarely gets asked plainly: why do purchase LLPAs and refinance LLPAs differ at all?

The official answer from the GSEs and FHFA is that refinances, particularly rate-and-term refinances, represent a slightly different risk profile because the borrower is actively shopping for a better rate rather than making a primary housing decision. The argument is tenuous at best. A borrower refinancing from 7.25% to 6.50% on a home they bought two years ago with 20% down and a 740 FICO, who has made every payment on time, is not meaningfully riskier than the buyer who originated that loan at origination. By every measurable risk indicator — seasoning, payment history, equity — they are more creditworthy, not less.

The more honest answer is what the Mortgage Bankers Association's own president implied in late 2025: the current structure was designed, at least in part, as a friction mechanism — to slow the refinance market, reduce prepayment speeds on mortgage-backed securities, and build capital at the GSEs. Those may be legitimate institutional goals. They are not the goals of the borrower. And they should be disclosed clearly to every person being told that refinancing later will be easy.

The irony is that the borrowers most affected — high-balance loan holders in expensive markets and second home owners — are frequently the ones with the financial sophistication to demand better answers. Ask for them. The rate sheet doesn't lie. LLPAs are published. The math is right there for anyone willing to look.

I look at it every day. And what I see is a system that has made buying the right decision, refinancing a more expensive one, and owning a vacation home one of the costliest choices a creditworthy borrower can make within the conventional mortgage market.

Buy right the first time. Lock in the best rate you can at purchase. Don't plan your financial future around a refinance that the GSEs have quietly made a great deal harder to execute than anyone told you it would be.

That's the honest conversation. I'm happy to have it with you.

---

Emmett Clark is a licensed mortgage professional with over two decades of experience serving buyers and homeowners across 18 states. NMLS 233747 | LoansByEmmett.com Rates and LLPA schedules referenced are from the Rocket Pro Wholesale rate sheet dated May 21, 2026. LLPAs are subject to change by the GSEs without notice. All scenarios are for illustrative purposes and assume a 30-year fixed conventional loan, primary or second home occupancy as noted, single-family property, and full documentation underwriting.