A 604 credit score is not a locked door. It’s a pricing tier.
That distinction matters more than almost anything else in this conversation. Plenty of homeowners assume a subprime score means no refinance at all, so they never call anyone, never check the numbers, and keep paying 7.4% on a loan they could probably replace. Others assume the opposite and get blindsided when the loan estimate arrives with a rate two points higher than the one quoted on the phone.
The truth sits in the middle. You can refinance a mortgage with bad credit. It will just cost you more, and how much more depends on details most borrowers never think to ask about.
What a lender actually weighs beyond the three-digit score
Credit scoring models differ, thresholds differ, and each lender layers its own underwriting rules on top. A 620 at one shop is a different loan than a 620 at another. Instead of fixating on the number, it helps to know what sits alongside it on the application.
- Loan-to-value. Equity is the strongest compensating factor you have. A 95% LTV with a 610 score is a hard sell. A 65% LTV with the same score is a much easier approval, because the lender’s risk of loss is small.
- Payment history in detail. A single medical collection from 2021 and four consecutive late mortgage payments in 2023 are not the same file, even if the scores land within ten points of each other.
- Seasoning. Most programs want two years from a bankruptcy discharge, three from a foreclosure or short sale, and four from a deed-in-lieu. Time served matters more than the score itself in many of these cases.
- Reserves and income stability. Six months of mortgage payments in the bank can carry a weak file across the line. So can a fifteen-year work history in the same industry.
Ask an underwriter what killed a recent denial and you’ll often hear about a debt-to-income ratio at 51% or an unexplained deposit, not the score.
Which refinance routes tolerate a low score
FHA streamline refinance
If your current loan is already FHA-insured, the streamline program is the most forgiving option available. Many lenders run it with no fresh appraisal and a reduced credit review, sometimes only a mortgage payment history check. FHA does allow scores down to 500 with a 10% down payment on purchases, and 580 and above at 3.5%, though refinance LTV rules differ. The catch is mortgage insurance, which you’ll keep paying.
VA Interest Rate Reduction Refinance Loan
The IRRRL is close to a no-questions-asked rate cut for veterans and active-duty borrowers. No appraisal in most cases, no credit underwriting in the traditional sense, and a certificate of eligibility is often all the documentation required. If you have any VA entitlement left and a VA loan already in place, this is the first phone call to make.
Conventional loans with loan-level price adjustments
Fannie Mae and Freddie Mac don’t ban low scores. They charge for them. A score under 640 triggers LLPAs that can add 1.5% to 3% of the loan amount in fees, or roughly a quarter to a half point on the rate. On a $310,000 loan, that’s real money, but it’s not a prohibition.
A lower rate can still lose you money
Here’s where borrowers get sloppy. They see a drop from 7.25% to 6.5% and sign, without checking whether the closing costs eat the difference. Run the break-even math honestly: divide total closing costs by the monthly savings, and see how many months it takes to get back to zero.
A realistic look at what a half-percent lower rate actually saves usually surprises people, because the answer is often less than the lender’s marketing implies. If the savings are thin—say $84 a month against $4,200 in costs—you’re looking at a 50-month payback on a loan you might not keep that long.
Paying discount points to buy the rate down is another place the arithmetic gets fuzzy. Points make sense when you plan to stay put for years and you’re certain about it. They’re a bad trade if a job move or a growing family is on the horizon. A quick pass through a refinance rate buydown calculator will tell you how many months of payments it takes before the points pay for themselves.
Costs that hit bad-credit borrowers hardest
Rate is only one line on the loan estimate. Watch these three:
Mortgage insurance. On a conventional loan under 80% LTV, private mortgage insurance premiums scale steeply with score. A 760 borrower might pay 0.35% annually; a 640 borrower could pay 1.1%. On a $300,000 loan, that’s an extra $1,875 a year. If your home value has climbed since you bought, an appraisal could drop you below the 80% threshold and solve the problem outright. A refinance with PMI calculator shows what the payment looks like either way, which is far more useful than guessing.
Escrow surprises. Your new monthly payment includes taxes and homeowners insurance, and those have been rising in much of the country. Comparing a quoted principal-and-interest figure to your current full payment is a common and expensive mistake. Use a refinance calculator that includes taxes and insurance so the comparison is apples to apples.
Extending the term. Stretching a 15-year loan back to 30 years cuts the payment dramatically but adds years of interest. A 15-year to 30-year refinance calculator lays out both numbers side by side. Sometimes the relief is worth it. Sometimes it quietly costs six figures.
Fixing your file before you apply
You may not need to wait two years. A few weeks of housekeeping can move a score 30 to 50 points, which can shift your pricing tier.
- Pull all three credit reports from AnnualCreditReport.com and dispute anything wrong. Duplicate collections and paid-off accounts still showing balances are common.
- Pay down revolving balances below 30% of the limit, then below 10% if you can. Utilization moves faster than any other factor.
- Ask a lender about rapid rescoring, which can push corrected data to the bureaus in days rather than weeks.
- Leave old, dormant accounts open. Closing them shortens your history and raises utilization.
- Don’t finance a car or open a store card in the two months before applying. New inquiries and new minimum payments both hurt.
Red flags worth walking away from
Bad-credit borrowers attract a specific kind of lender. Anyone demanding an upfront application fee before providing a loan estimate, promising to “erase” negative items, or quoting a rate without pulling credit should be dropped immediately. Loan estimates are free, and the three-day rescission right after closing exists for a reason.
Also push back on yield-spread premiums. On higher-rate loans, brokers sometimes earn a rebate from the lender for placing you at a higher rate than you qualified for. Ask directly whether your quoted rate includes any lender credit to the broker, and get the answer in writing.
What to do in the next seven days
Order your loan estimate from at least two lenders, ideally one bank and one independent mortgage broker, and compare the APR line rather than the interest rate. Get an updated home valuation, since equity is your strongest lever. Check whether your existing loan is FHA or VA, because if it is, the streamline path probably skips most of the obstacles described above.
Then decide with a calculator open, not a phone in your ear. A refinance that saves $140 a month and costs $3,900 is a good deal at 28 months and a mediocre one if you’re selling in two years. The score sets the price. Your equity, your timeline, and your paperwork decide whether the deal is worth doing at all.