A 604 credit score and four years of on-time rent-sized mortgage payments is not the profile most refinance ads are written for. Plenty of homeowners with bruised credit refinance anyway. They just don’t do it at the branch down the street. They do it with lenders who price risk instead of rejecting it, and the gap between those two worlds is mostly about knowing where to look and what paperwork to bring.
Here is how the approval actually works when your score is thin, and how to tell a fair deal from a fee trap before you sign anything.
What a 580 Score Actually Signals to an Underwriter
The number matters less than what sits behind it. An underwriter looking at a low-score file is trying to answer one question: is this someone who had a rough two years, or someone who doesn’t pay? A 590 with a 2021 medical collection and twelve clean months reads very differently from a 590 with three 30-day lates since last spring.
Three figures carry most of the weight:
- Loan-to-value. Equity is the cushion. At 65% LTV a lender can absorb a lot of risk. At 95% they can’t absorb any.
- Reserves. Cash left in the bank after closing. Six months of payments in savings softens an ugly score faster than almost anything else.
- Recency of late payments. Seasoning counts. A late from two years ago gets discounted. A late from last quarter does not.
There’s a longer explanation in this breakdown of what refinance lenders really look at when your credit is weak. The useful takeaway is that your score is one input among several, and a few of the others can be improved before you ever fill out an application.
The Four Places Bad Credit Mortgage Refinance Lenders Actually Operate
Credit unions and community banks
Portfolio lenders keep loans on their own balance sheet, which means they aren’t bound by Fannie Mae’s 620 floor or a national bank’s internal overlays. Many will do a manual underwrite for an existing member with a clean payment record and a decent down payment of equity. Membership often costs $5 and a local connection.
Non-QM and specialty lenders
These are the institutions built for files that don’t fit the government-backed box. Expect pricing 1 to 3 percentage points above conventional, and expect them to care much more about cash flow and equity than about a FICO band. A 610 borrower with $40,000 of accessible savings is a good client to them.
Wholesale mortgage brokers
One application, shopped across a dozen lenders with different guideline sets. This is genuinely useful when your file sits on the edge of three different rulebooks and you can’t tell which one will approve you. Ask the broker to show you the rate sheet they priced from.
Online lenders that advertise directly to your credit band
Sometimes the fastest route. Also the most expensive, most of the time. Compare the APR, not the headline rate, and check the NMLS ID before you hand over a single document.
Bank Statements Can Outrank a Credit Score
A self-employed contractor with a 615 score and $11,000 a month landing in a business account has options a salaried borrower with the same score doesn’t. Bank statement programs typically accept 12 or 24 months of deposits in place of tax returns, and they’ll tolerate lower scores when the deposits are steady and the equity position is strong. Asset depletion loans work the same way in reverse, converting a large investment balance into qualifying income.
If you’ve been turned down by two banks, the problem is almost certainly the loan product, not you.
The Real Cost of a Lower Score, in Dollars
Risk-based pricing is not subtle. On a $300,000 balance over 30 years, the gap between a 740 borrower at 6.4% and a 620 borrower at 7.6% is roughly $241 a month, or about $2,892 a year. That’s real money, and it’s the number to weigh against your current payment rather than the rate alone.
Before you get excited about a quote, run the arithmetic on what a smaller rate cut is actually worth to you. This look at what a half-percent lower rate really saves is a useful reality check, and the break-even math most lenders skip shows how long you need to stay in the home for the closing costs to wash out.
Points, Mortgage Insurance, and the Break-Even Question
Discount points are where bad-credit borrowers get talked into bad decisions. Paying 1.5 points to shave 0.375% off the rate can make sense over seven years and be a straight loss over three. If a loan officer pushes a buydown, ask for the break-even month in writing, then check it yourself with a refinance rate buydown calculator.
Mortgage insurance is the other quiet cost. If your current loan has PMI and your new one would too, the payment comparison gets murkier than the rate suggests. Model the full payment, insurance included, before you commit. A PMI refinance calculator will show you the real monthly figure rather than the one on the marketing flyer.
Red Flags That Show Up in This Corner of the Market
- Fees collected before you receive a Loan Estimate. The estimate is supposed to come first.
- Verbal promises of a rate that never appears on paper.
- Anyone suggesting you inflate income or leave a debt off the application.
- A prepayment penalty on a refinance. It exists in this market, so read for it.
- Pressure to sign the same day. A legitimate offer survives 48 hours of thought.
A Practical Checklist Before You Apply
Two weeks of preparation can move your pricing by half a point or more.
- Pull all three credit reports and dispute anything factually wrong. A removed collection can be worth 30 to 50 points.
- Pay down revolving balances below 30% utilization. This is the fastest legitimate score lever available to you.
- Collect two years of tax returns, two recent pay stubs or 12 to 24 months of bank statements, and a current mortgage statement.
- Estimate your LTV using a conservative home value, not last year’s Zestimate.
- Gather three written Loan Estimates inside a 14-day window so the credit pulls count as one inquiry.
- Ask each lender for the break-even month, in writing, on any points they quote.
If the First Lender Says No
A denial from a national bank tells you almost nothing about your options. It tells you that your file didn’t clear that institution’s overlays, which are often stricter than the actual agency guidelines underneath them. Get the denial reason in writing, then take it to a broker who can shop non-QM products and a credit union that underwrites manually.
Timing matters too. If you’re six months from a collection aging off, or three months from paying down a card, waiting can be the highest-return decision available. Sitting at a 7.9% rate for another year costs about $4,000 on a $280,000 balance. Spending three months getting to a 640 first might cost you nothing at all and save most of that. Run the numbers on both scenarios before you sign, and treat any lender who won’t help you run them as a signal in itself.