A 604 credit score won’t get you laughed out of a lender’s office. It gets you a narrower list of lenders, a different rate sheet, and a longer conversation about mortgage insurance. The pricing gap between a 604 borrower and a 760 borrower on the same house is real, usually somewhere between one and two percentage points, but it isn’t a locked door.
The awkward part is that the most heavily advertised names in lending are rarely the ones that handle low scores well. Large retail banks often layer their own credit overlays on top of FHA and VA minimums, which quietly pushes their real cutoff to 620 or even 640 while the government program underneath still allows 580. So a search for the best mortgage lenders for low credit scores tends to end somewhere less famous: a non-bank servicer, a regional credit union, or an independent broker with access to a dozen wholesale rate sheets.
What a Low Score Actually Costs You
Fannie Mae and Freddie Mac price conventional loans in credit bands. Cross from 619 into 620 and your loan-level price adjustment drops. Cross from 679 into 680 and it drops again. On a $320,000 loan, moving from the 620 band to the 760 band is worth roughly 1.25% to 2.75% of the loan amount in upfront pricing, which the lender converts into either discount points at closing or a higher note rate.
Government-backed loans run on a different rulebook. FHA insures mortgages down to a 500 FICO. At 580 and above, 3.5% down is enough. Between 500 and 579 the down payment requirement jumps to 10%, and plenty of FHA lenders set their own floor at 500 or 520 regardless. VA has no statutory score minimum, though almost every VA lender sets one, typically 580 or 620.
Two borrowers with the same 598 score can end up with quotes 0.75% apart. That isn’t the score talking. It’s the lender’s overlay, its pricing margin, and how much of the loan it plans to keep versus sell.
Lenders That Consistently Work With 500 to 620 Scores
Ask ten loan officers who takes a 540 and you’ll hear a handful of the same names. These are institutions that either hold loans on their own books or have built their business around government programs rather than around agency guidelines.
Non-bank specialists
- Carrington Mortgage Services writes FHA and VA loans down to a 500 FICO and runs a portfolio program for borrowers sitting in the 500s.
- Newrez, which services under the Shellpoint name, has a reputation for taking files other servicers decline, including recent derogatory marks with documented explanations.
- Angel Oak Mortgage Solutions is a non-QM shop. Bank statement loans, asset depletion, and 12-month seasoning after a bankruptcy are all on the menu, at a price.
- Freedom Mortgage is VA-heavy and routinely funds 580 to 620 files with high debt-to-income ratios.
- Guild Mortgage keeps a portfolio channel that occasionally beats agency pricing for borrowers just below the 620 line.
The detail that matters more than the logo is whether the lender will underwrite manually. Automated approvals stop at hard cutoffs. A human underwriter can look at a 590 with four years of on-time rent, twelve months of reserves, and a letter explaining a medical collection, then decide the file is fine. A closer look at the lenders where a 580 score still gets approved goes deeper on which of these actually read files by hand.
Credit unions and community lenders
Local credit unions give the least predictable answers and sometimes the best ones. A member-owned institution in a mid-sized city may hold a mortgage in portfolio for twenty years, which means it doesn’t much care what Fannie Mae thinks about a 601. Community development financial institutions go further, pairing below-market rates with down payment help for buyers in specific census tracts. The catch is membership and geography, since you usually need to live or work in the area. For a first purchase with a thin file and no derogatory marks, that local option often beats anything a national brand will quote, which is worth remembering when you weigh the real differences between lenders for first-time buyers.
FHA, VA, and USDA Do the Heavy Lifting
Almost every low-score approval runs through a government program. FHA funded a large share of purchase loans to borrowers under 640 for the simple reason that its credit floor is 500 and its down payment is 3.5%. VA is even more forgiving for eligible service members, with nothing down and no monthly mortgage insurance premium, though there is a one-time funding fee.
Not every FHA-approved lender is priced the same, and the spread is wider than most borrowers expect. Two lenders can both advertise an FHA 30-year fixed and differ by half a percent in rate and by thousands in fees. Overlays explain part of it. The rest is margin and compensation structure, and it’s worth reviewing how to tell a good FHA lender from an expensive one before you let anyone pull your credit.
USDA, the overlooked option
USDA Section 502 loans require no down payment and accept scores in the 600s with manual underwriting below that. The property has to sit in an eligible rural area, which covers more ground than most buyers assume, and household income has to fall under a county limit. If you’re looking at a small town or the outer edge of a metro, check the eligibility map before writing it off.
Why Star Ratings and Advertised Rates Mislead
Review scores for mortgage lenders are a shaky signal. The angriest reviews come from people whose loans fell apart, which is a service problem, and the happiest come from people who never read their closing disclosure. A 4.8-star average tells you almost nothing about whether that company prices FHA loans competitively for a 590 borrower. As the best-rated lenders aren’t always the ones with the most stars, the ratings deserve five minutes of your attention and no more.
Advertised rates are worse. The headline number on a lender’s website typically assumes a 740 score, 20% down, and a single-family primary residence. Your actual quote will carry a credit-based adjustment, plus mortgage insurance if the down payment falls under 20%.
What Sixty Days of Credit Work Can Buy You
There’s a real payoff in delaying an application by six to eight weeks, and it isn’t subtle. Moving a 612 up to 640 can shift a borrower out of the worst pricing tier and into a noticeably better one.
- Pay revolving balances below 10% of the limit rather than 30%. Utilization updates when the statement cuts, so timing matters.
- Ask a loan officer about a rapid rescore. It costs the lender a small fee and can update a score within a few business days once a payment posts.
- Dispute genuine reporting errors, but skip the shotgun approach. Blanket disputes of every account can freeze scoring models mid-underwrite.
- Leave old accounts open. Closing a card you’ve held for nine years shortens your file and hurts the score.
- Ask a collector for a goodwill deletion in writing before paying a stale medical bill.
If There’s a Foreclosure or Bankruptcy on the File
Seasoning rules matter more than the score in these files. FHA allows a new loan three years after a foreclosure and two years after a Chapter 7 discharge, with exceptions for documented hardship. VA shortens the foreclosure wait to two years. Conventional loans stretch to seven in most cases. This guide to buying again after a foreclosure lays out the timelines lender by lender, since several non-QM investors now accept 12 months of clean housing history.
Two Loan Estimates, One Clear Winner
Once quotes are in hand, compare them in a way that survives the sales pitch. Request a Loan Estimate from at least three lenders inside a two-week window so the credit pulls count as a single inquiry for scoring purposes. Then line up four numbers: the interest rate, points and origination charges in Section A, total lender fees, and the estimated monthly payment including mortgage insurance.
A working example. Lender one quotes 7.125% with $1,800 in origination and a $212 monthly mortgage insurance premium. Lender two quotes 6.75% with $4,300 in points and the same insurance. On a $300,000 loan the lower rate saves about $79 a month, so the extra $2,500 upfront takes roughly 32 months to recover. Stay in the house longer than that and lender two wins. Sell or refinance sooner and lender one does. That math, not the star rating or the first rate you hear, is what separates a fair deal from an expensive one.