So you’re retired. And now you want to buy a house without paying cash?
Yes, that is doable.
Asset Backed Mortgages
A typical bank wants to see predictable income in order to lend money. I suppose the logic is that if you get paid regularly then you will pay them regularly (actually, you will pay the company they sell the mortgage to.) These same banks will lend money to the self-employed but with more hoops to jump through to insure the income is “safe.”
So what about people who don’t have jobs of any kind, but happen to have a giant pile of money lying around, e.g. IRAs, 401ks, brokerage accounts, etc…?
That’s a situation that many banks just Can’t Even.
Bank – “Oh my, it appears you could pay cash for this house but our brain has short-circuited and we are unable to figure any of this out…. DENIED, DENIED, DENIED, ………
Which is weird… why would you rather lend money to somebody with a “good credit score” but spending 100% of their after-tax income vs somebody who could sell some stock and pay for a house 4 or 5 times over?
If this sounds frustratingly familiar, there are some banks who offer Asset Backed Mortgages or Asset Based Mortgages. As the name implies, instead of income they look at assets.
Schwab and Rocket Mortgage
When I started looking for non-traditional lending, the first and most prominent name I found was Schwab. They offer asset backed mortgages via Rocket Mortgage (formerly Quicken Loans.) The loans are fixed fee and applicants are assigned a mortgage broker who works exclusively with Schwab customers.
As a bonus, they recognize that asset-heavy households are lower risk on mortgage default… and interest rates reflect that. You can get as much as a 1% interest rate discount!
Rates are lower for asset-heavy households – (Learn more here.)
When I applied the turnaround time was quick – I had a phone call with my assigned broker within a day and a pre-approval letter by the next morning.
I simply explained that I wanted a ~$1 million mortgage and I was looking at homes in the range of $1.5 million. The broker can see the assets in your Schwab account so it is all very seamless. (At the time of application, my Schwab account had about $100 in it.)
Because you are working with Rocket Mortgage, you have access to all loan types (interest only, ARMs, 15-year, 30-year, etc…) and the rates are as competitive as you might expect from a major mortgage provider.
Preapproval Letter
I tried to explore deeper – how much total could I borrow? Is there a formula that determines this? Are assets in the brokerage account weighed differently than assets in retirement accounts? Etc… but my broker was not interested in anything except a sale. “Let me know when you have an accepted offer.”
I even blasted over an older article from when another early retiree was trying to buy a home (Getting a Mortgage When You Have Assets but No Income) because it explained a formula (basically the Freddie Mac asset depletion mortgage formula.)
Me: “Is this how it works?”
Broker: “No.”
I mean, I respect that the guy didn’t want to answer a bunch of questions without getting paid, but ultimately that was why I got a mortgage elsewhere. Sorry.
If anybody has more details on how this works, please comment below and I’ll update the post.
Freddie Mac Asset Depletion Mortgage
For awhile now (not sure how long), Freddie Mac has outlined an alternative to income based mortgages. Then late 2020 they updated this guidance. This is important because in many cases a mortgage companies entire business is just creating loans and selling them to Freddie Mac – The sole criteria for these companies is, “will FM buy it?”
The asset depletion mortgage goes like this – take 100% of your brokerage account minus home down payment and 70% of your retirement accounts and divide over the life of the loan (e.g. 360 month in a 30-year loan.) This is your “income.”
Now apply debt-to-income requirements (50% max for Freddie, lender may have tighter requirements) and you get your maximum mortgage payment (encompassing P&I, taxes, insurance, HOA, etc…)
From there you can reverse determine the maximum mortgage amount factoring in the FM required 20% down payment.
Example:
$1 million brokerage + 70% * $1 million IRA = $1.7 million
1.7 million / 360 = $4,722 monthly “income”
50% max debt-to-income = $2,361 mortgage payment
Now, reverse calculate the taxes and insurance, work through the amortization, carry the one, and we get… a house purchase price of ~$550,000 with a mortgage of ~$440,000. (2.75% 30-year fixed)
Or a home purchase of ~25% of total liquid net worth.
(Nice, but it’s a strange assumption that the underlying portfolio is expected to return 0% real over 30 years.)
In any case, based on Freddie’s guidance more lenders should be able to offer mortgages to the non-working members of society.
If in doubt, ask. “I want an asset depletion mortgage. I have a lot of assets but little/no income.”
Our mortgage
We paid cash for our house and then immediately refinanced to pull money out. Yeah, it’s weird.
Special note: you can’t do a (large) cash-out refinance as a Freddie Mac defined asset depletion mortgage. Purchase or traditional refinance only…
I started with Schwab / Rocket Mortgage but ended up with a random lender off of Bankrate’s refinance page. Or should I say, 6 random lenders.
I negotiated through 6 loan estimates before asking the best two to go head-to-head before signing an agreement with the winner.
This turned out to be a 2.75% 30-year fixed with total out-of-pocket fees of ~$0 (thanks to $3,000 in “Lender Credits”) and no escrow.
I’m not completely sure on what basis our mortgage was approved – I reported income equal to what is on our 1040 from the past 2 year (2019 and 2020, includes investment income and Schedule C) and had to provide extensive documentation on both blog income and assets / brokerage accounts.
I’m now working on a refinance to a 15-year fixed and will probably get at least $1,000 cash in my pocket for doing so.
Summary
For those of us who have chosen Leisure over Labor it is still possible to get a mortgage – no job or self-employment income required.
Companies like Schwab / Rocket Mortgage will provide asset backed loans. They will even provide a discounted interest rate. With Freddie Mac’s blessing, other lenders provide asset depletion mortgages based on expected “income” from an investment portfolio.
No matter which lender you go through it pays to get multiple loan estimates and ask brokers to compete for your business. After all, they are just going to sell it – there is no long term relationship here.
Best of luck getting your asset based mortgage – you’ve earned it.
Great post, I just completed a refi before I FIRE’d myself so that I wouldn’t have to go through these extra steps.
It’s good to know I have options in my the future.
So to clarify, you weren’t qualified as Asset Backed or Asset Depletion? You just qualified through traditional income means or whatever unique formulas the two lenders you shopped used?
I think a combination. I got pre-approved through Schwab based solely on assets.
Please help me understand…you said you only have $100 at Schwab, and it approved loan of nearly $1 million? thanks
All of our assets are at Fidelity and Vanguard. I use Schwab just as an ATM so only move cash there short term when I need to
Were we to have moved forward with Schwab I would have needed to move assets there.
Hv you tried margin loan instead !? You may be able to get rated 1% to 1.26% from ETrade/Schwab and/or Fidelity.
We know variable rate of margin (and portfolio performance) is a bit risky – especially if your loan approaching over 30-40% of portfolio value ..
But just an alternative- than selling securities at 15, 18 or even 20%+ tax rates (or worse yet, STCG rates).
So, with Schwab, would you have been eligible for the mortgage tax deduction? Will you be eligible when you refi?
With any of these options we would be able to get the tax deduction as long as it was for the initial purchase or for a refi of an existing mortgage.
With no mortgage any refinance is not tax deductible
But really, with SALT limit and a big standard deduction the mortgage interest deduction is largely useless
Ah yes. Blue state. Good point.
Not really a blue state thing – in 2022, for example, I’ll pay <$1k in state taxes and ~$10k in property taxes (and pay zero to the IRS plus get CTCs.) If I borrowed $750k at 3% or so the additional interest might just about equal the standard deduction... There are no real scenarios where I come out ahead by itemizing.
Even in a blue state, if you have mostly LTCGs or dividend income you can escape many taxes.
This was super handy. Thanks for posting. Do you think you would have gotten a better rate if you had gotten a traditional loan prior to purchase rather than paying cash and then doing the refi? Do you think asset based mortgages offer as competitive of rates as income based? I’ve been puzzling over this issue for a while.
I think you can do better going the asset route – just think of the Schwab discount thing as them buying points for you
I’m FIRE and paid $800k in cash for my house one year ago. Decided to apply for an asset depletion mortgage a couple of weeks ago to take most of that $800k out (a.k.a. a cash-out refi)
I talked to five banks, and they all quoted a higher interest rate than the prevailing 30-year fixed mortgage rate. Most were well over 4%. I finally found one for 3.625% + a 1% origination fee.
Jeremy’s approach was clearly superior — playing off multiple banks against each other.
But, Mighty Investor, as to your question whether it’s better–in general–to pay cash first then refinance vs. getting a mortgage at time of purchase, I believe you’re generally going to pay more interest with a cash out refi, and you might not be able to take 80% out. (I could only do 70% LTV to get my <4% rate.)
@Ty – How are you paying cash in full for a house and then taking out a mortgage if you already own the house? I know I’m missing something.
@Ken – Another way to think about it is that you’re taking out a loan where the collateral is your house. You can google “cash out refi” for the full story.
Thanks for your thoughts, Ty. Helpful!
@Mighty Investor – Glad to help. I had this same exact question for a long time.
You might also want to check out @Accidentally Retired’s blog post on this topic, which he links to below. Very comprehensive and matches my own experience.
Weird, but helpful for me in the future I bet. Thanks!
Thanks for posting this. It addresses a big question I had regarding FIRE.
What is your long-term strategy with the home equity? Will it to your children? Do a reverse mortgage? Sell the house eventually and rent?
Mentally I just assume the house money is gone forever.
We will probably sell the house when the kids are done with high school and then move to southern Spain
re: Spain… Can you get citizenship and a path to an EU passport? I know this is easily done via Portugal. My wife is Spanish and I’d love to live in Spain eventually. We’re in Japan now.
Is the $100 correct in the following statement?
“At the time of application, my Schwab account had about $100 in it.”
I added that just as a fun side note but it seems to be a distraction from the main point.
I only use our Schwab account as an ATM so keep minimal funds there. I told the broker I had millions elsewhere and would move them to Schwab if we went forward and that was sufficient for pre-approval
I went through the same thing in early 2008 after 5 years living abroad on volunteer stipend and investment earnings. The housing market was collapsing. I had 2x the house value in my regular brokerage acct (Non-IRA) and couldn’t get a mortgage at 3-4% anywhere. So I paid cash and did a cash out refinance at .5%. Used the cash to buy and investment property. Banks didn’t want a “sure thing.”
I’m glad Schwab/Rocket Mortgage figured this out. I also found a few other options as well in my search after running into roadblock after roadblock.
You are so right, the banks just can’t deal with someone who has enough assets to buy the house in cash.
WaFD Bank and other portfolio lenders will also do these mortgages, and some will even do investment mortgages for those without earned income.
Midfirst Bank will as well, though their terms weren’t as good last year when I went through it.
So the three options are the traditional and easiest Fannie/Freddie ones, Private Banks, and Portfolio Lenders. The latter two will have their own custom rules.
My full experience here as well: https://accidentallyretired.com/financial/asset-based-depletion-mortgage/223
Thanks, good to see what others have experienced going this route
Great post! Helpful as always! I wonder if you’d recommend to buy a house without an buyers agent if you know the area pretty well – I am just trying to change to a different house in my town? I understand the need of buyers agent in your case, but wonder if I could pull of buying without an buyers agent – who is, in my opinion, not necessary in my case… Thanks in advance!
.I would buy a house without an agent, sure. I sold my old house by owner, It’s mostly an automated process anyway.
What do you mean by automated process ?
I am a buyer in WA looking for a home but not may sellers trying to sell to a buyer without an agent. How did you approach the sale ? Did you call the agent or the owner ?
Great job on the blog. inspiring
It’s all standard forms and process. The escrow company will basically do all of the paperwork.
I sold my house through one of the “brokers” that just puts your property in the MLS. It cost around $500 afair.
Then in the first weekend 25 agents or so went through the property and I sold that day to one of their buyers. Once we agreed on price I just had to sign the escrow docs and I got my funds.
Assuming the seller’s agent would have gotten 3% I earned about $4,000/hour for my minimal effort.
Just curious as to why you would want to purchase without an agent being the seller pays the commission? At the very least, wouldn’t things be less of a hassle if you use an agent?
Represent yourself. Ask for the 3% buyer’s agent fee as a credit at close
Thanks much for your response Jeremy! I have an off-topic follow up question – from a pure financial perspective (big house is nice to have but not a must for a busy family of 5 with teenage kids), what do you at this time if you have large amount of cash (400k+) on your hands? My hunch is to average cost into index fund, instead of investing in a big house. But house and stock market are both at an all time high. Looking for a wise word of advise! Thanks so much!
Look at any historical stock chart – the market is always at all-time highs. Invest according to your asset allocation
Using brokage fund, you would need to sell stock (let’s say 1 mil). It will be a huge LTCG tax if not STCG.
Yes, we will have a hefty 2021 tax bill due to home purchase. At least I had 8 years of cap gain harvesting before hand
This is very helpful thank you
my pleasure
This is very useful. I wish it was more mainstream but now at least I know where to start. I like your summery of 25% portfolio home price reference.
Was the 20% down payment required?
My credit union waives mortgage insurance so I’m thinking less than 20% down payment. With an asset-backed mortgage it looks like I have to re-calculate things.
I don’t know how high they would go on LTV
I’m curious if and how the asset-based mortgages will count other paid-off property towards assets.
I guess I could take out a HELOC and put it in a brokerage, but I would hope they would consider an income-producing property as an asset.
Why in the FIRE community have a mortgage if you have the cash? Is the money better used in the market or such then the cost different of paying loan interest?
Also you mention Schwab didn’t earn your business, can you say who did?
Just pick any bank that will lend you money and offers the best terms. The mortgage broker we worked with sold the loan within days of closing so who is not important.
There are a lot of reasons to have a mortgage instead of paying cash – inflation hedge, tax consequences of converting assets to cash, you can invest the funds at a greater return than the interest, etc…
I’ve already invested most of the cash I got from the refi and am using the rest for cash flow management.
Im a little confused. What are the benefits to you personally in paying cash first and then doing a mortgage. It seems you didnt get some of the benefits. You have a big capital gains tax bill this year and ive heard that having a mortgage could exacerbate sequence of return risk. Not sure how doing it this way is an inflation hedge?
There were really only two benefits to me personally, speed and seller incentive. (See $ discussion here.)
>ive heard that having a mortgage could exacerbate sequence of return risk
Possibly – when comparing two situations:
a) a house with no mortgage because you paid cash
b) a house with a mortgage
In A you have a lower cost of living from a smaller portfolio. Is your withdrawal rate higher or lower than in B? If higher, then you have greater sequence of returns risk.
How much higher? If in A your withdrawal rate is 2% and B it is 4%, then clearly A is better.
But what if the withdrawal rate is 2% vs 2.2%? No big deal.
Or what about situation C – you rent the same house? That is highly similar to B and what we were doing until 2 months ago.
> Not sure how doing it this way is an inflation hedge?
Real estate is an inflation hedge because home and rent prices go up with inflation.
Debt is an inflation hedge because inflation makes the debt smaller.
I’m borrowing at 2.75% – inflation from time of purchase to now is 0.5%. For the calendar year it is 5.5%.
Timely article, I’ve just been investigating this myself as we prepare for my husband to retire in January (I’m retired already). Through just an internet search, I found an asset backed lender that offers these loans, and offers a more generous formula in valuing assets for income, namely, dividing the asset value (net of tax discount for some assets) by 84 months, rather than 360 months as the FNMA guidelines do. https://griffinfunding.com/non-qm-mortgages/asset-based-loans/
They only write mortgages in 14 states, so this lender is not available to everyone, but with a little searching they can find something similar.
Curious as to why you’re considering refinancing to a 15 yr.
Most areas don’t allow prepayment penalty’s, so you could pay off a 30 yr in 15.
Rates are lower on 15 yr mortgages but watch out for closing costs and municipality specific gotchas like mortgage recording tax.
Guess I’m saying….when you do it, please show the math. See, just gave you another idea to post about. 😁
No math – just r 15-year < r 30-yearI would only do it if it was profitable to do so, which it may no longer be as mortgage rates have risen these past few weeks
Great info–thanks for sharing! So, without providing any type of endorsements, are you comfortable with naming the 6 lenders you spoke with and the one you eventually went with?
Pick 6. Any 6 will do.
I hear you, but not all lenders are created equal. For example, I’d like to know:
– If a lender communicates well and is fully transparent
– What’s the overall ease in doing business with a particular lender
– If the lender has a history of closing on time without any hiccups
– Erc.
Dunno – I have only one anecdotal example which is no help at all.
You could look at the refinance thread on the bogleheads forum – lots of other anecdotal date points there. Or look at the star ratings and reviews on Bankrate – if they are in the top offers they generally have decent reviews.
Congratulations on your home purchase! I’ve really appreciated your tax articles over the years. I inferred from this and your previous post that you implied you had a six-figure capital gain even after tax-loss harvesting. Can you explain when AMT actually starts to kick in (I get different answers from different calculators post-TCJA exemption) given that I assume your marginal rate for LTCG is 18.8%? I’m in a similar position to you – most of my earned income is under the FEIE so I’d be selling appreciated stock to generate my down payment. Thank you so much!
Thank you sir. Yes, 18.8%.
So… the AMT is a mess and I didn’t predetermine this to the degree I should have because I was busy buying a house, but…
According to the Tax Policy Center, only 0.2% of households making less than $500k will be hit by the TCJA AMT (data.) That’s us!
(This even though you have to calculate the AMT once household income exceeds $114,600 MFJ 2021.)
When I went through form 6251 at a high level I calculated our AMT as small to zero. You still get the FEIE and there are still preferential tax rates for qualified dividends and long-term capital gains. I don’t know if I did it correctly.
I haven’t found a good AMT calc to double check our numbers. I think the best way may be to go through the process of filing an amended return for 2020 and just add an additional large capital gain to see what TurboTax spits out, but I haven’t prioritized doing this. I guess I will find out in a few months…
I have found the banks attitude towards people like ourselves ridiculous. When I wanted a $100K line of credit after retiring they looked a little funny at us. I asked them a simple question – “why do you favor a working person with no assets who could lose their job tomorrow, over a person like myself who is showing you enough assets that they don’t need your LOC?” They approved it but I am still perplexed at the lack of logic on their part when it comes to non-working people with plenty of assets at their disposal.
Hey Jeremy, I just had a similar confusing experience with my long time bank (15 yr client), which is a high net worth private bank. They refused to give me a line of credit or refi becasue I didn’t have earned income??? They said they only count 40%(!!) of your investment income as income, so I don’t qualify for anything basically. Yet I have over $2.2M in home equity (3 homes) and a big investment and reitrement accounts. And no debt. But I’m a “risk”, seriously!
It’s all very odd
Thanks for writing this up. We bought our retirement house before I quit (I was primary salary) and now are trying to decide whether to pay cash for renovations or do something else. Similar to this discussion I think. Perhaps I will just pay cash then try to refinance and pull some money out on the back end. I’m un-retiring to a part-time job anyways, so we might qualify on income again.
This might be impossible or be the dumbest idea ever. Do you think it might be possible to do an account closure IRA transfer but instead doing it electronically, you go the route where they cut you a check and you have 60 days to send it to a new custodian to avoid the taxes. What if you time it so that you purchase the house for cash, using the IRA money, and then immediately do a cash out mortgage and then deposit that money with the new custodian within the 60 day period? I’m a veteran so usually a cash out refi can be done up to the full 100% value of the home. I only ask because I have no brokerage account, only retirement accounts, and will be needing to perform the Roth IRA Conversion Ladder methodology. Thanks for the great article.
Possible, yes. Dangerous though – if you don’t get the money to the new custodian in time you will get hit with penalties and taxes.
It took about 60 days from home purchase to getting the check on the refi…
Banks are more more concerned on your cash flow instead of life time asset. Your mortgage holder need to pay investor monthly. So you need to show your monthly income is sufficient. Asset not equal to monthly cash flow.
that is how the story goes, yes
I hear you, but it still doesn’t make sense from a risk perspective being a person can lose their W2 job tomorrow and then have zero monthly income. Someone with, say, $2 million in a brokerage account still provides less risk.
Person with mortgaged home / job and no capital must work for food and shelter – millstone anchors.
A person with liquid capital can float away – leaving a underwater home.
Interesting stuff. Most mortgage brokers and banks use their ratios and if it does not work then they say no. But there are so many people with non-traditiional jobs these days the model must change.
Interesting. Thank you.
Thanks for this post! I’m in the middle of doing this right now.
Except I’m not doing a cash-out refinance, I’m able to offer cash to the buyer, but concurrently I’m applying for a regular asset-backed mortgage so on closing day I won’t actually have to bring cash to the deal (BUT am prepared to if something happens with lender).
I’m working with either JP Morgan Chase (where most of my assets are) or a local lender in the town where I’m buying the house. Both say they are able to use my assets vs income (my income is also quite low at this point).
You said you paid $0 in closing fees. What about all the regular government fees like transfer stamps and title fees like title insurance, etc? Thanks!
That was all offset by the $3k in lender credits.
Does Schwab/Rocket Mortgage care what stocks or etfs you have in your portfolio? Is that something they take into consideration when deciding to approve/deny you for an asset based loan?
They don’t care
Great article – thanks for sharing. This was really timely for me because we are just about to buy a house in the same price range. I used a bit of a different strategy, the Schwab pledged asset line, which gives you a credit line of about 70 percent of your assets, excluding retirement accounts. I did have to move my assets to Schwab but it was painless. Right now the interest rate is about 2.95. I plan to do a cash out refinance after we purchase just to avoid the risk of interest rates or falling asset values.
Great article – thanks for sharing. This was really timely for me because we are just about to buy a house in the same price range. I used a bit of a different strategy, the Schwab pledged asset line, which gives you a credit line of about 70 percent of your assets, excluding retirement accounts. I did have to move my assets to Schwab but it was painless. Right now the interest rate is about 2.95. I plan to do a cash out refinance after we purchase just to avoid the risk of interest rates or falling asset values.
I’m curious who you ended up going with.
I’m currently working with Rocket Mortgage and was told they count the retirement accounts at $0 until I am 59.5, then went down the path of income and recommended I get….a cosigner.
I think my early retirement stash was at $2.6mil at the time of writing…but 75% in various qualified accounts.
Does anyone have any comments on how to get HELOC after FI?
I am brainstorming the relocation house purchasing scenarios – and did not realize that getting the morg. for relocation before selling will be that hard (including getting the HELOC before the selling).
I like your thinking. Raymond James are also very forward thinking on this topic and work with their clients to be creative. We just did 100% mortgage using a combo of asset backed mortgage and SBL with them.
What is your interest in refinancing to a 15 year from a 30 year?
Lower interest rate
The PNC private loan portfolio (loans which PNC holds on its own books) should be more widely known among the FIRE crowd.
They delivered for us on multiple fronts.
They lend based on assets instead of just income. (I think they used 10% of our taxable accounts and 7% of our retirement assets and assumed that to be income).
They also lend on difficult parcels. Most mortgage originators wouldn’t touch our parcel with a house because 10 acres are rented to a farmer putting it outside of Fannie and Freddie guidelines. Such a loan can’t be sold off which is the business model for most mortgage shops.
A kicker – the rate PNC gave us on a 30 year fixed was lower than the myriad of loan originators out there.
Minimum loan size is $650,000.