Imagine a Colorado Springs–based couple, Jeremy and Irene. They’ve long dreamed of owning a vacation property in the foothills outside Hailey, Idaho. But after 10 years of diligent saving, Jeremy and Irene realize they have no idea how to manage such a large portfolio withdrawal.
Should they take the money out over time, as they search in earnest for their dream home, or wait, and withdraw it all at once? Which investments should they sell? How will the withdrawal affect their taxes? And what can they do to ensure the transaction doesn’t throw the rest of their portfolio off-balance?
“There’s plenty of advice out there about how to save for your goals,” says Rob Williams, CFP, CRPC, vice president of retirement income and wealth management at the Schwab Center for Financial Research, “but very little guidance regarding how to tap your investments once you reach one.”
As a result, many investors approach a sizable withdrawal as they would a smaller one—with potentially negative consequences for both their taxes and overall portfolio performance.
Here are three of the most common mistakes people make when managing a large portfolio withdrawal from a taxable brokerage account or, as the following example uses, an individual retirement account (IRA)—and how to avoid them.
Mistake #1: Withdrawing all at once
Selling substantial assets in a single calendar year—versus staggering the distribution over two or more years—increases your total taxable income and could be enough to bump you into a higher tax bracket.
“Depending on the size of the withdrawal, you might want to split it up over multiple years,” says Hayden Adams, CPA, CFP, director of tax and financial planning at the Schwab Center for Financial Research. “If you don’t, you could get hit with a big tax bill.” Particularly if you’re near the upper end of your tax bracket, spreading a large withdrawal across several years can often result in a significant savings, he says.
To help avoid a big bump in your tax bill, start by figuring out how much money you’ll need and how soon you’ll need it, and work backward from there. Then you can look at several strategies, such as tax-gain harvesting or topping off tax brackets, to get the cash you need with the least amount of tax impact.
Here’s how topping off a tax bracket could work to save taxes… Let’s say, Jeremy and Irene are both age 62 and have determined that they can afford a second home, which they plan on purchasing in 2023. They’ve determined that $50,000 is needed for the down payment and the funds will come from their traditional IRA. For 2021, they have $72,000 of income, which means their taxable income would be $46,900 after taking the $25,100 standard deduction for a married couple — putting them in the 12% marginal tax bracket.
They could take $34,150 out of their IRA this year, without being bumped into the next higher tax bracket of 22%. They would pay roughly $4,098 in taxes (at the 12% tax rate) on the withdrawal. Then the next year they could withdraw $22,668 from their IRA, paying about $2,720 in taxes (at the 12% tax rate) — giving them $50,000 after taxes, for the down payment.1
Mistake #2: Avoiding selling at a loss
Investors have a natural antipathy toward selling investments at a loss. This so-called loss aversion can cause us to overlook our underperforming investments when deciding which assets to sell. “It’s hard for many people to stomach losses, but they can actually be a boon, tax-wise,” Rob says,” if you hold the investment in a taxable brokerage account.
Not all underperforming assets are fit for the chopping block, but those with weak future prospects or that no longer fit your investment strategy are prime candidates. “When you sell an investment for less than you paid, you can use the capital loss to offset capital gains from the sale of other assets in a taxable brokerage account that have appreciated, potentially reducing your tax bill,” says Hayden. This strategy is called tax-loss harvesting, and can reduce taxes on your investments if done wisely.
What’s more, if your capital losses exceed your capital gains from investments held and sold in a taxable brokerage account, you can use those losses to potentially reduce your ordinary taxable income by up to $3,000. Anything above that can be carried forward over to future tax years.
A large withdrawal is also an ideal opportunity to rebalance your portfolio. As withdrawals and market fluctuations alter the proportions of your portfolio holdings, your asset allocation may stray from its target, causing some positions to be overweight and others underweight. “It’s important to keep your portfolio in line with your risk tolerance and time horizon,” Rob says.
Cutting your losses can cut your tax bill
Offsetting capital gains with capital losses—a.k.a. tax-loss harvesting—can potentially lower your taxes.
For illustrative purposes only. The long term capital gains rate of 20% assumes a combined 15% federal rate and 5% state rate. Investors may pay higher or lower long term capital gains rates based on their income and filing status.
Mistake #3: Neglecting your other goals
Jeremy and Irene are likely saving—and investing—for multiple goals, not only their vacation home in Idaho. For this, and other reasons, it makes sense to diversify your savings by the type of account you invest in, as well as the size, and timing, for your goals. This strategy is called tax-diversification.
“One of the benefits of tax diversification is having different types of accounts—taxable brokerage, traditional IRA and 401(k), Roth IRA and 401(k)—you can choose from to minimize the tax impact of a withdrawal,” Hayden says.
Contributions to Roth 401(k)s and Roth IRAs, for example, are made with after-tax dollars, meaning contributions won’t reduce your current taxable income, and you won’t owe taxes on appreciation, income or withdrawals in retirement (provided the account holder is over age 59½ and has held the account for five years or more).
Pre-tax contributions to tax-deferred accounts—which include 401(k)s, 403(b)s and traditional IRAs—generally reduce currant tax bill and aren’t taxed until you withdraw the money. Withdrawals are subject to ordinary income taxes, which can be higher than preferential tax rates on long-term capital gains from sale of assets in taxable accounts, and, if taken prior to age 59½, may be subject to a 10% federal tax penalty (barring certain exceptions).
Meanwhile, withdrawals from a taxable brokerage account may be subject to capital gains rates of 0% to 20%—plus an additional 3.8% Net Income Investment Tax for single filers with a modified adjusted gross income greater than $200,000 ($250,000 if you’re married filing jointly).
Of course, a particularly large withdrawal does not need to come from a single account. Rather, you can pick and choose, based on the overall composition of your portfolio and what makes the most sense for your situation.
When to consider borrowing
If you need access to capital but are hesitant to liquidate part of your portfolio because of tax consequences, such as a down market or other considerations, it might make sense to borrow to fund your goal.
Rob notes that if you were to borrow the funds at an interest rate that’s less than your expected portfolio return, you could come out ahead. Of course, there is no guarantee your portfolio will achieve its stated objective, and you should consider whether you are willing to assume the risk that it won’t.
If you borrow against your home, interest payments may be tax-deductible so long as you use the proceeds to improve your home or purchase a second home2, and your total itemized deduction is larger than your standard deduction. “That can further reduce the cost of borrowing,” Rob says.
You could also consider borrowing against the value of your investments with a margin loan, from a brokerage firm, or with a securities-based line of credit offered by a bank. Both involve risk, and it’s important to understand these risks before borrowing.3
Margin loans and bank-offered security-based lines of credit might make sense for sophisticated investors who have low-volatility assets to borrow against, who are in control of their debt, and for whom the level of risk is appropriate.
Entering into a securities-based line of credit and pledging securities as collateral involves a high degree of risk. Before you decide to apply for a security-based line of credit, make sure you understand the risks.
1 Example assumes a $34,150 withdrawal in year 1 followed by a $22,668 withdrawal in year 2, for a total withdraw of $56,818 over 2 years. $56,818 taxed at 12% equals $6,818 in taxes, leaving 50,000 after taxes for the down payment.
2The Tax Cuts and Jobs Act of 2017, enacted Dec. 22, suspends from 2018 until 2026 the deduction for interest paid on home equity loans and lines of credit, unless they are used to buy, build or substantially improve the taxpayer’s home that secures the loan.
The law imposes a lower dollar limit on mortgages qualifying for the home mortgage interest deduction. Beginning in 2018, taxpayers may only deduct interest on $750,000 of qualified residence loans. The limit is $375,000 for a married taxpayer filing a separate return. These are down from the prior limits of $1 million, or $500,000 for a married taxpayer filing a separate return. The limits apply to the combined amount of loans used to buy, build or substantially improve the taxpayer’s main home and second home.
3 For a bank-offered securities-based line of credit, the lending bank will generally require securities used as collateral to be held in a separate, pledged brokerage account held at a broker-dealer which may be an affiliate of the bank. The bank, in its sole discretion, generally determines the eligible collateral criteria and the loan value of collateral.
What You Can Do Next
Talk to a Financial Consultant about tailoring a large withdrawal to your individual circumstances.
Learn more about Schwab Bank’s security-based line of credit.
Make sure that you’re considering the potential tax implications of your investments and withdrawals. If your tax situation is complicated, you may want to consult a tax professional.