If you haven’t saved enough for retirement you need to prioritize that first. Also, cash is more flexible than home equity in case something else happens in life where you need it. Medical bills, a kid who needs help, whatever. Once the money is spent on the house you can’t get it back out again without borrowing against your equity, now at a much higher rate. Besides, a fixed 3% rate is so low. Even money markets are paying almost 5%. You make more just keeping it in cash. Unlike CDs, money markets are completely liquid.
A lot of people have mentioned tax changes and being able to deduct your mortgage, etc. One thing to keep in mind is the current rules expire at the end of 2025, and it reverts back to the old limits in 2026 unless Congress extends it. Of course this is just speculation, but unless there is a Republican sweep in 2024 an extension doesn’t seem likely to me. I just point that out because it may change the math for you in a couple years.
Another consideration is the social security trust fund is currently scheduled to run out in 2033 unless there is a patch before then. That doesn’t mean you won’t get benefits, but benefits will be cut to about 80% to match the money actually coming in at that time. Personally, I think there will be an 11th hour fix for this because it affects too many people who actually vote, but for planning purposes you may want to be conservative and figure your SS is cut to 80% in 2033 and then be pleasantly surprised if it is not. This weighs in favor of saving more of your money for retirement and not sinking it into your house right away.
There is no reason you have to make the decision now. Again, cash is flexible. I suggest hanging on to it for now and then reassessing when you get closer to retirement and you can do a deeper dive on your actual spending to see whether it’s really necessary or not. Or maybe take an intermediate position where you max out your retirement accounts first and send whatever is left to your mortgage to whittle away at it while you finish your career. Then if you decide to just pay it off in a lump sum on the eve of retirement you can do so, and the remaining balance will be even lower than it is now.
Finally, as an aside - please spend some time with your retirement accounts and learn how the money is invested, what the tax considerations are when you withdraw it, what fees you are paying within the account, etc. This is just my opinion, but learning all of that is way more important than spending time and energy on deciding whether it’s worth paying down a 3% mortgage early. I certainly understand that the mortgage feels more immediate and has an emotional component your retirement accounts may lack, but the state of your retirement accounts matters far more than your current mortgage does in the long run. For instance, your money may just be held in cash in a settlement account if you haven’t been paying attention and you may have lost out on all the growth that has taken place over the last few years. Or you may be sending your money to a fund with high fees. You would be shocked by the fees some funds charge. Many charge 1-2% per year. Dig into the what that actually costs you in real dollars and it’s highway robbery. Also, did you know that if you saved in a traditional 401k you have to pay tax on the money when you take it out? It’s not tax free forever, and those taxes have to be built in as part of your retirement budget. Please spend the time now to learn how all of this works and get your retirement in a good place. Then you can focus on the mortgage. It will still be there in a couple years, and it’s really the simplest piece of the overall puzzle because the rate is fixed.
I suggest lurking on the Bogleheads forums for awhile. You can learn a lot about retirement there and the wiki is pretty good.