Depends on what you all like to do.
I'm assuming that the lure of just exploring a new place or seeing some "minor" historical/cultural things is not high on your list or you wouldn't even have asked this question. If you have been sailing out of Galveston, my guess is that the port of Nassau itself is not any "better" than ports you've been to in Mexico or the Western Caribbean (personally, I've sailed from Galveston a couple of times, and felt all those port stops were nicer than Nassau). So, if things like seeing a pirate museum or old fort aren't appealing to you, and you would have other activities you'd like on board, there's not much reason to get off for the port itself. I don't ever hear people talk about Nassau being a city that is just incredible to see or anything. But, personally our family enjoyed a short carriage "tour" around the port area when our youngest was too young to do anything, and I once enjoyed the walk to the old fort.
I can't speak firsthand to the beaches of Nassau, but I haven't heard especially positive things about them, and I think most everyone would agree that
Castaway Cay is "better". So, unless you would just really like another beach day (again, not sure you would have asked the question if so), I doubt that's a strong reason to get off.
So, that leaves the "big" excursions, with Atlantis Aquaventure being the one that stands out to me. The waterpark there is fun and pretty different from others I've been to (but, wear water shoes - the pavement was dangerously hot...). Our family might not do it a second time, but we were all really glad we did that on one stop in Nassau. But, maybe that's not too appealing to your family, and it's not cheap (though, we did the Comfort Inn thing and it was much cheaper than Disney's excursion).
There are a few other excursions you can do there and it's worth reviewing them, but most of the other excursions seem to me to be similar to those that might have been available on your prior cruises (more private-ish beaches, dolphins, snorkeling, fishing, etc.). So, again you would probably know if those were especially appealing to you or different from what you've already done.
I've been to Nassau 3 times, gotten off the ship each time, and am glad I did so. But, if I were to go again, I very well might stay on the ship.