I do not know Le Havre specifically but as a French guy, I would recommend you take extra caution regarding your transportation options especially if you're going on your own — even if on a tour that is not Disney. Le Havre is primarily a cargo port, by no means intended to located near tourist attractions — unlike Civitavecchia for Rome or Piraeus for Athens. Whatever you do please make sure you know your way to your next port of call if you miss the ship. Maybe you won't miss it but I bet at least one party will not make it that day.
My personal suggestions nearby would be to check Étretat, a city very famous for its gorgeous cliffs. For those looking for a more lively atmosphere, Deauville has a very lovely boardwalk, think of it like Cannes / La côte d'Azur but in the Channel. Last but not least, Honfleur, just on the other side of the Seine river, is a truly amazing place, renowned for its many painters that depicted the port. Perfect spot to enjoy French cuisine in a place that is out of time!
By no means I am trying to frighten you and please visit Le Mont St-Michel if you can! Did you know that its shape was an inspiration for the castle at
Disneyland Paris? Definitely worth a visit!