Gosh I really don't know how far you want to travel but our winter house rental was in Madison, CT which is on the Long Island shore of CT.
Our rental was right on the shore and next to Hammonassett State Park which made for nice nature walks. It's a little town with a well stocked supermarket (Stop and Shop) and a nearby...Pathmark(?), produce shops, fishmongers, and such plus some nice restaurants, a movie house (mostly art films; you have to go one exit up on 95 to find the multiplex with current movies). It's surrounded by other small towns like Clinton, Bransford, Guilford that are 10 to 30 minutes away. The local patch gives you a heads up on what type of fundraisers the police and fire departments Rotary , Elks and similar clubs are doing. They tend to have fish frys, lobster dinners and the like. Around Christmas they have holiday cakes and cookies for sale. One of the nearby towns (Branford?) has an old fashioned bandshell in the middle of the town green where caroling takes place. Lots of antique and thrift shops mostly on US 1. Various nearby attractions such as a boat tour of the Thimble Islands (you'll have to check if it runs during the winter), Florence Griswold Museum
https://florencegriswoldmuseum.org/visit/programs-events/magic-of-christmas/
Mystic Seaport, the Aquarium, and Village are nearby and the seaport in particular looks beautiful during the holidays.
Bishops Orchard is so much fun even though you'll have missed the fruit picking. Great prepared food for those times you don't feel like cooking as well as fresh produce. Their cider doughnuts are
amazing.:
https://bishopsorchards.com/farm-market/products/
If you like Italian food on the quick (it's a deli/salumeria not a restaurant) try this place:
https://www.saldamarcosdeli.com/
Need some of the best seafood or fresh fish on the entire eastern seaboard (LOL someone in MD is gonna think I'm talking smack but really we rent a car just to grab stuff during the holidays). Really nice prepared food with the RI clam chowder not to be missed what with you not being in RI. Totally different from Manhattan or New England styles.
https://www.saldamarcosdeli.com/
Here's 2 links from CT Tourism that might help you better decide if you want to go there:
http://www.ctvisit.com/
https://maddendigitalbooks.com/ctvg18/
Another nearby area that could be fun is Essex, CT, in the Connecticut River Valley. Off the shore with it's own look and a bigger town center than any of the shore towns. During the summers we've taken car rides on small roads just for the beauty of the landscape. Probably 45 minutes from the shore.
For us the state is relaxing and up until recently where we spent our most of our 4th of Julys for 10 odd years.
Have fun where ever you decide to go
.
