Yes, you can see all of both HP sections in one day, even if it is busy. The baby will be allowed on the train, in the Ollivander wand shop for the wand choosing ceremony, can watch any stage shows, can go to any meet-n-greets, and can go through the tour only portion of FJ. For everything else, you will need to either use single rider line or child swap.
I'd start in US and head all the way back to Diagon Alley first. Ride Gringotts either single rider (since you will have to ride alone anyway) or use child swap if you can get there early enough that the standby line isn't crazy.
Some other things you may want to do in DA is to go in the robe shop and try on outfits for fun picture ops, check out Knocturn Alley, visit the sweet shop, visit the animal shop, and get a wand so you can do spells. Also, be careful of the dragon who shoots real fire; don't want to scare the little one.
Then, I'd take the train over to IOA and do Hogsmeade. You can all do the tour only option of the castle as a family without even needing a locker or anything. Then you can do single rider or child swap for the FJ ride depending on crowd. You can send mail by owl post if you bring a letter and stamp at the owlery. You can then have lunch in the 3 Broomsticks.
There are also stage shows in both parks you may want to see.
If you have time left over, I'd check out the play areas in both parks for toddlers. They are really cute and very nice. One of them has a neat ball pit.