Contemplating the same thing myself lately. I'm wanting to go to WDW this fall/winter. We've got 3: one will have turns 3yo in Oct, one will be 14 months, and one will be 2 months.
Right now, I'm leaning towards off-site, unless by some miracle we can come up with the $$$ for the Polynesian (monorail service and I'm not a fan of the Contemporary). To stay at a non-monorail Disney hotel and have to take a bus, with 3 kids and a heavy double stroller, would be just crazy IMO. I imagine you're thinking the same thing (with a double stroller and ECV).
Of course, off-site is no picnic either. You'd be utilizing several modes of transportation to get to your destination (car to WDW, then a shuttle from their parking lot to the entrance). Maybe there's more than that yet -- I can't remember, not having been there in 9 years. Somehow I have it in my mind that off-site would be easier (than using on-site bus service) because you can kind of move at your own pace.
I guess that (for us) the trouble factor is equal. And when that's the case, I go with cheapest. While one of the All-Star resorts might be just as cheap as something off-site, the food off-site is definitely cheaper than at Disney.
I wish I could be more help! Consider the costs of food/hotel/car offsite vs. the costs of food/hotel/airport shuttle onsite, consider daily transportation to/from the parks, consider if you're going to other non-Disney parks (Sea World?), consider if being onsite would "Disney-you-out" vs. make your experience magical, consider if you want the freedom that having a rental car provides.