We go to San Diego from Anaheim 2-3 times a year; here are a few things that we always enjoy doing.
The U.S.S. Midway tour
http://www.midway.org/site/pp.asp?c=coIMKTMCF&b=81432
We have actually done this twice and my wife and two daughters (18, 22) really enjoyed it.
Harbor cruise
http://www.sdhe.com/
Also very enjoyable and relaxing
There is Seaport Village
http://gothere.com/sandiego/seaport/
Lots of shops and restaurants, in the summer evening they have free concerts
And for good food and some Hollywood Star appeal you can go to the Kansas City Barbecue.
http://www.kcbbq.net/
This it the restaurant that was used in one of the scenes from the movie “Top Gun”
Where they are sitting inside a restaurant and “Goose” is playing the piano.
Lots of movie memorabilia on the walls.
Don’t be scared off by its appearance; it is a bar on one side a eating area on the other.
The food was very good, I would compare it to a “Cheers” that’s a little ruff around the edges.
When you walk in you grab your plastic silverware wrapped up in napkins sitting in a box right inside the door
When I asked for a refill of soda the waitress pointed me to the soda machine behind the counter and said “help yourself”
Since you have young children I do need to warn you that in the bar area there are a number of Women’s Support Mechanisms autographed and hanging from the ceiling.