I started going to my school while I was in high school, doing early entry classes. I was damned and determined to go elsewhere -- I applied to quite a few schools (Texas, Brown, Colorado, Minnesota, Meredith and Agnes Scott), but in the end, it came down to the fact that I really liked OU and didn't want to leave.
I chose some of the other schools I was applying to based on where I saw a lot of people from my town going and being very happy -- mainly, Texas and Colorado. They're close, but not too close, which was a draw for me. My mom went to Minnesota, and I was very interested in the small liberal arts school/all female school environment. Brown was a crapshoot; I had no clue I'd even get in, and while I liked it on my campus visit, I knew I wouldn't be happy there long term.
Had I left OU, it probably would have been Meredith or Agnes Scott -- a completely different experience (large private school vs. VERY small liberal arts schools), but one I still think I would have done well with. Trust your gut, and visit as many schools as you can. Make sure you have time to talk to students who are not official representatives of the university. Sit in on a couple of classes if you can. Talk to your school's guidance counselors -- find out where students from your school have gone and done well. Even if you don't want to go where everyone else has gone, knowing that people with your background have been successful in places you're looking at can be reassuring. See if you can get in touch with alumni of your high school and see what they're happy with and what they wish they had done or known.