In park surveys a sham?

[Lesley]

The first time I was approached by a CM with a survey pad in a Disney park I was impressed and very happy to answer their questions....it looked like a good customer service tactic at the time.

But as I read and hear more about Disney's reasoning behind certain decisions (Character Caravan, DinoRama, etc...) and hear about the kinds of questions others have been asked it sure is looking like they have devised surveys to tell them what they want to hear...that guests true opinions aren't really being valued.

Honestly, it seems like they are taking survey information and using it to see how bad they can really let things get and how much money they can get out of guests with minimal investment into improvements. When we complain later they just whip out the results of these surveys and say "sorry, but this is what guests say they want"

Any thoughts? The last survey I did at WDW was in regards to admission media and was done at MGM. Just makes me wonder what they're going to do to screw us as a result of their findings! I'm wondering if I need to stop participating in these...or if we just need to keep the Miranda warnings in mind "anything you say can and will be used against you by the Disney company"

 

[JeffJewell]

...about how there are three kinds of lies: lies, damned lies, and statistics. It gets even worse if you get to "manage" exactly what the statistics are measuring.

There is no doubt that the reason Disney performs these surveys is to determine where they can "safely" make budget cuts, and there is no doubt that survey questions can be slanted to favor a pre-determined outcome.

In December (this experience is similar to one related by D-R, I think it was), we got exit interviewed at Animal Kingdom, and the slant of the questions was clearly "did you get what you expected from your day Animal Kingdom?" We had precisely two reasons for going to AK: to see The Festival of the Lion King, and to see Tarzan Rocks! My family (matching what seems to be the majority opinion) thought TR! was okay, and thought FotLK was incredible.

My sister took the point in answering the questions, and because we'd had fun and gotten to see what we'd set out to see, all of the answers were "exceeded expectations." So, as far as the exit surveys measured my family's opinion, AK was a roaring success.

Unfortunately, the survey never took into consideration what our baseline "expectations" were. There was no followup question "if you thought it was so all-fired great, why are you leaving before lunchtime?" There was never a question "if AK wasn't included on your AP/Park Hopper, would you have bought a ticket?"

Bottom line, even though the survey read "exceeded expectations," our expectations for AK were much lower than our expectations for any of the other parks, we were in an out in just a few hours, we wouldn't have bothered if there had been an incremental charge for the park, and the only real money Disney made off of us was four large sodas shared between eight of us.

Although I doubt the survey writers were specifically chartered with the task of "get the guests to screw themselves with the answers," I do think the surveys are designed to make Disney look as good as possible, under the circumstances. If you're trying to cover your @$$ in front of your corporate bosses, that's a job well done.

If, however, you are honestly trying to figure out why your park is a ghost town late in the day, the surveys failed miserably.

Jeff

PS: And remember, these surveys are just tools; the results could be used for good or evil. It's unfortunate that the main use to which the tools have so far been applied is to answer the question "how little can we give the guest and still have them pay us for it," but that's the function of a human decision maker.

 

[barreloflaughs]

...about how there are three kinds of lies: lies, damned lies, and statistics. It gets even worse if you get to "manage" exactly what the statistics are measuring.

There is a great old Peanuts strip in which Lucy is standing at the pitchers mound listing one depressing statistic after another for poor old Charlie Brown. This many losses, this many home runs given up, this many runs against....finally Charlie Brown has heard enough and deadpans to Lucy:
"Tell your statistics to shut up".

Kind of sums up my feeling on these surveys! I always get the feeling that I'm being 'manipulated' to give the answer that the survey writer wants to hear!

carl
barrel of laughs

 

[PKS44]

the "old" saying is Mark Twain's...

Garbage in Garbage Out could also apply here. As a stockholder I am deeply disturbed by most recent moves and seriously considering dumping my stock and taking the loss...I have foolishly thought Disney and Eisner would figure out what is going wrong, but that appears to be a more hopeful optimism than based on any real signs of improvement on the horizon.

After AK and DCA- it amazes me that the guy in charge has not done a major bloodletting...before those two parks the history for Disney was open a new park and revenues and stock price increased...those two open and ...THUD! "Oops! Uh, it's not our fault, it's the economy... People are happy look at our surveys..."

I still don't undertand how those execs explain how OLC succeeds in Tokyo in the worst economy for the last 10 years that Japan has seen since WWII. "Oh, their market is different, they spend much more money per guest..." well duh! you mean the parks are more successful because the guests spend more money? Isn't that how it is supposed to work?

How about the genius who said "Let's put Millionaire on 4 x a week! if once is popular, 4 x will be 4 x as popular!" Another asset mishandled and lost because they were not clever enough to fill the schedule with something new..."but the survey said people liked the show..."

it is time to change management personel or style...or both

Paul

 

[Planogirl]

You know what Lesley? I've wondered about this.

A few years ago we were surveyed in MGM. I remember that many of the questions were general and one was whether or not we had used early entry. I said that we hadn't and no explanation was requested. Therefore, am I partially responsible for the demise of EE? We hadn't used it that day because we had somewhere else to be that morning but at the time we did use it on some days. And we always liked it since it seemed to pull a lot of the crowd to the EE park.

I don't think that I like surveys.

 

[raidermatt]

Therefore, am I partially responsible for the demise of EE?

Nah. If they were really using the survey to gather info and not justify a pre-determined conclusion, they would allow a small percentage for those who normally used EE, but didn't that day. Also, there would have been some people there who only used EE once in awhile, but happened to use it that day. So they would at least partially offset your group. Also, if properly done, the surveys would have been done on different days at different parks, so anomolies like your group wouldn't be over-represented.

I've asked this question before, but nobody has answered. I'm not sure if its because I'm on too many ignore lists, or if its just a dumb questions, but I'll ask it again anyway:

To find out how many people used EE, couldn't they just check their turnstile counts before letting the general public in? Would they really need to do a survey?

Maybe the survey was to find out how people who used EE felt about certain things vs. those who didn't. As opposed to just finding out how many people used EE.

 

[Eeyore2U]

To find out how many people used EE, couldn't they just check their turnstile counts before letting the general public in? Would they really need to do a survey?

That would be too easy!!!

 

[Planogirl]

LOL Eeyore2u!

And good points all, raidermatt. I just remember thinking about how it was odd that one of the lady's questions was about early entry and that she didn't ask for any details when I said that we didn't use it that day. She also asked if we were staying onsite or offsite, where, for how long, did we have hoppers, did we like sequels... (Just kidding about the last one. )

 

[Dizzyworld]

I had written to Guest Service about the removal of EE. They called me to talk for aprrox. 30 minutes about my letter. They said the reason was for stopping the EE mornings is when they asked people why they were at the park during EE, the response was for a character breakfast. This is why they leaped to saying people really wanted more interaction with the charaters. I said I'd like to see the questions and how they were slanted. Of course, not one question was read to me. The CM I talked to said they feed back has been very positive on the characters at the hotels. They said they have received very little negative feedback about not holding the EE.

The CM also said that when Universal opens its new hotel, they will stop offering the front of the line access.

The other thing I was told is that the people who read these boards are such a small group of people. Therefore, they don't represent any real numbers.

Talk about spin doctors........

People need to keep writing letters.

 

[OnWithTheShow]

I can tell you that we at the Disney MGM Studios pay close attention to guest surveys. For each month we recieve results from guest satisfaction surveys. These numbers compare areas within the park (attractions, merchandise, quick service) within the park and to other parks (along with # attractions, quality of experience etc) we are always striving to be number one in every area.

 

[Safari Steve]

That goes for the rest of property as well. The surveys are given at all parks with various topics (overall satisfaction, where you came from, what you liked most/least) and are considered an important tool for guest satisfaction. The idea is, if WDW knows what guests want, things can be improved and sharpened, which will lead to increased popularity. To say that the surveys are designed ultimately and exclusively to justify cutbacks is to say that the bottom line is not important. So now I'm seeing two different arguments from the same viewpoint: The bottom line is the only thing execs look at, and the bottom line doesn't matter as long as the surveys make us feel good. Guess which is ultimately true? Niether.

 

[larworth]

There is always the temptation to slant surveys to get a hoped for result. Which way a slant might occur probably depends on who’s paying the bill. Are the park survey departments on local park payroll? If so, it would be interesting to know what oversight corporate has in the process. You could see where a park manager who might want a bigger slice of the capital budget, would really like to have survey results that showed his park really needs a new attraction.

I figured survey results must be shared regularly with the cast, both for operational and motivational reasons. I understand why you guys have held out on us about trends (positive or negative) in guest sentiment, but maybe you can comment on how well you see the process is done.

It sounds like survey results are taken very seriously internally. I will assume from this the cast believes these are good reflections of guest sentiment? With the cycle between repeat customers being pretty long, how quickly can you get reliable feedback on how changes in park policy are being received?

 

[barreloflaughs]

dizzyworld wrote:

The other thing I was told is that the people who read these boards are such a small group of people. Therefore, they don't represent any real numbers.

I would be curious to know what they consider a significant number. At last count there were 36,735 registered users of the DIS discussion boards. This number obviously doesn't include those who lurk on the boards without registering (as I did for a year or so before registering). It also doesn't include folks who contribute/lurk exclusively on the myriad other Dis boards/mail lists. That adds up to a 'whole bunch of people' (sorry for the use of highly technical statistical terms there.

carl
barrel of laughs

 

[EUROPA]

The CM also said that when Universal opens its new hotel, they will stop offering the front of the line access.

Once again bad information from a CM. Universal will not be doing away with front of the line access once the new Pacific hotel opens. They will limit the amount of times that guest may use the front of the line access. But it will still be there for all onsite guest. Step on over to the Universal board this has been broght up many times.


Yes we do feel that Disney slants their survays to get answers that they like. We've taken quite a few and each time my wife and I commented that he survey did not ask important questions nore were they in anyway ment to help them deicide if something should go or stay.


After taking 2 years of stats in college you learn that you can form the questions to get the results that you are after.


We consider lossing EE a major blow to the staying On-site perks. My wife and I have no kids and get plenty of interaction with the anaimals in the parks. I really do wish they wouild bring this back in some fashion or another espically in the more crowded parks like MGM/Disney Stuidos and the Magic Kingdom.

 

[Lesley]

I also wrote to WDW about ee and was called (I'll give the guy points for persistence....my ds accidentally hung up on him once and we played phone tag for about a week!) It was a pleasant enough conversation, but it didn't really give me much information. Very interesting about the surveys... I just wonder how that particular survey was worded. We often would go for ee, take advantage of the less crowded early morning hours, then go to a character meal at about 10. So would my responses have indicated to them that I was there for a character meal? Couldn't they just as easily come to the conclusion that ee was a good thing because it drew folks into the park for character breakfasts (though this may have also indicated a need for more of those at the resorts instead...)

I'm glad to hear that guest opinions are taken seriously. I can see why surveys may be the most efficient way of getting those opinions, yet they are very subject to bias. It would be nice if they were more up front about why they are asking certain questions and gave guests a chance to add comments.

 

[raidermatt]

I understand the frustration at EE being gone, particularly if it was a big part of your touring plans. But really, Disney couldn't have just asked people if they wanted EE or not. I mean, I've never used it and probably never will, but if you ask me would I like Disney to offer EE, of course I'd say yes. Who knows, one day, I might actually use it.

But that kind of survey is just as useless as one that maybe slanted in some other direction. Its like asking guests if they would like lower prices. Well, duh!


Disney had to determine how many guests were using EE, and if they could find a better or less expensive way to meet those needs. Obviously, those who are upset think they messed up, but people who are happy or don't care generally don't speak up. So just because there are complaints on these boards, it doesn't automatically mean they represent a large percentage of guests.

Yes, there are over 36k registered users, and lots of lurkers. But some registered users haven't been to the boards in months, sometimes years. There's a poll on the parks board that's been up for months regarding the cutbacks. Its had 771 responses. 6.61% say they are mad and not going back to Disney. 14.79% were mad and undecided. 41.25% were mad but still going. 37.35% were not even upset.

What I find most significant about this poll, is that despite the supposed outrage by over 36,000 Dis'ers, only 771 have even bothered to respond to the poll! Disney gets nearly that many people on a single bus at MK's closing time!

If Disney is getting flooded with complaints as some speculate, and more importantly, if they lose on-site guests, they'll either backtrack or find something else as an on-site incentive. But its also very possible that there just aren't that many guests who really cared about EE. It doesn't make anybody wrong or right, it just means that there just may not be enough people who feel as strongly about EE as the more outspoken people here on these boards.

I'm not saying I know what the true bottom line impact of eliminating EE will be, only that its at least a possibility that it just won't have much impact at all.

 

[EUROPA]

Ha..ha the "poll" on the Parks board is just a bad as the Disney Polls. There are basically two choices on that Poll are you mad enough to let Early entry cancle you plans or not. That is exactly the same kind of crappy poll that Disney gives out. Its basically asking "how much can we cutt before you don't come back"?


BTW that is the first time that I saw the poll and I come here everyday. More people would notice things if they did not have 6 Important topics posted in every forum. (another beef of mine.l)

 

[raidermatt]

There are four choices:

Mad enough to cancel

Mad, but undecided

Mad, but still going

Not mad at all.

 

[King Triton]

The survey questions are slanted. With each survey, Disney should ask, "What would you change to improve this park? What do you like the best or least about this park?"

King Triton

 

[EUROPA]

Originally posted by raidermatt


There are four choices:

Mad enough to cancel

Mad, but undecided

Mad, but still going

Not mad at all.

Now read each choice and tell me what they mean ?

1 Mad enough to cancel. ( Real meaning : We've cut to much and your not coming back ")

2. Mad but Undecided . ( Real meaning: We might be able to cancle another perk and you still might come?

3 Mad but still going . ( Real meaning: Yeah we can cancle another perk and you will still come.)

4. Not mad at all . (Real meaning : wow this theme park stuff is easy. What if we made you pay to get in and pay to ride each ride also ?)