The Foibles and Flaws of Online Surveys

The Foibles and Flaws of Online Surveys

A copy of Think Like a Freak was an enjoyable and quickly consumed Christmas present, and it has led me to wonder out loud about the value of asking users for feedback on websites and apps.

A Traditional Method for Problem Solving

So you aren’t feeling well: a little nauseous, a bit tired and deflated. You’re guessing it’s nothing too serious – for the majority of your faculties are behaving as they should. But still, it’s always good to know, and you pop down to the GP for a quick check-up.

The queue in the waiting room means your GP probably won’t use all the tools at his disposal. And even though you know full well that he enjoys and endures the foibles and flaws that are incumbent in all men (and women), he delivers a reassurance and direction that comes from training, knowledge and, most importantly, experience.

Although you own your body and you are the only person who knows every pleasure and every rigour that it’s been through, you will listen and learn. It is an implicit trust based on the promise that you will gain from his expertise.

A Digital “Solution”

Now, let’s look at how this scenario might unfold if a common approach to resolving digital design concerns was applied to medical care; a real world acting out of how an online survey works. You know the type – the “please take a few minutes to tell us what you think of our site.”

To get started, you should go and stand in the middle of a motorway, and try to catch the attention of thousands of passing motorists by waving something in their eyes.

Of course the overwhelming majority of people will either not notice you, or will adapt such a pretense until they’ve passed you by.

For the tiny minority who do pull over, your only communication choice is now to announce the symptoms in their most basic form, and demand an on-the-spot diagnosis, while rejecting any further questions about context or history.

Even though few of them have medical training, the pullers-over can all relate to your problems at a certain level. They each do, after all, have bodies of their own, and have had some exposure to the various ailments that come from this privilege.

But you might notice a trend. Those who pull over tend to be either the highly altruistic, the kind of people who look out for each other and want to help even if they do not know how, or else the very frustrated, the kind of people who will feel the need to remonstrate with you for whatever it is you are doing in the middle of the road. Your friends will of course pull over too, but they’ll probably tell you what you want to hear.

After a few days you leave the road, armed with recommendations ranging from the “I know someone who had those symptoms, you have at best 24 hours to live”, to the “go for a walk in the fresh air, I find it’s the best cure for everything”, to the “were you not complaining about this last year too?… and sure you’re still here” and the now inevitable “don’t be standing in the middle of the road if you want to live longer”.

You now have a choice to make:

  1. Take the most common feedback and follow it through, whether you agree with it or not.
  2. Ignore what everybody says and go with what feels instinctively right for your body, after all you know it best.
  3. Visit a doctor and discuss the various recommendations that were made, and watch while he ignores and ridicules most of what you have pertained to learn.

At this point you could invoke a further digital method, and handpick a smaller group from those who stopped – including some who were obviously familiar with the ailments, and some who were not – and ask them to reach a consensus for you. At which point, you face the same three options as before.

Whether it’s their health in question, or their business, I’m guessing not many people will go with option 1.

We actually do use online surveys and focus groups at minus40… but only when they’re appropriate.

If you would like an honest assessment of your digital health, one that is based on experience and expertise, you should get in touch.