Good survey questions are the spine of your questionnaire. The way you formulate them is crucial for your research and data quality, but also for the number of participation you will receive. ‘For participation numbers?’, you might ask. Of course!
No one wants to scramble through millions of nonsensical questions, especially if they are volunteering their precious free time to answer your questionnaire. Therefore, by having the wrong questions, you might end up without any participants.
Just like a real spine - if your questions are faulty, the whole questionnaire will break down.
Understood. Then, how to phrase survey questions that actually work?
Below you’ll find some best-practice advice to follow if you want your questions to be any good!
1. Don’t start with the most complicated questions.
Alfred Hitchcock famously said: ‘I want a film that begins with an earthquake’. Although this might apply to movies, don’t apply this thinking for your survey. You will scare off participants that might feel shaken by the first questions and probably would not like to continue with your survey.
Instead: Start off brief and simple. Make questions naturally progress within the questionnaire, both in a logical order and the order of difficulty.
2. Don’t ask two questions at the same time.
Sometimes you want to kill two birds with one stone, especially if you want a short questionnaire. The thing though is, you cannot ask two things at once and receive reliable answers for both.
For example, consider questions such as: ‘How would you rate our website in terms of usability and visual appeal?’. Since usability and visual appeal are two completely separate entities, you cannot mention them in the same sentence and expect to receive liable data.
Instead: Split your question into two separate ones. Or, determine which part of the question is more important for your research and remove the unnecessary part.
3. Don’t ask the same question twice (or do).
This one may seem obvious. After all, who would ask the same thing over and over again? However, if your questionnaire is particularly long, you might overlook overlaps and repetitions. Especially, if questions are phrased differently, but ultimately ask the same thing.
Or maybe you put repetitions for purpose - for example, to rule out straight-liners and speedsters. Although it works in certain circumstances, do not overuse it. Don’t repeat every single question. This will annoy participants and skyrocket your drop-out rates.
Instead: The less the better. Go through your questionnaire again and remove any possible repetitions. For response control, use methods such as control questions to avoid repetitions.
4. Don’t use leading language.
Leading questions are the ones that suggest a certain answer. They usually include emotive language, such as: ‘How much did you like our page’ instead of ‘How would you rate our page’.
As you can see, they prompt a certain - usually positive - feeling, which will later transfer into the data and bias it. This might be a trick used for the company’s marketing purposes, a lazy way to get a wanted result, or simply inexperience in creating a reliable questionnaire. None of these three reasons is something a soon-to-be research expert like you wants to be associated with!
Instead: Remove all emotive language from your questions. Instead of ‘like’ or ‘dislike’ use ‘rate’ or ‘prefer’. Similarly, avoid using adjectives in your questions, especially if they can prompt a certain reaction.
5. Don’t use double negatives or vague language.
Double negatives twist the brain of participants and make questions either difficult to comprehend or too obvious, eg. “Are you opposed to a lack of feedback from our company?”
Similarly with the vague questions, which would, at best, confuse -- eg. ‘How do you feel about the feedback?' (which feedback? To what?)
Instead: Abstain from using negatives in your questions. Moreover, if you are not sure about the comprehension of your questionnaire, ask a trusted friend or colleague to review your questions.
6. Don’t choose a random scale.
Scale questions are very useful as they allow a nuanced view of your subject area. However, they also pose certain problems. For instance: how many points should my scale have? 4? 5? 7?
Different scales work provide different outcomes, so don’t just choose the scale randomly. For example, the larger the scale (eg. 7-point scale), the more detailed answer. Therefore, it is ideal if you looking for a diverse outlook for the investigated topic. But if you are a beginner researcher, a 5-point scale will probably suffice.
Now, you might ask, why not a 3 or 4-point scale? Well, the 3-point scale (Agree-So-so-Disagree) provides very little details on the actual opinion and may lead to extremes. However, if you need a very simple product feedback survey, you might find it useful.
The 4-point scale, on the other hand, does not have a central point and might lead to data distortions.
Instead: Most of the time, the odd-numbered scale works best, because they have a natural centre point, which can be used as a “maybe” or “average”. 5 and 7-point scales are the most commonly used and the most reliable.
7. Don’t force it.
Mandatory answers, especially in open or very specific questions, can easily backfire.
Some participants may write absolute nonsense or drop out. Not always is it caused by their negligence -- sometimes they may have nothing to say or the answer options you provided don’t apply to them. Or they did not understand your question at all.
Instead: Always include neutral answer options, such as “Maybe” or “I don’t know”. If some questions may not be suitable for everyone, make them optional.
That’s it! Hopefully, you now have an idea where to start with your questionnaire and what mistakes you should be mindful of. Please visit our blog for more survey-related tips and content!