You might have spent hours creating a perfect questionnaire only to find out that the responses you got from participants are incomplete, nonsensical, or biased. Understandably, you might feel demotivated or even betrayed by people who treated your precious survey this way.
You might wonder: are survey results always this much trouble? Do all researchers have to rummage through tons of unreliable responses to find a few fitting ones? It most certainly is sometimes the case, especially in the era of online surveys.
However, there are some tricks and best practices you can use to prevent you become one of the many:
1. Keep your eye on the target (group)
Before anything else, define your target group and the minimum number of participants you want to reach. Those are the two pillars for your survey results - if you get them wrong, your survey may take longer than appreciated.
First of all, make sure to include questions identifying your target group. Let's say you’re looking for young mums, and have stated so in the survey’s title or introduction. Chances are, some misfitting participants haven’t bothered to read it and decided to participate in the survey anyways. You should ask for their gender, age and if they have kids right at the beginning to make sure that you can sort out everyone who doesn't fit your desired profile (look at point 5: Clean your data).
On PollPool, your survey will only be displayed to people of the age, gender, and citizenship of your choice, which will make gatekeeping of your target group easier.
Moreover, set a safety margin for your participation’s numbers. Always add at least +10 or +15 for your minimal number of responses, so that you will reach your target even after clearing your data.
2. Minimize drop-outs
Good formatting can do wonders for your survey! Not only will it motivate your participants, but it will also make it harder to straightline (to learn what straightline means, look here), speed through, or drop out.
So, how should you format your questionnaire? First of all, avoid one-page surveys. They might seem quick and simple, but they are also a very easy target for unfit participants. Ideally, split your content among multiple short pages and keep the transition between questions and question topics smooth.
As for your questions, make them as clear and short as possible. Avoid multiple sentences in the row, difficult vocabulary or… certain fonts! Yes, it’s better to go with a classic, such as Times New Roman size 11 (who says plain vanilla can’t hit the spot sometimes?). It will make you look professional and won’t deform foreign diacritics. Also, make sure that the font size is well eligible on every device (and across target groups).
Last but not least: double-check your survey before and after you publish it. The design may vary between devices and sometimes the formatting may clash altogether after it was uploaded. And you certainly don’t want that to happen!
3. Share your survey in the right place
Putting your survey on social media and Facebook groups is, generally speaking, a necessity when starting to search for respondents. However, after you do so, you can no longer control who gains access to your questionnaire. Plagiarism or trolling may happen: useless responses may flood your results.
Carefully selecting the right platform you share your survey on may be time-consuming, but it’s worth it. For one, you might want to share your survey only to close friends on Facebook. What is more, you can do a check on the survey-sharing forums and groups, before your share your survey there. Make sure they are active and do not have a lot of spam-posting, as well as have a lot of participants fitting your target group.
Do not forget about other online spaces to gather people for your target group: forums, community apps etc. If you have a survey covering an obscure or under-researched topic, it might be welcomed and shared among the community. Either way, it’s worth looking closely at a desired participants’ habits, to make sure you are able to connect with them on some platform.
To learn more about how to find the right participants, read our article about 3 easy ways of recruiting survey participants.
4. Offer an incentive
This one is short: motivate your participants to respond by offering them a voucher or another type of prize (a shuffle, or a discount code). However, make sure that the prize can only be accessed after they complete the survey thoroughly.
On PollPool, we use a prize system to allow an exchange of survey participants. Our virtual currency, PollCoins, can be obtained after completing a survey on our page and further used for getting your own participants.
5. Clear your data
As I mentioned under point one, not all responses you will receive will be useful. Some people may not fit your target group, others might be straighliners or straight up trolls. In addition, many responses might be incomplete, and thus might need to be removed from your final data.
Cleaning and analyzing your data is the last step in improving your survey results. Any conclusions reached from the survey without this step is unconvincing. Don’t be fooled: your data set before and after cleaning can look completely different!
So, how to clean your data? Data filters are your best friend! If responses from the same person are inconsistent, or there is a strong tendency for only one answer option (eg. the first one), you might have spotted a speedster or a straightliner. Or, in simpler words - an uncaring participant.
That’s it for today! Make sure to read our other articles about survey creation, for example, our 10-part manual for the perfect survey or how to cope with an insufficient amount of survey participants.