5 Tips for Creating Engaging Surveys
Creating a survey can be a grueling task. Worse, far too many surveys result in inaccurate or inconclusive data, leaving a team with little added knowledge or understanding. Years of conducting research experiments has taught us important lessons to increase respondent accuracy. While there are many enhancements that can be made to a survey, the most important is increasing engagement. More engaging surveys lead to more willing participants, reliable and valid answers and ultimately cost-effective surveying tools. Here are 5 simple tips to create more engaging surveys:
1. Use Images
Humans are a visual species. About one third of our brain is devoted to processing visual information; this is why, for instance, graphs are such an effective tool for quickly understanding relationships in data and why photos and videos have much higher conversion rates than text alone.
Pictures or photographs associated with answer choices instantly make a question more compelling. When selecting your images, try including something humorous, visually appealing, or conceptually aligned with the question. To avoid selection bias between answer choices, make sure to choose images with the same composition, lighting, etc. whenever possible so people aren't picking a choice due to arbitrary factors. For instance, if you want people to decide between two car choices, don't choose an image with a supermodel in front of one car and some grumpy old man in front of the other.
When you start to look for images, check out one of the many royalty-free stock image libraries out there, like www.123rf.com. For more in-depth image selection considerations, check back for our upcoming blog post by subscribing to our blog.
2. Use the First Person
You probably already know that survey questions should be short, to the point, and active vs. passive. What you might not know is the psychological importance of using the first person. Statements like "I would ...." make questions feel shorter and immediately establish a link to a respondent’s prior experiences. Questions in the first person eliminate the time a respondent needs to imagine himself in the situation and associate that experience with an answer, leading to quicker and more accurate responses.
3. Use Concrete Answer Choices
Offer just a few direct choices that are very distinguishable. As a general research rule of thumb: the more specific and concrete, the better the measurement tool. Likert scales such as “On a scale of 1-7” or “Select of one the following: Very Likely, Somewhat Likely, Likely...” leave both you and the respondent at a loss. The respondent can’t clearly identify an option he relates to and you miss an opportunity for a more specific understanding of a question.
Vague or abstract answer choices such as Likert scales pose two fundamental problems. First, there is a lot of noise due to individual interpretation. Imagine a question that asks respondents how likely they are to purchase your product on a seven-point scale (where 7=Very Likley, 1=Very Unlikely). Two respondents with the exact same likeliness to purchase your product can choose two different answer choices because they interpret the value of the number choices differently.
The second problem with abstract answer choices is that they require an extra level of meta-cognition, in which they are forced to think about how they would respond to question instead of being asked to directly answer a question. Phrase a question as "I like this desert the best" instead of "I am most likely to choose this desert". This example is closely related to Tip #2, "Use the First Person."
4. Narrow the Scope
A simple, yet common mistake in survey writing is trying to cover a broad area of information through one survey, thinking you will get more bang for your buck. The effect is often overwhelming or fatiguing to a respondent, leading to inaccurate data, or worse, attrition -- where respondents don't even bother to finish. These two problems lead to another issue, which is sampling bias, in which you have skewed your sample size. For instance, if you were to survey over the phone, your sample of respondents would be skewed towards people who have a lot of extra time or are lonely, like older citizens. Instead of asking questions about a user's fashion tastes and sense of humor in the same survey, for example, just focus on sense of humor.
Besides issues with responses, there are also issues with data analysis that are more subtle, but that any accomplished researcher is aware of. For example, it can be harder to spot patterns if they are watered down by lots of irrelevant questions. Moreover, the larger your dataset the easier it is to mistake randomness for something meaningful (what statisticians call overfitting, or mistaking noise for signal). If you flip a coin 50 times, you are much more likely to get a run of 5 heads in a row (even though chances are always 50-50 of flipping heads), than you would if you flipped the coin 5 or 10 times.
5. Usability Test Beforehand
One of the main goals of a survey creator should be to create an experience that is as easy and seamless as possible. The last thing you want is participants losing focus on what they are doing because of the confusing information design, user experience, or user interface of your survey. To test the usability, have three co-workers or friends who were not involved in the creation of the survey dry run the most confusing parts of the survey. If possible, watch them take the survey and notice where they get confused or stuck. Try not to say anything while they are taking the survey, as you want them to feel as if they were taking the survey as they most naturally would. You will uncover the majority of your survey's most problematic usability issues by watching just three people.
What steps do you take when preparing a survey? What has been your most successful use of surveys for collecting insight? Comment below or tweet @tiptap with your response.