Learn why respondents leave your survey halfway through and what you can do to encourage them to answer the full survey.
When collecting survey data, you can either only collect the submitted answers (as in when the respondents have answered all required questions and successfully submitted the survey) or you can collect all the answers given, even if the respondents leave without submitting their survey responses first. At Extellio, we do the latter.
The benefit of collecting all surveys answers is that you will end up with more data. However, that means that not all respondents will have answered the full survey. You will therefore have more data for the first few questions in the survey than you'll have for the last few questions. The share of respondents leaving the survey without answering all required questions is the drop-off rate. If only 80% answer all required questions, the drop-off rate is 20%.
In this article, we'll first go through common reasons for a high drop-off rate, how to calculate your drop-off, and then what you can do to encourage respondents to answer the full survey.
Why respondents leave halfway through
As Extellio collects all survey answers, even if the respondent didn't submit their answers, basically all surveys will have a drop-off. That is perfectly normal. However, the drop-off rate will differ between surveys. Some will have very loyal respondents that will take the time to answer the entire survey, and some will have more impatient respondents. However, the survey itself also affects the drop-off rate. Extellio can't increase the loyalty of the respondents, but we can help to improve your survey. Here are some reasons why the respondents leave surveys without submitting their answers:
- Long survey
Survey respondents are generally impatient, and even more so when answering an online survey. The respondents may be fine with answering the first five questions, but when they realize that you have ten more, they may give up and leave the survey. - Text answers
Text answers (questions where respondents write their own answer) are great when you can't predict how the respondents will answer, or don't want to influence how they answer. However, they also take more time than selecting from a list of options. As the respondents are impatient, realizing they have to write a lot can make them leave. - Unexpected questions
Follow-up question are great for gaining more knowledge from only the relevant respondents. However, as only the respondents that have answered the "right" options will see these questions, they will receive more questions than they perhaps expected too. New questions that keep poppig up can make the survey feel never-ending. - Personal questions
It is a common mistake to collect demographic data (age, gender, city, education, etc) without planning or needing to use the data. These are also questions that can be perceived as personal, and some respondents don't want to give you this information. If these questions are required, they will instead choose to leave the survey.
How to calculate your drop-off rate
There is a simple way of getting an estimate of your drop-off rate. Calculate the share of respondents who answered the last required question compared to the first required question. This will not be an exact calculation, but it will be close enough. If only 64% answered both the first and last question, that means about a third of your respondents leave your survey before the last required question.
Unfortunately, there is no clear limit for when a drop-off rate becomes too high (partly because it is also dependent of industry and loyalty), but staying below 25% is a good goal. If it's higher than that, it's time to start evaluating how you can improve your survey.
How to decrease the drop-off rate
As mentioned, the drop-off is not only dependent of the survey, but there are some things you can do to your survey that should decrease the drop-off rate:
- Keep the survey short
As the respondents are impatient, and you have asked them to give up their time for your benefit, keep the survey short to maintain their interest. There are a few ways of keeping the number of questions down:- Only ask relevant questions
When creating a survey, you should have the purpose of it clear to you. Why do you have this survey? What do you want to know? When you have the answers to those questions, it will be much easier for you to ask relevant questions in your survey. If some of the questions you have written down or included don't help you to get the answers you need, then those questions should not be in the survey. Never include questions "for the sake of it". The purpose of the survey may come to change later on, and then you can change your questions. - Use several surveys if needed
Even though you have a clear purpose and only a list of relevant questions, you may still end up with too many questions. That's when it may be a good idea to split it up into several surveys. One general survey for the general questions, and then one or more targeted surveys for the more specific questions. For example, you may want to know a lot about how the website visitors experience the product pages, and instead of adding 3-5 extra questions to the general survey, you can complement with a targeted survey on the product pages only, where you ask these specific questions. That way, you'll have two shorter surveys instead of one long. - Avoid demographic questions
Unless demographic data is needed to give you the answers you need, or will be included in the analysis in any way, demographic questions (age, gender, education, city, etc) should not be included in the survey. Information such as device, browser used, and country, for example, is automatically collected when the respondents answer the survey.
- Only ask relevant questions
- Only use text answers when needed
Text answers are great when you cannot predict the answer or don't want to influence how the respondents answer, but text answers require more of the respondents time. Therefore, you should use them sparingly. If you notice that two different text answer questions give you approximately the same information, you can remove one of them. If you have text answer questions with very few answers (usually the case for follow-up questions where the share of the option controlling them is small), you can again often remove the question (unless those 12 answers are useful despite being few). - Use a logical structure
If you want the respondents to take the time to answer your survey, the respondents need to know that you have taken the time to think about the survey and the questions that you ask. If the questions are in the wrong order, follow-up questions are in the wrong place, or you jump between subjects in a confusing way, you have not given the respondents the respect they deserve for answering your survey.
- Give the respondents fair expectations
Use the intro text to inform the respondents of how long it will take to answer the questions, or how many questions there are, as well as a clear purpose of the survey. The longer the survey, the more important it is to give accurate expectations. If you tell the respondents that it will take "a couple of minutes" and it takes 5 minutes, you have given them the wrong expectation. Realizing this can make them leave the survey.
The general rule for decreasing the drop-off rate is to shorten the survey. If you compare not only the first and last required questions but also some of the questions in between, you may be able to pinpoint if there is a particular question that seems to cause the respondents too leave. Sudden drops in responses are often caused by follow-up questions that unexpectedly increases the number of questions to answer. That could indicate the need of an additional survey, as you may still want to ask those questions.