Factors that affect our responses often work subconsciously and remain obscure for a respondent themselves and for an interviewer.
Biased questions can be spotted and rephrased, but response bias is much harder to recognize. It is often caused by reasons beyond an interviewer’s control: students’ mood and general condition, their willingness to answer, social norms, etc.
Even if you cannot get rid of it, it’s better to keep it in mind. This knowledge will help you look at your survey from an attendee’s viewpoint and give you hints on how to gather information more effectively.
It takes place when a respondent uses their reply to the first question as a basis for a reply to the next question.
Let’s look at the order of the following fields:
Knowledge of the topic can be considered as a part of a trainer's performance, which means that the detailed question here precedes the general question. In this situation, when a student is asked to rate the performance in general, the first thing that comes to their mind is how knowledgeable the trainer was because it has been mentioned just before. And they might give their rating based on it alone, ignoring all the other aspects of the performance.
It happens because our mind always looks for a shortcut. To find an answer, we go through the data stored in our memory. Of course, we stumble upon the most recent information first. If it suits our purposes, we tend to use it and don’t try to look further.
But the most recent information is not always what we need. An attendee might be impressed with the trainer’s profound knowledge, but have complaints about the presentation style or the way the trainer interacted with the audience. If they rate the performance high thinking only of knowledge, some serious problems might go unnoticed.
What to do? Placing a general question first is the easiest solution. It helps get rid of the assimilation effect because this way you don’t give respondents any hints.
If there are several detailed questions related to one topic, it’s safe to put them before a general question. A student will have a broader picture and will be less susceptible to the assimilation effect.
If, for some reason, you still need to put one detailed question first, and a general question second, try providing an explanation for the second case. For example, elaborate on what you mean by performance in brackets. Not a very elegant way out, but still better than invalid results.
It’s opposite to the assimilation effect. When answering a general question, respondents exclude the aspect they were asked about before. That’s because they think that if you have already mentioned this one aspect in particular, you mean something else in the next field.
In their answer about the event’s venue, respondents might not take transportation into account. Thus, you will get confusing data when students had various troubles with getting to the location, but at the same time they gave the venue 5 stars out of 5.
What to do? Contrast effect can be avoided the same way as the assimilation effect — either put a general question first, or precede a general question with several detailed questions.
People want to look good in front of others. Even without realizing it, they usually deny traits that can be viewed negatively by a society. This makes it difficult to receive honest answers to delicate questions and to make accurate statistics.
It’s considered that learning and self-improvement are essential qualities of any successful person. Even if a student doesn't have any intention to learn anything new, there is a high chance they won’t admit it and give you a positive answer that would be socially approved.
What to do? Making a questionnaire anonymous helps people relax and share honest thoughts.
If anonymity is not an option, at least assure people that you will keep this information private — for example, in an attached letter or during an event itself.
You can also try to rephrase a sentence to make it sound less aggressive.
If you list possible directions and include “I haven’t decided yet” or “I don’t know” options, it will help two groups of people. You will give hints to those who want to grow but haven’t yet decided how to proceed. And you will add a way out for those who do not intend to raise their skills for now.
Some students give answers based on their assumption of what you need the most. They do not reflect on their actual thoughts and emotions, instead trying to pick the most favorable answer.
Some respondents might think that they help you by choosing “Yes”, so they will mark this reply solely because of this reason. At the same time, they could be satisfied with the current location and don’t see any need changing it.
What to do? Do not leave ground for assumptions and stay as neutral as possible. Ask attendees to share their impressions on the current location or write their suggestions on what can be improved on this matter.
If you get a lot of negative feedback, changing a location can really be a good idea, even if you don’t ask about it directly.
It’s also called the agreement bias. It takes place mostly because respondents don't want to waste any mental effort on the survey so they just agree with everything.
It has its counterpart in the dissent bias, when a respondent always chooses “No”.
When fields of the similar structure and with similar answers go one after another, some of the respondents choose the easiest way out by marking the same response in every field.
What to do? Maintain the balance by using different question types. It will help students not to get bored while answering them.
Do not ask for agreement or disagreement and offer different options instead.
There is still a chance that some students give random replies, but others will feel interested to share their real thoughts.
It stems from the same reason as the acquiescence bias — respondents do not feel motivated enough to think about their replies and mark the same option everywhere.
It happens with rating questions, but can affect multiple choice as well.
What to do? The advice is the same here — balance of question types. It’s better to avoid several rating scales in a row or matrix questions, asking to rate several aspects.
It will be harder for students to click on the same answer, if multiple choice is followed with an open-ended question, then star rating. Or when options in multiple choice are different from the previous field.
Writing an event feedback form is not just asking the right questions. You also need to understand how the human mind works. Knowledge of psychological and sociological sides of a survey can help you better guide your respondents without them falling to biases or becoming bored.
It will transform evaluations from a mere formality into a really powerful tool that will support your events and give them additional value.
And to enhance this tool even further, you can automate the whole process of collecting feedback — sending evaluation requests, structuring and storing reviews and converting them into testimonials.
You can easily do it with Workshop Butler — a software that offers complex solutions designed for training business specifically.
Request a free trial now to learn more about its advantages.