How to Avoid Respondent Bias in Quantitative Research

Posted on 15 July 2020 Catherine Chipeta

Bias in market research is expected, but you can minimise its effects on your study. Learn the common types of survey bias and how to avoid them.

Response bias in market research

Response bias is bias that occurs due to a range of predispositions and conditions (whether knowingly or not) which may influence survey respondents’ answers. It is an unavoidable occurrence in market research (Suchman 1962) but researchers can limit its effects on survey results by understanding what causes it and taking proactive measures. By implementing effective survey design and ensuring questions are both well written and well-formatted, researchers can be more confident that respondents’ answers are more accurate and autonomous (Summers and Hammonds 1969).

In this article, we explore some of the common types of response bias, and how to minimise bias in a study.

Acquiescence bias or ‘Yes’ bias

Acquiescence bias or friendliness bias is a response bias which occurs when a respondent feels inclined to answer a question in a positive or agreeable way. This can happen when the respondent selects options which seem “right”.

For example, in a customer satisfaction survey, the respondent may select “Very satisfied” because it is the most positive option and pleasing to the researcher. Acquiescence bias can also occur if respondents are fatigued and begin to answer questions with minimal thought. It is more prevalent in Asian cultures than in Western countries and varies by other characteristics of respondent groups (Johnson et al. 2005).

To avoid acquiescence bias, the researcher should focus on writing questions that do not lead the respondent to determine that there is a correct or positive answer. Therefore, the researcher should avoid asking questions based on if the respondent agrees with a statement or not. The dual negative-positive scale helps avoid this bias, making results more comparable across countries and subgroups.

Acquiescence bias examples

In the first example, we see that the question leads the respondent to respond positively, by asking them to agree to a positive statement. Conversely, the second example takes a more neutral tone without suggesting to the respondent that there is a “correct” answer.

❌ Bad Example ✔️ Good Example
How much do you agree with the following statement:
"I am very excited by the launch of this new product"
5 - Strongly Agree
4 - Somewhat Agree
3 - Neither Agree nor Disagree
2 - Somewhat Disagree
1 - Strongly Disagree
How excited are you by the launch of this new product?
5 - I am very excited
4 - I am somewhat excited
3 - I am neutral
2 - I am not very excited
1 - I am not excited at all

Acquiescence bias is not only limited to Likert style questions (as shown above), but should also be considered when using binary style formats, such as “Agree/Disagree”. The researcher should review and adjust any question that might lead the respondent to believe that there is a favorable response.

❌ Bad Example ✔️ Good Example
Do you agree with the following statement:
"I found the new product intuitive to use"?
  • Agree
  • Disagree
Did you find the new product intuitive to use?
  • I found the product intuitive
  • I found the product to have a learning curve

Need high quality responses?

We can source the right participants to prevent research bias.

Get respondents

Social desirability bias

Social desirability bias is an imbalance caused by respondents who choose answers based on what they think is socially acceptable (Lindsay and Roxas 2011). This results in answers which either depict a higher number of “desirable” responses or a lower number of “undesirable” responses.

Questions about topics such as health, income, politics, and religion tend to be affected by social bias. For example, respondents might answer the question “How often do you drink alcohol?” with a lower frequency than is actually true. To reduce this form of bias, researchers should anonymise respondents and assure confidentiality, whilst using neutral and non suggestive question wording.

Habituation bias

Habituation bias is a form of response bias which affects respondents when questions are repetitive or phrased similarly. This lowers assertiveness, causing respondents to answer questions based on similar questions they have previously answered.

For example, a survey that uses the question “On a scale of 1-5, how likely would you buy this product”, is likely to suffer from habituation bias. To prevent this, researchers must differentiate question wording, and use an engaging tone to keep respondents alert.

Confirmation bias

Confirmation bias is characterised as the human tendency to seek out information that supports their pre-existing beliefs or opinions and ignore information that does not. It affects researchers during the analysis process as they may use respondent data to validate their original hypothesis but ignore any data that contradicts it.

To reduce the effects of confirmation bias, researchers must remain open-minded and consider all data when evaluating existing hypotheses whilst acknowledging it could be disproven during analysis.

Extreme response bias

Extreme response bias is a common issue for Likert Scale questions, which occurs when respondents answer a question in the extreme way, regardless of if it reflects their actual views. It is usually the result of another underlying form of bias.

For example, if a respondent is affected by acquiescence bias and selects the most “positive” answers during a customer satisfaction survey, this is also an instance of extreme response bias. Habituation bias from habit can also cause extreme response bias as respondents may choose the lowest or highest option as a habit or due to fatigue.

Researchers should ensure there is question variation, unsuggestive phrasing, and respondent anonymity to help reduce extreme response bias.

Non-response bias

Non-response bias or participation bias is a frequently occurring bias, where potential respondents do not participate in or complete a survey (Shultz and Luloff 2009). This can happen for a number of reasons such as respondent fatigue, privacy concerns, complex survey design, poor question wording, or if a survey is irrelevant to the respondent.

For example, if a survey asks too many open response questions, respondents may get overwhelmed and opt not to complete it. To avoid non-response bias, researchers should keep surveys as simple as possible, with clear wording and instruction and seek respondents who are relevant to the survey topic.

Key tips on how to reduce bias in quantitative research

  • Write your questions in a neutral tone to ensure that the respondent is not led to believe that there is a correct answer.

  • Avoid asking if a respondent agrees/disagrees with a statement, as the respondent may be more likely to agree.

  • In some cases, a binary response question may be better suited as a different question type to avoid bias.

  • Respondents may interpret the first option as the “correct answer”, so it is important to randomise the order levels where possible.

Avoid bias in market research

One powerful end-to-end platform for fast and reliable market research.

Start for free