How does Conjointly ensure high quality panel responses?

Conjointly takes sample quality seriously. Our researchers and fieldwork team are committed to delivering quality assured completes through a combination of automated and manual quality checks.

Prior to launching your study

We have a series of automated quality checks that the system performs on all our choice-based and MaxDiff studies.

When we are building a predefined panel for your study, we may also choose to include a set of bias check questions. We usually include these questions if the question that qualifies respondents are susceptible to acquiescence bias (‘Yes’/‘No’ questions or questions asking for degree of openness towards a phenomenon/product.) For more information on survey biases, please check out: How to Avoid Respondent Bias in Quantitative Research

After launching your study

Once you have hit the launch button, our fieldwork team members will review the launch request and perform (but not limited to) the following steps:

• Coding the screener survey based the predefined panel that you have selected, including setting up quotas

• Checking through the inputs from your survey (notifying you on any suggestions we have to fix and/or improve your survey)

• Implementing appropriate survey redirects to ensure a smooth respondent flow (from screener survey to your main experiment to the appropriate end pages)

• Several rounds of pre-launch testing to ensure that:

  • display and screen-out logic as well as quota flows are working as intended

  • survey inputs are displaying correctly (especially in multi-language setups)

  • alpha features (if any are being used) are tested thoroughly and you are notified of the potential things to take note of when using the features

  • GET variables (including participant IDs and quota variables) are recorded correctly

  • participants are being redirected back to their respective panels so they can claim their participation reward

• To prevent duplicated and fraudulent responses from entering the survey, we will enable Disable respondents from the same IP address and Disable respondents from the same cookie settings.

Once all of the steps above are performed, we will proceed to soft launch. Typically, during this process, we will be inviting 50-100 responses (depending on your study) into the survey. For more information on the sample recruitment process, please check out our guide on how to get participants for your study

During soft launch

Soft launch is a crucial part to the fieldwork process as during this phase, our team will look through the data collected and check for:

• Length of interview (to see if participants are attentive to the survey in general and not speeding through it)

• Inconsistent sample parameters (e.g. location, age, gender, etc.)

• Inconsistent answers across questions (e.g. if there are questions around frequency product category and their sub categories)

• Poor quality open-ended answers (we will look out for copy pasted/repeating comments, answers that does not make sense, etc. )

Once we’ve checked the data quality during the soft launch phase, we will exclude any responses that are deemed as low quality, we will then proceed with the full launch, where we will invite the remaining sample ordered.

During full launch

When a study is in the “full launch” phase, it is often a waiting game for you, but behind the scene, our fieldwork team is hard at work at ensuring that your sample is of the best quality possible. To briefly illustrate the process, our fieldwork team will be:

• monitoring the overall progress of data collection, to ensure that qualified participants are steadily entering the survey

• monitoring and filling out the required quotas as specified by the predefined panel. During this process, the speed of data collection may slow down as we encounter challenging quotas. This expected slowdown in speed is typically reflected in the average speed of the panel. However, occasionally, some quotas may be even more challenging than what we initially expected. When this happens, our research team will notify you of any potential delay/request for certain relaxations on any quotas if needed

• checking out the preliminary results of your studies to make sure there is nothing out of the ordinary

Closing fieldwork

Once we’ve reached the target sample size and fill out all the required quotas, our fieldwork team will be:

• performing another round of quality checks (similar to the steps in “During soft launch”). Poor quality responses will be removed and replaced. This is why sometimes you may notice that the response count is lower and then gets filled up again after a short period of time

• pre-setting up key segments that you may need (mainly from the specified quotas)

• uploading other respondent variables. Typically answers to the questions asked in the screener survey (and other respondent background data if available)