Quality assurance for Predefined panels

You receive genuine, attentive responses from qualified participants with every Predefined panel through automated fraud detection and meticulous hand-checking by Conjointly’s expert fieldwork team at each stage of your study.

The entire process comes with complete transparency as you can track your fieldwork progress in real-time. The following sections outline the key quality assurance procedures at each phase of your study.

Before you launch your study

Quality assurance begins before your study even launches through careful respondent selection and feasibility analysis. Respondents sign up to Conjointly’s panel networks and complete extensive profiling questions covering demographics, shopping habits, spending behaviours, and household characteristics. This groundwork enables precise targeting and cost efficiency.

When you request specific respondents, Conjointly incorporates bias check questions alongside the qualifying questions. This approach helps mitigate the risk of acquiescence bias,, which is the tendency for respondents to select favourable answers to qualify for the survey. These unrelated questions serve as quality indicators, helping identify participants who may be providing superficial or inconsistent responses during the screening process.

When you launch a study

Once a study is launched, Conjointly fieldwork team will review the launch request and perform several steps, including but not limited to the following:

• Coding the screener survey based on your selected Predefined panel and setting up quotas.
• Reviewing your survey inputs and providing you with suggestions for fixes and improvements.
• Implementing survey redirects to ensure a smooth survey flow from the screener survey to your main experiment and appropriate end pages.
• Conducting multiple rounds of pre-launch testing to ensure that:
  • Display and screen-out logic, along with quota flows are working as intended.
  • Survey inputs are displaying correctly, especially in multi-language setups.
  • Thorough testing of alpha features in use, while notifying you of essential considerations for using these features.
  • GET variables, including participant IDs and quota variables, are recorded correctly.
  • Redirection of participants back to their respective panels for claiming their participation rewards.
• Preventing duplicated and fraudulent responses by enabling all relevant response quality management options.

To learn more, you may also check out this intuitive guide on Conjointly’s respondent recruitment process.

During the soft launch of your study

After completing all the steps outlined above, your project will proceed to the soft launch phase. During this phase, Conjointly invites 50–100 respondents, depending on your study, to participate in the survey.

During the survey, automated checks are automatically implemented to identify fraud or inattentiveness behaviours.

Throughout the soft launch phase, the fieldwork team carefully examines the collected data and monitors:

• Survey completion time to assess participants’ attentiveness and ensure they are not speeding through the survey.
• Sample parameters such as location, age, gender, etc., to ensure they are consistent with your request.
• Consistency in responses across questions to prevent contradictory answers.
• Open-ended answers to identify if there are copied or repeated comments, nonsensical answers, etc.

During the full launch of your study

Your study is fully launched, and Conjointly invites the remaining sample you have ordered. The fieldwork team will be:

• Monitoring the overall progress of data collection to ensure qualified participants consistently enter the survey.
• Monitoring and filling out the required quotas as specified by the Predefined panel. During this process, the speed of data collection may slow down, especially when we encounter challenging quotas. This expected slowdown in speed is typically reflected in the average speed of the panel.
• Notifying you of any potential delay or request to relax specific quotas when the quotas are more challenging than initially expected.
• Reviewing the preliminary results of your studies to ensure response quality is consistent with the established standards.

Closing fieldwork

Upon reaching the target sample size and meeting all the required quotas, the fieldwork team will:

• Perform another round of quality checks similar to those in the soft launch phase. Poor-quality responses will be removed and replaced, which may lead to fluctuations in the response count.
• Pre-set up key segments based on your specified quotas.
• Upload other respondent variables, which are typically answers to the screening questions along with additional respondent background data if available.

FAQs

What automated quality checks does Conjointly use?

Respondents are automatically marked as low quality if any of the following signs are detected:

• Avoidance of network detections (when respondents deliberately block our network detection systems, indicating potential intent to circumvent our quality control measures).
• Quick completion of the full survey (e.g. within 20 seconds).
• Rapid selection of conjoint choice sets (e.g. within 1 second).
• Presence of a mouse on the device, but no mouse movement is detected.
• Lack of scrolling when it’s needed to read the full question.
• Submission of unusual answers in open-ended questions.
• Any violation against your enabled response quality management options.

Additionally, the system will automatically remove participants with 70% duplicated open-ended answers.

You can review respondent status, including the reasons of them being marked as low-quality on the Participants page.