Setting up quotas

You can set quotas in your experiments to collect a certain number of responses and prevent oversampling.

The five methods of quota management

Method 1: When sourcing respondents through Conjointly

When you recruit respondents using Predefined panels, the number of responses you specify is automatically applied as quota to prevent oversampling.

Quotas in Self-serve sample should be considered soft quotas (i.e. they are adhered to on a best-effort basis). If you need to add hard quotas, please use Methods 3, 4, or 5 described below.

Method 2: Enable Autopause function

For bring your own respondents and email campaigns, you can limit the number of respondents entering your survey using the Autopause function by following these simple steps.

• Open your experiment and click on the Advanced settings tab.
• Under the Autopause heading, check the Enable autopause box.
• Enter your preferred number of respondents in the Desired number of responses field.
• Click the Save Changes button on the bottom-right of your screen.

Method 3: Add survey flow controls that will terminate respondent flow when quotas are met

You can add quotas into survey flow controls to specify the number of qualifying respondents. The type of the survey flow control will determine the action taken when the quota is met. The options are:

• Screen out: Respondents are tagged as screened out and will not be included in analysis.
• Complete survey: Respondents will be tagged as having completed the survey.
• Flag as low quality: Respondents will be flagged as low quality and will not be included in analysis.

Method 4: Using the Quota check survey flow control

The Quota check survey flow control allows you to set up quotas based on any conditions. It has its own quota fulfilment counter, but it does not terminate respondents (unlike Method 3). Therefore, it needs to be used in logic or formulas.

Method 5: Add quotas based on answers in multiple-choice and dropdown questions

You can set quotas based on answers to multiple choice and dropdown menu questions by following these steps:

• Open the Settings menu by clicking the button next to the attribute you wish to set a quota for.
• Select Redirect (if quota is met) from the Action menu.
• Specify the Quota size and the Redirect URL.

In the example above, our survey will stop accepting male respondents, once the quota of 150 is met.

Monitoring quota fulfilment

We strongly recommend that you track quota progress on the Quota fulfilment tab in your participants page. This tab contains two separate tables.

Responses within each quota

This table shows how many respondents included in analysis are within the limit of each quota. The table displays quota descriptions, limits, breakdowns by participant sources, and fulfillment percentages.

Responses over each quota

This table tracks how many respondents tried to enter a survey when quotas were full. Columns display quota descriptions, limits, over-quota counts by participant source, and running totals. Rows represent quotas from survey controls, answer selections, and paused participant sources.

Both tables can be exported for further analysis.

Nuances of quota management

There are several important considerations when working with quotas:

• Quota counters on the platform are cached for up to one minute in order to render surveys fast. It means that respondents who come into the survey at once (within the same minute) will be assigned the same sequential number and quota controls will work the same for them.

• Quota fulfilment is recorded immediately after the survey is submitted (not when the respondent enters the page). In practice, it means that if you have many respondents answering your survey at the same time, there will likely be some overfulfilment of quotas. In most cases the level of overfulfilment is negligible.

• When you modify quotas after the experiment has been launched:

  • If you are setting up a quota for an option in a multiple choice question (Method 5) and some respondents have already selected that option, your new quota will not count the already-recorded responses. It means that if you already have 30 respondents who selected A and you need a total of 100, your quota value should be 70 (not 100).

  • The same principle works for survey flow controls (Methods 3 and 4) because a new quota fulfilment counter is initiated at the time of specifying the quota for the first time.

  • The only exception is Autopause (Method 2, which does not have its own counter separate from the count of respondents in a source) and therefore will count all responses in its source from the beginning.

  • However, if you had a quota of a certain size and you update the quota size (up or down), the quota will still count the already-recorded responses (because the quota fulfilment counter already exists).

• Only respondents that are good for analysis (i.e. those that can and should be included in the report, excluding those struck out by quality filters) are counted towards quotas. If you include or exclude complete respondents from analysis, the quota counters will be updated accordingly. For example, if you have a quota of size 100 and you have already collected 100 complete responses, but then you decide to exclude 10 of them from analysis, your quota will now show that you have 90 complete respondents in the quota and 10 more are needed to fulfil the quota.