Technical and methodological limits on Conjointly

Conjointly has limits on experiment complexity, both for technical reasons and to preserve the user experience of both users and respondents.

Each experiment type and analysis method on Conjointly supports a maximum level of complexity, referred to as the technical limit. This is a hard limit on the maximum number of attributes, levels, etc., that is supported.

While the technical limit is the maximum permissible limit for each method, creating experiments with this level of complexity can result in other issues due to hardware limitations of the Conjointly user and survey respondents. These issues may include the experiment settings page crashing due to a lack of user memory, or an excessive loading time for respondents trying to access the questionnaire.

To ensure an optimal experience for both users and survey respondents, Conjointly establishes a recommended limit on the maximum number of attributes, levels, etc., included in experiments. These limits are derived from analysing the performance of experiments, using an environment similar to that of the average user of our platform.

Survey participant limit on Conjointly

Surveys on Conjointly have a recommended entry limit of 100,000 total participants. Exceeding this limit may result in excessive loading times for users navigating the report. Experiments have a technical entry limit of 300,000 respondents. Once exceeded, experiments will no longer be able to continue collecting responses.

Audience size limit on Conjointly

My Audiences on Conjointly can inlcude a maximum of 50,000 contacts.

Maximum number of segments per survey

You can use Segmentation to categorise your respondents based on different variables and break down your report by segments accordingly. Conjointly supports a maximum of 50 segments per report.

Hierarchical Bayes participant limit

Conjointly uses Markov Chain Monte Carlo Hierarchical Bayes (MCMC HB) estimation for discrete choice experiments (DCE). For all experiments containing DCE methods on Conjointly the recommended limit is 10,000 complete responses. Exceeding 10,000 complete responses is considered bad practice, and may result in technical issues.

If you are looking to exceed this limit in your experiment, please reach out to one of our researchers to discuss alternative solutions.

Methodological concerns

Both the technical and recommended limits on Conjointly were designed to be considerably higher than that required for the vast majority of research projects. When approaching these limits, consider that designing an experiment in such a way is likely not methodologically recommended. In these scenarios, considering alternative approaches will likely result in more robust results and less technical concerns.

For example:

• When you are looking to run a Brand-Specific Conjoint with close to 100 levels, consider a 2-stage study:
  1. Using a methodology such as a Generic Conjoint or Claims Test to reduce the number of levels.
  2. Running Brand-Specific Conjoint on the winning features of stage 1.
• When you are setting up a Brand-Price Trade-Off with a large number of SKUs (e.g. 100+ SKUs), consider reducing that number by:
  • Removing SKUs that have small volume shares (less than 1%) with the assumption preferences for those SKUs will be represented by the larger SKUs within the brand. You should do this consistently across your brands as well as competitor brands.
  • Only including SKUs with similar pack sizes. Apart from reducing the number of SKUs, adopting this guideline will often lead to making your results more robust. For example, a single pack 100ml SKU will likely be in minimal competition with a multipack 8x100ml SKU. There are likely to be purchased in different location as well as occasions, which will likely introduce some noise to the results.
  • Removing SKUs that are from a different category as the main category you are focusing on.

If you are in doubt when setting up your studies, please do not hesitate to book a call with one of our researchers.

Technical and recommended limits by method

Generic Conjoint

Brand-Specific Conjoint

MaxDiff Analysis

Brand-Price Trade-Off

TURF Analysis Simulator

Preference share simulator

To learn more about the experimental design on Conjointly, please refer to the technical points on DCE with Conjointly.