How to set up a benchmark for your studies to know if a feature will be successful or not?


To assess the success or failure of a product in a study within a conjoint analysis, usually the easiest way is to include benchmark products (competition or your current product) into your experiment.

This is because the output of a conjoint, preference scores, are scaleless values that should be interpreted relative to each other. By including benchmark products from competition or your current product, you will be able to test the performance of your conceptual feature relative to one that is already in the market.

When you are interpreting the results for a conjoint analysis with said setup (including a benchmark competition or current SKU), you can use the confidence interval functionality on Conjoint.ly to see if your new features are significantly more preferred than the benchmark. If so, then you can say that the new feature/product will likely be a successful feature/product.

Benchmark - Checking confidence interval of preference

In the example above, the NPD - Conjoint.ly Orange is not significantly different from the benchmark competition - Fresh Lemon and Fresh Orange, we can therefore say that it will likely not be more successful than the benchmark.