How does the Conjoint.ly Claims Test decide on claims to show for diagnostics/association questions?
To allow for robust read on an aggregate level, our Claims Test is set to show the diagnostic questions randomly. That said, we do have an adaptive setting that prioritises winning claims (but not on top 5 only).
We would not recommend showing associations/diagnostics to only the top claims of each respondent as it will introduce bias to the study because:
- We are more likely to get high ratings (largely 5/5) for these metrics because consumers are more likely to give higher ratings about claims that they like more (preference is correlated to diagnostic metrics).
- It is also very likely that the middle / less preferred claims will not be shown to enough respondents. Hence, they will have wider confidence intervals.
This question from our users was answered on 23 May 2021. If there is anything else you'd like to know, please do not hesitate to contact us.