How does the Conjoint.ly Claims Test decide on claims to show for diagnostics/association questions?
To ensure that your study is robust, Conjoint.ly Claims Test is set to show the diagnostic questions randomly. That said, there is the adaptive setting that prioritises winning claims (but not on top 5 only) while maintaining the robustness.
It is not recommended to show associations or diagnostics to only the top claims of each respondent as it will introduce bias to the study:
- You are more likely to get high ratings (largely 5/5) for these metrics because consumers are more likely to give higher ratings for claims that they like more (i.e. preference is correlated to diagnostic metrics).
- It is also likely that the middle or less preferred claims will not be shown to enough respondents. Hence, they will have wider confidence intervals.