How does the Conjointly Claims Test decide on claims to show for diagnostics/association questions?
To ensure that your study is robust, Conjointly Claims Test is configured to show diagnostic questions randomly (out of the claims that were shown in the conjoint part of the survey). That said, there is the adaptive setting that prioritises winning claims (but not on top 5 only) while maintaining the robustness.
It is not recommended to show associations or diagnostics for only the top claims of each respondent as it will introduce bias to the study:
- You are more likely to get high ratings (largely 5/5) for these metrics because consumers are more likely to give higher ratings for claims that they like more (i.e. preference is correlated to diagnostic metrics).
- It is also likely that the middle or less preferred claims will not be shown to enough respondents. Hence, they will have wider confidence intervals.