What is the ideal sample size for my survey?
Also: What can I do if I receive fewer responses than the recommended sample size?
- To get a general overview with 15-20 claims, a sample size of N=200 would be sufficient.
- If you are looking to do segmentation on the report, we recommend having at least N=100 in each of the segment. For example, if you want to have a segment of males (assuming 40% or N=100 allocated to the male quota and 60% or N=150 to the female quota), then we should aim for a total sample size of N=250.
What is the drawback of collecting less than the recommended sample size?
The implication of having lower sample size would be the robustness of the data (i.e. wider confidence intervals).
The extent to which survey answers/recommendations would potentially be affected is dependent on the survey itself (whether respondents have strong and clear preferences or not). Ahead of analysis of completed results, we do not know how many responses will be needed. Our sample size estimation is based on experience from past studies and aims at 90% confidence intervals. Reports are set up for 90% CIs because we find it most useful in business settings.
Our platform displays the recommended sample size for your experiment on the left-hand panel (please see below).
What should I do if I receive fewer responses than the recommended sample size?
If you foresee that the number of responses you will be collecting is lower than the current recommended sample size, you can make the following changes, which will lower the recommended sample size:
Selecting the “Identify and zoom in on top claims/variants” setting (for [Claims Test](/products/claims-test/)/[Product Variant Selector](/products/product-variant-selector/) only):
Increasing the number of choice sets per respondent (which will make the survey longer). With this setting, we would also recommend forcing respondents to stay for a few seconds in the choice-based exercise (you can do so from the same tab: “Advanced settings” --> “Response quality warnings”)
- Increasing the number of alternatives per choice set up to 5.
This question from our users was answered on 26 September 2019. If there is anything else you'd like to know, please do not hesitate to contact us.