Automated script review for survey improvements


Script review is currently an α alpha feature. Have you got any suggestions for how we can improve the functionality? We are always looking for feedback, so feel free to get in touch!

Get recommendations to improve your survey design before launch. The Script Review tab is available at any stage of your survey setup and uses large language models to identify issues that could affect response quality, such as:

  • Spelling and grammar in question text, options, and labels
  • Scale inconsistencies across questions (direction, labeling, terminology)
  • Missing answer options (e.g. “Don’t know”, “Prefer not to say”)
  • Ambiguous wording that could confuse respondents
  • Question order issues that affect survey flow
  • Methodological problems (leading questions, response bias, unclear definitions)
  • Data quality improvements (screening questions, appropriate response scales)

Please note that if your team has disabled LLM features, Script Review will not be available.

To use Script Review:

  1. Navigate to the Script Review tab and click Start review to begin the analysis.

    Start Script Review

  2. On the pop-up Large Language Model (LLM) chat, the system automatically sends a prompt to identify typos, inconsistencies, errors, and opportunities for improvement.

  3. Review the results and continue the conversation to ask follow-up questions, request clarification on specific recommendations, or explore additional improvements for your survey.

Example recommendations

1. Typos & Grammatical Errors

No spelling mistakes, punctuation errors, or grammatical issues were identified in the survey text.

2. Inconsistencies

Scale Direction Inconsistency:

  • Q7 uses a 1-5 scale where 1 = "Not at all important" and 5 = "Very important" (positive direction)
  • Q8 uses a 1-5 scale where 1 = "Strongly disagree" and 5 = "Strongly agree" (positive direction)
  • While both scales run in the same direction, the labeling approach differs unnecessarily between importance ratings and agreement ratings

Terminology Inconsistency:

The survey alternates between "functional foods or consumer goods" and "consumer goods" without clear distinction of what constitutes each category, potentially confusing respondents about scope

3. Errors (Logical & Methodological)

Ambiguous Product Categories:

The term "functional foods or consumer goods with specific nutritional benefits" is vague and could be interpreted differently by respondents (e.g., vitamins, fortified cereals, protein bars, supplements)

Missing Answer Options:

  • Q3 lacks a "Don't know/Can't remember" option for respondents who may not recall their research habits from 5 years ago
  • Q6 includes "Other (please specify)" but the write-in field is labeled simply as "Other" without clear instruction

Potential Response Bias:

Q8 assumes that increased information availability exists, which may lead respondents rather than allowing them to indicate whether they've noticed such an increase

4. Opportunities for Improvement

Clarity & Simplicity:

  • Define "functional foods" and "consumer goods with nutritional benefits" in the introduction with specific examples (e.g., "such as protein bars, fortified cereals, probiotic yogurts, or vitamin supplements")
  • Simplify Q4 by providing more intuitive time ranges that align with typical shopping behaviors

Question Order:

  • Move the age demographic question (Q5) to the end of the survey, as demographic questions are typically placed last
  • Consider placing Q7 (importance rating) before the behavioral frequency questions to establish context

Respondent Experience:

  • Add a progress indicator or estimated completion time in the introduction
  • Consider shortening Q9 or making it optional, as open-ended questions can increase abandonment rates
  • Provide clearer instructions for Q6's "Other" option (e.g., "Other (please specify below)")

Data Quality:

  • Add a "Don't know" or "Not applicable" option to Q3 for respondents who cannot make the 5-year comparison
  • Consider adding a screening question early to ensure respondents actually purchase these types of products

For more guidance on creating effective surveys, see Conjointly’s guide on survey scripting best practices.