Recall test

You can measure how well respondents remember a brand, product, or concept shown earlier in your survey with the Recall test question. Respondents type what they recall, and their answer is scored against a benchmark value that only you can see, using a Levenshtein distance (edit distance) score to grade how close the match is.

Respondent view

Recall test questions use a standard free-text interface like a standard Short text question, with no special formatting or file uploads required. You can evaluate any stimulus (e.g., product concepts, messaging, or pricing) by placing a question that displays the content prior to the recall test, or by including the question inside a monadic block.

The example video below demonstrates this setup for an adcept evaluation, showing an Intro Text question used to display the ad before asking respondents what they recall:

What you get in the report

Each Recall test question adds descriptive statistics and a Levenshtein distance chart to your report, based on how closely each response matches your benchmark value.

Before scoring, both the benchmark value and respondent answers are normalised (extra whitespace, dots, and commas removed, and case ignored) so minor typos or formatting differences don’t unfairly lower a respondent’s score.

The chart groups responses into score bands with a pennant marking the mean score. Lower scores indicate closer matches:

• 0 (exact match)
• 1 (minor difference)
• 2 (minor differences)
• 3+ (major differences)

In monadic blocks and crosstabs, this question reports the mean Levenshtein distance score alongside the number of respondents shown and the number who answered. It isn’t available in TURF or Timeseries analysis, and doesn’t yet export to PowerPoint.

Setting up a Recall test question

Follow these steps to set up a Recall test question.

1. Add question

Navigate to the Add questions tab and click on the Add question button at the bottom of the page.

On the Add or import questions pop-out, select the Add a new question tab, find Recall test under the “Open-ended input” group, and click on the question card to add it to the experiment.

2. Edit question options

Once you add the question, you can adjust the following settings:

1. Insert the question text in the text box. You can also include formatting and images.
2. Enter the benchmark value, the text your respondents’ answers are scored against. This can be up to 1,000 characters and is never shown to respondents, only used to calculate their recall score.
3. Set the possible number of characters respondents can enter using the From and To fields, to require a minimum answer length or cap it.

If you’re using this question inside a monadic block, you can insert a [View] tag in the question text, and select a monadic view for the benchmark value as well. If the view you select for the benchmark value contains only an image with no text, you won’t be able to save your changes, since there’s nothing for the recall score to compare against.

Once the survey is live, you can still edit the question text, benchmark value character limits, and other settings, but the benchmark value itself is locked and can’t be changed.

FAQs

Can this question be used for segmentation, weighting, or other analysis?

It works with segmentation and weighting, using operators such as equal, not equal, contains, and does not contain (case-sensitive and case-insensitive versions of each), alongside the standard shown/answered operators. It can also be referenced in continuous weighting schemes. It appears in crosstabs, showing the mean Levenshtein distance score across segments.

Does this question count toward duplicate response detection?

No. Recall test answers are excluded from duplicate response checks. Because respondents who correctly recall details will naturally enter similar or identical text, checking this question would risk flagging genuine, accurate responses as duplicates.

Can this question be added to a block?

Yes, to any block except Kano.