Question Wording

One of the major difficulties in writing good survey questions is getting the wording right. Even slight wording differences can confuse the respondent or lead to incorrect interpretations of the question. Here, I outline some questions you can ask about how you worded each of your survey questions.

Can the Question be Misunderstood?

The survey author has to always be on the lookout for questions that could be misunderstood or confusing. For instance, if you ask a person for their nationality, it might not be clear what you want (Do you want someone from Malaysia to say Malaysian, Asian, or Pacific Islander?). Or, if you ask for marital status, do you want someone to say simply that they are either married or not married? Or, do you want more detail (like divorced, widow/widower, etc.)?

Some terms are just too vague to be useful. For instance, if you ask a question about the “mass media,” what do you mean? The newspapers? Radio? Television?

Here’s one of my favorites. Let’s say you want to know the following:

What kind of headache remedy do you use?

Do you want to know what brand name medicine they take? Do you want to know about “home” remedies? Are you asking whether they prefer a pill, capsule or caplet?

What Assumptions Does the Question Make?

Sometimes we don’t stop to consider how a question will appear from the respondent’s point-of-view. We don’t think about the assumptions behind our questions. For instance, if you ask what social class someone’s in, you assume that they know what social class is and that they think of themselves as being in one. In this kind of case, you may need to use a filter question first to determine whether either of these assumptions is true.

Is the time frame specified?

Whenever you use the words “will”, “could”, “might”, or “may” in a question, you might suspect that the question asks a time-related question. Be sure that, if it does, you have specified the time frame precisely. For instance, you might ask:

Do you think Congress will cut taxes?

or something like

Do you think Congress could successfully resist tax cuts?

Neither of these questions specifies a time frame.

How personal is the wording?

With a change of just a few words, a question can go from being relatively impersonal to probing into your private perspectives. Consider the following three questions, each of which asks about the respondent’s satisfaction with working conditions:

• Are working conditions satisfactory or not satisfactory in the plant where you work?
• Do you feel that working conditions are satisfactory or not satisfactory in the plant where you work?
• Are you personally satisfied with working conditions in the plant where you work?

The first question is stated from a fairly detached, objective viewpoint. The second asks how you “feel.” The last asks whether you are “personally satisfied.” Be sure the questions in your survey are at an appropriate level for your context. And, be sure there is consistency in this across questions in your survey.

Is the wording too direct?

There are times when asking a question too directly may be too threatening or disturbing for respondents. For instance, consider a study where you want to discuss battlefield experiences with former soldiers who experienced trauma. Examine the following three question options:

• How did you feel about being in the war?
• How well did the equipment hold up in the field?
• How well were new recruits trained?

The first question may be too direct. For this population it may elicit powerful negative emotions based on their recollections. The second question is a less direct one. It asks about equipment in the field, but, for this population, may also lead the discussion toward more difficult issues to discuss directly. The last question is probably the least direct and least threatening. Bashing the new recruits is standard protocol in almost any social context. The question is likely to get the respondent talking, recounting anecdotes, without eliciting much stress. Of course, all of this may simply be begging the question. If you are doing a study where the respondents may experience high levels of stress because of the questions you ask, you should reconsider the ethics of doing the study.

Other Wording Issues

The nuances of language guarantee that the task of the question writer will be endlessly complex. Without trying to generate an exhaustive list, here are a few other questions to keep in mind:

• Does the question contain difficult or unclear terminology?
• Does the question make each alternative explicit?
• Is the wording objectionable?
• Is the wording loaded or slanted?