Determining survey questions and their wording

Fév 24, 2019 | Improve your surveys

One of the most difficult aspects of creating a survey is determining the questions and their exact wording.  Even minor changes in the wording can lead to misinterpretation of meaning or confusion. This, in turn, can lead to biased or incorrect data collection, that we dislike.

Writing clear questions takes experience and may be a delicate process for beginners. This article will  broaden your question wording skills.  Below are a series of questions you should ask yourself about your questions and why you need to consider them before sending your questionnaire. So let’s get started…

Are your questions clear? –  You know what you mean, but, is it transparent for everyone else?

 

Good questions are clear, for everyone, even if they are broad.

 

To be clear you do not need to be extremely specific. For example:

“Have you taken a vacation on a weekend, when the weather  was  unpleasant and you discovered your shoes weren’t quite suited to the occasion?”

See what I mean?  That question is NOT CLEAR, but it’s very specific: it adds variables of context, footwear and so on… but what is the question actually asking?  Having too many specifics means that the respondent is going to have to work harder to figure out what their answer is… or they’ll simply drop out of your survey.

So if too many specifics make a question unclear how can we rephrase the above question?

How about: “Have you ever taken a weekend vacation?” or “Do you take vacations on the weekend?”

These two examples bring me to my next point: misinterpretation.

Notice that these two examples have slightly different wording, but they’re asking the same thing right? Well… no. For example, the second question asks if you “take your vacations on the weekend” which could be interpreted to mean “I always use my weekends to go on a trip”. It could also mean “I have taken advantage of the weekend to take a vacation.” Or “I only ever take vacations over the weekend”… Insert other interpretations here.

It’s silly, and with controversy or difficult topics, for instance this problem gets far more insightful. The point is that if the respondent may be able to interpret the question in many different ways then you need to change your wording.  It is best to ask your friends, colleagues to read your questions and interpret the meaning first, this will enable you to see if your wording isn’t quite correct.

 

Misinterpretation is one thing but are you just being downright pedantic?

 

The question is: Are you using terminology that is very specific? Are you using precise technical terms or “using big words” to explain something simple?  Know your audience and realize that no one likes to feel dumb. Use common language and terms people can easily understand. Now if you’re sending your questionnaire exclusively to a group of theoretical physicists concerning black hole thermodynamics or the photoelectric effect… then… well you can use whatever terminology you feel appropriate for that situation… I’m afraid I can’t help you there.

 

Wht r u talkin bout?

 

It is also very important not to use “text speak”, profane speech, or group specific vernacular.

If you use language or vocabulary that is specific to a particular demographic you will alienate your other respondents. For example, if you start talking about Angry Birds… you’re going to lose some people. It is difficult to control how many of your respondents will understand specific cultural references. For example, you could bring up topics such as M.A.S.H., Power Blades, turn-tables, 8-tracks and so on… Each of these would alienate a different group. The exception to this is if you targeted a specific age group or demographic when you published your questionnaire.

Rude language is never !&*#!!? acceptable in a survey. Do not drop f-bombs or include any other profanity in your questionnaire.

 

Is your question actually two?

 

It is important not to ask two (or more) questions in one.  “Obviously!” I hear you protest. Well you’d be surprised how often I see questions like this:

“What do you think of the food and decor of our restaurant?”

Well… make up your mind. Do you want to know about the food or the decor?  They are different things entirely.  Make sure that each of your questions is in fact ONE question.

 

Woah! Did we just change subjects?

 

You can include many different types of questions within the same questionnaire. However, you need to make sure you make transitions clear.  Your respondent is going to be confused and possibly give wrong answers if you ask questions with completely different topics one after then other.

“Where was your last vacation?”

“Do you like Nutella?”

What? Is Nutella related to my last vacation? Are they trying to ask me if I eat Nutella when on vacation?… Add pages to your questionnaire or separate the topics with a subtitle to make it clear to the respondent that you’re moving on to something new.

For now I’ll leave you with these questions to ponder. This list of questions is simply a guide to help you optimize your questionnaires by starting with clear questions.

Even after asking yourself all these questions (and those from part 2 of this article) about your questions there will still be some trial and error.  However, after a couple of questionnaires you’ll start to get the hang of it.

 

Asking the Right Questions

 

When was this exactly?

 

Does your question depend on a certain time frame? There’s a big difference between:

  • Do you watch TV?
  • Do you watch TV every day?
  • How many hours a week do you watch TV?

You need to determine what kind of data you hope to glean and change the wording of your question accordingly.  Note that the more specific you ask your respondent to get the more likely you are to collect “incorrect” data; this is also true of large time-frames.  If you ask your respondent how many cumulative minutes per month they watch TV, you’ve just shot yourself in the foot.  They’re never going to calculate that! It’s too specific and too large a time frame.

There’s also a difference between “in the past week” and “during the course of an average week”.  Realize that if you ask a question with this first time frame you run the risk of getting period specific data.  For example perhaps a respondent went camping the week before answering your questionnaire and therefore watched no TV whatsoever, but on average they watch 15 hours of TV a week.  Or perhaps the week before you sent your survey the Olympics were on and everyone had their TVs on more than they usually would.  That being said it may be easier for the respondent to calculate last week, rather than conceptualize how much they watch in the average week.  You need to be conscious of the difference between these wordings in order to choose the one appropriate for the kind of data you are hoping to get.

 

What units do you want your respondents to answer in?

 

When you asked for how large their kitchen is, did you mention whether you wanted the answer in square feet or square meters? If you asked about salaries, for example, did you want them to write 60 K or 60 000? Be sure that you note the units you want your respondent to use and, if appropriate, an example of what it should look like.

 

Are you making your respondent uncomfortable?

 

Are your questions too personal?  Sometimes we ask questions that make respondents feel uncomfortable even if they are aware the questionnaire is anonymous (medical history, religion, sexual history…). A simple change in wording can turn a question from relatively impersonal to probing and inappropriate.

Keep social norms and morays in mind.  Don’t ask questions that the respondent will feel are too personal to answer.

 

Are you being offensive?

 

Depending on who your respondents are you might need to double check whether your questions are “PC”.  Some groups are easier to offend than others.  Each culture, country, group, etc… has their own triggers that upset them.  Be aware of this and make sure you aren’t upsetting anyone.  You may get unwanted backlash from that kind of mistake.

 

Are you angling for an answer?

 

Are you asking loaded questions? Is your wording likely to get you biased results?  For example, there is a subtle difference between “Please indicate how satisfied you are by our services:” and “Are you very satisfied by our services?”  The second is slightly biased and will lead to different results from the more neutral first example.

 

In conclusion:

 

By now you’re puzzled: “My goodness! Since when has it been so complicated to ask a simple question?!”  It’s a lot to take in, but the main thing to remember is that you need to read your question through the eyes of your respondent. This list of questions you should ask yourself is simply a guide to help you optimize your questionnaires by starting with clear questions. Even after asking yourself all these questions about your questions there will still be some trial and error. However, after a couple of questionnaires you’ll start to get the hang of it.

 

Source: Trochim, William M. The Research Methods Knowledge Base, 2nd Edition. Internet WWW page, at URL: https://www.socialresearchmethods.net/kb/ (version current as of October 20, 2006).

Click and create your online questionnaire!

How to completely get rid of being a spammer?

How to completely get rid of being a spammer?

  Are you a spammer ?   I know you don’t consider yourself a spammer, but spam is not a question of  who you are, but what you do when you send emails.  Spam often occurs  unwittingly and without malicious intent. Take into account the rules and regulations...

Matrix style Question— Why you should use them?

  The “Matrix” question type is a multi-dimensional version of “multiple choice” or “scale” type questions.  It is presented in a tabular form and the respondent must provide one answer per line.  Different types of questions Matrix radio button This question type...