Epidemiology: Conducting Surveys, Including Questionnaire Design
A survey is a study design that collects the same data on each case in the sample. This produces a standard set of data for each subject that can be analysed statistically to look for patterns and relationships between the variables assessed. Examples include large national household surveys, such as the Health Survey for England, as well as smaller surveys, for example asking hospital patients about their experiences. Surveys can be used to gather information on a range of topics relevant to healthcare, such as health behaviours and public attitudes.
If well designed, with a large enough representative sample, surveys can generate good generalisable data. However, surveys cannot answer questions about causation; they may generate hypotheses for further research though.1
Designing a survey2
There are several steps which should be considered when designing a survey:
- Establish the goals of the project - what do you want to learn? This will determine who you survey and what you will ask them.
- Determine your sample – different sampling methods are covered elsewhere. However, there are two principal components to choosing a sample:
- What kind of people will you interview? Who are you target population?
- How many people will you interview? This may involve a trade-off between the benefits of a large sample versus the costs of interviewing a large number of people.
- Personal face-to-face interviews
- Telephone surveys
- Mail surveys
- Internet surveys
The principal documentation in a survey is the questionnaire itself, or the paperwork used to record subjects’ responses. However the document linked here, describing the Malaria Indicator Survey3 carried out by the World Health Organisation and other groups, illustrates the range of other documents that can be produced to accompany a survey.
This includes the questionnaires used with participants, but also manuals for both the interviewers and their supervisors, other documentation for training, and guidelines for sampling. Consequently the methods are transparent and could be replicated by other investigators. However, more importantly, it helps to ensure that the methods used are rigorous and there is minimal variation in the techniques used by different interviewers.
Questionnaires may be used as the sole instrument for the collection of survey data or in combination with other instruments of data collection. The degree to which a questionnaire produces data that is relevant and valid to a studies goals and objectives will depend on how well the questionnaire is designed, how well the questions are constructed and how well it is administered.
A valid questionnaire measures what it claims to measure. For example, a self completed questionnaire that seeks to measure food intake may be invalid because it measures what they say they have eaten, not what they have actually eaten.4 Careful consideration of the aims and objectives of the study is crucial if the questionnaire is to yield responses that are both valid and reliable.
Perhaps the most important guidance at the design stage is ‘KISS - keep it short and simple’. If you present a 20 page questionnaire most potential respondents will give up in horror before even starting.2
The language used in a questionnaire should be clear and simple, with short sentences. Abbreviations and jargon should be avoided, and the language should be appropriate for the target audience. Professional production methods will also convey the impression that the questionnaire is important.
The questionnaire should start with non-threatening, interesting items, and ideally the most important questions should be included in the first half as respondents may return incomplete questionnaires. Questions should be grouped in coherent categories.
The question format should be varied to prevent participants producing repetitive answers as their attention wanes, also known as ‘habituation’.
Types of question
The first task when generating questions is to choose the variables you want to measure. This may be done purely theoretically, for example, covering the issues you think are important, by using a focus group or interviews to determine what is important to the target group, or finally by identifying important variables via a literature search. Questions may generate either numerical answers, which can be analysed quantitatively, or text-based examples, which may require a qualitative style of analysis.
There are two main types of question: 4
- Closed questions limit the response to a specified list of possible answers. This may either be ‘Yes/No’ or a multiple choice format. This has a number of advantages for the researcher including providing a set of standard responses that enable researchers to produce aggregated data quickly. However, the richness of potential responses is lower.4 Closed questions may also fail to include all potential responses, leading to biased results. Where appropriate, closed questions should also include an opened ended question for 'Other' or an option for 'Don't Know'.
- Open questions allow the respondent to answer freely. However, if opened ended questions are used, the methods for analysing these responses is more complex.4
Participants may also be asked to mark their answer on a rating or agreement scale. Qualitative study designs, which may involve a more in depth interview with participants, are discussed elsewhere on this website.
Maximising response rates
In order to produce results that are as representative as possible, it is important to aim for the highest response rate possible. Potential barriers to participant accuracy and honesty include:1
- Excess mental demands – for example, difficulty understanding the question, difficulty in recalling moods and events over time.
- Biases in answering the question – for example, social desirability (seeking to present themselves in the best light) or end avoidance, respondents choosing not to give extreme answers on a continuous scale
Other tips to improve response rates include:
- Notify participants in advance with a letter of introduction outlining the purpose of the study
- Clear and simple layout
- Questions should be clear and concise and avoid the use of technical jargon, long, leading or negative questions
- Inclusion of a stamped addressed envelope if conducting a postal survey, or collection of questionnaires if feasible
- Ensure anonymity where possible, especially if the questionnaire includes sensitive items
- Follow-up of non-responders by telephone or letter
- Rewards for completing the questionnaire, such as a free gift or donation to charity, may also improve responses
There is an extensive literature about the science of questionnaire design and readers are directed to references 2, 4 and 5 below for further reading.
- Green J, Browne J. Principles of Social Research. Open University Press, 2005.
- http://www.surveysystem.com/sdesign.htm - Accessed 31/01/09
- http://www.searo.who.int/LinkFiles/Malaria_Indicator - Accessed 31/01/09
- Boynton, PM, Selecting, designing, and developing your questionnaire. BMJ 2004;328:1312-1315.
- Boynton, PM, Administering, analysing, and reporting your questionnaire. BMJ 2004;328:1372-1375.
© Helen Barratt, Maria Kirwan 2009