Different types of questions require different study designs.
There are many sorts of questions that research can address.
Different questions require different study designs. To find out what living with a condition is like, a qualitative study that explores subjective meanings and experiences is required. In contrast, a qualitative study relying only on the subjective beliefs of individuals could be misleading when trying to establish whether an intervention or treatment works. The best design for effectiveness studies is the randomised controlled trial (RCT), discussed below. In a randomised control trial, a hierarchy of evidence exists by which different methods of collecting evidence are graded as to their relative levels of validity. When testing a particular treatment, subjective anecdotal reports of benefit can be misleading and qualitative studies are therefore not appropriate. An extreme example was the fashion for drinking Radithor® a century ago. The death of one keen proponent, Eben Byer, led to the 1932 Wall Street Journal headline, ‘The Radium Water Worked Fine until His Jaw Came Off.'
A cross-sectional survey is a useful design to determine how frequent a particular condition is. However, when determining an accurate prognosis for someone diagnosed with, say, cancer, a cross-sectional survey (which observes people who have the disease and describes their condition) can give a biased result. This is because by selecting people who are alive, a cross-sectional survey systematically selects a group with a better prognosis than average because it ignores those who died. The design needed for a prognosis question is an inception cohort - a study that follows up a recently diagnosed patient and records what happens to them.
It is important to recognise that different questions require different study designs for critical appraisal; first, because you need to choose a paper with the right type of study design for the question that you are seeking to answer and, second, because different study designs are prone to different biases. Thus, when critically appraising a piece of research it is important to ask first of all: did the researchers use the right sort of study design for their question? It is then necessary to check that the researchers tried to minimise the biases (that is, threats to internal validity) associated with any particular study design; these differ between studies.
The Critical Appraisal Skills Programme (CASP) aims to help people develop the skills they need to make sense of scientific evidence. CASP has produced simple critical appraisal checklists for the key study designs. These are not meant to replace considered thought and judgement when reading a paper but are for use as a guide and aide memoire.
CASP have checklists for study designs that address all these types of questions:
All CASP checklists cover three main areas: validity, results and relevance. The validity questions vary according to the type of study being appraised, and provide a method to check that the biases to which that particular study design is prone have been minimised. (The first two or three questions of each checklist are screening questions. If it is not possible to answer ‘yes' to these questions, the paper is unlikely to be helpful and, rather than read on, you should try and find a better paper.) Each critical appraisal e-learning unit interactive learning module in this package will take you through the theory, provide the CASP checklist and a paper for you to read and appraise using the checklist and then work through a guided exercise using the paper and checklist. There is a short quiz that you can use to test yourself or reinforce learning at the end of each unit module.