LibGuides: Critical appraisal for Health Professionals: 3. Checklists

Using the checklists

The Cochrane Glossary defines critical appraisal as:

"The process of assessing and interpreting evidence by systematically considering its validity, results and relevance."

The checklists will help you consider these three areas as part of your critical appraisal. See the following tabs for an overview.

This means that you need to examine the extent to which the study is free from bias.

There will be particular biases to look out for, depending on the study type.

For example, the checklists will help you to scrutinise:

Was the study design appropriate for the research question?
How were participants selected? Has there been an attempt to minimise bias in this selection process?
Were potential ethical issues addressed?
Was there any failure to account for subjects dropping out of the study?

The checklists will help you assess:

How was data collected and analysed?
Are the results reliable?
Are the results statistically significant?

The following e- resources, developed by the University of Nottingham, may be useful when appraising quantitative studies:

Finally, the checklists will assist you in determining:

Can you use the results in your situation?
How applicable are they to your patient?

Where do I look for this information?

Most articles follow the IMRAD format; Introduction, Methods, Results and Discussion (Greenhalgh, 2014, p. 28), with an abstract at the beginning.

The table below shows where in the article you might look to answer your questions:

Question	Where do I look?
What was the research question?	Abstract Introduction
What was the research design?	Abstract Methods
Who was recruited to the study?	Methods
What was the primary outcome being measured?	Abstract Introduction Methods
What were the findings?	Results

Greenhalgh, T. (2014). How to read a paper: the basics of evidence-based medicine. 5th ed. Chichester: Wiley.

Checklists and tools

Checklists for different study types

There are different checklists for different study types, as each are prone to different biases.

The following tabs will give you an overview of some of the different study types you will come across, the sort of questions you will need to consider with each, and the checklists you can use.

Not sure what type of study you're looking at? See the Spotting the study design guide from the Centre for Evidence-Based Medicine for more help.

What is a systematic review?

A review of a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant research, and to collect and analyse data from the studies that are included in the review. Statistical methods (meta-analysis) may or may not be used to analyse and summarise the results of the included studies.

From Cochrane Glossary

Some questions to ask when critically appraising a systematic review:

Do you think all the important, relevant studies were included?
Did the review’s authors do enough to assess quality of the included studies?
If the results of the review have been combined, was it reasonable to do so?

From: Critical Appraisal Skills Programme (2018). CASP Systematic Review Checklist. [online] Available at: https://casp-uk.net/casp-tools-checklists/. Accessed: 22/08/2018

Checklists you can use to critically appraise a systematic review:

What is a randomised controlled trial?

An experiment in which two or more interventions, possibly including a control intervention or no intervention, are compared by being randomly allocated to participants. In most trials on intervention is assigned to each individual but sometimes assignment is to defined groups of individuals (for example, in a household) or interventions are assigned within individuals (for example, in different orders or to different parts of the body).

From Cochrane Glossary

Some questions to ask when critically appraising RCTs:

Was the assignment of patients to treatments randomised?
Were patients, health workers and study personnel ‘blind’ to treatment? i.e. could they tell?
Were all of the patients who entered the trial properly accounted for at its conclusion?
Were all participants analysed in the groups in which they were randomised, i.e. was a Intention to treat analysis undertaken?

From: Critical Appraisal Skills Programme (2018). CASP Randomised Controlled Trial Checklist. [online] Available at: https://casp-uk.net/casp-tools-checklists/. Accessed: 22/08/2018

Checklists you can use to critically appraise an RCT:

What is a qualitative study?

Qualitative research is designed to explore the human elements of a given topic, where specific methods are used to examine how individuals see and experience the world...Qualitative methods are best for addressing many of the why questions that researchers have in mind when they develop their projects. Where quantitative approaches are appropriate for examining who has engaged in a behavior or what has happened and while experiments can test particular interventions, these techniques are not designed to explain why certain behaviors occur. Qualitative approaches are typically used to explore new phenomena and to capture individuals’ thoughts, feelings, or interpretations of meaning and process.

From Given, L. (2008) The SAGE Encyclopedia of Qualitative Research Methods. Sage: London.

Some questions to ask when critically appraising a qualitative study:

What was the selection process and was it appropriate?
Were potential ethical issues addressed, such as the potential impact of the researcher on the participants? Has anything been done to limit the effects of this?
Was the data analysis done using explicit, rigorous, and justified methods?

From: Critical Appraisal Skills Programme (2018). CASP Qualitative Checklist. [online] Available at: https://casp-uk.net/casp-tools-checklists/. Accessed: 22/08/2018

Checklists you can use to critically appraise a qualitative study:

CASP checklists

Watch the video for an example of how to critically appraise a qualitative study using the CASP checklist:

Cardiff University critical appraisal checklists

What is a cohort study?

An observational study in which a defined group of people (the cohort) is followed over time. The outcomes of people in subsets of this cohort are compared, to examine people who were exposed or not exposed (or exposed at different levels) to a particular intervention or other factor of interest. A prospective cohort study assembles participants and follows them into the future. A retrospective (or historical) cohort study identifies subjects from past records and follows them from the time of those records to the present. Because subjects are not allocated by the investigator to different interventions or other exposures, adjusted analysis is usually required to minimise the influence of other factors (confounders).

From Cochrane Glossary

Some questions to ask when critically appraising a cohort study

Have there been any attempts to limit selection bias or other types of bias?
Have the authors identified any confounding factors?
Are the results precise and reliable?

From: Critical Appraisal Skills Programme (2018). CASP Cohort Study Checklist. [online] Available at: https://casp-uk.net/casp-tools-checklists/. Accessed: 22/08/2018

Checklists you can use to critically appraise a cohort study:

What is a case-control study?

A study that compares people with a specific disease or outcome of interest (cases) to people from the same population without that disease or outcome (controls), and which seeks to find associations between the outcome and prior exposure to particular risk factors. This design is particularly useful where the outcome is rare and past exposure can be reliably measured. Case-control studies are usually retrospective, but not always.

From Cochrane Glossary

Some questions to ask when critically appraising a case-control study:

Was the recruitment process appropriate? Is there any evidence of selection bias?
Have all confounding factors been accounted for?
How precise is the estimate of the effect? Were confidence intervals given?
Do you believe the results?

From Critical Appraisal Skills Programme (2018). CASP Case Control Study Checklist. [online] Available at: https://casp-uk.net/casp-tools-checklists/. Accessed: 22/08/2018.

Checklists you can use to critically appraise a case-control study:

What is a case report?

A study reporting observations on a single individual.

From Cochrane Glossary

Some questions to ask when critically appraising a case report:

Is the researcher’s perspective clearly described and taken into account?
Are the methods for collecting data clearly described?
Are the methods for analysing the data likely to be valid and reliable?
Are quality control measures used?
Was the analysis repeated by more than one researcher to ensure reliability?
Are the results credible, and if so, are they relevant for practice? Are the results easy to understand?
Are the conclusions drawn justified by the results?
Are the findings of the study transferable to other settings?

From: Roever and Reis (2015), 'Critical Appraisal of a Case Report', Evidence Based Medicine and Practice Vol. 1 (1)

Checklists you can use to critically appraise a case report:

CEBM critical appraisal of a case study