Critical appraisal of the literature. Why do we care?

PRACTICAL SCENARIO

Investigators conducted a noninferiority, double-blind clinical trial involving 4,215 patients with mild asthma, randomly assigned to receive twice-daily placebo plus budesonide-formoterol used as needed vs. maintenance therapy with twice-daily budesonide plus terbutaline as needed. They found that budesonide-formoterol used as needed was noninferior to twice-daily budesonide concerning the rate of severe asthma exacerbations but was inferior in controlling symptoms.¹

HOW TO CRITICALLY APPRAISE THE MEDICAL LITERATURE

As clinicians, when we read a paper reporting the benefit of a given intervention, we make a judgment regarding whether we should use those results to inform how we care for our patients. In our example, after reading the paper, we ask ourselves: should a clinician working in a public hospital in Brazil start prescribing budesonide-formoterol as needed rather than maintenance budesonide for her patients with mild asthma? What criteria should guide her decision to adopt a new intervention? One may think that if a study is published in a high-impact, peer-reviewed journal, it is of high quality and should therefore be used to guide clinical decision making. However, if the population included in the study or the context is different from her population, that may not be the case. Therefore, examining the external validity of a study is critical to informing local practice.

Other commonly used criteria are related to evaluating the quality of the evidence by evaluating the type of study design used. The pyramid of evidence puts meta-analyses at the top (as providing the highest quality of evidence), followed by systematic reviews and randomized controlled trials; then come observational studies (cohort, case-control, and cross-sectional studies); whereas case reports and case series are categorized as offering the lowest quality of evidence. Although those criteria may be helpful, making a detailed appraisal of a paper, taking into account aspects other than the study design, is a skill that researchers and clinicians can learn and apply when reading the literature.

Critical appraisal is the systematic evaluation of clinical research papers that helps us establish if the results are valid and if they could be used to inform medical decision in a given local population and context. There are several published guidelines for critically appraising the scientific literature, most of which are structured as checklists and address specific study designs.² Although different appraisal tools may vary, the general structure is shown in Table 1.

Table 1
How to appraise medical literature.

QUESTION	WHAT TO LOOK FOR
Does this study address a clearly focused, important question?	The research question should be clearly stated, and the scope of the study should be focused
Was the study design appropriate for the research question?	The chosen design should be suited to answering the research question
Did the study use valid methods to address this question?	Adequate participant allocation, intervention administration, and outcome assessments
Was systematic bias avoided or minimized?	The groups being compared should be as similar as possible except for the intervention/exposure being studied
Was the primary outcome adequately evaluated?	Assessments should be blinded when possible, measured objectively, and performed for all (or most) participants
Are these valid, significant results applicable to my patient or population?	The study intervention should be available, affordable, and acceptable in your clinical context

The items in Table 1 are a guide to appraising the content of a research article. There are also guidelines for appraising the quality of reporting of health research which focus on the reporting accuracy and completeness of research studies.³ These two types of appraisal (content and reporting) are complementary and should both be used, because it is possible that a research paper has high reporting quality but is not relevant to the context in question.

KEY MESSAGE

Critical appraisal of the literature is an essential skill for researchers and clinicians, and there are easy-to-use guidelines. Clinicians have the responsibility to help patients make health-related decisions, which should be based on high-quality, valid research that is applicable in their context.

REFERENCES

• ¹
  Bateman ED, Reddel HK, O’Byrne PM, Barnes PJ, Zhong N, Keen C, et al. As-Needed Budesonide-Formoterol versus Maintenance Budesonide in Mild Asthma. N Engl J Med. 2018;378(20):1877-1887. https://doi.org/10.1056/NEJMoa1715275
  » https://doi.org/10.1056/NEJMoa1715275
• ²
  Critical Appraisal Skills Programme-CASP [homepage on the Internet]. Oxford (UK): CASP; c2018 [cited 2018 Nov 1]. CASP Checklists. Available from: https://casp-uk.net/casp-tools-checklists/
  » https://casp-uk.net/casp-tools-checklists
• ³
  Equator network [homepage on the Internet]. Oxford (UK): Centre for Statistics in Medicine, University of Oxford [cited 2018 Nov 1]. Available from: https://www.equator-network.org/
  » https://www.equator-network.org

Publication Dates

• Publication in this collection
  Nov-Dec 2018