How to prepare a systematic review and meta-analysis: the methodological approach

Abstract

Aim: This article aimed to provide to the authors a summary of the methodological approach to prepare a systematic review and meta-analysis.

Methods:

The instructions were established to support authors in preparing systematic reviews and meta-analyses, according to the required recommendations.

Conclusion:

The researchers should keep in mind that conduct a systematic review involves rigorous methodological criteria to identify and synthesize all relevant studies on a given topic defined a priori.

Keywords:
systematic review; meta-analysis; evidence-based health care

Introduction

Since the concept of evidence-based medicine was established (later extended to evidence-based health), systematic reviews are considered the highest evidence level to guide health decision-making. However, the methodological quality in conducting and the transparency in reporting a systematic review allow readers to interpret and reproduce the results, improving the usability of the evidence¹. The effort to perform a systematic review is wasted if the authors do not clearly report the methods applied and how the findings were obtained². Indeed, systematic reviews are essential to map, critically assess and synthesize data (with or without meta-analyses) from primary studies focusing on a specific research question. Differently, narrative reviews are mainly descriptive and is focused on a subset of studies based on author selection³. This article aimed to provide to the authors a summary on how to prepare a systematic review according to the required recommendations.

Methods

These instructions were developed to support authors in preparing systematic reviews, according to the required recommendations. Systematic reviews submitted to this journal or elsewhere will be considered for appraisal only if they comply with the following guidelines:

1. Formulating the research question.

A systematic review should be based on a specific and structured research question. Some strategies may be useful to assist authors in formulating the research question, and to determine the eligibility criteria for primary studies. Some study designs are more suitable than others for answering a particular question. For example, the acronym PICO (Population / Intervention / Comparator (s) / Outcomes) - for clinical trials, or PECO (Population / Exposure / Comparator (s) / Outcomes) - for observational studies.

It is important to emphasize that each component of the structured research question should be presented in detail in the systematic review, especially the outcomes of interest and the tools of measurement.

2. Registering the systematic review protocol.

Systematic review protocols should be registered on the PROSPERO database (International database of prospectively registered systematic reviews)⁴ to ensure a comparison between the proposed methods and what has been published and try to avoid duplicating efforts with repeated research questions. The PROSPERO registration number should be presented on the methods section of the systematic review.

3. Planning and conducting a systematic review.

It is recommended that the steps of a systematic review of interventions follow the Cochrane Handbook for Systematic Reviews of Interventions⁵; and the PRISMA statement (Preferred Reporting Items for Systematic Reviews and Meta-Analysis)⁶ to improve the quality of the report.

4. Searching for studies.

Systematic reviews include a sensitive, comprehensive, and reproducible search to identifying studies that meet the eligibility criteria. The search for studies should not be restricted by date, language, or publication status. The search strategies used in the electronic databases must be fully presented in the paper, and with the date of the search. It is also recommended searching in additional sources such as trial register databases and grey literature, as well as manual search. The development of search strategies is a complex step and often requires the experience of an information specialist. Inadequate search strategies may fail to identify relevant studies, leading to bias in the results.

5. Selection of the studies.

The study selection process must be carried out in two stages, by two independent authors. The first step is the selection of the references retrieved through the search strategies, by title and abstracts. In the second stage, the studies that showed potential for eligibility must be analysed in full. Disagreements need to be resolved by a third author. The PRISMA flowchart of the study selection process should be presented in the Results section of a systematic review⁶.

6. Assessing the risk of bias.

The assessment of methodological quality (risk of bias) should be performed for all included studies. Appropriate tools for each study design must be identified and applied by two authors independently. For example, the Cochrane Bias Risk tool (RoB table) for randomized clinical trials, ROBINS-I for non-randomized studies, among others.

7. Synthesizing data.

Data synthesis process ought to be described and presented in detail, emphasizing the required aspects to conduct a meta-analysis (when appropriate - homogeneous studies and available data), such as types of outcome data, estimate effect measures, confidence interval, effect model, software used, heterogeneity analysis, subgroup and sensitivity analysis. Meta-analysis is the statistical combination of data from two or more included studies. The authors should determine what studies are similar enough to be pooled in meta-analysis, under a comparison for each outcome. In some circumstances, when it is not possible to perform the meta-analysis, the results of the included studies should be summarized narratively or using visual presentations including graphs and tables.

8. Assessing the certainty of the evidence.

When concluding the qualitative and quantitative syntheses of the included studies in a systematic review, it is important to assess the certainty of the body of evidence using the GRADE approach (Grading of Recommendations, Assessment, Development, and Evaluations)⁷. The summary of findings table (SoF table) should be created for a given comparison, for each important outcome.

9. Interpreting results.

Review authors should interpret the results, considering the balance between benefits and harms, in order to communicate the conclusions of the systematic review appropriately and help people to make better informed health decisions. It is also important to distinguish between “no evidence of effect” from “evidence of no effect”. For example, no evidence of effect indicates that there is insufficient data to draw conclusions about an intervention, but it does not always mean that the intervention is not effective (no effect).

Conclusion

The researchers should keep in mind that conduct a systematic review involves rigorous methodological criteria to identify and synthesize all relevant studies on a given topic defined a priori. As with all health studies, it must be well planned, conducted in a systematic, transparent, and methodologically rigorous way, and reported in a way that allows its reproducibility and applicability.

References

• ¹
  Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014; 383: 267-276.
• ²
  Hoffmann TC, Oxman AD, Ioannidis JP, Moher D, Lasserson TJ, Tovey DI, et al. Enhancing the usability of systematic reviews by improving the consideration and description of interventions. BMJ. 2017; 358: j2998.
• ³
  Uman LS. Systematic reviews and meta-analyses. J Can Acad Child Adolesc Psychiatry. 2011;20(1):57-59.
• ⁴
  PROSPERO. Available on https://www.crd.york.ac.uk/prospero/
  » https://www.crd.york.ac.uk/prospero/
• ⁵
  Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane, 2019. Available from www.training.cochrane.org/handbook
  » www.training.cochrane.org/handbook
• ⁶
  Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine / Public Library of Science 2009; 6: e1000097. Available on http://prisma-statement.org/PRISMAStatement/FlowDiagram
  » http://prisma-statement.org/PRISMAStatement/FlowDiagram
• ⁷
  Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011 Apr;64(4):383-94.

• Associate Editor:

  Angelina Zanesco. UNESP/Rio Claro, SP, Brasil.
• *
  Professor at Postgraduate Program in Health and Environment. Universidade Metropolitana de Santos (UNIMES).

Publication Dates

• Publication in this collection
  02 Apr 2021
• Date of issue
  Mar 2021

History

• Received
  13 Dec 2020
• Accepted
  15 Dec 2020