Systematic review: general notions

Abstracts

A revisão sistemática é uma metodologia útil em saúde, dado que possibilita identificar as melhores evidências e sintetizá-las, para fundamentar propostas de mudanças nas áreas de prevenção, diagnóstico, tratamento e reabilitação. Este artigo tem como objetivo oferecer subsídios para o planejamento da revisão sistemática respondendo às seguintes perguntas: O que é Prática Baseada em Evidências? O que é revisão sistemática? A revisão sistemática deve estar ancorada em pesquisas qualitativas ou quantitativas? Quais são as fontes de informação pertinentes para identificar as evidências? O que é evidência? Quais os procedimentos para validar a revisão sistemática? Como avaliar a qualidade das evidências? Qual metodologia é empregada para a síntese das evidências? Como planejar a revisão sistemática?

Medicina baseada em evidências; Enfermagem baseada em evidências; Metanálise; Pesquisa em enfermagem; Pesquisa qualitativa

REFLECTION

Systematic review: general notions

Revisión sistemática: nociones generales

Mônica Cecilia De-la-Torre-Ugarte-Guanilo^I; Renata Ferreira Takahashi^II; Maria Rita Bertolozzi^III

^IMidwife. Master in Collective Health Nursing. Doctoral Student at School of Nursing, University of Sao Paulo. Sao Paulo, SP, Brazil. [e-mail]

^IINurse. Master in Sciences. PhD in Nursing. Full-Professor of the Collective Health Nursing Department at School of Nursing, University of Sao Paulo. Sao Paulo, SP, Brazil. [e-mail]

^IIINurse. Master and PhD in Public Health. Full-Professor of the Collective Health Nursing Department at School of Nursing, University of Sao Paulo). Sao Paulo, SP, Brazil. [e-mail]

^{Correspondence addressed to:}

ABSTRACT

Systematic review is a useful methodology in health, as it makes it possible to identify the best evidence and sumamrize them so as to found proposals for changes in the areas of prevention, diagnosis, treatment and rehabilitation. The purpose of this article is to provide support for planning systematic reviews by answering the following questions: What is evidence-based practice? What is systematic review? Should systematic reviews be anchored on qualitative or quantitative research? What are the pertinent information sources to identify the evidence? What is evidence? What are the procedures to validate a systematic review? How can the quality of evidence be evaluated? What methodology is used to summarize the evidence? How should a systematic review be planned?

DESCRIPTORS: Evidence-based medicine; Evidence-based nursing; Meta-analysis; Nursing research; Qualitative research.

RESUMEN

La revisión sistemática es una metodología útil en salud, dado que posibilita identificar las mejores evidencias y sintetizarlas para fundamentar propuestas de cambio en las áreas de prevención, diagnóstico, tratamiento y rehabilitación. Este artículo tiene como objetivo ofrecer ayuda para la planificación de la revisión sistemática, respondiendo a las siguientes preguntas: ¿Qué es la Práctica Basada en Evidencias? ¿Qué es la revisión sistemática? ¿La revisión sistemática debe estar respaldada por investigaciones cualitativas o cuantitativas? ¿Cuáles son las fuentes de información pertinentes para identificar las evidencias? ¿Qué es evidencia? ¿Cuáles son los procedimientos para validar la revisión sistemática? ¿Cómo evaluar la calidad de las evidencias? ¿Qué metodología es utilizada para la síntesis de las evidencias? ¿Cómo planificar la revisión sistemática?

DESCRIPTORES: Medicina basada en evidencia; Enfermería basada en la evidencia; Metanálisis; Investigación en enfermería; Investigación cualitativa.

INTRODUCTION

The need to improve the quality of health practice and teaching reflected on the way that studies are selected and apprehended. In the past, proposals for improvement were based on primary studies. Today, the copious number of scientific productions on one same theme requires systematic reviews (SR) to be performed with the purpose to capture, acknowledge and synthesize scientific evidence (SE) to found qualified practice proposals in health and implement Evidence-Based Practice (EBP).

EBP was initially limited to the clinical environment, but has now expanded to different health areas, embracing themes involving areas such as prevention, diagnosis, treatment, rehabilitation, health policy development, and health service management. This gives evidence of the growing demand to develop SR in hospitals and superior education, as well as by health policy makers and in health service administrators.

An SR should be carefully planned to guarantee the validity of the results, as it provides the foundations of the proposed changes. Therefore, the objective of this article is to guide researchers in planning an SR by answering the following questions: What is EBP? What is SR? Which studies should Systematic Reviews be anchored to: qualitative or quantitative? What are the pertinent information sources to identify SE in an SR? What is SE? What procedures are used to validate an SR? How can the quality of SE be evaluated? What is the methodology used to synthesize SE? How should an SR be planned?

What is Evidence-Based Practice?

It is a movement that appeared due to the need to improve clinical practice and the quality of teaching. It emerged from the need to synthesize the outstanding amount of scientific information and has the purpose to obtain information to found proposals of improvement, implementation and evaluation of the obtained results with a view to improve health care and teaching⁽¹⁾.

EBP appeared first in Canada, in the 1980's, and was limited to clinical medicine, hence the reference to Evidence-Based Medicine - EBM. Later, it was embraced by other fields, including nursing, mental health, physiotherapy, occupational therapy, education and psychotherapy, and addressed subjects related to prevention, diagnosis, treatment and rehabilitation. EBP is also used by health policy makers and health service administrators^(1-3).

Because of its importance, EBP was implemented in different countries including Australia, New Zealand, United States (USA), China, South Africa, Taiwan, Chile, Belgium, Spain, England, and Brazil. EBP is promoted by international institutions such as The Cochrane Collaboration, The Campbell Collaboration and the Joanna Briggs Institute (JBI), which have 15,000 collaborating centers in over 100 countries^(3-5). Other national initiatives, as those in the USA, also promote EBP, in which the National Institutes of Health develop clinical guidelines based on meta-analyses of the best available evidence. The United Kingdom has different EBP Centers that guide the practices of the country's National Health System, such as the Center for Evidence-Based Medicine (Oxford), the Center for Evidence-Based Child Health (London), Center for Evidence-Based Surgical Practice (Manchester), Center for Evidence-based Pathology (Nottingham), and others. England counts with the National Institute for Health and Clinical Excellence (NICE)⁽⁵⁾.

What is Systematic Review?

The SR is a means to obtain the foundations for EBP. It is a rigorous methodology proposed to: identify the studies on one specific theme, applying explicit and systematized search methods; evaluate the quality and validity of those studies, as well as the applicability in the context where changes will be made, so as to select the studies to provide SE and provide their synthesis, with a view to facilitate implementation in EBP^(3-4). Each of these moments is planned on the SR protocol, considering the validation criteria to minimize bias and assign quality to the methodology. All procedures must be registered at the moment they are performed so the SR can be reproduced and verified by other researchers, thus becoming a consistent methodology to ground EBP^(4,6). Therefore, SR differs from a traditional review, also referred to as narrative literature review, in that it answers a more objective question. In order to overcome any possible bias, it is established that each phase be planned using a rigorous protocol about SE search and selection, its evaluation and applicability and, the synthesis and interpretation of the data obtained with the SE⁽⁷⁾.

Which studies should Systematic Reviews be anchored to: qualitative or quantitative?

In research, a qualitative or quantitative approach depends on the study objective and research question. A qualitative approach, for example, is adequate when the aim is to understand the meanings that a heart transplant has for patients with Chagas disease⁽⁸⁾. A quantitative approach, on the other hand, is indicated, for instance, to evaluate the late effects of an interdisciplinary program for patients with fibromyalgia syndrome and compare the effects with the traditional treatment⁽⁹⁾. Similarly, for SR the approach also depends on the objective and proposed question⁽³⁾. According to JBI⁽³⁾, SR using a quantitative approach usually aim at answering questions related to the evaluation of health interventions (regarding therapy, prognosis, prophylaxis, effectiveness, cost, cost-efficiency, cost-minimization, cost-benefit or cost-utility) and of social and educational policies and practices, health services, policy makers, educators, students and researchers. The qualitative approaches of SR permit researchers to understand or interpret issues of social, emotional, and cultural issues, as well as the behaviors, interactions and experiences that take place in the health care environment or in the society, based on the occurrence of a phenomenon, besides supporting the proposition of new theories. It evinces that both approaches are important for developing SR, with the view to answer the broad scope of health care issues and support policy interventions, as recommended by the World Health Organization⁽¹⁰⁾. The objective of SR is to provide SE to the government and health professionals of every health care system level.

What are the pertinent sources of information to identify Scientific Evidence in a Systematic Review?

SE can be obtained from experience, inference or deduction of experts in a specific area, as well as from results of rigorous studies, either quantitative or qualitative. More credibility is given to the SE obtained through research developed with theoretical and methodological rigor. Nonetheless, experts' opinion can best represent SE in cases they are inexistent⁽¹¹⁾.

Choosing the database to find SE depends on the criteria established for the SR. Therefore, databases should be chosen aiming to obtain the best SE. Some of the criteria that authors⁽¹²⁾ take into consideration when selecting databases are: a) Type of study indexed by the database: qualitative or quantitative; b) Knowledge areas: single or multi-disciplinary; c) Addressed theme: specific to the addressed theme or not; d) Access to the databases; c) Available time and budget to obtain the results, i.e., on line, or by borrowing from libraries, etc.

Choosing sources that answer the needs of the SR can be more difficult when only the most commonly used health databases are know (LILACS and MEDLINE). We highlighted some of the databases provided by the Coordination for the Improvement of Higher Education Personnel (in Portuguese: Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES) and the University of São Paulo Integrated Library System (SIBiNet).

CHART 1

More information on databases according to the country of study, databases specific in biology, health promotion, pharmacology, nursing, social and collective health, as well as thesis and dissertation databases can be ser consulted elsewhere^(3,13).

Once the reviewer chooses the databases, there may be some difficulty to find full-text studies. The following are some WebPages that offer free access to full-text studies:

• Portal de Periódicos de Livre Acesso-LivRe: http://livre.cnen.gov.br/Inicial.asp

• Portal da Pesquisa: http://www.portaldapesquisa.com.br/databases/sites

• SciELO: http://www.scielo.br

• Sistema Regional de Información en Línea para Revistas Científicas de América Latina el Caribe, España y Portugal (México)-Latindex: http://www.latindex.org

• Información Bibliotecológica Latinoamericana. Portal do Centro Universitario de Investigaciones Bibliotecológicas (CUIB) de la Universidad Nacional Autónoma de México-INFOBILA: http://infocuib.laborales.unam.mx/~ibt/infoweb.html

• Directory of Open Access Journal. Portal da Lund University Library (Sweden): http://www.doaj.org/

• Open J. Gate. Portal de INFORMATICS (India) LIMITED: http://www.openj-gate.com/Search/QuickSearch.aspx

• Rede Bibliodata. Portal da Fundação Gertulho Vargas (Brazil): http://www8.fgv.br/bibliodata/

• Biblioteca Digital de Teses e Dissertações da Universidade de São Paulo: www.teses.usp.br

In quantitative SR it is recommended to extend the search sources to the maximum, thus including governmental publications, abstracts in congress annals, theses or studies that are not indexed by commercial editors, besides the electronic databases^(3,14). However, in qualitative SR it is recommended to select the indispensible sources or those most closely related to the study theme, besides adjusting and estimating the investigator's capacity to analyze the available articles. That is because the numerous articles make it difficult to deepen the analysis and may pose a threat to the validation of the SR⁽²⁾. However, some authors propose to make the widest selection possible of databases considered appropriate for the focus of the analysis⁽³⁾.

What is Scientific Evidence?

SE evidence is the result of objective and scientific research, obtained using procedures that incorporated validity criteria, minimizing the bias level. In order to be considered as SE, study results must comply with the FAME criteria: Feasibility, Appropriateness, Meaningfulness, Effectiveness. Feasibility occurs when the result can be applied in a certain context considering the physical, cultural and financial aspects. Appropriateness occurs when an intervention is appropriate for a certain situation. Meaningfulness occurs when the patient or target population sees the intervention as a positive experience, personally and in terms of their opinions, values, thoughts, beliefs and interpretations. Effectiveness refers to the extension to which an intervention reaches the expected effect⁽¹¹⁾. The JBI introduced the SUMARI software, which helps to evaluate the FAME criteria⁽³⁾.

What procedures are used to validate a Systematic Review?

The procedures to validate a SR generate results useful for practice. In qualitative SR, the procedures must guarantee the descriptive, interpretative, theoretical and pragmatic valitidy⁽²⁾. Descriptive validity refers to the identification of relevant studies using all the accessible search means. Interpretative validity consists of recognizing the correspondence between the data registered by the reviewer and his or her interpretation of the study content. Theoretical validity concerns the credibility of the methods developed to reach the synthesis of the SE that the reviewer provided. Pragmatic validity refers to the utility, applicability and transference of the knowledge to practice⁽²⁾.

In quantitative SR, although under different names, procedures are also suggested to optimize synthesis validity. The procedure to validate qualitative^(2-3) and quantitative SR^(4,13,15-16) are listed in Chart 2.

How can the quality of Scientific Evidence be evaluated?

With regards to quantitative SR it is indispensible that reviewers have deep knowledge on different methods, statistical analysis, measures or measuring instruments to determine the quality of each study. Different scales have been develop to assist in study evaluation: Delphi, Pedro, OTSeeker, Maastricht Criteria and Jadad Scale⁽¹⁾. The Critical Appraisal Skills Programme - CASP, has also developed an instrument that helps to make an evaluation⁽¹⁷⁾.

Because there is still no consensus about which criteria validate qualitative studies, some EBR researchers recommend using a systematized methodology evaluation using standardized instruments, such as the CASP, for qualitative studies⁽¹⁸⁾.

What is the methodology used to synthesize Scientific Evidence?

In quantitative SR, the synthesis of SE can be descriptive or by meta-analysis. Meta-analysis is a statistical analysis resulted by combining the original article results to produce a single measure of the therapeutic intervention effect, of the accuracy of the diagnosis test, or the studied risk factor. By combining SE it is possible to increase the size of the analyzed population, reduce the confidence interval, reduce the probability of a random result, make a better estimate of the final result, adjust the magnitude of its value and increase the strength of the SE⁽⁶⁾. That analysis is possible is there is similarity between the population considered in the SE or in case it is the same intervention, if there is homogeneity between SE results, considering the way it was measures as well as the direction of the effects favoring one of the compared groups⁽⁶⁾. When meta-analysis is not possible, the SR is descriptive. In that case, the synthesis is a textual summary of the SE characteristics and information considered relevant. That modality has less scientific value compared to meta-analysis⁽³⁾.

As to qualitative SR, it is observed there is a diversity of methodologies that permit to synthesize the SE results. Though each methodology has its own particularities, they can be complementary, with characteristics that juxtapose. Some methodologies prioritize the construction or explanation of theories and some aim at describing a specific phenomenon⁽¹⁹⁾. Meta-study⁽²⁰⁾ and Meta-synthesis⁽²⁾ are examples of those methodologies. The JBI outlines other methodologies, such as: Meta-ethnography, Narrative Synthesis, Thematic Synthesis and - aggregation⁽³⁾.

Some methodologies, such as the Integrative Review, permit to include experimental, non-experimental, empirical and theoretical studies, thus incorporating the contribution of different perspective of one same phenomenon and permitting a more thorough understanding^(21-22).

FIGURE 1

How should an SR be planned?

First, it is important to verify if there is any SR on the theme on Cochrane Library, PubMed or on the JBI library. If any SR on the theme is identified, consider the following: has it been updated over the last three years? Do the methods reflect the specific criteria that are relevant for the studied theme? Is there any specific knowledge gap, in terms of population or intervention?⁽³⁾. If the answer to any of these questions is yes, it is evident there is a need to perform a new SR.

Planning corresponds to designing a protocol, which lists the methodological steps to be followed in the SR to reduce the risk of bias, promote clearness of the methods and processes, and permit peer reviews using the established methods. With the purpose to meet the international SR standards, there are protocols for quantitative and qualitative SR^(3,15). Despite the specificities of those protocols, they do share some characteristics, for instance: justification of the SR, objectives, inclusion criteria, type of study included, search strategy, criteria used to evaluate study validity, forms of extracting and synthesizing, forms of presenting the results, and the conflict of interests statement.

The protocol must be submitted to the Cochrane Review Group or to the Evidence Synthesis Groups (JBI), who can request changes to improve the methodology. When accepted, the protocol is published in the respective databases^(3-4,15).

The Cochrane Collaboration developed the Review Manager Software (RevMan) to help design the RS protocol and development, which helps meet the norms and guidelines, improve the analytical methods and identify errors⁽²³⁾. The JBI also developed the JBI-QARI, JBI-MAStARI, JBI-ACTUARI, and JBI-NOTARI software to manage, evaluate, extract and synthesize SE, aimed at SR of qualitative and quantitative studies, as well as those of economic studies and expert opinion texts and information notes, respectively⁽³⁾.

CONCLUSION

Scientific production, in various health areas, shows a progressive densification, which highlights Systematic Reviews in the process of health improvement, as it makes a substantial contribution to identify the best Scientific Evidence and incorporate tem to practice in health services, teaching, management, and in creating health policies. Nevertheless, developing Systematic Reviews demands a high amount of time and intellectual effort, thus requiring careful planning to optimize time and effort.

The present article presents some fundamental aspects for developing a Systematic Review - cognitive elements, and the necessary human and technical resources - which should be known by the researcher before beginning the process.

Nurses, due to their essential work in health care, should be committed with health promotion, protection, prevention, and recovery, whether in processes of health care, management, teaching or research. That implies the importance to progressively incorporate quantitative and qualitative Systematic Reviews in decision-making processes, with a view to make the incorporation of Scientific Evidence to practice feasible.

REFERENCES

Received: 11/18/2009

Approved: 12/02/2010

• 1. Sampaio RF, Mancini MC. Estudos de revisão sistemática: um guia para síntese criteriosa da evidência científica. Rev Bras Fisioter [Internet]. 2007 [citado 2009 jun. 10];11(1):83-9. Disponível em: http://www.scielo.br/pdf/rbfis/v11n1/12.pdf
• 2. Sandelowski M, Barroso J. Handbook for synthesizing qualitative research. Brainerd: Bang Printing; 2007.
• 3. Joanna Briggs Institute (JBI). Joanna Briggs Institute Reviewers' Manual [Internet]. Adelaide; 2008 [cited 2009 June 10]. Available from: www.joannabriggs.edu.au
• 4. Green S, Higgins JPT, Alderson P, Clarke M, Mulrow CD, Oxman AD. Introduction. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 updated March 2011[Internet]. Melbourne: The Cochrane Collaboration; 2011 [cited 2010 Sept 01]. Available from: www.cochrane-handbook.org
• 5. Joanna Briggs Institute (JBI). Welcome to the Joanna Briggs Institute [Internet]. Adelaide; 2008 [citado 2010 Sept 03]. Available from: www.joannabriggs.edu.au
• 6. Moacyr N, Wanderley B, editores. Prática clínica baseada em evidência. São Paulo: Elsevier; 2001. Revisão sistemática e metanálise; p.145-56.
• 7. Cordeiro AM, Oliveira GM, Rentería JM, Guimarães CA. Revisão sistemática: uma revisão narrativa. Rev Col Bras Cir. 2007;34(6):428-31.
• 8. Sadala MLA, Stolf NAG, Bicudo MAV. Heart transplantation: the experience of patients with Chagas disease. Rev Esc Enferm USP [Internet]. 2009 [cited 2009 Nov 05];43(3):588-95. Available from: http://www.scielo.br/pdf/reeusp/v43n3/en_a13v43n3.pdf
• 9. Souza JB, Bourgault P, Charest J, Marchand S. Escola inter-relacional de fibromialgia: aprendendo a lidar com a dor: estudo clínico randomizado. Rev Bras Reumatol [Internet]. 2008. [citado 2009 nov. 05]; 48(4):218-25. Disponível em: http://www.scielo.br/pdf/rbr/v48n4/v48n4a05.pdf
• http://www.who.int/bulletin/volumes/87/3/09-063719.pdf
• 11. Pearson A, Wiechula R, Court A, Lockwood C. A re-consideration of what constitutes "evidence" in the healthcare professions. Nurs Sci Q. 2007;20(1):85-8.
• 12. Barroso J, Gollop CJ, Sandelowski M, Meynell J, Pearce PF, Collins LJ. The challenges of searching for and retrieving qualitative studies. West J Nurs Res. 2003;25(2):153-78.
• 13. Lefebvre C, Manheimer E, Glanville J. Searching for studies. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.2 [Internet]. Melbourne: The Cochrane Collaboration; 2009 [cited 2010 Sept 01]. Available from: www.cochrane-handbook.org
• 14. Sousa MR, Ribeiro AL. Revisão sistemática e meta-análise de estudos de diagnóstico e prognóstico: um tutorial. Arq Bras Cardiol. 2009;92(3):241-51.
• 15. Green S, Higgins JPT. Preparing a Cochrane review. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 [Internet]. Melbourne: The Cochrane Collaboration; 2008 [cited 2010 Sept 01]. Available from: www.cochrane-handbook.org
• 16. Higgins JPT, Deeks JJ. Selecting studies and collecting data. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.1 [Internet]. Melbourne: The Cochrane Collaboration; 2008 [cited 2010 Sept 01]. Available from: www.cochrane-handbook.org
• 17. The University of Kent. Critical Appraisal of the Journal Literature. Critical Appraisal Skills Programme (CASP). Evaluation tool for quantitative studies [Internet]. England: Public Health Resource Unit; 2006 [cited 2007 Oct 15]. Available from: http://www.fhsc.salford.ac.uk/hcprdu/tools/quantitative.htm
• 18. The University of Kent. Critical Appraisal of the Journal Literature. Critical Appraisal Skills Programme (CASP). 10 questions to help you make sense of qualitative research [Internet]. England: Public Health Resource Unit; 2006 [cited 2007 Oct 15]. Available from: http://www.phru.nhs.uk/Doc_Links/Qualitative%20Appraisal%20Tool.pdf
• 19. Finfgeld DL. Metasynthesis: the state of the art-so far. Qual Health Res. 2003;13(7):893-904.
• 20. Paterson BL, Thorne SE, Canam C, Jillings C. Meta-study of qualitative health research: a practical guide to meta-analysis and meta-synthesis. California: Sage; 2001.
• 21. Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546-53.
• 22. Silveira RCCP, Galvão CM. O cuidado de enfermagem e o cateter de Hickman: a busca de evidências. Acta Paul Enferm. 2005;18(3):276-84.
• 23. Deeks JJ, Higgins JPT, Altman DG. Analysing data and undertaking meta-analyses. In: Higgins JPT, Green S, editores. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.1 [Internet]. Melbourne: The Cochrane Collaboration; 2008 [cited 2010 Sept 01]. Available from: www.cochrane-handbook.org

Correspondence addressed to:

Mônica Cecília De-la-Torre-Ugarte-Granilo

Rua da Consolação, 2125, Apto. 72 - Consolação

CEP 01301-100 - São Paulo, SP, Brazil

Publication Dates

• Publication in this collection
  25 Oct 2011
• Date of issue
  Oct 2011

History

• Received
  18 Nov 2009
• Accepted
  02 Dec 2010