Overview of systematic reviews - a new type of study. Part II

Overview de revisões sistemáticas - um novo tipo de estudo. Parte II

Abstracts

CONTEXT AND OBJECTIVE:

Overviews of Systematic Reviews (OoRs) are a new type of study in which multiple evidence from systematic reviews (SRs) is compiled into an accessible and useful document. The aim here was to describe the state of the art and critically assess Cochrane OoRs that have been published.

DESIGN AND SETTING:

Descriptive study conducted at a research center.

METHODS:

The OoRs identified through the filter developed in Part I of this study were evaluated in five domains: methodological quality; quality of evidence; implications for practice; general profile of OoRs; and length of work.

RESULTS:

All 13 OoRs included had high methodological quality. Some OoRs did not present sufficient data to judge the quality of evidence; using sensitivity analysis, the quality of evidence of the OoRs increased. Regarding implications for practice, 64% of the interventions were judged as beneficial or harmful, while 36% of them showed insufficient evidence for judgment. It is expected (with 95% confidence interval) that one OoR will include 9,462 to 64,469 patients, 9 to 29 systematic reviews and 80 to 344 primary studies, and assess 6 to 21 interventions; and that 50 to 92% of OoRs will produce meta-analysis. The OoRs generated 2 to 26 meta-analyses over a period of 18 to 31 months.

CONCLUSION:

The OoRs presented high methodological quality; the quality of evidence tended to be moderate/high; most interventions were judged to be beneficial/harmful; the mean length of work was 24 months. The OoR profile adds power to decision-making.

Review [publication type]; Study characteristics [publication type]; Decision making; Evidence-based practice; Evidence-based medicine

CONTEXTO E OBJETIVO:

Overviews de revisões sistemáticas (OoRs) representam um novo tipo de estudo que compila múltiplas evidências de revisões sistemáticas (SRs) em um documento acessível e útil. O objetivo foi de descrever o estado da arte e avaliar criticamente as OoRs Cochrane publicadas.

DESENHO E LOCAL:

Estudo descritivo realizado em centro de pesquisa.

MÉTODOS:

As OoRs identificadas através do filtro desenvolvido na parte I deste estudo foram avaliadas por cinco domínios: qualidade metodológica, qualidade da evidência, implicações para a prática, perfil geral das OoRs e tempo de execução.

RESULTADOS:

As 13 OoRs incluídas apresentaram alta qualidade metodológica. Algumas OoRs não apresentavam dados suficientes para julgar a qualidade da evidência; com a análise de sensibilidade, a qualidade evidência nas OoRs aumentou. Implicações para prática foram julgadas como benéficas ou danosas em 64% das intervenções; em 36% das intervenções há evidências insuficientes para o julgamento. É esperado (com intervalo de confiança de 95%) que uma OoR inclua 9.462 a 64.469 pacientes, 9 a 29 revisões sistemáticas e 80 a 344 estudos primários, e avaliem entre 6 e 21 intervenções; que 50 a 92% das OoRs realizem metanálise. As OoRs geraram entre 2 e 26 metanálises em um período de 18 a 31 meses.

CONCLUSÃO:

As OoRs apresentam alta qualidade metodológica; a qualidade da evidência tende a ser alta/moderada; as intervenções, em sua maioria, foram julgadas como benéficas/danosas; o tempo de execução foi de 24 meses em média. O perfil das OoRs potencializa a tomada de decisão.

Revisão; Características dos estudos; Tomada de decisões; Prática clínica baseada em evidências; Medicina baseada em evidências

INTRODUCTION

Overviews of Systematic Reviews (OoRs) are a new type of study that has been proposed by the Cochrane Collaboration in order to compile multiple evidence from systematic reviews (SRs) into a single document that is accessible and useful. Each OoR focuses on a problem or medical condition for which two or more SRs have addressed potential interventions and their outcomes.^{1 - 6}

One SR rarely addresses all potential interventions for a condition, and healthcare policymakers may have difficulty in finding, evaluating, comparing and summarizing the information from all the relevant SRs.^{3 , 4} Thus, the main objective of OoRs is to serve as a friendly front end for the Cochrane Collaboration with regard to healthcare decision-making. The relevant SRs are integrated and/or summarized into a single document, i.e. an OoR. This, in theory, allows the reader to have access to an integrated summary of a long list of studies included in Cochrane SRs.^{1 - 6} Therefore, the primary audience for OoRs are healthcare decision-makers, such as healthcare professionals, policymakers and informed consumers who, through the Cochrane Library, seek evidence on treatments for various health conditions.^{1 , 2}

The first part^{1 , 2} of this series of three articles on OoRs focused on the growth of publications with the best level of evidence available for healthcare decision-making. It provided justifications for implementing this new type of study, as well as defining who the target audience are. Furthermore, a filter was created and applied in order to search for specific OoRs in the Cochrane Library.

This second part of the series continues to address this topic by describing the state of the art (state of knowledge) of Cochrane Collaboration OoRs, through a critical assessment of Cochrane OoRs. In Part III, a new hierarchy for the pyramid of evidence will be proposed, taking this new type of study into consideration.

OBJECTIVE

To critically assess Cochrane Overviews of Systematic Reviews, through analyzing the characteristics, approaches and methodological aspects of this type of study.

METHODS

This descriptive study was conducted at the research center of a federal university in Brazil and within one of its postgraduate study programs.

We performed a search for OoRs in the Cochrane Library, as described in Part I^{1 , 2} of this series. The flowchart for the OoRs is shown in Figure 1. The inclusion criteria were that the studies needed to be OoRs and to have been published in the Cochrane Database of Systematic Reviews, which is one of the six directories of the Cochrane Library.⁷ Protocols for Cochrane OoRs that have been published were excluded.

Figure 1
Preferred Reporting Items for Systematic Reviews and Meta-Analysis flowchart for Cochrane Overviews of Systematic Reviews (OoRs).

After OoR selection, two of the present authors (VS and AJG) read them, extracted data and assessed the quality. Differences in data collection information were resolved by reaching a consensus.

The data extracted from each OoR were organized using a specific form that sought information on the research question and objective, date of search, number of studies included, participants, interventions, main outcome, methodological quality of the review, quality of evidence and authors' conclusion.

We used five items to critically assess the OoRs: (1) methodological quality, using the AMSTAR tool (Assessing the Methodological Quality of Systematic Reviews);⁸ (2) quality of evidence, assessed using the GRADE tool (Grades of Recommendation, Assessment, Development and Evaluation) or any other method reported through OoRs; (3) implications for practice (the evidence confirms that the intervention presents benefits; the evidence confirms that the intervention presents harm/risk; or absence of evidence for a recommendation); (4) general profile of OoRs included (patients/OoR, SRs/OoR, studies/OoR, interventions/OoR, meta-analysis/OoR and search strategy); (5) Length of work, i.e. the time taken to publish the OoR (in years), obtained from the date of registration of the title (via Archie), date of publication of the protocol (via the Cochrane Library) and date of publication of the OoR (via the Cochrane Library).

Data synthesis was performed using descriptive statistics. Contingency tables were used to summarize dichotomous data as frequencies and proportions. Quantitative data were summarized using the mean and standard deviation. Sensitivity analysis were performed to assess the robustness of the results.

RESULTS

The search filter for OoRs that was developed in Part I^{1 , 2} of this study was updated, validated⁹ [sensitivity = 1.00 (95% CI = 0.86 to 1.00); specificity = 0.99 (95% CI = 0.99 to 1.00)] and used (on November 5, 2013). Through this process, 1207 titles were retrieved, of which 95% were excluded because they had not been published in the Cochrane Database of Systematic Reviews. Ninety-three references were checked, but 52 of these were systematic reviews and were excluded; and another 26 potential OoRs^{10 - 35} were excluded because they were at the protocol stage. At the end of the selection process, 13 OoRs^{26 , 35 - 46} were included, were used for data extraction and were critically assessed. Figure 1 presents the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) flowchart for the OoRs.

The characteristics of the thirteen OoRs^{26 , 35 - 46} included in this study are described in detail in ^{Appendix 1}, which includes the main conclusions from the approaches used in each study. The synthesis on the data extracted from these 13 OoRs,^{26 , 35 - 46} with the overall assessment, is presented in Table 1.

Table 1
Overall assessment of Cochrane Overviews of Systematic Review (OoRs)

	Overall assessment
Methodological qualitya	Classification of quality	General analysis
	% (n/N)	Mean (SD); range; N
High (8 to 11)	100% (13/13)	9.8 (0.9); 8-11; 13
Moderate (4 to 7)	0% (0/13)
Low (≤ 3)	0% (0/13)
Quality of evidenceb	General analysis % (n/N)	Sensitivity analysisc % (n/N)
High	11% (232/2141)	21% (171/801)
Moderate	16% (348/2141)	30% (242/801)
Low	20% (425/2141)	31% (251/801)
Very low	11% (246/2141)	16% (128/801)
Not clear to judge	42% (890/2141)	1% (9/801)
Implication for practiced	% (n/N)
Beneficial intervention	52% (26/50)
Harmful intervention	12% (6/50)
Insufficient evidence	36% (18/50)
General profile	Mean (SD); range; N
Patients/OoR e	36965.7 (40939.6); 2323-117501; 11
Studies/OoR e	211.9 (207.8); 21-700; 12
SRs/OoR e	18.9 (16.3); 3-54; 12
Interventions/OoR	13.3 (10.2); 3-38; 13
Meta-analysis/OoR	% (n/N)	Mean (SD); range;Nf
	76.9% (10/13)	13.5 (14.4); 5-46; 8
Additional search in CDSR g	% (n/N)	Mean (SD); range; Nh
	39% (5/13)	5.8 (3.5); 2-11; 5
Length of work (months)	Mean (SD); range; N
Total time taken to publish	24.5 (10.3); 8-43; 13
Time taken to publish the protocol i	13.8 (8.4); 5-30; 11
Time taken to publish the OoR i	12.4 (8.4); 3-25; 11

• ^aAMSTAR conducted;^8bfrequency based on the number of outcomes or studies assessed; ^cOoRs^41,43,48 in which more than 10% of the studies/categories could not be clearly judged regarding the quality of the evidence were excluded from the sensitivity analysis; ^djudgments based on main outcome conclusion; ^esome data could not be determined (patients/OoR,^39,41 SRs/OoR⁴2 and studies/OoR⁴¹⁾, even by looking for results tables in the OoRs or SRs included, thus reducing the number of OoRs in the analysis; ^fstudies in which no meta-analysis was conducted^37,41,43-45 were not included; ^gCDSR is the abbreviation for Cochrane Database of Systematic Reviews; honly OoRs^39,41-43,46 in which additional searches in the CDSR were conducted were included in the analysis; ithe protocols for these OoRs^37,39 were not found in the Cochrane Library or in Archie, thus reducing the number of OoRs in the analysis; SD = standard deviation; range = minimum to maximum; SR = systematic review.

All the OoRs that were included^{26 , 35 - 46} had high methodological quality, reaching between 9 and 11 points out of the 11 points possible in AMSTAR (Table 1). However, some OoRs lost points regarding methodological quality for the following reasons: they did not record the protocol³⁶ or, if it was cited, did not make it available;³⁸ they only included searches in the Cochrane Database of Systematic Reviews (CDSR);^{26 , 35 - 37 , 39 , 43 , 44 , 46} they did not consider the quality of evidence in formulating conclusions;⁴⁶ they were unable to "meta-analyze" the data;^{40 , 42 , 43 , 46} and they did not assess the risk of bias among the reviews included.³⁶

With regard to judging the quality of evidence, it was found that many categories/studies were unclear, and thus the proportion of the categories/studies that presented high-quality evidence was low. If the OoRs^{40 , 42 , 46} in which more than 10% of the studies/categories could not be clearly judged regarding the quality of evidence were excluded, the proportion of studies/categories that were of high quality doubled. The sensitivity analysis drastically altered the proportions of the other categories regarding the quality of evidence (Table 1).

Through the outcome of implications for practice, we found that about 64% of the interventions were judged to be beneficial or harmful. However, there was insufficient evidence to judge the interventions in 36% of them (Table 1).

Regarding general factors, with a 95% confidence interval, we expect that each new OoR will include between 9,462 and 64,469 patients, between 9 and 29 SRs and between 80 and 344 primary studies, and will assess between 6 and 21 interventions. Between 50 and 92% of OoRs that pool data qualitatively will generate between 2 and 26 meta-analyses. Additional details are analyzed in Table 1.

Most of the OoRs (62%) conducted searches for systematic reviews only in the Cochrane Database of Systematic Reviews (CDSR). However, those that conducted external searches, in addition to searching in the CDSR, used between one and ten databases (Table 1).

Table 1 also shows that, from the date when the OoR title was registered, the authors spent about two years (about one year for planning/preparing the protocol and another year for implementing the full study) until the date of publication.

Only 23% (3/13) of the OoRs did not identify the study type in the title. The same also occurred in the protocols for OoRs (6/28).

DISCUSSION

The Cochrane Collaboration's OoRs are supervised by one of the 16 methods groups, the Comparing Multiple Interventions Methods Group. This methods group of the Cochrane Collaboration was established in 2004 and initially was called the Umbrella Reviews Working Group.⁴⁷

Other milestones during the development of the OoRs methodology occurred in 1996, 1997, 1998 and 2005. Between February 1997 and December 1998, the first series of OoRs,^{48 - 51} four reviews relating to pregnancy were published as a partnership between researchers from the UK Cochrane Centre and the World Health Organization. In 1996, Julian Higgins and Anne Whitehead published the first article to describe the standard Bayesian approach towards multiple-treatment meta-analysis (MTM).⁵² In 2005, in the 13^th Cochrane Colloquium in Melbourne, Georgia Salanti made a presentation on MTM methodology,⁵³ co-authored with Julian Higgins and Valeria Marinho. This won the prize for best oral presentation and helped to popularize the technique.

In this context, the present systematic review describes how far the knowledge of the select group of Cochrane OoRs^{26 , 35 - 46} published in the CDSR has reached. Although we only included 13 OoRs,^{26 , 35 - 46} these studies published in the CDSR by the Cochrane Collaboration stand out through their methodological rigor. This was highlighted in two studies^{54 , 55} that compared the methodological rigor of Cochrane systematic reviews with non-Cochrane systematic reviews. On January 24, 2011, in recognition of the contribution to healthcare that the Cochrane Collaboration has made,⁵⁶ the World Health Organization lauded the Collaboration as a non-governmental organization, with a seat in the World Health Assembly and voting rights, in order to help manage healthcare worldwide.

As examples of Cochrane systematic reviews,^{54 , 55} all the OoRs included in this review^{35 - 46} had high methodological quality and therefore less risk of bias, according to the results obtained through AMSTAR.⁸ However, the reasons why not all the OoRs included obtained the maximum score need to be discussed:

1. The protocols for two OoRs^{36 , 38} were not found in the Cochrane Library or in Archie, even though one of them cited the protocol.³⁸ Registration of the protocols for OoR, just as for clinical trials and systematic reviews, allows assessment of methodological quality, provides transparency in conducting the study and minimizes occurrences of publication bias or selective reporting of outcomes.

2. Eight OoRs^{26 , 35 - 37 , 39 , 43 , 44 , 46} restricted the search to only one database, i.e. the CDSR. Moreover, we did not consider this to be a potential source of bias, taking the view that the search methods for systematic reviews in the CDSR are rigorous and comprehensive (including no restrictions on date, publication status or language), and therefore that there was a high probability of including all the relevant primary studies.

3. Four OoRs^{40 , 42 , 43 , 46} could not match any of the systematic reviews included in their quantitative data synthesis, through either direct or indirect meta-analysis, and therefore summarized and integrated the evidence qualitatively.

4. One OoR³⁶ did not evaluate or discuss the risk of publication bias among the systematic reviews included.

Although AMSTAR⁸ was developed to assess the methodological quality of systematic reviews, it needs to be borne in mind that many of the items that it assesses are also present in OoRs, even though OoRs are different studies. Thus, AMSTAR⁸ can be considered to be an analogous tool for assessing OoRs, in which external validity is preserved. Development of an instrument for judging the methodological rigor and consistency of OoRs will be important for reducing the uncertainties in decision-making from these studies.

One challenge identified in the present study was the lack of standardization in the method for assessing the quality of evidence. OoRs^{26 , 35 - 46} have used various methods of assessment (e.g. GRADE,⁵⁷ Cochrane risk of bias tool,^{58 , 59} Jadad et al.,⁶⁰ or even review-specific criteria) or, when using the same method, have described it in different ways, thereby making the judgment uncertain. In this context, two OoRs^{40 , 42} can be highlighted. Despite the high methodological quality of the study by Ryan et al.,⁴⁰ it only described how the systematic reviews included had assessed the quality of evidence, thus making it difficult to assess this outcome. Many studies (890/2141) may not have made judgments regarding the quality of evidence. In the second of these OoRs, Jones et al.⁴² used the Cochrane risk of bias tool^{58 , 61} to assess the quality of the evidence in Cochrane reviews, while for non-Cochrane reviews they used Jadad et al.⁶⁰ Despite the differences in the concepts of quality of evidence and methodological quality (which were not among the objectives of this discussion), these authors reported the proportion of studies that presented high quality. Thus, the description was not sufficiently clear or standardized for a judgment to be made regarding the overall quality of evidence.

To assess the quality of evidence, the most consistently used method among the OoRs assessed^{26 , 35 - 46} was the GRADE approach.⁵⁷ Basically, this tool classifies evidence into four levels: (1) high quality - it is unlikely that future research will change the estimated effect; (2) moderate quality - future research may have a major impact on the estimated effect and change the result; (3) low quality - further research is very likely to have an important impact on the estimated effect and change the results; and (4) very low quality - there is no certainty in the estimated effect. In the GRADE approach to the classification of evidence, the assessment can be lowered or raised according to certain methodological factors. Methodological limitations, inconsistent results, imputations of evidence, imprecision of results and publication bias diminish the quality of evidence. The magnitude of the effect, controlling for confounders and dose-response gradients are methodological factors that may raise the level of evidence.

Even though there has been criticism regarding the lack of methodological rigor in this type of study,^{62 , 63} all the Cochrane OoRs^{35 - 46} assessed methodological quality, thus allowing judgment of the effects of interventions based on the main outcomes found. About two-thirds of the OoRs^{26 , 35 - 46} presented evidence to judge how beneficial or harmful the main outcomes were. These results are based on various systematic reviews and thus integrate and summarize evidence from several interventions relating to a given condition, from a high number of clinical trials and patients. The evidence in these OoRs^{26 , 35 - 46} was released over a period of about two years from the time of registration of the title to the date of publication. In the future, it will be important to also determine the time taken to update the OoRs.

Furthermore, comparing the results from this descriptive systematic review of published studies with other OoRs^{62 , 63} that assessed this new type of study, the Cochrane OoRs seem to have higher quality and methodological rigor than non-Cochrane OoRs. To reduce the inconsistency of OoRs, it is important that authors follow the existing methodological recommendations.^{3 , 5 , 61}

When extracting data, attention was drawn to two factors: (1) three studies^{37 , 39 , 43} did not declare the type of study, i.e. "overview of reviews" (there was a similar proportion with regard to protocols for OoRs: 6/28), which may be a limiting factor in that it becomes more difficult for readers to easily identify the type of study; (2) Singh et al.⁴¹ only included primary studies from the list of references of systematic reviews and conducted search strategies in bibliographic databases.

Another feature that may pose a challenge in conducting this type of study is the complex statistical methods required for integrate and summarizing the evidence. The methodological basis for network meta-analysis, also known as multiple-treatment meta-analysis and mixed-treatment comparisons, was established in 1996.⁵² In view of the challenge of implementing these statistical techniques, a special issue of Research Synthesis Methods was published on this topic in 2012 (see volume 3, number 2). For researchers who wish to conduct an OoR, it is highly recommended that the information provided by the Cochrane Comparing Multiple Interventions Methods Group, the Multiple-Treatment Meta-analysis website <http://www.mtm.uoi.gr/> and the special issue of Research Synthesis Methods should be consulted.

This new type of study certainly attracts everyone's attention, as demonstrated from data in Evidence-Based Child Health.⁵ In 2009, OoRs were downloaded 3.05 times more than other types of study. There has also been a tendency towards growth of this frequency over the years (2006 versus 2009), by a rate of 1.84 times. The importance of this type of study can be understood through the great interest of its audience.

Despite the limitation of the small number of OoRs included and the absence of some information that could not be obtained, it could be seen that this type of study presents the following potential benefits: (1) it allows consumers to know about and understand the concepts and application of such studies and their ability to integrate and summarize various interventions for a disease or health condition; (2) for healthcare policymakers, summarizing all interventions into a single document allows them to keep up to date in the era of information globalization^{1 , 2} and, thus, this type of study serves as a friendly front end for healthcare decision-making based on the best available evidence, thereby changing professional practice and healthcare policies; and (3) it allows researchers to address the gaps, weaknesses and strengths of each study and think about future research strategies based on what still needs to be explored.

In Part III, a new hierarchy for the pyramid of evidence will be proposed, taking this new type of study into consideration.

CONCLUSION

OoRs have high methodological quality and high quality of evidence, as assessed in general. Moreover, about two-thirds of the conclusions had sufficient evidence to judge that the implications for practice were either beneficial or harmful. In order to do this, the mean length of work required was 24 months.

The profile of OoRs reduces uncertainty in decision-making. This new type of study demonstrates the ability to compile multiple evidence from systematic reviews in one handy and useful document that is able to address all potential interventions for a condition, thereby allowing decision-makers to locate, evaluate, compare and summarize the evidence from systematic reviews or primary studies.

Although OoRs are able to integrate and summarize multiple interventions for a problem in a single document, there is still a need to standardize the methods for this new type of study.

REFERENCES

• ¹
  Silva V, Grande AJ, Martimbianco AL, Riera R, Carvalho AP. Overview of systematic reviews - a new type of study. part I: why and for whom? Sao Paulo Med J. 2012;130(6):398-404.
• ²
  Silva V, Grande AJ, Martimbianco ALC, Riera R, Carvalho APV. Overview de revisões sistemáticas - um novo tipo de estudo. Parte I: Por que e para quem? [Overview of systematic reviews - a new type of study. Part I: Why and for whom?] Diagn Tratamento. 2012;17(4):195-200.
• ³
  Becker LA, Oxman AD. Chapter 22: Overviews of reviews. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration; 2011. Available from: http://handbook.cochrane.org/chapter_22/22_overviews_of_reviews.htm. Accessed in 2014 (Apr 9).
  » http://handbook.cochrane.org/chapter_22/22_overviews_of_reviews.htm
• ⁴
  Becker LA. Cochrane Overviews of Reviews. Available from: https://www.socialtext.net/overviews/cochrane_overviews_of_reviews. Accessed in 2014 (Apr 9).
  » https://www.socialtext.net/overviews/cochrane_overviews_of_reviews
• ⁵
  Thomson D, Russell K, Becker L, Klassen T, Hartling L. The evolution of a new publication type: Steps and challenges of producing overviews of reviews. Research Synthesis Methods. 2010;1(3-4):198-211. Available from: http://onlinelibrary.wiley.com/doi/10.1002/jrsm.30/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/jrsm.30/abstract
• ⁶
  Salanti G, Becker L, Caldwell D, et al. Evolution of Cochrane Intervention Reviews and Overviews of Reviews to better accommodate comparisons among multiple interventions. Report from a meeting of the Cochrane Comparing Multiple Interventions Methods Groups; Milan. March 2011. Available from: http://cmimg.cochrane.org/sites/cmimg.cochrane.org/files/uploads/CMIMG%20summary%20of%20meeting%20Milan%20March%202011.pdf. Accessed in 2014 (Apr 9).
  » http://cmimg.cochrane.org/sites/cmimg.cochrane.org/files/uploads/CMIMG%20summary%20of%20meeting%20Milan%20March%202011.pdf
• ⁷
  The Cochrane Library. About the Cochrane Library. Available from: http://www.thecochranelibrary.com/view/0/AboutTheCochraneLibrary.html. Accessed in 2014 (Apr 9).
  » http://www.thecochranelibrary.com/view/0/AboutTheCochraneLibrary.html
• ⁸
  Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10.
• ⁹
  Silva V, Puga MES, Grande AJ, et al. Search filter for retrieving overviews of systematic reviews in The Cochrane Library. In: 21st Cochrane Colloquium, Auckland, 2013. Available from: http://2013.colloquium.cochrane.org/abstracts/search-filter-retrieving-overviews-systematic-reviews-cochrane-library. Accessed in 2014 (Apr 9).
  » http://2013.colloquium.cochrane.org/abstracts/search-filter-retrieving-overviews-systematic-reviews-cochrane-library
• ¹⁰
  Amato L, Del Giovane C, Ferri M, et al. Acceptability, efficacy and safety of pharmacological interventions for cocaine dependence: an overview of Cochrane reviews. Cochrane Database of Systematic Reviews. 2012;(3):CD009696. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009696/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009696/abstract
• ¹¹
  Brown J, Farquhar C, Dias S. Endometriosis: an overview of Cochrane Reviews. Cochrane Database of Systematic Reviews. 2012;(1):CD009590. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009590/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009590/abstract
• ¹²
  Cahill K, Stevens S, Perera R, Lancaster T. Pharmacological interventions for smoking cessation: an overview and network meta-analysis. Cochrane Database Syst Rev. 2013;(5):CD009329.
• ¹³
  Cates CJ, Wieland LS, Oleszczuk M. Safety of regular formoterol or salmeterol in adults with asthma: an overview of Cochrane reviews. Cochrane Database of Systematic Reviews. 2013;(1):CD010314. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010314/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010314/abstract
• ¹⁴
  Disler RT, Inglis SC, Davidson PM. Non-pharmacological management interventions for COPD: an overview of Cochrane systematic reviews. Cochrane Database of Systematic Reviews. 2013;(2):CD010384. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010384/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010384/abstract
• ¹⁵
  Dumville JC, O'Meara S, Bell-Syer SEM. Dressings for treating foot ulcers in people with diabetes: an overview of systematic reviews. Cochrane Database of Systematic Reviews. 2013;(4):CD010471. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010471/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010471/abstract
• ¹⁶
  Evans JR, Virgili G, Gordon I, et al. Interventions for neovascular age-related macular degeneration. Cochrane Database of Systematic Reviews. 2009;(1):CD007650. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD007650/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD007650/abstract
• ¹⁷
  Filippini G, Del Giovane C, Vacchi L, et al. Immunomodulators and immunosuppressants for multiple sclerosis: a network meta-analysis. Cochrane Database of Systematic Reviews. 2011;(1):CD008933. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD008933/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD008933/abstract
• ¹⁸
  Guay J, Choi P, Suresh S, et al. Neuraxial blockade for the prevention of postoperative mortality and major morbidity: an overview of Cochrane systematic reviews. Cochrane Database of Systematic Reviews. 2012;(9):CD010108. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010108/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010108/abstract
• ¹⁹
  Hazlewood GS, Barnabe C, Tomlinson G, Marshall D, Bombardier C. Methotrexate monotherapy and methotrexate combination therapy with traditional and biologic disease modifying anti-rheumatic drugs for rheumatoid arthritis: A network meta-analysis. Cochrane Database of Systematic Reviews. 2013;(1):CD010227. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010227/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010227/abstract
• ²⁰
  Hughes RAC, Lunn MPT, Frost C, van Schaik IN. Treatments for chronic inflammatory demyelinating polyradiculoneuropathy (CIDP): an overview of systematic reviews. Cochrane Database of Systematic Reviews. 2013;(2):CD010369. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010369/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010369/abstract
• ²¹
  Kew KM, Li T. Long-acting inhaled therapy (beta-agonists, anticholinergics and steroids) for COPD: an overview and network meta-analysis. Cochrane Database of Systematic Reviews. 2013;(10):CD010178. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010178.pub2/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010178.pub2/abstract
• ²²
  Lockhart P, Holland NJ, Swan I, Teixeira LJ. Interventions for Bell's Palsy (idiopathic facial paralysis). Cochrane Database of Systematic Reviews. 2011;(2):CD008974. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD008974/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD008974/abstract
• ²³
  Moore RA, Derry CJ, Derry S. Sumatriptan (all routes of administration) for acute migraine attacks in adults: an overview of Cochrane reviews. Cochrane Database of Systematic Reviews. 2011;(5):CD009108. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009108/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009108/abstract
• ²⁴
  Moore RA, Derry S, McQuay HJ. Topical analgesics for acute and chronic pain in adults. Cochrane Database of Systematic Reviews. 2010;(7):CD008609. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD008609/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD008609/abstract
• ²⁵
  Moore RA, Wiffen PJ, Derry S, et al. Non-prescription (OTC) oral analgesics for acute pain - an overview of Cochrane reviews. Cochrane Database of Systematic Reviews. 2013;(10):CD010794. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010794/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010794/abstract
• ²⁶
  O'Connell NE, Wand BM, McAuley J, Marston L, Moseley GL. Interventions for treating pain and disability in adults with complex regional pain syndrome. Cochrane Database Syst Rev. 2013;(4):CD009416.
• ²⁷
  Ota E, Souza JP, Tobe-Gai R, et al. Interventions during the antenatal period for preventing stillbirth: an overview of Cochrane systematic reviews. Cochrane Database of Systematic Reviews. 2012;(1):CD009599. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009599/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009599/abstract
• ²⁸
  Roqué i Figuls M, Martínez García L, Martinez-Zapata MJ, et al. Interventions for treating overweight or obesity in adults: an overview of systematic reviews. Cochrane Database of Systematic Reviews. 2013;(8):CD010665. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010665/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010665/abstract
• ²⁹
  Shah PB, Meleveedu R, Elayaraja S, James S, Sathiyasekaran BWC. Interventions for treating community-acquired pneumonia: an overview of Cochrane systematic reviews. Cochrane Database of Systematic Reviews. 2011;(10):CD009343. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009343/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009343/abstract
• ³⁰
  Shah PB, Meleveedu R, James S, Elayaraja S, Sathiyasekaran BWC. Interventions for treating hospital-acquired pneumonia: an overview of Cochrane systematic reviews. Cochrane Database of Systematic Reviews. 2011;(10):CD009370. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009370/abstract?deniedAccessCustomisedMessage=&amp;userIsAuthenticated=false. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009370/abstract?deniedAccessCustomisedMessage=&amp;userIsAuthenticated=false
• ³¹
  Udell JE, Drahota A, Dean TP, Sander R, Mackenzie H. Interventions for preventing falls in older people: an overview of Cochrane Reviews. Cochrane Database of Systematic Reviews. 2011;(4):CD009074. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009074/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD009074/abstract
• ³²
  Visser J, McLachlan MH, Fergusson P, Volmink J, Garner P. Supplementary feeding for food insecure, vulnerable and malnourished populations - an overview of systematic reviews. Cochrane Database of Systematic Reviews. 2013;(6):CD010578. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010578/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010578/abstract
• ³³
  Welsh EJ, Spencer S, Saba T, Evans DJ, Fowler SJ. Interventions for bronchiectasis: an overview of Cochrane systematic reviews. Cochrane Database of Systematic Reviews. 2013;(2):CD010337. Available from: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010337/abstract. Accessed in 2014 (Apr 9).
  » http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD010337/abstract
• ³⁴
  Wiffen PJ, Derry S, Moore RA, et al. Antiepileptic drugs for neuropathic pain and fibromyalgia - an overview of Cochrane reviews. Cochrane Database Syst Rev. 2013;11:CD010567.
• ³⁵
  Singh JA, Christensen R, Wells GA, et al. Biologics for rheumatoid arthritis: an overview of Cochrane reviews. Cochrane Database Syst Rev. 2009;(4):CD007848.
• ³⁶
  Keus F, Gooszen HG, van Laarhoven CJ. Open, small-incision, or laparoscopic cholecystectomy for patients with symptomatic cholecystolithiasis. An overview of Cochrane Hepato-Biliary Group reviews. Cochrane Database Syst Rev. 2010;(1):CD008318.
• ³⁷
  Amato L, Minozzi S, Davoli M. Efficacy and safety of pharmacological interventions for the treatment of the Alcohol Withdrawal Syndrome. Cochrane Database Syst Rev. 2011;(6):CD008537.
• ³⁸
  Flodgren G, Eccles MP, Shepperd S, et al. An overview of reviews evaluating the effectiveness of financial incentives in changing healthcare professional behaviours and patient outcomes. Cochrane Database Syst Rev. 2011;(7):CD009255.
• ³⁹
  Moore RA, Derry S, McQuay HJ, Wiffen PJ. Single dose oral analgesics for acute postoperative pain in adults. Cochrane Database Syst Rev. 2011;(9):CD008659.
• ⁴⁰
  Ryan R, Santesso N, Hill S, et al. Consumer-oriented interventions for evidence-based prescribing and medicines use: an overview of systematic reviews. Cochrane Database Syst Rev. 2011;(5):CD007768.
• ⁴¹
  Singh JA, Wells GA, Christensen R, et al. Adverse effects of biologics: a network meta-analysis and Cochrane overview. Cochrane Database Syst Rev. 2011;(2):CD008794.
• ⁴²
  Jones L, Othman M, Dowswell T, et al. Pain management for women in labour: an overview of systematic reviews. Cochrane Database Syst Rev. 2012;3:CD009234.
• ⁴³
  Payne C, Wiffen PJ, Martin S. Interventions for fatigue and weight loss in adults with advanced progressive illness. Cochrane Database Syst Rev. 2012;1:CD008427.
• ⁴⁴
  Farquhar C, Rishworth JR, Brown J, Nelen WL, Marjoribanks J. Assisted reproductive technology: an overview of Cochrane Reviews. Cochrane Database Syst Rev. 2013;8:CD010537.
• ⁴⁵
  Cates CJ, Oleszczuk M, Stovold E, Wieland LS. Safety of regular formoterol or salmeterol in children with asthma: an overview of Cochrane reviews. Cochrane Database Syst Rev. 2012;10:CD010005.
• ⁴⁶
  Cahill K, Stevens S, Perera R, Lancaster T. Pharmacological interventions for smoking cessation: an overview and network meta-analysis. Cochrane Database Syst Rev. 2013;5:CD009329.
• ⁴⁷
  Becher L, Thomson D, Caldwell D. Addressing multiple treatments I: Cochrane Overviews. Available from: http://cmimg.cochrane.org/sites/cmimg.cochrane.org/files/uploads/Becker%20-%20Intro%20to%20Overviews%20Keystone.pdf. Accessed in 2014 (Apr 9).
  » http://cmimg.cochrane.org/sites/cmimg.cochrane.org/files/uploads/Becker%20-%20Intro%20to%20Overviews%20Keystone.pdf
• ⁴⁸
  Gülmezoglu M, de Onis M, Villar J. Effectiveness of interventions to prevent or treat impaired fetal growth. Obstet Gynecol Surv. 1997;52(2):139-49.
• ⁴⁹
  de Onis M, Villar J, Gülmezoglu M. Nutritional interventions to prevent intrauterine growth retardation: evidence from randomized controlled trials. Eur J Clin Nutr. 1998;52 Suppl 1:S83-93.
• ⁵⁰
  Villar J, Gülmezoglu AM, de Onis M. Nutritional and antimicrobial interventions to prevent preterm birth: an overview of randomized controlled trials. Obstet Gynecol Surv. 1998;53(9):575-85.
• ⁵¹
  Kulier R, de Onis M, Gülmezoglu AM, Villar J. Nutritional interventions for the prevention of maternal morbidity. Int J Gynaecol Obstet. 1998;63(3):231-46.
• ⁵²
  Higgins JP, Whitehead A. Borrowing strength from external trials in a meta-analysis. Stat Med. 1996;15(24):2733-49.
• ⁵³
  Salanti G, Higgins J, Marinho V. How to determine the best treatment: a mixed-treatment-comparisons meta-analysis (MTM) of trials of topical fluoride therapies for the prevention of dental caries [abstract]. In: Proceedings of the XIII Cochrane Colloquium; 2005 Oct 22-26; Melbourne, Australia; 2005. Available from: http://cochrane.bvsalud.org/doc.php?db=cmr&amp;id=CMR-7814&amp;lib=COC. Accessed in 2014 (Apr 9).
  » http://cochrane.bvsalud.org/doc.php?db=cmr&amp;id=CMR-7814&amp;lib=COC
• ⁵⁴
  Collier A, Heilig L, Schilling L, Williams H, Dellavalle RP. Cochrane Skin Group systematic reviews are more methodologically rigorous than other systematic reviews in dermatology. Br J Dermatol. 2006;155(6):1230-5.
• ⁵⁵
  Moseley AM, Elkins MR, Herbert RD, Maher CG, Sherrington C. Cochrane reviews used more rigorous methods than non-Cochrane reviews: survey of systematic reviews in physiotherapy. J Clin Epidemiol. 2009;62(10):1021-30.
• ⁵⁶
  Bero L, Ghersi D. Making systematic reviews global. Cochrane Database Syst Rev. 2011;(8):ED000020.
• ⁵⁷
  Schünemann HJ, Oxman AD, Vist GE, et al. Chapter 12: Interpreting results and drawing conclusions. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration; 2011. Available from: http://handbook.cochrane.org/chapter_12/12_interpreting_results_and_drawing_conclusions.htm. Accessed in 2014 (Apr 9).
  » http://handbook.cochrane.org/chapter_12/12_interpreting_results_and_drawing_conclusions.htm
• ⁵⁸
  Higgins JPT, Altman DG, Sterne JAC. Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration, 2011. Available from: http://handbook.cochrane.org/chapter_8/8_assessing_risk_of_bias_in_included_studies.htm. Accessed in 2014 (Apr 9).
  » http://handbook.cochrane.org/chapter_8/8_assessing_risk_of_bias_in_included_studies.htm
• ⁵⁹
  Carvalho APV, Silva V, Grande AJ. Avaliação do risco de viés de ensaios clínicos randomizados pela ferramenta da colaboração Cochrane [Assessment of risk of bias in randomized controlled trials by the Cochrane Collaboration tool] . Diagn Tratamento. 2013;18(1):38-44.
• ⁶⁰
  Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996;17(1):1-12.
• ⁶¹
  Smith V, Devane D, Begley CM, Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol. 2011;11(1):15.
• ⁶²
  Pieper D, Buechter R, Jerinic P, Eikermann M. Overviews of reviews often have limited rigor: a systematic review. J Clin Epidemiol. 2012;65(12):1267-73.
• ⁶³
  Hartling L, Chisholm A, Thomson D, Dryden DM. A descriptive analysis of overviews of reviews published between 2000 and 2011. PLoS One. 2012;7(11):e49667.

• Sources of funding: None
• Conflict of interest: None
• *The Portuguese version of this manuscript was published in the journal Diagnóstico &amp; Tratamento, Volume 19, Issue Number 1, 2014
• Acknowledgements: The authors VS, APVC, ALCM and AJG thank Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (Capes) for granting scholarships

ANEXO A

Publication Dates

• Publication in this collection
  07 Nov 2014
• Date of issue
  May-Jun 2015

History

• Received
  24 Dec 2013
• Reviewed
  24 Apr 2014
• Accepted
  06 May 2014