ABSTRACT
Objectives
We sought to determine whether disease representation in the Cochrane Database of Systematic Reviews (CDSR) reflects disease burden, measured by the Global Burden of Disease (GBD) Study as disability-adjusted life-years (DALYs).
Materials and Methods
Two investigators performed independent assessment of ten men’s health and urologic diseases (MHUDs) in CDSR for systematic review and protocol representation, which were compared with percentage of total 2010 DALYs for the ten conditions. Data were analyzed for correlation using Spearman rank analysis.
Results
Nine of ten MHUDs were represented by at least one CDSR review. There was a poor and statistically insignificant positive correlation between CDSR representation and disease burden (rho = 0.42, p = 0.23). CDSR representation was aligned with disease burden for three conditions, greater than disease burden for one condition, and less than disease burden for six conditions.
Conclusions
These results yield high-quality estimates to inform future research prioritization for MHUDs. While prioritization processes are complex and multi-faceted, disease burden should be strongly considered. Awareness of research priority setting has the potential to minimize research disparities on a global scale.
Men’s Health; Urologic Diseases; Neoplasms; Infertility, Male
INTRODUCTION
In order to achieve effective clinical research, scarce research funds must be distributed to appropriate diseases in order to maximize health benefits to the represented population. Systematic approaches to inform research prioritization include identifying and prioritizing research questions, recognizing existing research, and setting goals for primary research (11. Fleurence RL. Setting priorities for research: a practical application of ‘payback’ and expected value of information. Health Econ. 2007;16:1345-57., 22. Nasser M, Welch V. Prioritization of systematic reviews leads prioritization of research gaps and needs. J Clin Epidemiol. 2013;66:522-3.). A derivative of this approach is to value major diseases, injuries, and risk factors based on their burden to society (33. Shiffman J. Knowledge, moral claims and the exercise of power in global health. Int J Health Policy Manag. 2014;3:297-9.). Spearheaded by the Institute for Health Metrics and Evaluation (IHME), the Global Burden of Disease (GBD) 2010 Study estimates the burden of 291 diseases and injuries across 187 countries from 1990 to 2010 (44. Murray CJ, Ezzati M, Flaxman AD, Lim S, Lozano R, Michaud C, et al. GBD 2010: design, definitions, and metrics. Lancet. 2012;380:2063-6., 55. Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380:2095-128. Erratum in: Lancet. 2013;381:628.). The metric of disability-adjusted life years (DALYs), in which 1 DALY is equivalent to 1 year of healthy life lost, allows for descriptive global epidemiology of a wide array of disease states. The following ten men’s health and urologic diseases (MHUDs) were studied by GBD on the basis of prevalence, common case definitions, and data availability: tubulointerstitial nephritis, pyelonephritis, and urinary tract infections; kidney and other urinary organ cancers; urolithiasis; male infertility; benign prostatic hyperplasia; prostate cancer; testicular cancer; hydrocele due to lymphatic filariasis; dysuria/bladder pathology/hydronephrosis due to schistosomiasis; and bladder cancer (Figure-1).
Square pie chart representing percent of total DALY for ten men’s health and urologic diseases; area of each square/rectangle represents percentage of total burden.
Systematic reviews are the cornerstone of evidence-based medicine, yet few efforts have been made to assess whether the prioritization of systematic reviews reflect global disease burden (66. Clarke M, Hopewell S, Chalmers I. Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting. Lancet. 2010;376:20-1.). The Cochrane Database of Systematic Reviews (CDSR) produces systematic reviews and protocols (published proposals for future systematic reviews) across all medical specialties as well as health systems, public health, and child development. Cochrane systematic reviews undergo exhaustive editorial processing, are more methodologically rigorous, and are updated more frequently than non-Cochrane reviews and paper-based journals (77. Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, Moher M, et al. Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA. 1998;280:278-80., 88. Collier A, Heilig L, Schilling L, Williams H, Dellavalle RP. Cochrane Skin Group systematic reviews are more methodologically rigorous than other systematic reviews in dermatology. Br J Dermatol. 2006;155:1230-5.). Prior studies have evaluated the association between broad categories of disease burden with randomized trials and Cochrane systematic reviews (99. Emdin CA, Odutayo A, Hsiao AJ, Shakir M, Hopewell S, Rahimi K, et al. Association between randomised trial evidence and global burden of disease: cross sectional study (Epidemiological Study of Randomized Trials—ESORT). BMJ 2015;350 :h117.
10. Yoong SL, Hall A, Williams CM, Skelton E, Oldmeadow C, Wiggers K, et al. Alignment of systematic reviews published in the Cochrane Database of Systematic Reviews and the Database of Abstracts and Reviews of Effectiveness with global burden of disease data: a bibliographic analysis. J Epidemiol Community Health. J Epidemiol Community Health. 2015 Jul;69(7):708-14
11. Pederson H, Okland T, Boyers LN, Karimkhani C, Rosenfeld RM, Nasser M, et al. Identifying otolaryngology systematic review research gaps: comparing Global Burden of Disease 2010 results with Cochrane Database of Systematic Review content. JAMA Otolaryngol Head Neck Surg. 2015;141:67-72.-1212. Karimkhani C, Trikha R, Aksut B, Jones T, Boyers LN, Schlichte M, et al. Identifying gaps for research prioritisation: Global burden of external causes of injury as reflected in the Cochrane Database of Systematic Reviews. Injury. 2016;47:1151-7.). This study will assess whether the CDSR representation of ten MHUDs corresponds to GBD 2010 disability estimates.
MATERIALS AND METHODS
ICD-10 code definitions for the ten MHUDs have been previously published and were used to generate search terms, which were entered into the “title, abstract, keywords” CDSR search function (55. Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380:2095-128. Erratum in: Lancet. 2013;381:628., 1313. The Cochrane Collaboration. Search the Cochrane library. Available from: http://www.thecochranelibrary.com/view/0/index.html. Accessed January 15, 2015.
http://www.thecochranelibrary.com/view/0...
). Systematic reviews and protocols were considered to determine MHUD representation in CDSR, according to abstract subject content. Online publication date, the number of studies included in each systematic review, and the particular Cochrane review group that published the review or protocol were collected.
Two authors (T.O and H.P.) collected data independently during February 2015. DALY metrics for each of the ten MHUDs, expressed as percentages of total DALY’s of all 291 conditions measured in GBD 2010, were obtained from the GBD Compare interactive time plot, available at <http://viz.healthmetricsandevaluation.org/gbd-compare/>. Spearman rank correlation analysis was performed to assess statistical dependence between CDSR representation and disease burden. Rho, a coefficient ranging from-1 (strong negative correlation) to +1 (strong positive correlation), is interpreted with a two-tailed p-value. A line-of-best-fit was also generated between CDSR representation and % of total DALYs.
As this study did not involve human subjects, institutional review board approval was not necessary.
RESULTS
Nine of the ten MHUD conditions studied in GBD 2010 were represented by at least one systematic review. A total of 116 systematic reviews and protocols published by nine Cochrane review groups represented the ten MHUDs (Supporting Tables 1 and 2 for included and excluded titles, respectively). The majority of reviews and protocols covered tubulointerstitial nephritis, pyelonephritis, and urinary tract infections (n=46). Hydrocele due to lymphatic filariasis had no representation in CDSR. Of the ten MHUDs, benign prostatic hyperplasia had the greatest global disease burden (0.2%) while male infertility had the lowest (0.007%) Table-1.
Men’s health and urologic diseases studied by GBD 2010 with corresponding ICD-10 codes, search terms, number of systematic reviews (R) and protocols (P) in CDSR, percent of total DALYs (arranged in order of decreasing % of total DALY), and number of studies included in Cochrane reviews.
Reviews and protocols representing the ten MHUDs were published by the following Cochrane review groups: Prostatic Diseases and Urologic Cancers Group (n=43); Renal Group (36); Incontinence Group (1414. World Health Organization. Neglected tropical diseases. Available from: http://www.who.int/neglected_diseases/diseases/en/. Accessed March 5, 2015.
http://www.who.int/neglected_diseases/di...
); Menstrual Disorders and Subfertility Group (88. Collier A, Heilig L, Schilling L, Williams H, Dellavalle RP. Cochrane Skin Group systematic reviews are more methodologically rigorous than other systematic reviews in dermatology. Br J Dermatol. 2006;155:1230-5.); Pregnancy and Childbirth Group (77. Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, Moher M, et al. Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA. 1998;280:278-80.); Infectious Diseases Group (33. Shiffman J. Knowledge, moral claims and the exercise of power in global health. Int J Health Policy Manag. 2014;3:297-9.); Pain, Palliative and Supportive Care Group (22. Nasser M, Welch V. Prioritization of systematic reviews leads prioritization of research gaps and needs. J Clin Epidemiol. 2013;66:522-3.); Gynecological Cancer Group (22. Nasser M, Welch V. Prioritization of systematic reviews leads prioritization of research gaps and needs. J Clin Epidemiol. 2013;66:522-3.); and Neonatal Group (11. Fleurence RL. Setting priorities for research: a practical application of ‘payback’ and expected value of information. Health Econ. 2007;16:1345-57.).
Spearman rank correlation testing between CDSR representation and DALY metrics revealed poor positive correlation that was statistically insignificant (rho=0.41, p=0.21). The majority of the MHUDs (66. Clarke M, Hopewell S, Chalmers I. Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting. Lancet. 2010;376:20-1.) were under-represented in CDSR as compared to GBD DALY (Figure-2). Most of the systematic reviews and protocols (58.6%) were published from 2011 to 2015 while 37.9% were published from 2000 to 2010; only 4 reviews were published prior to 2000. Maintaining systematic reviews up-to-date is critical to deliver consensus statements on current world literature that ultimately impact clinical decisions and patient outcomes.
Comparison of men’s health and urologic disease representation in the Cochrane Database of Systematic Reviews with percent of 2010 DALYs from 291 conditions studied by GBD 2010.
Representation of tubulointerstitial nephritis, pyelonephritis, and urinary tract infections exceeded GBD disease burden. This disease category also had the greatest number of cumulative studies informing its systematic reviews (529). The one systematic review representing testicular cancer, entitled “Screening for testicular cancer,” found no randomized controlled trials in the literature. Systematic reviews that find no suitable trials to address their objectives uncover areas for much-needed, high-quality research.
The World Health Organization (WHO) classifies two of the MHUDs as neglected tropical diseases: dysuria/bladder pathology/hydronephrosis due to schistosomiasis and hydrocele due to lymphatic filariasis (1414. World Health Organization. Neglected tropical diseases. Available from: http://www.who.int/neglected_diseases/diseases/en/. Accessed March 5, 2015.
http://www.who.int/neglected_diseases/di...
). It is important to note that just as the DALY metrics reported for these two diseases include only burden due to the MHUD morbidity (dysuria, bladder pathology, hydronephrosis, hydrocele), systematic reviews were only considered representative if they included assessment of the MHUD pathology.
DISCUSSION
We acknowledge several limitations of our study. The scope of CDSR systematic reviews is subject to variability. For instance, authors may prepare one large review of multiple interventions (lumping) or several reviews of individual interventions (splitting). Therefore, treating a systematic review or protocol as one measurement unit may not be entirely accurate for every topic. While beyond the scope of this limited study, further exploration is warranted into potential underrepresentation of certain conditions.
There remains a lack of transparency in publications and databases on the quality of data and criteria involved in prioritization decisions (1515. Oxman AD, Schünemann HJ, Fretheim A. Improving the use of research evidence in guideline development: 2. Priority setting. Health Res Policy Syst. 2006;4:14.). Other important factors in priority setting include availability of research funds, knowledge gap, and impact on disadvantaged populations. Research prioritization is also inherently political and dependent on financial backing, which further demonstrates the importance of a transparent process. Attention and awareness of priority setting has the potential to minimize research disparities and, ultimately, impact populations at a global scale.
ACKNOWLEDGEMENTS:
Tyler Okland and Chante Karimkhani are contributed similarly as first authors.
Luc Coffeng MD, Department of Public Health at Erasmus MC University Medical Center, Rotterdam, Nederlands.
Megan Coggeshall BA, Institute for Health Metrics and Evaluation, University of Washington, Seattle, WA.
REFERENCES
-
1Fleurence RL. Setting priorities for research: a practical application of ‘payback’ and expected value of information. Health Econ. 2007;16:1345-57.
-
2Nasser M, Welch V. Prioritization of systematic reviews leads prioritization of research gaps and needs. J Clin Epidemiol. 2013;66:522-3.
-
3Shiffman J. Knowledge, moral claims and the exercise of power in global health. Int J Health Policy Manag. 2014;3:297-9.
-
4Murray CJ, Ezzati M, Flaxman AD, Lim S, Lozano R, Michaud C, et al. GBD 2010: design, definitions, and metrics. Lancet. 2012;380:2063-6.
-
5Lozano R, Naghavi M, Foreman K, Lim S, Shibuya K, Aboyans V, et al. Global and regional mortality from 235 causes of death for 20 age groups in 1990 and 2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380:2095-128. Erratum in: Lancet. 2013;381:628.
-
6Clarke M, Hopewell S, Chalmers I. Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting. Lancet. 2010;376:20-1.
-
7Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, Moher M, et al. Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA. 1998;280:278-80.
-
8Collier A, Heilig L, Schilling L, Williams H, Dellavalle RP. Cochrane Skin Group systematic reviews are more methodologically rigorous than other systematic reviews in dermatology. Br J Dermatol. 2006;155:1230-5.
-
9Emdin CA, Odutayo A, Hsiao AJ, Shakir M, Hopewell S, Rahimi K, et al. Association between randomised trial evidence and global burden of disease: cross sectional study (Epidemiological Study of Randomized Trials—ESORT). BMJ 2015;350 :h117.
-
10Yoong SL, Hall A, Williams CM, Skelton E, Oldmeadow C, Wiggers K, et al. Alignment of systematic reviews published in the Cochrane Database of Systematic Reviews and the Database of Abstracts and Reviews of Effectiveness with global burden of disease data: a bibliographic analysis. J Epidemiol Community Health. J Epidemiol Community Health. 2015 Jul;69(7):708-14
-
11Pederson H, Okland T, Boyers LN, Karimkhani C, Rosenfeld RM, Nasser M, et al. Identifying otolaryngology systematic review research gaps: comparing Global Burden of Disease 2010 results with Cochrane Database of Systematic Review content. JAMA Otolaryngol Head Neck Surg. 2015;141:67-72.
-
12Karimkhani C, Trikha R, Aksut B, Jones T, Boyers LN, Schlichte M, et al. Identifying gaps for research prioritisation: Global burden of external causes of injury as reflected in the Cochrane Database of Systematic Reviews. Injury. 2016;47:1151-7.
-
13The Cochrane Collaboration. Search the Cochrane library. Available from: http://www.thecochranelibrary.com/view/0/index.html Accessed January 15, 2015.
» http://www.thecochranelibrary.com/view/0/index.html -
14World Health Organization. Neglected tropical diseases. Available from: http://www.who.int/neglected_diseases/diseases/en/ Accessed March 5, 2015.
» http://www.who.int/neglected_diseases/diseases/en/ -
15Oxman AD, Schünemann HJ, Fretheim A. Improving the use of research evidence in guideline development: 2. Priority setting. Health Res Policy Syst. 2006;4:14.
-
FINANCIAL SUPPORTThere was no direct funding to the current study. The Global Burden of Disease study received funding from the Bill and Melinda Gates Foundation (PI: Christopher J.L. Murray). Lindsay Boyers, Mark Sawyer, and Robert Dellavalle are employees of the U.S. Department of Veterans Affairs. The U.S. Department of Veterans Affairs had no role in the design and execution of the study. Robert Dellavalle is supported by grants from the CDC and National Institutes of Health. Tyler Okland, Chante Karimkhani, Hannah Pederson, Lindsay Boyers, Mohsen Naghavi, and Mark Sawyer report no relevant disclosures. Any opinions expressed herein do not necessarily reflect the opinions of the CDC or the Department of Veterans Affairs.
Publication Dates
-
Publication in this collection
Mar-Apr 2017
History
-
Received
04 Apr 2016 -
Reviewed
20 Sept 2016 -
Accepted
06 Jan 2017