Quality assessment of clinical practice guidelinesfor Chagas disease

Abstract

INTRODUCTION:

The development of clinical practice guidelines (CPGs) has increased; this study aimed to assess the quality of CPGs for the management of Chagas disease.

METHODS:

Following a systematic search of the scientific literature, two reviewers assessed the eligible guidelines using the Appraisal of Guidelines Research and Evaluation (AGREE) II instrument.

RESULTS:

Five CPGs were included. The AGREE domains of scope/purpose, stakeholder involvement, and clarity of presentation were rated well, and the domains of applicability and editorial independence received poor ratings.

CONCLUSIONS:

The quality of CPGs for Chagas disease is poor, and significant work is required to develop high-quality guidelines.

Chagas disease; Clinical Practice Guidelines; AGREE

Chagas disease, which is a parasitic infection, is considered a neglected tropical disease, and it has the greatest socioeconomic impact in Latin America. It affects the poorest people and causes substantial deterioration in health status⁽¹⁾. However, it is a low-priority disease for public health authorities and the pharmaceutical industry, and safe and effective treatments are lacking.

In Colombia, the development of clinical practice guidelines (CPGs) has recently increased; however, a quality assessment of the guidelines has not been conducted, despite the effect on proliferation. Many of the CPGs that address similar issues appear to have significant inconsistencies and discrepancies in the recommendations.

Because CPGs enable rapid transfer between research and practice, the validity and reliability of guidelines are especially relevant^{(2) (3)}. The use of a validated instrument, such as the Appraisal of Guidelines Research and Evaluation (AGREE) II⁽³⁾, to analyze the quality of developed guidelines helps to identify the factors that could improve the development of CPGs. The quality of CPGs for Chagas disease that is available globally has not been systematically evaluated. Therefore, this study aimed to assess the quality of available guidelines for Chagas disease, to obtain the information necessary to improve the quality of CPGs for this important disease.

An electronic search was conducted of the published literature in Publicações Médicas (PubMed), Scientific Electronic Library Online (SciELO), and Google Scholar using the keywords Chagas disease or Trypanosoma cruzi, Guidelines, Consensus, and Practice Guideline; all literature published in any language before January 2014 for studies conducted with humans only was included. A secondary search was conducted by reviewing the reference lists of the retrieved CPGs.

For a publication to be included, it had to be a CPG that involved the management of patients with Chagas disease. CPGs were defined as documents developed according to scientific criteria to facilitate decision-making in health care and containing recommendations for the prevention and treatment of Chagas disease. Manuals and protocols of institutions were not considered CPGs.

The AGREE II was used to analyze the quality of each guideline. It consists of 23 items, with responses on an ordinal scale, from 1 to 4⁽³⁾. The items are organized into six key domains, each assessing a separate dimension of the quality of the guideline: scope/purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. An overall assessment of the CPG quality is also included.

Two expert reviewers individually and independently assessed the CPGs using AGREE II. Any disagreements were resolved by a third reviewer. Agreement between reviewers was calculated using the kappa coefficient (κ) for all of the guidelines and each of the guidelines. The kappa coefficient was interpreted according to the guidelines proposed by Landis and Koch⁽⁴⁾. Statistical analyses were performed using Stata(r) v11.0 (Stata, College Station, TX).

Of the 8,566 publications regarding the management of patients with Chagas disease that were retrieved (Figure 1 ), five guidelines were selected for the analyses^{(5) (6) (7) (8) (9)} (Table 1 ).

Figure 1:
Flow diagram of the search and selection process of clinical practice guidelines for Chagas disease.

Table 1:
Clinical practice guidelines for Chagas disease management included in the present systematic review

In the guideline entitled Diagnosis and Treatment of Patients with Chagas Disease by the Ministry of Health in the Presidency of the Nation of Argentina⁽⁵⁾, the description of the systematic review methodology was not explicit enough to be reproducible. The guideline mentioned that systematic searches and informal expert consensus were conducted and that the recommendations were based on the supporting evidence. The domains of scope and purpose and stakeholder involvement were scored higher than the other domains (Table 2 ). In the I Latin American guidelines for the diagnosis and treatment of Chagas heart disease ⁽⁶⁾ the description of the methodology was also not explicit enough to be reproducible. The recommendations were based on evidence. It was the only guideline that achieved higher scores for clarity of presentation and editorial independence. The guideline entitled Comprehensive Care Clinic Patient with Chagas Disease by the Ministry of Social Protection of the Republic of Colombia⁽⁷⁾ did not report that a systematic review of the literature was conducted. However, a review of other guidelines was reported. Some of the recommendations were based on the results of a Brazilian consensus. The scope and purpose and stakeholder involvement domains obtained higher scores than the other domains. The guideline entitled Diagnosis, Treatment and Prevention of Chagas Disease by the Ministry of Health of the Government of Chile⁽⁸⁾ did not report the databases in which the systematic search was conducted. However, the terms that were used for the literature search were described. The recommendations were not based on the supporting evidence. The Brazilian consensus in Chagas Disease ⁽⁹⁾ was based on expert opinions and, although it was not described, there was an explicit relationship between the recommendations and supporting evidence. The clarity of presentation domain achieved a higher score than the other domains.

Table 2:
Domain-standardized scores of each clinical practice guidelines, as assessed using the Appraisal of Guidelines Research and Evaluation (AGREE) II instrument

The overall agreement between the reviewers was high: κ = 0.91 (95% confidence interval: 0.89-0.96).

Aspects related with the scope and purpose, stakeholder involvement, and clarity of presentation of the guidelines received high scores; the poorest scores were for aspects related to the applicability and editorial independence. This could be due to insufficient information about the latter items.

Numerous methodological documents were available that lacked descriptions of their development and evidence for the recommendations. This is not uncommon and might be explained by the lack of specific algorithms to search databases⁽¹⁰⁾. We believe that the search strategy employed in the present study included all relevant sources that allowed the identification of documents that could be defined as CPGs. Regarding the evaluation of the guidelines, the results are similar to those of other studies in which the domains of purpose/participation and presentation received the highest scores^{(11) (12)}. Also, the applicability of the recommendations for clinical practice was rated as poor and was essentially not addressed in the guidelines. In terms of methodological rigor, none of the evaluated guidelines had structured the clinical questions that serve as the basis for a systematic review of the literature⁽¹³⁾. This reflects the lack of standardized processes in the development of these CPGs, which resulted in documents of varying quality.

Importantly, all guidelines scored low in applicability, mainly because an explicit statement of the potential organizational barriers in applying the recommendations was lacking, and the costs resulting from the implementation of the recommendations in clinical practice were not considered. In addition, the key criteria for monitoring and/or an audit were not described. Moreover, the guidelines did not indicate if editorial independence from the financial institution existed, and the conflicts of interest for the group members involved in the guideline development were not reported.

The evaluation of guidelines for Chagas disease is very important for Colombia, because most patients with this disease are located in rural areas where there are organizational barriers that prevent the implementation of recommendations. The implementation of CPGs and the evidence in general requires individual and organizational changes^{(14) (15)}.

Because the ratings for the majority of the domains were 40-60%, the overall assessment of the guidelines was moderate overall quality or recommended with modifications. With some modifications, such as additional information regarding the methodology, the guidelines could be considered for use, especially with the lack of other available clinical information. With a disease, such as Chagas disease, that is considered a public health problem, it is a paradox that a high-quality CPG was not retrieved.

In conclusion, the CPGs currently available for the management of Chagas disease have important limitations, particularly in the areas of development, measures of implementation, and audit of the proposed measures. Significant changes are required to generate high-quality guidelines that can be used as reliable tools for clinical decision-making.

• ¹
  World Health Organization (WHO). Chagas disease (American trypanosomiasis). (Cited 2014 December 03). Available at: who.int/mediacentre/factsheets/fs340/en/
  » https://www.who.int/mediacentre/factsheets/fs340/en/
• ²
  Holloway KA, Henry D. WHO essential medicines policies and use in developing and transitional countries: an analysis of reported policy implementation and medicines use surveys. PLoS Med 2014; 11:e1001724.
• ³
  Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, et al. AGREE II: advancing guideline development, reporting, and evaluation in health care. Prev Med 2010; 51:421-424.
• ⁴
  Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33:159-174.
• ⁵
  Ministerio de Salud de la Nación. Guías para la atención al paciente infectado con Trypanosoma cruzi (Enfermedad de Chagas). Buenos Aires; 2012. (Cited 2014 December 03). Available at: http://www.msal.gov.ar/chagas/images/stories/Equipos/Guia_Nacional_Chagas_version_27092012.pdf
  » http://www.msal.gov.ar/chagas/images/stories/Equipos/Guia_Nacional_Chagas_version_27092012.pdf
• ⁶
  Andrade JP, Marin Neto JA, Paola AA, Vilas-Boas F, Oliveira GM, Bacal F, et al. I Latin American Guidelines for the diagnosis and treatment of Chagas' heart disease: executive summary. Arq Bras Cardiol 2011; 96:434-442.
• ⁷
  Ministerio de la Protección Social. Guía para la atención clínica integral del paciente con enfermedad de Chagas. Bogotá; 2010. (Cited 2014 December 03). Available at: http://www.ins.gov.co/temas-de-interes/Chagas/02%20Guia%20Clinica%20Chagas.pdf
  » http://www.ins.gov.co/temas-de-interes/Chagas/02%20Guia%20Clinica%20Chagas.pdf
• ⁸
  Ministerio de Salud. Guía Clínica "Guías de Diagnóstico, Tratamiento y Prevención de la Enfermedad de Chagas". Santiago; 2010. (Cited 2014 December 03). Available at: http://ivl.ispch.cl/_Documentos%5CTrypanosoma%5CGu%C3%ADa_Clinica_Enf_de_Chagas_2011.pdf
  » http://ivl.ispch.cl/_Documentos%5CTrypanosoma%5CGu%C3%ADa_Clinica_Enf_de_Chagas_2011.pdf
• ⁹
  Ministério da Saúde. Consenso Brasileiro em Doença de Chagas. Rev Soc Bras Med Trop 2005; 38 (supl III):7-29.
• ¹⁰
  Misra S, Barth JH. How good is the evidence base for test selection in clinical guidelines? Clin Chim Acta 2014; 15:27-32.
• ¹¹
  Wu CM, Wu AM, Young BK, Wu D, Chen A, Margo CE, et al. An evaluation of cataract surgery clinical practice guidelines. Br J Ophthalmol 2015; 99:401-404.
• ¹²
  Schildmann EK, Schildmann J, Kiesewetter I. Medication and Monitoring in Palliative Sedation Therapy: A Systematic Review and Quality Assessment of Published Guidelines. J Pain Symptom Manage 2014; S0885-S03924.
• ¹³
  Hsu J, Brozek JL, Terracciano L, Kreis J, Compalati E, Stein AT, et al. Application of GRADE: making evidence-based recommendations about diagnostic tests in clinical practice guidelines. Implement Sci 2011; 6:62.
• ¹⁴
  Hazlewood GS, Akhavan P, Schieir O, Marshall D, Tomlinson G, Bykerk V, et al. Adding a "GRADE" to the quality appraisal of rheumatoid arthritis guidelines identifies limitations beyond AGREE-II. J Clin Epidemiol 2014; 67:1274-1285.
• ¹⁵
  Wiseman R, Cohen K, Gray A, Jamaloodien K, Kredo T, Miot J, et al. AGREE to disagree: critical appraisal and the publication of practice guidelines. S Afr Med J 2014; 104:345-346.

Publication Dates

• Publication in this collection
  may-jun 2015

History

• Received
  26 Oct 2014
• Accepted
  12 Dec 2014