Comparison of the methodological quality and transparency of Brazilian practice guidelines

Comparação da qualidade metodológica e transparência das guias de prática clínica brasileiras

Abstract

This study aims to compare the differences between clinical practice guidelines (CPGs) of the Ministry of Health (MoH) and those of other Brazilian health institutions. A systematic review of Brazilian CPGs was carried out. CPGs with recommendations for the pharmacological treatment of non-communicable disease (NCDs) were included. CPG methodological quality and transparency was independently assessed by 2 reviewers using the AGREE II. CPGs were rated as high, moderate, and low quality (ranging from A to C). Twenty-six CPGs were assessed for quality. MoH CPGs were published more recently, and were of better quality than the others: 6/6 (100%) were rated as Moderate-A. Although CPGs presented a wide range of methodological quality and transparency, MoH CPGs presented better consistency in the preparation method. To avoid confusion and to improve the quality of care within finite resources in Brazil, and to avoid potential bias, conflicts of interest, national CPGs used within SUS should be developed by Conitec with partners who have no conflict of interest.

Key words
Practice guidelines; Chronic disease; Primary health care; Technology assessment biomedical; Delivery of health care

Resumo

O objetivo deste estudo é comparar as diferenças entre as guias de prática clínica (GPCs) do Ministério da Saúde (MS) e as de outras instituições de saúde brasileiras. Foi realizada uma revisão sistemática das GPCs brasileiras. Foram incluídas GPCs com recomendações para o tratamento farmacológico de doenças crônicas não transmissíveis elencadas (DCNTs). A qualidade metodológica e a transparência das GPCs foram avaliadas de forma independente por 2 revisores utilizando o AGREE II. As GPCs foram classificadas como alta, moderada e baixa qualidade (variando de A a C). Vinte e seis GPCs foram avaliadas quanto à qualidade. As GPCs do MS foram publicadas mais recentemente, e apresentaram melhor qualidade do que as outras: 6/6 (100%) foram classificadas como Moderada-A. Embora as GPCs tenham apresentado uma ampla gama de qualidade metodológica e transparência, as GPCs do MS apresentaram melhor consistência no desenvolvimento. Para evitar confusão e melhorar a qualidade do cuidado com os recursos limitados no Brasil e, para evitar viés, conflitos de interesse, GPCs nacionais usadas no SUS devem ser desenvolvidas, sobretudo, pela Conitec e parceiros sem conflitos de interesse.

Palavras-chave
Guias de prática clínica como assunto; Doença crônica; Atenção primária à saúde; Avaliação da tecnologia biomédica; Assistência à saúde

Introduction

Health systems have used Health Technology Assessment (HTA) tools and principles to guide decisions related to inclusion and exclusion of health technologies, such as the incorporation of drugs¹. The World Health Organization (WHO) and the Pan American Health Organization (PAHO) have encouraged the development of HTA in Latin America and the Caribbean since the 1990s, when many health systems were reformed in the region, including in Brazil². Only 2011, the National Committee for Technology Incorporation (Conitec) was created in Brazil³. The main purpose of Conitec is to assist the Ministry of Health (MoH) in incorporating or excluding health technologies for the Unified Health System (SUS)³. Conitec is also in charge of the National List of Essential Medicines (RENAME) update, the development of clinical practice guidelines (CPGs) to guide health professionals and managers regarding the medication access and availability within the SUS^3-5. Since the implementation of the Conitec, the number of drugs incorporated or excluded from the SUS has increased^5,6. However, due to the great demand for drug incorporation to the SUS, Conitec prioritizes the development of CPGs with most financial impact on the SUS^4,5.

In Brazil, the SUS provides drugs for primary care which are selected and dispensed by the municipalities⁷. Ideally, all drugs included in the SUS should be incorporated together with a clinical practice guideline (CPG) to assist health care professionals. However, municipalities do not have the resources needed to develop or adapt CPGs^8,9, and the Conitec is unable to meet the national demand. Consequently, health professionals follow CPGs published by different health institutions, and this raises concerns about the quality of the CPGs that are available.

Other Brazilian health institutions, governmental or non-governmental, such as specialty societies, have been preparing and publishing CPGs for many years. The MoH’s Department of Primary Care publishes the “Primary Care Notebooks” however, they are not listed on the MoH or Conitec CPG websites. The Brazilian Medical Association (AMB) developed the “Guidelines Project,” which establishes medical conducts for diagnosing and managing various diseases¹⁰. Medical societies such as the Brazilian Society of Cardiology and the Brazilian Society of Pulmonology also draft and publish CPGs^11,12. However, concerns have been reported regarding the quality of the Brazilian CPGs^13,14. Previous studies have found low methodological quality and moderate to low transparency in the Brazilian CPGs^13,14. Therefore, in this study we compare methodological quality and transparency differences between MoH/Conitec CPGs and those of other Brazilian health institutions to provide evidence to improve CPGs in Brazil.

Methods

Study design

This study is a complementary analysis of a systematic review and assessment of the methodological quality and transparency of the Brazilian CPGs¹⁴. This systematic review was registered in PROSPERO.

In sum, a comprehensive search was conducted for CPGs containing pharmacological treatments for the non-communicable diseases (NCDs) listed below. On October 30, 2015, a systematic search was carried out in three databases: MEDLINE (by PubMed), LILACS (by the Virtual Health Library), and the Cochrane Library. The specific CPG website, the National Guideline Clearinghouse, available at guidelines.gov, was searched on September 9, 2015. A manual search was also done on Google and on the MoH/Conitec and AMB websites on September 9, 2015.

CPGs featuring pharmacological treatments for the following NCDs were included: Asthma, dementia, depression, type 2 diabetes, coronary disease and/or stable angina, chronic obstructive pulmonary disease, atrial fibrillation, hypercholesterolemia, benign prostatic hyperplasia, hypertension, heart failure, osteoarthritis, osteoporosis, and gastroesophageal reflux disease. CPGs were excluded if they were for local use or only for specific populations, such as pediatric and pregnant women. Year of publication or language restrictions were not applied.

One reviewer did the systematic search; two reviewers applied the eligibility criteria independently.

Data were extracted by one reviewer and reviewed by a second one. Data extracted from the CPGs that were included were publisher (MoH, specialty societies, BMA), year of publication, and NCDs addressed.

Discrepancies at any stage were resolved by consensus between the reviewers. When necessary, a third reviewer was included in the process.

As previously described¹⁴, 661 studies were found in the systematic search, of which 26 met the eligibility criteria and were assessed for methodological quality and transparency. Details on the studies that were included, excluded, and on the PRISMA checklist can be found in a previously published study¹⁴.

CPG Quality Assessment

The CPGs’ methodological quality and transparency were assessed independently by 2 reviewers based on the AGREE II Appraisal of Guidelines for Research & Evaluation instrument14. AGREE II was chosen because it is a widely used instrument and has been adapted and validated for Brazilian Portuguese^13,15-18. AGREE II comprises 23 items grouped into 6 domains and 2 global assessment items that allow reviewers to evaluate the CPG’s overall quality and recommend or not recommend its use¹⁹. The 6 domains that comprise AGREE II are: 1) Scope and purpose (items 1-3), 2) Stakeholder involvement (items 4-6), 3) Rigor of development (items 7-14), 4) Clarity of presentation (items 15-17), 5) Applicability (items 18-21), and 6) Editorial independence (items 22-23)¹⁹. Each item is rated on a 7-point scale, in which 1 means totally disagree and 7 totally agree¹⁹. Each domain’s final score is given by the percentage of the sum of the scores of the items of each domain in relation to the maximum score for that domain¹⁹. Thus, each domain’s scores are independent and should not be summed in a single final score¹⁹.

Discrepancies in AGREE II scores were assessed based on Kappa coefficients²⁰ and on McMaster University’s concordance calculator²¹. First, the Kappa coefficient was calculated to identify CPGs with discrepant scores, and those with a Kappa below or equal to 0.4 were considered as discrepant. Next, the domain with a high level of discrepancy was identified using McMaster University’s calculator. Finally, domains identified with high discrepancies were discussed and independently reassessed by reviewers. A third reviewer was included as needed.

Analysis of the data

As described previously¹⁴, the CPGs were rated as of high, moderate or low general quality and, then, from A to C, as shown in Figure 1.

Figure 1
Overall quality rating of clinical practice guidelines according to AGREE II domain scores. AGREE II: Second version of the tool for clinical guideline evaluation.

The CPGs were stratified per publisher: MoH, specialty societies or BMA. The difference between each publisher’s domain score medians was analyzed by means of the Kruskal-Wallis statistical test. P values less than 0.05 were considered statistically significant.

Two sensitivity analyses were carried out to determine if the year of publication influenced the AGREE II score:

1. Comparison between the CPGs published after or in 2009 (the year AGREE II was published);

2. Comparison between the CPGs published after or in 2013 (the year of the oldest MoH CPG included in the study was published);

Results

Characteristics of CPGs by publisher are described in Table 1. In total, 26 CPGs were assessed: 6 of the Ministry of Health, 10 of the specialty societies, and 10 of the BMA. Chronic obstructive pulmonary disease was the only NCD for which a CPG had been published by all publishers. MoH GPCs were published more recently.

Table 1
Characteristics of clinical practice guidelines with pharmacological treatment for chronic non-communicable diseases, stratified by the publishers (n = 26).

Characteristic	Publisher
	Ministry of Health	Specialty societies	Brazilian Medical Association
Total CPGs	6	10	10
Year of publication
Mode	2013	2012	2011
Older	2013	2004	2004
Most recent	2015	2015	2012
Number of CPGs published after 2009	6	9	7
Non-communicable disease covered
Asthma	1	1
Rheumatic arthritis	1	1
Depression			1
Type 2 diabetes		2	2
Coronary disease		1
Alzheimer's disease	1		1
Gastroesophageal reflux disease			1
Chronic obstructive pulmonary disease	1	1	1
Atrial fibrillation		1
Hypercholesterolemia	1	1
Benign prostatic hyperplasia			1
Hypertension		1	1
Heart failure		1
Osteoarthritis			1
Osteoporosis	1		1

• * CPG: Clinical practice guideline.

Figure 2 shows that MoH CPGs were of better quality than the others: 6/6 (100%) with Moderate-A quality. Only 3 out of 10 of the specialty societies’ CPGs were rated as Moderate-B, while 7 out of 10 BMA CPGs were rated as Low-B quality.

Figure 2
General clinical practice guideline quality rating per publisher.

Except for the clarity of presentation domain, MoH CPGs got higher scores in all domains when compared to other publishers’ (Figure 3). In addition, that MoH CPG domains got more homogeneous scores than the other publishers’ CPGs.

Figure 3
Distribution of AGREE II domain scores per publisher. Total of 26 CPGs, of which 6 are from the Ministry of Health, 10 from the medical specialties, and 10 from the Brazilian Medical Association. AGREE II: Second version of the tool for clinical guideline evaluation.

Again, except for the clarity of presentation domain, Figure 4 shows that MoH CPGs had domains that scored better, even when compared to CPGs published after 2009 or after 2013.

Figure 4
Median AGREE II domain scores per publisher and year of publication, after 2009 or after 2013. Total of 26 CPGs assessed, of which 6 from the Ministry of Health, and all of them were published after 2013, 10 from the medical specialties, and 10 from the Brazilian Medical Association. AGREE II: Second version of the tool for clinical guideline evaluation.

Discussion

Our findings suggest that Brazilian CPGs have a wide range of methodological quality and transparency, depending on the publisher. The concern about the need to improve CPG quality is not recent. A 2000 study assessed 431 CPGs prepared by specialty societies and found unsatisfactory quality²². Recently, another study assessed CPGs for the pharmacological treatment of bipolar disorder and found that those prepared by specialty societies got lower scores than those prepared by other institutions²³. In this study, the assessed CPGs had low or moderate rigor of development, but those prepared by the MoH got higher scores than the others, as was the case in previous reports^17,23.

It is interesting to note that, in addition to MoH CPGs being of moderate quality and getting higher scores when compared to those of other publishers, except in the clarity of presentation domain, MoH CPG scores were more homogeneous than the others. This may be explained by the fact that MoH follows methodological guidelines to draft and publish CPGs^24-27. Another explanation is that only the MoH CPG developer group presented methodologists specialized in CPG development and systematic review. In fact, as noted by Burgers et al.²⁸, in a review of 86 guidelines from 11 countries, CPGs prepared within a program organized and coordinated by government institutions were of better quality.

There is, however, another possible explanation. It can be inferred that MoH GPCs were published more recently; therefore, CPG preparers may have followed recommendations featured in current studies when drafting the CPGs, such as the AGREE recommendations themselves. In fact, a systematic review of 626 CPGs published between 1980 and 2007 found that CPG quality has improved over the years¹⁷. However, in the present study, the sensitivity analysis showed that MoH CPGs still got higher scores in most domains when compared to CPGs that were only published after 2009 or 2013.

In addition, results from two other domains should be highlighted: Stakeholder involvement and applicability. As discussed previously¹⁴, insofar as the CPGs development group is concerned, no publisher explicitly considered the patients’ preferences when developing the CPGs, and only the MoH seems to bring together a multidisciplinary team to prepare its CPGs. The specialty societies included only physicians from the same area. The BMA brought together mixed professionals, but the group comprised mainly physicians of different specialties. In addition to the multidisciplinary MoH CPG preparer group, the MoH promotes public consultations in which anyone can analyze and make suggestions for improving the CPGs.

The applicability domain was the one that scored lowest, corroborating previously reported results^17,18,23,29. Knowing that the applicability domain is directly related to the CPG’s potential for implementation, publishers need to be about the implementation of recommendations in a real life context. Lack of concern with applicability is a critical issue, since CPGs should not only be a synthesis of the evidence, but also be able to translate scientific knowledge for use in clinical practice³⁰.

Editorial independence was the only domain with no significant difference among publishers. MoH CPGs did not report whether the views of the funding source influenced CPG content. Although most CPGs revealed the authors’ conflicts of interest, information about how the conflicts of interest would be managed, the explicit statement of the source of funding and of the conflicts of interest were absent. However, these results should be interpreted with caution because the assessment with AGREE is based on the reports indicated in the CPGs¹⁹, which may not truly reflect the CPG preparation process. By contrast, it is known that at least 80% of physicians have some relationship to pharmaceutical companies, such as the psychological effects and expectations of reciprocity³¹. Recently, study conducted among cancer CPGs reported that 86% of the authors had at least one financial conflicts of interest, meaning an average greater than $200,000 (range $0-$2,756,713) in industry research payments³². Furthermore, reports that financial ties to the pharmaceutical companies have not been fully disclosure by CPGs’ authors³³ have raised the concerns about the credibility of the authors’ conflict of interest declaration. Consequently, awareness of the importance of developing and implementing CPGs created by organizations and authors that have no conflict of interest must be reinforced³⁴.

In addition, it should be noted that the publishers used different terms to designate the CPGs. For example, specialty societies and the BMA have often used consensus terms or guidelines instead of CPGs. Despite publishing methodological guidelines based on AGREE II, GRADE, and ADAPTE^25,27,35, the Ministry of Health also preferred another term: clinical protocols and therapeutic guidelines. This lack of standardization and use of the term CPG may be related to the lack of understanding of the characteristics that should constitute a high-quality CPG and, thus, reduces methodological quality and transparency.

The development of CPG is complex³⁶, and demands time and financial resources³⁷. Alternatives such as CPG adaptation have been reported in other countries as an efficient way to compile CPGs^38,39. For example, middle-income countries such as South Africa and Kazakhstan have developed strategies for adapting CPGs to their context based on other high quality CPGs^38,39. The Ministry of Health of Saudi Arabia, in collaboration with McMaster University, developed 10 CPGs within 4 months using methods for adapting high-quality CPGs^40,41. As mentioned, most Brazilian municipalities have little resources available to prepare or adapt CPGs^8,9. A solution that would allow the development of high-quality GPCs for local realities would be the establishment of partnerships between the MoH and universities or specialty societies to adapt high-quality CPGs, following suit of successful experiences carried out in other countries.

The generalization of these results is subject to the documents related to the CPGs that were assessed. According to the AGREE II User Manual, all documents related to the CPG should be taken into account in the methodological assessment of quality and transparency¹⁹. Despite our extensive research for supplementary materials, however, some document might be lost in the quality assessment process, which would undermine AGREE scores. For example, Brazilian Society of Cardiology CPGs are published in scientific databases such as PubMed, and a more user-friendly version of the CPG is posted on the society’s official website. As previously reported^13,14, MoH CPGs are published in two versions. The first is published in the Official Gazette of the Brazil without the flowcharts and tables that improve applicability and clarity of presentation domain scores. The second version is the most complete and user-friendly version of the CPG, and is compiled in the official MoH books. However, when the book version is not available, the CPG is deprived of the second version. In fact, in this study, a lot of the information about the CPG development process was found in the preface to the MoH books, as previously reported^13,14.

In addition, another limitation of this study is the non-assessment of the MoH “Notebooks of Basic Care” series. This series was not included in the study because, with the enactment of federal Act 12,401/2011 and of decree 7646/2011, the Conitec was put in charge of developing CPGs^4,6.

Finally, there is room to improve the MoH CPG quality in coming years, since Conitec published the first edition of the “Methodological Guidelines: Preparation of Clinical Guidelines” in 2016³⁵. This guideline follows the best evidence available for CPG preparation, such as AGREE II, and uses GRADE to formulate recommendations. However, until this new era of CPGs is published, health professionals must be cautious in choosing CPGs to guide their clinical practice.

This study’s findings suggest that MoH CPGs have better methodological quality and transparency than other Brazilian CPGs. However, despite the need to improve MoH CPGs in nearly all domains, MoH CPGs got more homogeneous scores, suggesting better consistency in the preparation method the MoH/Conitec adopts. To avoid confusion and to improve the quality of care within finite resources in Brazil, and to avoid potential bias, conflicts of interest, national CPGs used within SUS should be developed by Conitec with partners who have no conflict of interest. As a result, this may ensure and enhance methodological quality to benefit the Brazilian public healthcare system.

References

• ¹
  Silva HP, Petramale CA, Elias FTS. Advances and challenges to the Brazilian policy of health technology management. Rev Saude Publica 2012; 46(Supl. 1):83-90.
• ²
  Pan American Health Organization (PAHO). Developing Health Technology Assessment in Latin America and the Caribbean Whashington: PAHO; 1998.
• ³
  Brasil. Lei no 12.401, de 28 de abril de 2011. Altera a Lei no 8.080, de 19 de setembro de 1990, para dispor sobre a assistência terapêutica e a incorporação de tecnologia em saúde no âmbito do Sistema Único de Saúde - SUS. Diário Oficial da União 2011; 29 abr.
• ⁴
  Capucho HC, Salomon FCR, Vidal ÁT, Louly PG, Santos VCC, Petramale CA. Incorporação de tecnologias em saúde no Brasil: novo modelo para o Sistema Único de Saúde. BIS, Bol Inst Saúde 2012; 13(3):215-22.
• ⁵
  Caetano R, Silva RM, Pedro EM, Oliveira IAG, Biz AN, Santana P. Incorporação de novos medicamentos pela Comissão Nacional de Incorporação de Tecnologias do SUS, 2012 a junho de 2016. Cien Saude Colet 2017; 22(8):2513-2525.
• ⁶
  Santos-pinto CDB, Ventura M, Pepe VLE, Osorio-de-Castro CGS. Novos delineamentos da Assistência Farmacêutica frente à regulamentação da Lei Orgânica da Saúde. Cad Saude Publica 2013; 29(6):1056-1058.
• ⁷
  Brasil. Portaria GM/MS no 1.555, de 30 de julho de 2013. Dispõe sobre as normas de financiamento e de execução do Componente Básico da Assistência Farmacêutica no âmbito do Sistema Único de Saúde (SUS). Diário Oficial da União 2013; 31 jul.
• ⁸
  Machado CV, Lima LD, Andrade CLT. Federal funding of health policy in Brazil: trends and challenges. Cad Saude Publica 2014; 30(1):187-200.
• ⁹
  Araújo CEL, Gonçalves GQ, Machado JA. Os municípios brasileiros e os gastos próprios com saúde: algumas associações. Cien Saude Colet 2017; 22(3):953-963.
• ¹⁰
  Nobre MRC, Bernardo WM. Diretrizes AMB/CFM. Rev Assoc Med Bras 2002; 48(4):290-290.
• ¹¹
  Sociedade Brasileira de Pneumologia e Tisiologia (SBPT). Consensos e diretrizes da SBPT [Internet]. [cited 2017 Apr 19]. Available from: https://sbpt.org.br/consensos-e-diretrizes-da-sbpt/
  » https://sbpt.org.br/consensos-e-diretrizes-da-sbpt/
• ¹²
  Sociedade Brasileira de Cardiologia. Diretrizes da Sociedade Brasileira de Cardiologia [Internet]. Consensos / Diretrizes. 2017 [cited 2017 Apr 19]. Available from: http://publicacoes.cardiol.br/2014/diretrizes.asp
  » http://publicacoes.cardiol.br/2014/diretrizes.asp
• ¹³
  Ronsoni RDM, Pereira CCA, Stein AT, Osanai MH, Machado CJ. Avaliação de oito Protocolos Clínicos e Diretrizes Terapêuticas (PCDT) do Ministério da Saúde por meio do instrumento AGREE II: um estudo piloto. Cad Saude Publica 2015; 31(6):1157-1162.
• ¹⁴
  Molino CDGRC, Romano-Lieber NS, Ribeiro E, Melo DO. Non-communicable disease clinical practice guidelines in Brazil: A systematic assessment of methodological quality and transparency. PLoS One 2016; 11(11):e0166367.
• ¹⁵
  Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE, Littlejohns P, Makarski J, Zitzelsberger L; AGREE Next Steps Consortium. AGREE II: Advancing guideline development, reporting and evaluation in health care. CMAJ 2010; 182(18):E839-842.
• ¹⁶
  Khan GSCC, Stein AT. Adaptação transcultural do instrumento Appraisal of Guidelines for Research & Evaluation II (AGREE II) para avaliação de diretrizes clínicas. Cad Saude Publica 2014; 30(5):1111-1114.
• ¹⁷
  Alonso-Coello P, Irfan A, Solà I, Gich I, Delgado-Noguera M, Rigau D, Tort S, Bonfill X, Burgers J, Schunemann H. The quality of clinical practice guidelines over the last two decades: a systematic review of guideline appraisal studies. Qual Saf Heal Care 2010; 19(6):e58.
• ¹⁸
  Knai C, Brusamento S, Legido-Quigley H, Saliba V, Panteli D, Turk E, Car J, McKee M, Busse R. Systematic review of the methodological quality of clinical guideline development for the management of chronic disease in Europe. Health Policy (New York) 2012; 107(2-3):157-167.
• ¹⁹
  AGREE Next Steps Consortium (2009). The AGREE II Instrument - Electronic version [Internet]. [cited 2017 Apr 19]. Available from: http://www.agreetrust.org
  » http://www.agreetrust.org
• ²⁰
  Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33(1):159-174.
• ²¹
  McMaster Univeristy. McMaster AGREE II score calculator Excel spreadsheet [Internet]. [cited 2015 Jul 8]. Available from: http://fhswedge.csu.mcmaster.ca/cepftp/qasite/CriticalAppraisal.html
  » http://fhswedge.csu.mcmaster.ca/cepftp/qasite/CriticalAppraisal.html
• ²²
  Grilli R, Magrini N, Penna A, Mura G, Liberati A. Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet 2000; 355(9198):103-106.
• ²³
  Castellani A, Girlanda F, Barbui C. Rigour of development of clinical practice guidelines for the pharmacological treatment of bipolar disorder: systematic review. J Affect Disord 2015; 174:45-50.
• ²⁴
  Brasil. Ministério da Saúde (MS). Diretrizes metodológica: Diretriz de Avaliação Econômica 2ª ed. Brasília: MS; 2014.
• ²⁵
  Brasil. Ministério da Saúde (MS). Diretrizes metodológicas?: Sistema GRADE - Manual de graduação da qualidade da evidência e força de recomendação para tomada de decisão em saúde Brasília: MS; 2014.
• ²⁶
  Brasil. Ministério da Saúde (MS). Diretrizes metodológicas: elaboração de revisão sistemática e metanálise de ensaios clínicos randomizados Brasília: MS; 2012.
• ²⁷
  Brasil. Ministério da Saúde (MS). Diretrizes metodológicas: ferramentas para adaptação de diretrizes clínicas Brasília: MS; 2014.
• ²⁸
  Burgers JS, Cluzeau FA, Hanna SE, Hunt C, Grol R. Characteristics of high-quality guidelines: evaluation of 86 clinical guidelines developed in ten European countries and Canada. Int J Technol Assess Health Care 2003; 19(1):148-157.
• ²⁹
  Kredo T, Gerritsen A, van Heerden J, Conway S, Siegfried N. Clinical practice guidelines within the Southern African Development Community: a descriptive study of the quality of guideline development and concordance with best evidence for five priority diseases. Health Res Policy Syst 2012; 10:1.
• ³⁰
  Gagliardi AR, Berta W, Kothari A, Boyko J, Urquhart R. Integrated knowledge translation (IKT) in health care: a scoping review. Implement Sci 2015; 11(1):38.
• ³¹
  Keller F, Marczewski K, Pavlovic D. The relationship between the physician and pharmaceutical industry: Background ethics and regulation proposals. Croat Med J 2016; 57(4):398-401.
• ³²
  Mitchell AP, Basch EM, Dusetzina SB. Financial Relationships With Industry Among National Comprehensive Cancer Network Guideline Authors. JAMA Oncol 2016; 2(12):1628.
• ³³
  Cosgrove L, Bursztajn HJ, Krimsky S, Anaya M, Walker J. Conflicts of interest and disclosure in the American Psychiatric Association's clinical practice guidelines. Psychother Psychosom 2009; 78(4):228-232.
• ³⁴
  Schott G, Dünnweber C, Mühlbauer B, Niebling W, Pachl H, Ludwig W-D. Does the Pharmaceutical Industry Influence Guidelines? Dtsch Arztebl Int 2013; 110(35-36):575-583.
• ³⁵
  Brasil. Ministério da Saúde (MS). Diretrizes Metodológicas: elaboração de diretrizes clínicas Brasília: MS; 2016.
• ³⁶
  Atkins L, Smith JA, Kelly MP, Michie S. The process of developing evidence-based guidance in medicine and public health: a qualitative study of views from the inside. Implement Sci 2013; 8:101.
• ³⁷
  Grimmer K, Dizon J, Louw Q, Kredo T, Young T, Machingaidze S. South African Guidelines Excellence (SAGE): Efficient, effective and unbiased clinical practice guideline teams. S Afr Med J 2016; 106(5):440-441.
• ³⁸
  McGowan J, Muratov S, Tsepke A, Issina A, Slawecki E, Lang ES. Clinical practice guidelines were adapted and implemented meeting country-specific requirements - The example of Kazakhstan. J Clin Epidemiol 2016; 69:8-15.
• ³⁹
  Kredo T, Machingaidze S, Louw Q, Young T, Grimmer K. South African Guidelines Excellence (SAGE): What's in a name? S Afr Med J 2016; 106(1):18-20.
• ⁴⁰
  Rayess ZA, Wiercioch W. A new approach to CPG adaptation in Saudi Arabia: Adaptation of practice guidelines to a country-specific context using the GRADE/DECIDE evidence to decision framework. In: GIN Conference; 2014; Melbourne.
• ⁴¹
  Al-Hameed F, Al-Dorzi H, Al-Momen A, Algahtani F, Al-Zahrani H, Al-Saleh K, Al-Sheef MA, Owaidah TM, Alhazzani W, Neumann I, Wiercioch W, Brozek J, Schunemann H, Akl EA. The Saudi Clinical Practice Guideline for the treatment of venous thromboembolism. Outpatient versus inpatient management. Saudi Med J 2015; 36(8):1004-1010.

Publication Dates

• Publication in this collection
  26 Sept 2019
• Date of issue
  Oct 2019

History

• Received
  22 July 2017
• Accepted
  28 Mar 2018
• Published
  30 Mar 2018