Brazilian society of rheumatology methodological guide for the development of evidence-based clinical guidelines in rheumatology

Abstract

Clinical practice guidelines (CPG) are developed to align standards of health care around the world, aiming to reduce the incidence of misconducts and enabling more effective use of health resources. Considering the complexity, cost, and time involved in formulating CPG, strategies should be used to facilitate and guide authors through each step of this process. The main objective of this document is to present a methodological guide prepared by the Epidemiology Committee of the Brazilian Society of Rheumatology for the elaboration of CPG in rheumatology. Through an extensive review of the literature, this study compiles the main practical recommendations regarding the following steps of CPG drafting: distribution of working groups, development of the research question, search, identification and selection of relevant studies, evidence synthesis and quality assessment of the body of evidence, the Delphi methodology for consensus achievement, presentation and dissemination ofthe recommendations, CPG quality assessment and updating. This methodological guide serves as an important tool for rheumatologists to develop reliable and high-quality CPG, standardizing clinical practices worldwide.

Keywords
Clinical practice guideline; Methodological guide; Rheumatology

Background

Clinical practice guidelines (CPG) are developed to align standards of health care around the world, aiming to reduce the incidence of misconducts and enabling more effective use of health resources [¹].

The process of creating a CPG with reliable and high-quality recommendations requires methodological rigor in the fulfillment of a series of steps in a systematic and organized manner [²].

The initial step is to define which clinical question and population will be addressed by CPG and to select which outcome variables should be assessed to answer that question [³]. A systematic review is conducted to retrieve the best available evidence. The data obtained are synthesized and analyzed, generating a body of evidence and an estimate of effect. After grading the quality of the evidence and the strength of the recommendations, CPG authors seek consensus for final drafting of the guideline. Finally, proposed guidelines should be disseminated and implemented, and their content should be regularly updated as new evidence emerges [⁴].

In the field of rheumatology, CPG have special relevance. The growing knowledge about the pathophysiology and natural history of immune-mediated rheumatic diseases, the modernization of complementary research tools and the profusion of new therapeutic options generate a wide range of evidence and, with them, many clinical doubts [⁵]. The CPG are intended to help rheumatologists to keep acquainted with the best evidence and use it as a basis for patient care decisions [⁵].

The participation of the clinical rheumatologist with expertise in managing the population of interest is essential. Thus, the team of authors of the CPG should count not only on professionals with extensive knowledge about its methodology, but also on rheumatologists with extensive clinical know-how [⁵]. At the same time, it is important for rheumatologists to have some degree of technical and methodological knowledge to be able to critically evaluate the recommendations in the CPG [⁵].

The literature has many studies and tools related to the methodology and quality analysis of CPG. This information, however, is scattered in different articles and publications and may present a complex and poorly understood technical language. Therefore, rheumatologists may be very reluctant to participate in the development of CPG or to trust and adhere to the proposed recommendations [², ⁵].

Objectives

The Epidemiology Commission of the Brazilian Society of Rheumatology provides this methodological guide for the elaboration, development, quality assessment and updating of CPG in rheumatology.

Based on an extensive narrative review of the literature, the objective is to clarify this process in a practical and accessible way for the clinical rheumatologist to support the interpretation and quality evaluation of CPG, as well as to encourage rheumatologists to participate in the elaboration of CPG.

Drafting the scope of the guideline

The development of CPG starts with a clear definition of the scope of the guideline, the roadmap that will guide and delimit all the next steps to be taken [²].

This stage is of crucial importance, from which the authors will define the size of the project and the working team; the target population and the setting or scenario where the guideline will be implemented; and which study designs and outcome measures should be included [²].

It is a complex, expensive and time-consuming process. For better use of resources, it is suggested that the scope of the guideline addresses topics in the following situations: (a) There is a need to update existing guidelines to incorporate new diagnostic technologies or new therapeutic options; (b) The development of a new guideline will result in a potential improvement in the quality of patient care; (c) There is good quality evidence to support the practices and treatments to be recommended by the guideline; (d) The management of a given health condition is surrounded by uncertainties, resulting in significant practical inconsistencies among health professionals; (e) There are strategic areas for the health system or a health institution that require better and consistent care protocols [⁶].

Modifying and adapting a good quality pre-existing guideline rather than building a new one from scratch can be an excellent approach to reduce duplication of effort, enhance efficiency, and improve CPG utilization [⁷, ⁸].

The international collaboration ADAPTE has developed the Resource Toolkit for Guideline Adaptation, a manual that supports the adaptation of CPG to environments other than the original, with a systematic approach and careful contextualization [⁷].

Working groups assignment: the management committee, the elaboration group and the panelist group

The members of the guideline elaboration team must compose a management committee, an elaboration group, and a panelist group. These groups are composed of methodologists, health economists, a systematic review team, clinical experts, and administrative support team. They might work collaboratively themselves, involving the consumer and stakeholders.

The management committee must include members with expertise in systematic review, epidemiology, and public health, and who are familiar with the research subject, as well as members of the institution that commissioned the guideline. It is a committee that oversees all steps throughout the process. It should be the first group to be formed, and it is recommended to include 4–10 people. The management committee’s functions are to coordinate the guideline scope elaboration process; to identify and invite the members to compose the elaboration group; to monitor the development of the guideline; and to review and approve the final version of the guideline [⁹].

The elaboration group must be composed of specialists with experience in the critical evaluation of scientific articles, in evidence-based health and in the performance of systematic reviews. This profile includes epidemiologists or methodologists with knowledge on evidence synthesis for health-related decision making [⁹]. It should include 8–12 people who will perform the following main tasks: to search and critically evaluate the evidence to support the recommendations; to formulate recommendations; to select and train panelists in terms of working methods, and to assess and incorporate suggestions from external reviews. The elaboration group must also have a leader who will participate in the management committee, facilitating interaction among the working group members so that the process is carried out in a collaborative environment [⁹].

The panelist group should be multidisciplinary and include health managers, health professionals, specialists in the guideline theme, economists, and patients, providing the point of view of those to whom the recommendations are directed. It is recommended that members have diversified knowledge to assess the realities of the different geographic regions and their populations. In addition, it is important to consider both the vision of the primary care professional and the one who works in specialized care, as well as the access to technology both by the public system and the complementary system. Ideally, this group should have 6 to 10 members and should assist other members involved in the guideline development and formulation of the recommendations [⁹].

It is important to point out that everyone involved in the guideline elaboration must complete a conflicts of interest (CI) statement before the work begins. Such statements must be available to the entire drafting team. The declaration of financial and non-financial competing interests must be mandatory. Any change in the CI during the guideline development must be communicated to the management committee and shared with the entire team. A member with a substantial CI should not be the leader of the elaboration group or participate in the management committee. A member of the elaboration group or panelist group with potential CI can still participate in the process, if those CI are transparently disclosed. CI must be declared in the appendix of the guideline [⁴].

Research question and the PICO model

The research questions (or key questions) must befit the scope defined by the guideline management committee. The PICO model, an acronym for Population, Intervention, Comparator, and Outcomes, captures the key elements of the research question, enabling the design of a broad and clear research strategy, consistent with CPG objectives [⁹, ¹⁰].

P: Population

It corresponds to the population, problem or health condition. “Who are the relevant population?” “What are the characteristics of the population?” “Are there subgroups that need to be considered?” The most challenging decision when framing the research question is to define the population for which the intervention will be applied [¹⁰, ¹¹]. For example, in addressing the effect of a new immunobiological medication for the treatment of rheumatoid arthritis, one might include only patients who did not respond to other available immunobiologi-cal medications or who did not respond to nonbiologic disease-modifying antirheumatic drugs (DMARDs). The magnitude of effect on key outcomes may be different depending on the population chosen. In that case, the guideline will generate misleading estimates for at least some subpopulations of patients and interventions [¹⁰].

I: Intervention

“Which intervention will be evaluated?” The description of the intervention must mention its availability in the Brazilian Unified Public Healthcare System (SUS), and/ or its coverage by supplementary health, regulated by the National Health Agency [¹²]. Registration and evidence of intervention with the Brazilian regulatory agency, Agência Nacional de Vigilância Sanitária (ANVISA), must be reported. Information from regulatory agencies in other countries is also recommended [¹²].

C: Comparator

“What is the main intervention considered standard to compare with the new intervention under consideration?” The comparator generally refers to the standard care for the condition being studied or placebo. Whenever possible, the comparator should always be the one already available at the Brazilian Unified Public Healthcare System (SUS) or supplementary health care system for the same clinical situation [¹²].

The comparator should be clear and evident, facilitating interpretation of the recommendations proposed by the CPG [¹⁰]. When there are multiple comparators, it should be clear whether all agents are equally recommended or whether there is superiority of one over the other [¹⁰].

O: Outcomes

“What is really important to the patient?” Final (hard) outcomes such as mortality, survival, morbidity, quality of life, treatment complications or adverse effects should be prioritized over surrogate outcomes [¹⁰]. Harms associated with diagnostic tests or treatment strategies, patient-reported outcome and outcomes that evaluates public health impacts are of growing relevance in literature [¹⁰]. It is important to note that the choice of outcomes of interest in a study is subject to cultural and regional influences, and if there is little evidence on an important outcome, this fact should be acknowledged rather than excluding this outcome [¹⁰].

Guideline panels using GRADE will consider the importance of outcomes in three steps. First, a preliminary classification of outcomes must be done before the review of the literature [¹⁰]. Using the GRADE method, the elaboration group and the panelist group must analyze all outcomes chosen for each PICO question and classify them with scores between 1 and 9. On this scale, outcomes with scores of 7–9 are classified as “critical” for decision making; those with scores between 4 and 6 are classified as “important but not critical”; and those classified with scores between 1 and 3 are classified as low importance for decision making. The hierarchy of outcomes will be important in judging the quality of the body of evidence by GRADE.

It is also important to define the type of study design that will be considered to better answer each question. Therefore, the variant called PICOS is also used, where the letter “S” stands for “study design”.

Search, identification and selection of relevant studies

After assembling the PICO question, the next step in developing a guideline is to search the literature for relevant studies that meet the eligibility criteria [¹³].

CPG authors should specify in advance which study designs best answer the research question, what specific characteristics of the population, the intervention and the comparison elements best serve the purpose of the guideline and other important studies characteristics must be observed for their inclusion, such as language and publication situation [¹³].

These eligibility criteria must be broad enough to encompass the great diversity of studies available in the literature but restricted enough so that the studies can be grouped and compared in the data synthesis and analysis stage [¹³].

The search phase is the major foundation that will ground the quality of a guideline. A well-conducted and systematized search strategy ensures that most relevant papers are retrieved, minimizing reporting bias and achieving more reliable estimates of effects and uncertainties [¹³]. It is recommended that authors work closely with a librarian or health information specialist from the beginning of the literature search [¹³].

Databases and other sources

A broad, clear, and reproducible search strategy should be devised, covering the most established health-related databases and also unpublished data sources, ongoing studies and grey literature [⁹]. The more databases that are searched, the more laborious the review will be, so optimizing this choice is critical [¹³, ¹⁴].

MEDLINE and EMBASE are the most used databases worldwide, and there is little overlap in retrieving references on musculoskeletal disorders between them [¹³, ¹⁴]. Other important search sources are The Cochrane Library, the LILACS (a highly recommended Latin-American database), and subject-specific databases such as CINAHL, PsycINFO or PEDro [¹³, ¹⁵].

Trials registers (such as ClinicalTrials.gov), grey literature (reports, dissertations, theses, and conference abstracts) and manual searching of reference lists of included studies must also be part of the search strategy [¹³].

Descriptors, Boolean operators and search filters

A search strategy must contain the appropriate terms and descriptors to find the elements that constitute the PICO question. Databases may be searched using a combination of two retrieval approaches [¹³].

• Text words: search for direct free-text terms occurring in the title or abstract.

• Descriptors: or subject terms used by each database to “officially” label a particular concept.

The use of descriptors aims to increase the number of retrieved studies without substantially increasing the number of non-relevant references [¹⁴]. Each database has its own controlled vocabulary: MeSH (MEDLINE and Cochrane Library), EmTREE (EMBASE) and DeCS (LILACS).

Once all free-text terms and controlled vocabulary terms are chosen, they must be combined by logical Boolean operators: AND, OR and NOT. The AND operator combines terms so that the database retrieves studies that contain all of them. The OR operator matches keywords so that the database retrieves studies that contain one or all of them in the search results. When using the OR operator, all keywords must be enclosed in parentheses. The NOT operator is used to exclude keywords from search results and should be avoided whenever possible because of the risk of inadvertently removing relevant records from the search set [¹³].

It is also possible to use search filters to further delimit the studies retrieved by language, age group, study design and others. Search filters in checkbox format are available on databases websites but it is highly recommended to use filters validated and tested by groups of experts such as Cochrane and the InterTASC Information Specialists Group [¹⁶].

Tools such as the Peer Review of Electronic Research Strategies (PRESS) checklist can help with the steps of designing the search strategy, which must be reported in sufficient detail so that it can be reproduceable [¹³, ¹⁷].

Study selection

The study selection process must be carried out by two independent reviewers, and disagreements may require a third person arbitration. Studies retrieved from all sources and databases are merged, duplicates are removed, and researchers review titles and abstracts and then full texts, selecting studies that meet the eligibility criteria for inclusion. This process must be documented step by step using the PRISMA flow diagram [¹³, ¹⁸].

Several tools and software such as Rayyan QCRI, DistillerSR and Covidence have been developed to facilitate the process of study selection [¹³, ¹⁹–²¹].

Assessment of the quality of the body of evidence

Assessment of the quality of the included studies should be done by two independent reviewers, and disagreements may require a third person arbitration or resolved by consensus. There are tools specially developed for each study design, for example AMSTAR for systematic reviews; QUADAS-2 for accuracy studies; Newcastle– Ottawa tool for case control or cohort studies; ROBINS-I for non-randomized studies of interventions and RoB 2 for clinical trials [¹³, ²²–²⁶].

The GRADE assessment

GRADE tool allows authors to assess the quality of evidence and the strength of recommendations in a given study [²⁷, ²⁸]. It provides guidance in the process of determining the outcomes of interest, summarizing the evidence, and formulating a recommendation in an outcome-focused manner. The GRADE classification is made for each outcome and quality may differ from one outcome to another [²⁷, ²⁸]. Furthermore, GRADE classification makes it possible to assess the degree of recommendation of a certain conduct in clinical practice, as well as for decision-making in public and private health policies.

The GRADE tool rates the quality of evidence in one of four levels—high, moderate, low, and very low [²⁷, ²⁹, ³⁰]. The quality of an evidence may be compromised by imprecision (very large confidence intervals), inconsistency (or heterogeneity), the indirect nature of the evidence (correspondence to PICO and applicability), and publication bias [²⁷]. Likewise, there are factors that can increase confidence in effect estimates: when there is a large magnitude of effect, when plausible residual confounders and biases would reduce the demonstrated effect or increase the effect if no effect was observed, or when there is an evidence of a dose–response gradient [²⁷].

Direction and strength of the recommendations

The GRADE approach rates a recommendation as strong or weak. To determine the strength of recommendation, it is necessary to critically evaluate the desirable and undesirable effects of an alternative strategy, analyze the quality of the primary studies, the preferences of the scenario where the new strategy will be applied and the rational use of resources [³¹–³³].

The GRADE evidence profile tables should be used to inform the final decision on the quality of the body of evidence for each outcome and to make the quality rating explicit and reproducible [³³].

Achieving consensus: the Delphi model

Some health issues are under-explored due to ethical or logistical difficulties, generating low-quality, insufficient, or contradictory evidence. Writing a guideline on these topics represents a major challenge, since it is not possible to gather such evidence in a body and analyze it through objective techniques such as GRADE [³⁴, ³⁵].

In these scenarios, the development of recommendations can be based on the clinical experience of highly qualified professionals using The Delphi Method, a structured technique, which anonymously and systematically collects the opinions of panelists, generating a reliable consensus with statistical value [³⁵].

The Delphi process is based on the application of interactive questionnaires to a panel of professionals, for several rounds until the divergence between opinions has been reduced to a satisfactory level [³⁶]. The possibility of application in virtual format allows the participation of specialists from different regions of the globe and reduces costs. The anonymity of responses mitigates the influence of the most prominent panelists, allowing a more homogeneous and proportional participation of all professionals [³⁷–³⁹]. Other important advantages of the method are providing the participants with feedback for their contributions, the possibility of reviewing the experts’ answers and the formation of heterogeneous groups, with different clinical experiences [³⁷–³⁹].

Limitations of the Delphi model includes the possibility of bias due to personal interests or preconceived opinions of both investigators leading the process or participants [³⁴, ⁴⁰].

The sequence for developing the Delphi model is succinctly described in the Table 1.

Table 1
Checklist for applying the Delphi model [³⁶]

Planning	To define the researcher
Execution	To develop the preliminary evaluation instrument and define the consultation objectives
	To select the panelists
	To perform a survey of the evidence in the literature on the chosen theme
	To define what is established as consensus and threshold values
	First round: Start with an open-ended questionnaire. The researcher structures the answers for the second round. When basic information is structured and available, an already structured questionnaire can be used in the first round (modified Delphi technique)
	Process and evaluate first-round results through statistical measures. Provide to the participants statistical feedback from the group
	Subsequent rounds: each participant receives a questionnaire with the items from the previous round summarized by the researcher.
	Participants can review their opinions or justify them. Rounds can be stopped when the consensus level is reached

Writing and reporting a guideline

After gathering and analyzing the evidence of each PICO question, the guideline full text must be prepared, describing minutely all the steps adopted to reach the conclusion and pertinent recommendations. Summarized versions of the guideline should be made for specific audiences [⁴¹].

Submitting a guideline draft for external review or public consultation may be recommended. The management committee evaluates the revised text and ultimate adjustments can be made by the elaboration group prior to publication of the guideline final version.

Dissemination and implementation of a guideline

To incorporate the recommendations of a guideline into clinical practice it is essential to propagate the study results to a variety of healthcare professionals and patients. Special importance is given to the Cochrane EPOC group, which provides active strategies and other arrangements for implementing health guidelines recommendations [⁴²].

Updating a guideline

Since knowledge is dynamic and is constantly evolving, efforts to keep CPG updated are necessary to maintain the validity of recommendations. Methods of CPG development have progressed substantially in the past 20 years, but guidance for updating CPG remains heterogeneous and poorly described in methodological handbooks [⁴³]. Therefore, there is no consensus about how frequently CPG should be updated, although most authors recommend assessing the need for an update every three to five years [⁴³].

Table 2 summarizes a systematic approach for deciding whether CPG might need updating and a simplified updating process framework [⁴³, ⁴⁴].

Table 2
Basic steps for the updating process of clinical practice guideline [⁴³, ⁴⁴]

Define the multidisciplinary group responsible for updating the specific CPG Identify new relevant evidence: interval may vary from 6 months to 5 years Assess if there was any change in the following set of circumstances: Evidence on the existing benefits and harms of interventions Outcomes considered important Available interventions Evidence that current practice is optimal Values placed on outcomes Resources available for health care In case of a positive answer to any of the above items from step 3, proceed to step 4 Updating process: systematic literature search, evidence selection, evidence synthesis, evidence assessment External review Publication and dissemination

Quality assessment of guidelines

A high-quality guideline should ensure that potential biases are adequately addressed and that the resulting recommendations are feasible, with internal and external validity.

Currently, the main tools available for the reporting of guidelines development are the Appraisal of Guidelines for Research and Evaluation II (AGREE II) instrument and the Reporting Items of Practice Guidelines in Healthcare (RIGHT) statement [⁴⁵, ⁴⁶].

The AGREE II instrument consists of 23 items comprising 6 quality domains and has 3 main objectives: to provide a methodological strategy for the elaboration of a clinical guideline, to guide how it should be adequately reported and to assess its quality [⁴⁵, ⁴⁷]. Details on how to calculate these scores as well as translated versions are available on the AGREE Enterprise website [⁴⁸].

The RIGHT statement includes 22 items considered essential for good reporting of CPG [⁴⁷, ⁴⁹]. Both AGREE II and RIGHT may be used interchangeably to improve the comprehensiveness, completeness, and transparency of CPG divulgence [⁴⁶]. Their main features are summarized on Table 3.

Table 3
RIGHT Statement and AGREE Reporting Checklist domains and essential items [⁴⁷, ⁴⁹]

RIGHT Statement (2017)	AGREE reporting Checklist (2016)
Basic information	Domain 1. Scope and Purpose
1. Title/subtitle	1. Objectives
2. Executive summary	2. Questions
3. Abbreviations and acronyms	3. Population
4. Corresponding developer	Domain 2. Stakeholder Involvement
Background	4. Group membership
5. Description ofthe health problem(s)	5. Target population preferences and views
6. Aim(s) of the guideline and objectives	6. Target users
7. Target population(s)	Domain 3. Rigour of Development
8. End users and settings	7. Search methods
9. Guideline development groups	8. Evidence selection criteria
Evidence	9. Strengths and limitations of the evidence
10. Health care questions	10. Formulation of recommendations
11. Systematic reviews	11. Consideration of benefits and harms
12. Assessment of the certainty of the body of evidence	12. Link between recommendations and evidence
Recommendations	13. External review
13. Recommendations	14. Updating procedure
14. Rationale for recommendations	Domain 4. Clarity of Presentation
15. Evidence to decision processes	15. Specific and unambiguous recommendations
Review and quality assurance	16. Management options
16. External review	17. Identifiable key recommendations
17. Quality assurance	Domain 5. Applicability
Funding and declaration of interests	18. Facilitators and barriers
18. Funding source(s) and role(s) of the funder	19. Implementation advice/tools
19. Declaration and management of interests	20. Resource implications
Other information	21. Monitoring/ auditing criteria
20. Access	Domain 6. Editorial Independence
21. Suggestions for further research	22. Funding body
22. Limitations ofthe guideline	23. Competing interests

Another instrument of great relevance in reiterating the quality of CPG is the GIN-McMaster Guideline Development Checklist, which consists of 18 topics addressing all stages of a guideline elaboration, from planning to implementation and evaluation [⁵⁰]. The purpose of this instrument is not to replace the AGREE II, but to ensure that, following the steps described in the checklist, key items are covered and higher scores are achieved in the credibility assessment tools [⁵¹].

Conclusions

Guidelines provide support for decision-making in clinical practice. Guideline development is based on the selection and synthesis of the best available evidence, using a systematic and transparent approach in the judgment of the quality of evidence and strength of recommendations. The methodological guide proposed in this review could be used as an important tool for creating, elaborating, and updating CPG and consensus in rheumatology. Figure 1 presents a roadmap listing the steps described in this paper for developing a guideline.

Fig. 1
Brazilian Society of Rheumatology methodological guide for the development of evidence-based CPGs in rheumatology roadmap

• Funding
  Not applicable.
• Declarations
  Ethics approval and consent to participate
  Not applicable.
• Consent for publication
  Not applicable.
• Publisher’s Note
  Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Availability of data and materials

Not applicable.

Acknowledgements

Not applicable.

Abbreviations

• AGREE  Appraisal of Guidelines, REsearch and Evaluation
• AMSTAR  Assessment of multiple systematic reviews tool
• ANVISA  Agência Nacional de Vigilância Sanitária
• CGP  Clinical practice guideline
• CI  Conflicts of interest
• CINAHL  Cumulative Index to Nursing and Allied Health Literature
• DeCS  Descritores em Ciências da Saúde
• EMBASE  Excerpta Medica dataBASE
• EmTREE  Embase subject headings
• EPOC  Cochrane Effective Practice and Organisation of Care
• GIN  Guidelines International Network
• GRADE  Grading of Recommendations Assessment, Development, and Evaluation
• InterTASC  Collaborative group for “Technology Appraisal Support Collaboration”
• MEDLINE  The National Library of Medicine journal citation database
• MeSH  Medical subject headings
• PEDro  The Physiotherapy Evidence Database
• PICO  Population, intervention, comparator and outcomes
• PRESS  Peer Review of Electronic Research Strategies
• PRISMA  Preferred Reporting Items for Systematic reviews and Meta-Analyses
• PRO  Patient-reported outcome
• PsycINFO  American Psychological Association’s database
• QUADAS-2  Quality Assessment of Diagnostic Accuracy Studies RIGHT Reporting Items for practice Guidelines in HealThcare RoB 2 Risk-of-bias tool for randomized trials
• ROBINS-I  Risk Of Bias In Non-randomised Studies of Interventions SUS Brazilian Unified Public Healthcare System

References

• ¹
  The American College of Rheumatology Clinical Practice Guidelines [Internet]. [cited 2022 Aug 30]. Available from: https://www.rheumatology.org/Practice-Quality/Clinical-Support/Clinical-Practice-Guidelines
  » https://www.rheumatology.org/Practice-Quality/Clinical-Support/Clinical-Practice-Guidelines
• ²
  Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P. Guidelines international network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012;156(7):525-31.
• ³
  Neumann A, Vandvik P, Agoritsas T, Alonso-Coello P, Rind D, Santesso N, et al. How to use a patient management recommendation: clinical practice guidelines and decision analyses. In: Guyatt G, Rennie D, Meade M, Cook D, editors., et al., Users’ guides to the medical literature: a manual for evidence-based clinical practice. 3rd ed. McGraw-Hill Education; 2015.
• ⁴
  World Health Organization. WHO handbook for guideline development. 2nd ed. Geneva: World Health Organization; 2014.
• ⁵
  Homik JE, Suarez-Almazor ME. Clinical guidelines: a must for rheumatology? Best Pract Res Clin Rheumatol. 2000;14(4):649-61.
• ⁶
  Ferreira CBT. Diretrizes Metodológicas: Elaboração de Diretrizes Clínicas. Revista Brasileira de Cancerologia. 2016;62(2):165-6.
• ⁷
  The ADAPTE Collaboration. [Internet]. [cited 2022 Aug 30]. Available from: https://g-i-n.net/
  » https://g-i-n.net/
• ⁸
  Fervers B, Burgers J, Voellinger R, Brouwers M, Browman G, Graham I, et al. Guideline adaptation: an approach to enhance efficiency in guideline development and improve utilisation. BMJ Qual Saf. 2011;20(3):228-36.
• ⁹
  Diretrizes Metodológicas: Elaboração de Revisão Sistemática e Meta-análise de Ensaios Clínicos Randomizados. Brasília, DF: Ministério da Saúde; 2021.
• ¹⁰
  Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, et al. GRADE guidelines: 2: framing the question and deciding on important outcomes. J Clin Epidemiol. 2011;64(4):395-400.
• ¹¹
  Guyatt G, Meade M, Agoritsas T, Richardson W, Jaeschke R. What is the question? In: Guyatt G, Rennie D, Meade M, Cook D, editors. Users’guides to the medical literature: essentials of evidence-based clinical practice. 3rd ed. McGraw-Hill Education; 2015.
• ¹²
  Diretrizes Metodológicas: Elaboração de Pareceres Técnico-Científico. 4th ed. Brasília, DF: Ministério da Saúde; 2014.
• ¹³
  Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, et al., editors. Cochrane handbook for systematic reviews of interventions. version 6.3. Cochrane; 2022.
• ¹⁴
  Suarez-Almazor ME, Belseck E, Homik J, Dorgan M, Ramos-Remus C. Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough. Control Clin Trials. 2000;21(5):476-87.
• ¹⁵
  Clark OAC, Castro AA. Searching the Literatura Latino Americana e do Caribe em Ciências da Saúde (LILACS) database improves systematic reviews. Int J Epidemiol. 2002;31(1):112-4.
• ¹⁶
  ISSG Search Filters Resource [Internet]. [cited 2022 Aug 26]. Available from: https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home
  » https://sites.google.com/a/york.ac.uk/issg-search-filters-resource/home
• ¹⁷
  McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;1(75):40-6.
• ¹⁸
  Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;29:372.
• ¹⁹
  Covidence: Better systematic review management [Internet]. [cited 2022 Aug 26]. Available from: https://www.covidence.org/
  » https://www.covidence.org/
• ²⁰
  DistillerSR|Systematic Review Software|Literature Review Software [Internet]. [cited 2022 Aug 26]. Available from: https://www.evidencepartners.com/products/distillersr-systematic-review-software
  » https://www.evidencepartners.com/products/distillersr-systematic-review-software
• ²¹
  Rayyan: Intelligent Systematic Review: Rayyan [Internet]. [cited 2022 Aug 26]. Available from: https://www.rayyan.ai/
  » https://www.rayyan.ai/
• ²²
  Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;21:358.
• ²³
  Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529-36.
• ²⁴
  The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses [Internet]. [cited 2022 Aug 26]. Available from: https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp
  » https://www.ohri.ca/programs/clinical_epidemiology/oxford.asp
• ²⁵
  RoB 2: A revised Cochrane risk-of-bias tool for randomized trials | Cochrane Bias [Internet]. [cited 2022 Aug 26]. Available from: https://methods.cochrane.org/bias/resources/rob-2-revised-cochrane-risk-biastool-randomized-trials
  » https://methods.cochrane.org/bias/resources/rob-2-revised-cochrane-risk-biastool-randomized-trials
• ²⁶
  Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;12:i4919.
• ²⁷
  The GRADE Working Group. GRADE handbook for grading quality of evidence and strength of recommendations. [Internet]. Updated Oct 2013. Schünemann H, Brożek J, Guyatt G, Oxman A, editors. Available from guidelinedevelopment.org/handbook; 2013 [cited 2022 Dec 14]. Available from: https://gdt.gradepro.org/app/handbook/handbook.html
  » guidelinedevelopment.org/handbook» https://gdt.gradepro.org/app/handbook/handbook.html
• ²⁸
  Guyatt G, Oxman AD, Sultan S, Brozek J, Glasziou P, Alonso-Coello P, et al. GRADE guidelines: 11: making an overall rating of confidence in effect estimates for a single outcome and for all outcomes. J Clin Epidemiol. 2013;66(2):151-7.
• ²⁹
  Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, et al. GRADE guidelines: 3: rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401-6.
• ³⁰
  Andrews JC, Schünemann HJ, Oxman AD, Pottie K, Meerpohl JJ, Coello PA, et al. GRADE guidelines: 15: going from evidence to recommendationddeterminants of a recommendation’s direction and strength. J Clin Epidemiol. 2013;66:726-35.
• ³¹
  de Abreu MM, Levy RA, Wahl D. Using the GRADE system in diagnostic strategy: the case of antiphospholipid syndrome. Lupus. 2014;23(12):1292-4.
• ³²
  Abreu MM, Danowski A, Wahl DG, Amigo MC, Tektonidou M, Pacheco MS, et al. The relevance of “non-criteria” clinical manifestations of antiphospholipid syndrome: 14th international congress on antiphospholipid antibodies technical task force report on antiphospholipid syndrome clinical features. Autoimmun Rev. 2015;14(5):401-14.
• ³³
  Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, et al. Rating quality of evidence and strength of recommendations: going from evidence to recommendations. Br Med J. 2008;336(7652):1049.
• ³⁴
  Mahajan V, Linstone HA, Turoff M. The Delphi method: techniques and applications. J Mark Res. 1976;13(3):317.
• ³⁵
  Nasa P, Jain R, Juneja D. Delphi methodology in healthcare research: How to decide its appropriateness. World J Methodol. 2021;11(4):116-29.
• ³⁶
  Rocha-Filho CR, Cardoso TC, Dewulf N de LS. Método e-delphi modificado: um guia para validação de instrumentos avaliativos na área da saúde. Brazil Publishing; 2019.
• ³⁷
  Munaretto LF, Corrêa HL, da Cunha JAC. A study on the characteristics of the Delphi method and focus group as techniques to obtain data in exploratory research. Revista de Administração da UFSM. 2013;6(1):9-24.
• ³⁸
  Massaroli A, Martini JG, Lino MM, Spenassato D, Massaroli R. Método DELPHI como referencial metodológico para a pesquisa em enfermagem. Texto Contexto Enfermagem. 2018;26(4):1110017.
• ³⁹
  von der Gracht HA. Consensus measurement in Delphi studies: review and implications for future quality assurance. Technol Forecast Soc Change. 2012;79(8):1525-36.
• ⁴⁰
  Green B, Jones M, Hughes D, Williams A. Applying the Delphi technique in a study of GPs’ information requirements. Health Soc Care Commun. 1999;7(3):198-205.
• ⁴¹
  Diretrizes Metodológicas: Elaboração de Diretrizes Clínicas. 2nd ed. Brasília, DF: Ministério da Saúde; 2020.
• ⁴²
  Cochrane effective practice and organisation of Care [Internet]. [cited 2022 Aug 23]. Available from: https://epoc.cochrane.org/
  » https://epoc.cochrane.org/
• ⁴³
  Vernooij RWM, Sanabria AJ, Solà I, Alonso-Coello P, Martínez GL. Guidance for updating clinical practice guidelines: a systematic review of methodological handbooks. Implement Sci. 2014;9(1):1-9.
• ⁴⁴
  Shekelle P, Woolf S, Grimshaw JM, Schünemann HJ, Eccles MP. Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues ofenhancing guideline implementability and accounting for comorbid conditions in guideline development. Implement Sci. 2012;7(1):1-7.
• ⁴⁵
  Cluzeau F, Burgers J, Brouwers M, Grol R, Mäkelä M, Littlejohns P, et al. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care. 2003;12(1):18-23.
• ⁴⁶
  Tokalić R, Viđak M, Buljan I, Marušić A. Reporting of clinical practice guidelines: practical testing of AGREE and RIGHT checklists. J Gen Intern Med. 2020;35(7):2167.
• ⁴⁷
  Brouwers MC, Kerkvliet K, Spithof K. The AGREE Reporting Checklist: a tool to improve reporting of clinical practice guidelines. BMJ. 2016;8:352.
• ⁴⁸
  AGREE II: AGREE Enterprise website [Internet]. [cited 2022 Aug 26]. Available from: https://www.agreetrust.org/resource-centre/agree-ii/
  » https://www.agreetrust.org/resource-centre/agree-ii/
• ⁴⁹
  Chen Y, Yang K, Marušić A, Qaseem A, Meerpohl JJ, Flottorp S, et al. A reporting tool for practice guidelines in health care: the RIGHT statement. Ann Intern Med. 2017;166(2):128-32.
• ⁵⁰
  Schünemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, et al. Guidelines 20: systematic development of a comprehensive checklist for a successful guideline enterprise. Can Med Assoc J. 2014;186(3):E123.
• ⁵¹
  GIN-McMaster Guideline Development Checklist [Internet]. [cited 2022 Aug 30]. Available from: https://cebgrade.mcmaster.ca/guidecheck.html
  » https://cebgrade.mcmaster.ca/guidecheck.html

Publication Dates

• Publication in this collection
  01 Sept 2023
• Date of issue
  2023

History

• Received
  01 Sept 2022
• Accepted
  05 Mar 2023
• Published
  15 Mar 2023