Protocol for pelvic organ prolapse treatment with vaginal pessaries

Abstract

Objective

To develop a clinical protocol for the conservative treatment of pelvic organ prolapse with vaginal pessaries.

Methods

Developmental research conducted in the period from July 2015 to January 2016 and performed in the following steps: refinement of topics/protocol issues; establishing recommendations for research and updates; peer review. The analysis was by statistical program and the Content Validity Index (CVI).

Results

The protocol was developed and evaluated by professionals of the area through the Delphi technique regarding criteria of objectives, content and presentation, and relevance. The total CVI of each domain and the overall CVI were calculated. The total Content Validity Index for the objectives domain was 1.00, for content and presentation criterion was 0.98, and for the relevance domain was 0.96. The overall Content Validity Index obtained was 0.98. Thus, there was agreement among participants of the Delphi technique with value above 0.85, and the clinical protocol was considered valid.

Conclusion

When health professionals use the clinical protocol, they will have a better foundation in practice and offer a higher quality care, since this is a valid and scientifically based tool.

Pelvic floor disorders; Pelvic organ prolapse; therapy Pessaries; Protocols; Validation studies

Resumo

Objetivo

Desenvolver um protocolo clínico para o tratamento conservador do prolapso de órgãos pélvicos com pessário vaginal.

Métodos

Pesquisa de desenvolvimento ocorrida de julho de 2015 a janeiro de 2016 e realizada em etapas: refinamento dos tópicos/questões do protocolo; estabelecimento de recomendações para pesquisa e atualização; revisão por pares. A análise se deu por programa estatístico e pelo Índice de Validade de Conteúdo.

Resultados

O protocolo foi desenvolvido e avaliado por meio da técnica Delphi quanto aos critérios objetivos, conteúdo e apresentação e relevância por profissionais da área, sendo calculado o Índice de Validade de Conteúdo total de cada domínio e global. O Índice de Validade de Conteúdo total do domínio objetivos foi 1,00, do critério conteúdo e apresentação foi 0,98 e do domínio relevância, 0,96. Obteve-se o Índice de Validade de Conteúdo global de 0,98. Dessa forma, verificou-se concordância entre os participantes da técnica Delphi, com valor acima de 0,85, considerando o protocolo clínico válido.

Conclusão

Acredita-se que os profissionais de saúde, ao utilizar o protocolo clínico, terão maior embasamento na prática, oferecendo um cuidado de maior qualidade, pois é uma ferramenta válida e pautada cientificamente.

Distúrbios do assoalho pélvico; Prolapso de órgão pélvico/terapia; Pessários; Protocolos; Estudos de validação

Resumen

Objetivo

Desarrollar un protocolo clínico para el tratamiento conservador del prolapso de órganos pélvicos con pesario vaginal.

Métodos

Investigación de desarrollo realizada entre julio de 2015 y enero de 2016, efectuada en etapas: refinación de tópicos/preguntas del protocolo; establecimiento de recomendaciones para investigación y actualización; revisión por pares. Análisis ejecutado mediante programa estadístico e Índice de Validez de Contenido.

Resultados

El protocolo fue desarrollado y evaluado utilizándose la técnica Delphi respecto a los criterios objetivos, contenido y presentación, y relevancia por profesionales del área, calculándose el Índice de Validez de Contenido total de cada dominio y el global. El Índice de Validez de Contenido total del dominio objetivos fue 1,00; el del criterio contenido y presentación, del 0,98; y el del dominio relevancia, 0,96. El Índice de Validez de Contenido global fue de 0,98. Así, se verificó concordancia de la técnica Delphi entre los participantes, con valor superior a 0,85; considerándose válido el protocolo clínico.

Conclusión

Al ser utilizado por los profesionales de salud, el protocolo clínico les brindará mayor fundamentación en la práctica, permitiéndoles ofrecer mejor calidad de atención, pues es una herramienta válida y elaborada científicamente.

Trastornos del suelo pélvico; Prolapso de órgano pélvico/terapía; Pesarios; Protocolos; Estudios de validación

Introduction

The prevalence of pelvic organ prolapse (POP) varies widely worldwide, as it depends on the female population studied and the criteria for entry, and is estimated at 30% to 50% of multiparous women over 50 years of age.⁽¹⁾

Data from a specialized outpatient clinic in Fortaleza, Ceará, revealed that out of 85 patients with pelvic floor dysfunction (PFD), 58.8% had stage II prolapse, 14.1% stage III and 2.4% stage IV, and more than half of patients presented anterior wall vaginal defects (55.3%).⁽²⁾

The surgical approach is the most frequent type of treatment, but conservative treatments, such as the use of vaginal pessaries have been gaining prominence.^(3,4)

Pessaries are recommended as a first-line, low-cost and low-risk treatment option, and indicated for a variety of signs and symptoms related to prolapse. Its use is a viable and effective option, since long-term users (over 12 months) have reported high levels of satisfaction and control of the condition with the device.⁽⁵⁾

Professionals assisting women who use pessaries should be attentive to some care practices in order to avoid possible complications. Although in national studies there are no publications on nurses’ role in the conservative treatment of POP, internationally, this has been reported for some time.^(6-9) The importance of these professionals’ assistance has also been mentioned for the success in the use of pessaries.⁽⁶⁾

The existence of a clinical protocol can contribute to the guidance and standardization of insertion and follow-up visits, promotion of specific care, early detection of alterations, minimization of complications, improvement of quality of life and enable the targeting and standardization of actions.⁽¹⁰⁾

Protocols are guidelines for practice, and their adoption is supported by the Ministry of Health, which provides for a greater appropriation of the health problem to which it reports by allowing that professionals’ actions have technical and scientific support and they feel more self-confident in their practices.⁽¹¹⁾

Since there are no clear guidelines for pessary management so far,⁽¹²⁾ was developed a clinical protocol for the conservative treatment of pelvic organ prolapse with vaginal pessaries. The existence of a valid and reliable technology such as the clinical protocol is relevant and will be beneficial so that professionals working in the area can improve care for women with POP through the organization of service and assistance.

Methods

In order to conduct research for the development of potentially applicable and useful technologies in the existing teaching-learning methods, developmental research was chosen for the present study.⁽¹³⁾

The development of the protocol occurred from July 2015 to January 2016, and was based on the steps of Ribeiro (2010),⁽¹⁴⁾ who sought to reach consensus for the protocol development process in order to guarantee its quality. However, the ideas of Werneck, Faria and Campos (2009)⁽¹⁵⁾ for developing clinical protocols were also used.

The steps were divided into: 1) refining topics/issues; 2) conducting review of the scientific literature; 3) establishing recommendations for research and updating of the guideline/protocol; 4) ensuring peer review.

In the first step, were defined the objective, justification and topics contained in the protocol. In the period between July and October 2015, the Electronic Brainstorming technique was used for defining these topics with five invited health professionals, who research and/or act on the guidance of the use of vaginal pessaries through an online platform. The Urogynecology outpatient clinic of a tertiary hospital in Fortaleza-CE, a reference in the care of women with PFD, was also visited in July 2015. An interview was conducted with some vaginal pessary users in order to find the main doubts about the use of the device, and the needs to be satisfied in the consultation. The researcher’s experience on the subject was also taken into consideration through insertion in the service.

The second step involved following procedures in order to perform an integrative review⁽¹⁶⁾ in the month of November, 2015 through journals indexed in the following computerized databases: US National Library of Medicine (PUBMED)/Medical Literature Analysis and Retrieval System Online (MEDLINE), Latin American and Caribbean Literature in Health Sciences (LILACS), SCOPUS, COCHRANE, Center for Reviews and Dissemination (CRD), WEB OF SCIENCE, Brazilian Network of Health Technology Assessment (REBRATS) and published books. From the reading of these materials, were selected contents that served as support for the protocol development.

Note that during the protocol development, preference was given to the use of studies with higher levels of evidence (LE) and grade of recommendations (GR), according to the Oxford Centre for Evidence-Based Medicine classification.⁽¹⁷⁾

After the integrative review guided by group suggestions that arose in the Brainstorming session, in December 2015, the protocol was developed in text and represented in graphical form as flowcharts with algorithms when necessary, which comprised the third step of the process. For the selection of illustrations and layout, NBR 14.724 and 6.029 norms were followed. In the protocol content, the references used for its development, the databases investigated, and the classification of LE and GR adopted were made available in order to facilitate its future update.

The last step was obtaining feedback on the proposed protocol by ensuring an opportunity to review the document and identify potential difficulties for its implementation. In January 2016, the Delphi technique was used to this end with collaboration of seven health professionals who research and/or act in the Urogynecology area and/or in the orientation of the use of vaginal pessaries. The protocol was evaluated according to criteria of objectives, content and presentation, and relevance. Values from 1 to 4 were assigned, in which 1 is inappropriate; 2 is partially appropriate; 3 is appropriate; 4 is totally appropriate, and suggestions were made when deemed necessary.

The analysis of agreement among participants of the Delphi technique was through the Content Validity Index (CVI). As the number of professionals in this technique was greater than six and the protocol is new, the minimum value assigned to agreement was 0.85.⁽¹⁸⁾

The study was submitted in accordance with Resolution number 466/12 of the National Health Council to the Research Ethics Committee (COMEPE) through the Brazil Platform for due consideration, and was approved under opinion number 1.116.853.⁽¹⁹⁾

Results

At the step of refining topics/issues of the clinical protocol, were developed the objective of the clinical protocol, the justification for the development of the protocol and the topics of mandatory contents. These topics and subtopics were built through electronic brainstorming.

As for the profile of health professionals who participated in the electronic brainstorming, four are female and one is male. Three are nurses, one is a physiotherapist and the other is a doctor. One has a PhD degree, two have masters’ degree and two are specialists. All have knowledge/experience in the ‘vaginal pessary’ subject, present published works and participate in research groups in the area of Urogynecology.

In this same step, at another moment, were interviewed four vaginal pessary users aged between 57 and 74 years, prolapse stage ranging from III to IV, and who have been using the device for at least three years. The ideas and opinions of these women helped in the extraction of topics for the protocol development.

In the integrative review step, the delimitation of the guiding topics of the review was achieved through the electronic brainstorming, interview with pessary users and the researcher’s experience, as explained in the previous step.

Inclusion criteria for selection of articles were the following: to cover the guiding topics of the protocol theme, available electronically and published in English, Portuguese and Spanish. Repeated and unavailable articles were excluded. There was no restriction of publication year.

In the LILACS, CRD and REBRATS databases, was used the descriptor “pelvic organ prolapse” alone and the association of “pessaries” and “pessary” with the Boolean operator OR. In the PUBMED/MEDLINE, SCOPUS, COCHRANE and WEB OF SCIENCE databases, was used a unique combination of “pelvic organ prolapse” with the Boolean operator AND added to “pessaries” and “pessary” with the Boolean operator OR.

As a result, 3,630 articles were retrieved. After selection of articles based on inclusion and exclusion criteria, the search was completed with 44 articles for analysis, synthesis and inclusion in the protocol. Publications were dated between years 2002 to 2015, 39 were from international journals in English, and the LE ranged from 1A to 5 and the GR from A to D.

In the third step, was developed the clinical protocol itself. The text development process was judicious with the aim to facilitate the reading and management of the protocol during its use, and happened in a logical sequence. Throughout the protocol, LEs and GRs were made available so that readers were aware of the type of information selected in integrative review studies.

The final version of the protocol had 71 pages. The content was divided into 13 sessions with their sub-sessions related to the construction process, general considerations on the subject and the complete assistance for treatment with vaginal pessaries. Eleven tables were inserted; two indicating the LE and GR of the protocol, seven identifying each type of pessary with indications, advantages and disadvantages, one about the self-care options for women using the device, and one with possible complications in the use and management of the device. Fourteen figures were included too, of which three are flowcharts. Figure 1 shows some pages of the protocol.

Figure 1
Some pages of the Clinical Protocol for Use of Vaginal Pessaries

In the last step, the Delphi technique was used for the clinical protocol review. Regarding the characteristics of health professionals who participated in the Delphi technique, five are female and two are male. Four are nurses, three are stomatherapists, two are physiotherapists, and one is a doctor. Four have masters’ degrees and three are specialists. Two judges resided outside the state; one is from Curitiba and the other is from São Paulo. All have experience in PFD and knowledge on the ‘vaginal pessary’ subject, have published works and participate in research groups in the area of Urogynecology.

Table 1 shows total values of CVI according to the clinical protocol evaluation criteria.

Table 1
Values of Total Content Validity Index and Overall Content Validity Index according to the clinical protocol evaluation criteria

Objectives criterion	CVI*	Content and presentation criterion	CVI*	Relevance criterion	CVI*
Item 1	1.00	Item 1	1.00	Item 1	1.00
Item 2	1.00	Item 2	1.00	Item 2	1.00
Item 3	1.00	Item 3	1.00	Item 3	0.85
Item 4	1.00	Item 4	1.00	Item 4	1.00
-	-	Item 5	1.00	-	-
-	-	Item 6	1.00	-	-
-	-	Item 7	1.00	-	-
-	-	Item 8	1.00	-	-
-	-	Item 9	1.00	-	-
-	-	Item 10	0.85	-	-
Total CVI	1.00		0.98		0.96
Overall CVI	0.98

• *CVI – Content Validity Index

The ‘objectives’ criterion refers to the purposes, goals or ends to be achieved with use of the protocol. In this regard, the evaluators rated the clinical protocol as 3 – Appropriate, and 4 - Totally appropriate, and the total CVI of the domain was 1.00.

The ‘content and presentation’ criterion refers to the form of presentation of the guidelines. This includes general organization, structure, presentation strategy, consistency, and formatting. The content and presentation evaluated were rated by most professionals as 3 - Appropriate and 4 - Totally appropriate. However, an evaluator rated the item referring to the number of pages as 2 - Partially appropriate. The total CVI for this criterion was 0.98.

The ‘relevance’ criterion, in turn, refers to the characteristic evaluating the significance level of the presented protocol. Reporting to this criterion, the evaluators attributed 3 - Appropriate and 4 - Totally appropriate, and the total CVI of this domain was 0.96. The overall CVI was 0.98. Thus, there was agreement among participants of the Delphi technique in the items, domains and in a global way with a value of 0.85 or more, hence the review step was considered appropriate.

Professionals have also made some suggestions for improving the quality of the protocol before the final evaluation. These suggestions were analyzed and adopted, as shown in chart 1.

Chart 1
Some changes made in the protocol from the evaluators’ suggestions

Evaluators’ suggestions	Changes made
Reformulation of protocol considerations	The target audience of the protocol was defined as health professionals working in the area of PFD rather than in Urogynecology.
Insertion of figure	The figure of the pessary was inserted next to the explanation of each specific type
Replacement of term in English	The name of the pessary type ‘Donut’ was changed to ‘Rosca’
Insertion of table	A table was inserted with the main intercurrences in the use of pessaries and their management
Detailing/Reformulation of content	POP classifications were better detailed (POP-q); The steps for selecting and adjusting the pessary were rearranged; One more item was included in the steps for insertion of the pessary; Inclusion of the possibility of the stomatherapist nurse acting together with the physiotherapist in the conservative treatment with training of the pelvic floor musculature

• *PFD – Pelvic Floor Dysfunction; *POP - Pelvic Organ Prolapse; *POP-q - Pelvic Organ Prolapse Quantification System

Discussion

The development of technologies aimed at improving health professionals’ technical-scientific knowledge should be encouraged, because they arouse interest in the search for updated information, and innovate the care practice.

Considering the relevance of the topic and the fact that this area is still little known within Nursing, the development and validation of this protocol may expose this practice and encourage nurses to enter this type of service, since these professionals can manage the conservative treatment with pessary effectively. In addition, it might increase indicators of quality of care, and serve as a guide for evidence-based health care.

The main challenge in the development of protocols, is ensuring their reliability so that health professionals feel comfortable to follow their recommendations. In this regard, protocol development methods have sought to increase the transparency and quality of the process and stimulate the participation of those interested throughout each step.⁽²⁰⁾ The development of justification, objectives and content with technical-scientific reference as a starting point is also important in the development of a protocol.⁽¹⁵⁾

The brainstorming technique becomes useful at this starting point, since it involves the development of a group’s collaborative creativity and consequent organization of these ideas to a given process. This technique has been applied increasingly along with the internet, virtual meetings and specific software for documenting ideas.⁽²¹⁾ Electronic Brainstorming involves group members (ranging from five to seven) at computer terminals, who type their ideas and have full access to the ideas of the other participants.^(22,23) Thus, it is easy to join ideas of important contents for construction of technologies.

Studies show the importance of this guided listening through the use of brainstorming with the service staff in order to assess the needs and ideas for protocol creation, meetings and interviews with clients and discussions with professionals. Furthermore, the use of online resources is also beneficial and effective for the development of technologies, as it enables and encourages both interaction and collaboration of participants.^(24-26)

According to Catunda et al. (2017),⁽²⁷⁾ despite variations in protocol development methods, there are common steps, among which the participation of target patients of the protocol for assisting in the process. However, the theoretical references do not specify an ideal number, and there are variations between studies.⁽²⁷⁾

After collaboration of all participants in the first step of the protocol development, each comment/suggestion was analyzed, and topics and subtopics were extracted from the mandatory content, namely: definition, risk factors and types of PFD; definition, quantification, risk factors and treatments of POPs; aspects related to vaginal pessary such as definition, comparison with surgical treatment, complications in quality of life, Brazilian panorama, types, indication, advantages, disadvantages, barriers to use, how to approach the patient, multidisciplinary team, first consultation, insertion consultation, subsequent consultations, anamnesis, physical examination, complementary exams, estrogenization, guidance on the device and care, measurement, insertion, removal, possible complications and return period.

The review of the scientific literature is the search and critical analysis of the publications. It is a strategic step in the development of protocols, in which finding the best evidence on the proposed subject is essential for the construction of consistent and higher quality protocols that are focused on the methodology used, critical analysis of the literature used, levels of evidence, grade of recommendation, entities participating in the validation and the form of validation.^(11,14)

In other studies of protocol development, was also conducted a review of the scientific literature for the collection and selection of content through computerized databases.^(28-30) Therefore, this step is the basis for selecting the content of the clinical protocol, because the dissemination of scientific evidence is fundamental in order that professionals can guide their practices.

Aiming at the quality of the protocol, were used the review of the scientific literature, ideas of professionals of the area and of device users, and insertion of the researcher into the service. This integration of the best available evidence in the literature, professionals’ clinical experience, patients’ preferences and resources available at the institution are characteristics of the Evidence-Based Practice (EBP).⁽³¹⁾

EBP is the conscious, explicit and judicious use of the best available evidence in patient care decision-making. It is focused on problem-solving grounded on the best scientific evidence with the aim to improve care, identify and promote effective practices for minimizing gaps between the production of evidence and its application in patient care.⁽¹¹⁾

Still in the search for quality, it is expected that protocols are clear and easy to read, have good formal quality and evidence-based content, follow a logical and progressive order, and are useful and relevant to the target audience.⁽¹¹⁾

Thus, the content of the protocol was divided into the following sessions: Preface; Justification; Protocol Considerations; Procedure of Search for Scientific Evidence; Classification System of Levels of Scientific Evidence and Grade of Recommendation; General Definitions on the Topic; Vaginal Pessary: general considerations; Assistance to the Vaginal Pessary Consultation; First Consultation for Evaluation of Women with Vaginal Pessary Therapeutic Indication; Vaginal Pessary Insertion Consultation; Follow-up Consultation for Women using Vaginal Pessary; References; Annex and Appendices with their respective sub-sessions.

The explicit and concise representation of processes is essential in order to improve the organization and facilitate the protocol management by professionals. Thus, for the order and establishment of action flows of a protocol focused on health outcomes, it is advisable to use algorithms represented in the form of flowcharts.^(11,15)

The flowcharts of the protocol were titled: 1) Algorithm for evaluation of women with therapeutic indication of the pessary; 2) Algorithm of the insertion consultation of the pessary; 3) Algorithm of the follow-up consultation of women using the pessary.

Another important aspect in the protocol development is the experts review in order to check organizational effects of the implementation of recommendations that is a preparation for its future adoption. This is considered the first dissemination of the protocol, when researchers have the opportunity to address the issues raised by reviewers before finalization of the process.⁽³²⁾

To this end, in the present study was used the Delphi technique, which is a systematized method of judging information. Its aim is to reach the consensus of opinions on a certain subject of knowledge of a group of experts through articulated validations in rounds of questionnaires repeated until reaching consensus among participants of 70 to 80%, or a determined percentage duly justified by the researcher.^(33,34)

The CVI is a widely used method in the health area for measurement of the proportion or percentage of judges in agreement on certain aspects of the instrument and its items.⁽¹⁸⁾

Regarding the number of participants, there must be at least seven and a maximum of 30 subjects. However, the decision on this number is based on aspects such as the nature of the object of study, which may point to a greater or lesser availability of participants.⁽²⁹⁾

In the study by Sousa and Turrini (2012)⁽³⁵⁾ was also applied the Delphi technique with the objective to assist in the construction of an educational material for patients undergoing orthognathic surgery, and to evaluate the pertinence of information contained in this technology with the multiprofessional team. The educational material evaluation was carried out in relation to coherence/pertinence and illustration of the information by means of a Likert-type instrument. The minimum value used for agreement between the ten judges was 0.85.⁽³⁵⁾

A methodology that uses the Delphi technique can be useful in the construction of educational technologies, since participants’ contributions are rich for improvement of the final work. The experts made relevant suggestions of changes for improving the protocol, even though the overall CVI was satisfactory in the study (0.98). These changes were related mainly to contents and presentation of the protocol in order to facilitate the understanding of professionals that use it. In addition, was established the applicability of the educational material for clinical practice.

Conclusion

The clinical protocol for the use of vaginal pessaries was developed through rigorous steps that included consultation with health professionals from different areas with expertise in PFD, consultation with pessary users, researcher’s experience in practice, and the search for evidence in the scientific literature. In the review step, the protocol was evaluated by means of the Delphi technique, and the total CVI of each domain and overall CVI were calculated. The total CVI of the objectives domain was 1.00, of the content and presentation criterion was 0.98, and of the relevance domain was 0.96. The overall CVI was 0.98. Thus, there was agreement among Delphi participants with a value above 0.85, and the clinical protocol was considered valid for professional clinical practice.

Acknowledgements

To the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) for research funding through Universal 14/2014, process number 459445/2014-6.

Referências

• ¹
  Hagen S, Stark, D. Conservative prevention management of pelvic organ prolapse in women. Cochrane Database Syst Rev. Oxford, 2011; 2011 Dec 7;(12):CD003882.
• ²
  Vasconcelos CT, Vasconcelos Neto JA, Bezerra LR, Augusto KL, Karbage SA, Frota IP, et al. Pelvic floor dysfunctions: clinical and sociodemographic profile of urogynecologic outpatients. Rev Eletrôn Gestão Saúde. 2013;4(1):1484-98.
• ³
  Atnip S, O’dell K. Vaginal support pessaries: Indications for use and fitting strategies. Urol Nurs. 2012; 32(3):114-24.
• ⁴
  Braekken IH, Majida M, Engh ME, Bø K. Can pelvic floor muscle training reverse pelvic organ prolapse and reduce prolapse symptoms? An assessor-blinded, randomized, controlled trial. Am J Obstet Gynecol. 2010;203(2):1-7.
• ⁵
  Tenfeld S, Tell D, Thomas TN, Kenton K. Quality of life in women who use pessaries for longer than 12 months. Female Pelvic Med Reconstr Surg. 2015;21(3):146-9.
• ⁶
  Hanson LA, Schulz JA, Flood CG, Cooley B, Tam F. Vaginal pessaries in managing women with pelvic organ prolapse and urinary incontinence: patient characteristics and factors contributing to success. Int Urogynecol J Pelvic Floor Dysfunct. 2006;17(2):155-9.
• ⁷
  McIntosh L. The role of the nurse in the use of vaginal pessaries to treat pelvic organ prolapse and/or urinary incontinence: a literature review. Urol Nurs. 2005; 25(1):41-8.
• ⁸
  Nager CW, Richter HE, Nygaard I, Paraiso MF, Wu JM, Kenton K, et al. Incontinence pessaries: size, POPQ measures, and successful fitting. Int Urogynecol J Pelvic Floor Dysfunct. 2009; 20(9):1023-28.
• ⁹
  Bugge C, Hagen S, Thakar R. Vaginal pessaries for pelvic organ prolapse and urinary incontinence: a multiprofessional survey of practice. Int Urogynecol J. 2013; 24(6):1017-24.
• ¹⁰
  O’dell K, Atnip S. Pessary care: Follow up and management of complications. Urol Nurs. 2012;32(3): 126-36.
• ¹¹
  Pimenta CA, Pastana ICA, Sichieri K, Solha RK, Souza W. Guia para construção de protocolos assistenciais de enfermagem. São Paulo: COREN-SP; 2015.
• ¹²
  Alperin M, Khan A, Dubina E, Tarnay C, Wu N, Pashos CL, et al. Patterns of pessary care and outcomes for medicare beneficiaries with pelvic organ prolapse. Female Pelvic Med Reconstr Surg. 2013;19(3):142-7.
• ¹³
  Matta AE, Silva FP, Boaventura EM. Design-based Research ou Pesquisa de Desenvolvimento: metodologia para pesquisa aplicada de inovação em educação do século XXI. Rev FAEEBA – Educação e Contemporaneidade. 2014;23(42): 23-36.
• ¹⁴
  Ribeiro RC. Clinical guidelines: how to evaluate its quality? Rev Soc Bras Clín Méd. 2010;8(4): 350-5.
• ¹⁵
  Werneck MA, Faria HP, Campos KF. Protocolo de cuidado à saúde e organização do serviço [Internet]. Horizonte (MG): Coopmed; 2009. [citado 2018 Out 13]. Disponível em: https://www.nescon.medicina.ufmg.br/biblioteca/imagem/1750.pdf
  » https://www.nescon.medicina.ufmg.br/biblioteca/imagem/1750.pdf
• ¹⁶
  Mendes KD, Silveira RC, Galvão CM. Integrative literature review: a research method to incorporate evidence in health care and nursing. Texto Contexto Enferm. 2008;17(4):758-64.
• ¹⁷
  Howick J, Phillips B, Ball C, Sackett D, Badenoch D, Straus S, et al. Oxford Centre for Evidence-Based Medicine (CEBM). Levels of evidence and grades of recommendation (March 2009). [Internet]. Oxford: CEBM; 2009. [cited 2018 Oct 13]. Available from: https://www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009
  » https://www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009
• ¹⁸
  Alexandre NM, Coluci MZ. Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciênc Saúde Coletiva. 2011;16(7):3061-8.
• ¹⁹
  Brasil. Ministério da Saúde. Conselho Nacional de Saúde. Comissão Nacional de Ética em Saúde. Resolução CNS nº 466/12. Normas para pesquisa envolvendo seres humanos. Diário Oficial da União, Poder Executivo, Brasília (DF); 2012.
• ²⁰
  Mega TP, Lopes AC, Santos VC, Petramale CA. Protocolos clínicos e diretrizes terapêuticas no SUS: histórico, desafios e perspectivas. Rev Eletrôn Gestão Saúde. 2014;4(6):3275-85.
• ²¹
  Massuchetto V, Sousa WH. Collaborative decision support systems: a proposition of a Web methodology with brainstorming and strategy table. In: Conferencia Internacional sobre Sistemas de Informação e Gestão e Tecnologia, 12. São Paulo; 2015. 21p.
• ²²
  Linsey JS, Becker B, editors. Effectiveness of brainwriting techniques: comparing nominal groups to real teams. London: Design Creativity; 2010. p.165-71.
• ²³
  Furnham A. The brainstorming myth. Bus Strategy Rev. 2000;11(4):21-8.
• ²⁴
  Borba EE. O gerenciamento de projeto na construção e implantação de um protocolo de assistência na área da saúde para pessoas com deficiência intelectual. Rev Ciênc Gerenciais. 2012;16(23):107-26.
• ²⁵
  Frota NM, Barros LM, Araújo TM, Caldini LN, Nascimento JC, Caetano JA. Construção de uma tecnologia educacional para o ensino de enfermagem sobre punção venosa periférica. Rev Gaúcha Enferm. 2013;34(2):29-36.
• ²⁶
  Rosini I, Salum NC. Protocolo de cuidados para punção aspirativa por agulha fina de mama e tireoide. Texto Contexto Enferm. 2014;23(4):1059-67.
• ²⁷
  Catunda HL, Bernardo EB, Vasconcelos CT, Moura ER, Pinheiro AK, Aquino PS. Methodological approach in nursing research for constructing and validating protocols. Texto Contexto Enferm. 2017;6(2): e00650016.
• ²⁸
  Gubert FA, Santos DA, Pinheiro MT, Brito LL, Pinheiro SR, Martins MC. Protocolo de Enfermagem para consulta de puericultura. Rev RENE. 2015;16(1): 81-9.
• ²⁹
  Pelegrino FM, Bolela F, Corbi IS, Carvalho AR, Dantas RA. Protocolo educativo para pacientes em uso de anticoagulante oral: construção e validação. Texto Contexto Enferm. 2014; 23(3):799-806
• ³⁰
  Paes GO, Leite JL, Mello EC. Cuidando do cliente com distúrbio respiratório agudo: proposta de um protocolo assistencial para tomar decisões em enfermagem. Online Braz J Nurs. 2011; 10(1):1-8.
• ³¹
  Pedrosa KK, Oliveira IC, Feijão AR, Machado RC. Evidence-based nursing: characteristics of studies in Brazil. Cogitare Enferm. 2015; 20(4):733-41.
• ³²
  Cullum N, Haynes B, Ciliska D, Marks S. Enfermagem baseada em evidências: uma introdução. Porto Alegre: Artmed; 2010. 382p.
• ³³
  Coutinho SS, Freitas MA, Pereira MJ, Veiga TB, Ferreira M, Mishima SM. O uso da técnica Delphi na pesquisa em atenção primária à saúde: revisão integrativa. Rev Baiana Saúde Pública. 2013;37(3):582-96.
• ³⁴
  Scarparo AF, Laus AM, Azevedo AL, Freitas MR, Gabriel CS, Chaves LD. Reflexões sobre o uso da Técnica Delphi em Pesquisas na Enfermagem. Rev RENE. 2012;13(1):242-51.
• ³⁵
  Sousa CS, Turrini RNT. Validação de constructo de tecnologia educativa para pacientes mediante aplicação da técnica Delphi. Acta Paul Enferm. 2012;25(6): 990-6.

Publication Dates

• Publication in this collection
  Nov-Dec 2018

History

• Received
  1 June 2018
• Accepted
  15 Nov 2018