Delphi technique in dialogue with nurses on acupuncture as a proposed nursing intervention

Abstracts

Objective:

Present the theoretical assumptions of the Delphi technique, its applicability and operationalization in nursing research, from a model proposed to conduct a Master's thesis on acupuncture as a proposed nursing intervention.

Methods:

This is an analytical essay of the reflection type.

Results:

In recent decades, the research developed in the nursing field has been seeking methods and techniques capable of dealing with different questions and producing answers for both the healthcare field and the development of nursing science, in order to consolidate its know-how in line the dictates of modern science.

Conclusion:

The Delphi method seeks the consensus among renowned expert opinions on a complex issue, permitting the validation of propositions submitted to them, turning into an alternative for the development of studies in which there are few data on the research subject or which demand projection for the future.

Nursing; Nursing research; Methodology; Acupuncture

Objetivo:

Apresentar os pressupostos teóricos da técnica Delphi, sua aplicabilidade e operacionalização nas pesquisas em enfermagem, a partir de um modelo proposto para condução de uma dissertação de mestrado sobre a acupuntura como proposta de intervenção de enfermagem.

Métodos:

Trata-se de um ensaio analítico do tipo reflexão.

Resultados:

Nas últimas décadas, as pesquisas desenvolvidas na área de enfermagem vêm buscando métodos e técnicas capazes de lidar com diferentes questões e produzirem respostas tanto para o campo assistencial quanto para o desenvolvimento da ciência de enfermagem, a fim de consolidar seu saber-fazer em consonância aos ditames da ciência moderna.

Conclusão:

A metodologia Delphi busca o consenso de opiniões de especialistas de notório saber a respeito de um problema complexo, permitindo a validação de proposições que lhes são apresentadas, se constituindo como alternativa para o desenvolvimento de estudos em que existam poucos dados acerca do assunto pesquisado ou que demandem projeção para o futuro.

Enfermagem; Pesquisa em enfermagem; Metodologia; Acupuntura

Objetivo:

Presentar los presupuestos teóricos de la Técnica Delphi, su aplicabilidad y operacionalización, a partir de un modelo creado para la realización de una disertación de Maestría sobre la acupuntura como intervención de Enfermería.

Métodos:

Ensayo analítico reflexivo.

Resultados:

En las últimas décadas, las investigaciones desarrolladas en el campo de la Enfermería han buscado métodos y técnicas capaces de hacer frente a diferentes preguntas y respuestas, tanto en el campo asistencial, como para el desarrollo de la ciencia de la Enfermería, con el fin de consolidar sus conocimientos de acuerdo a los dictamos de la ciencia moderna.

Conclusión:

La metodología Delphi busca el consenso de la opinión de expertos sobre un tema complejo, lo que permite la validación de las propuestas que se les presenten, convirtiéndose en una alternativa para el desarrollo de estudios que no tengan muchos datos sobre el tema o que exijan proyección para el futuro.

Enfermería; Investigación en Enfermería; Metodología; Acupuntura

INTRODUCTION

The objective in this reflection is to present the theoretical premises of the Delphi technique, its applicability and operation in nursing research, based on a model proposed to conduct a Master's thesis about acupuncture as a proposed nursing intervention.

Scientific production in Brazilian nursing has considerably increased in the last three decades, mainly enhanced by the advance of stricto sensu graduate programs, which serve as a privileged locus for research support and development, reflections, analyses and theoretical-practical criticism on the nurses' know-how. Therefore, appropriate knowledge on the methodological, epistemological and social aspects related to the scientific research one intends to develop is vital for the development and success of nursing research.

Since the 1990's, qualitative research methods have been present in nursing research, centered on the nurses' interest in presenting responses to the complex phenomena of health issues¹. Nevertheless, the use of methods and technique that permit research development in the light of modern science, the mastery and response to its emerging issues, and the consequent adaptation to the scientific paradigms it uses, represent challenges that need to be overcome by the researchers who study the nature and science of nursing².

These methods have been employed in view of the limits of investigating and understanding, through statistical means, the phenomena focused on perception, intuition and subjectivity, related to the meaning of human relations and their theoretical and practical actions^3,4. The theoretical foundations and the constitution of a research corpus that guides the identification of what has been investigated and what gaps remain to be completed, like through the establishment of the state-of-the-art and through systematic and integrative reviews of the literature for example, have become fundamental to guide studies in nursing science.

In view of complex situations for which there is no established research corpus available, or even an incipient corpus, the nursing researchers use methods and techniques that can support the development of their research, permitting advances in the construction of knowledge. That is the case of the Delphi technique, originally created in 1948 and disseminated as from the 1960's, whose initial objective was the development of a technique to permit the enhancement of experts' opinions in technological forecasts⁵.

Conceptually, in its original formulation, the Delphi method took the form of a technique capable of seeking expert opinions on future events^5,6. Today, besides seeking a consensus and support futurists in specific areas like the industry and the military, oral knowledge areas have used the Delphi technique as a research tool to solve complex problems, including nursing⁷.

The technique is particularly recommended when one has no data about the theme in vogue or these cannot be safely projected for the future, in view of the expected changes in the determining factors of future trends⁶. In other words, it can be used in case of incomplete knowledge on the nature or components of a given situation.

In view of these characteristics, the Delphi technique was applied in a thesis research that originated this article, which aimed to promote a cross-paradigmatic dialogue about the use of acupuncture as a potential and holistic technology that needs to be integrated into the set of nursing interventions. The discussions were related to the theme acupuncture, knowledge that, although not characteristic of nursing, has been used in the area for some time in Brazil, as a form of therapeutic intervention. The goal of the millenary treatment system is to diagnose illnesses, seen as energy imbalances, and to cure by stimulating the body's self-healing strength⁸. Today, it is understood as an complementary and integrative health practice (CIHP), originating in Traditional Chines Medicine (TCM), which is globally accepted and whose results have been scientifically proven, based on the application of needles, heat (moxibustion) or laser on acupuncture points, called acupoints, and on orientations for the adoption of healthy life practices, in the attempt to harmonize the energy flow. Its diagnostic and treatment form differ from that adopted in the biomedical model, as it rests on the holistic perception of man, on the relation established with the natural context and on the influence it exerts on health⁹.

As a profession, although nursing aims to develop care by approaching the human being in a comprehensive and dynamic manner, its diagnostic language and intervention practices are related to the theoretical-philosophical reference frameworks of the biomedical model, which is dominant in health and, therefore, distinct from the models that guide the acupuncture diagnosis and its intervention. Nevertheless, like acupuncture, in theory, nursing care departs from the principle that its actions are not concentrated on care for the disease, but on the human being and his/her interrelations with the natural midst⁸.

Thus, the discussion in that research was based on the theoretical-philosophical reference frameworks of TCM and the biomedical model, and their influences on the nursing diagnosis and intervention forms, in the light of the opinions of two groups of nurses: nursing diagnosis experts and nurse acupuncturists. The choice of the Delphi technique to produce the research data was due to the intention to establish a dialogue between the selected participant groups, given the need for theoretical support, linked to these nurses' expertise, in order to contribute to the discussion on the theme. Reflecting on the strategies developed in the application of the Delphi technique seems appropriate here, as it can be of interest to other researchers whose research problems are related to the characteristics of this technique. In addition, the lack of dissemination about its use in nursing is taken into account.

OUTLINING THE THEME: UNDERSTANDING THE DELPHI TECHNIQUE

The Delphi technique is a research tool that seeks a consensus of expert opinions about a complex problem. The search for consensus occurs systematically, in which the experts, also known as judges, judge information, called propositions, presented by the researcher on a previously formatted questionnaire, based on the research problem and its objectives, in which a synthesis is presented, addressing the main trends regarding the theme. The technique can be applied to quantitative or qualitative data^5,6.

Its basic characteristics are anonymity, the statistical representation of the result distribution and the feedback of the group's answers, linked to two basic premises that sustain the development process of the technique: the flexibility and the experts' contributions to the group (Figure 1).

Figure 1
Representative scheme of characteristics of the Delphi technique. Rio de Janeiro, 2013.

In that sense, in view of a complex problem in which little or no data are available to built concepts, practices, techniques and interventions, as well their respective validations in the nursing area, this research technique shows to be a feasible, low-cost alternative, with expressive and reliable results, as it combines the opinions of experts with diversified experiences, even if geographically distant^7,10.

Studies highlight that the harshest criticism against the use of the Delphi technique is precisely the consensus. Depending on the way the results and the feedback to the participants are presented, there is a risk of creating forced or artificial consensuses, mainly if the respondents passively accept other experts' opinion¹¹.

To avoid this kind of event, the group should be diversified, with very distinct origins in the form of approaching and discussing the research theme. Therefore, the careful selection of the participants and the selected experts' critical-reflexive capacity are valuable issues that need to be reflected in the choice of the technique and the elaboration of the inquiries, which should definitely be in accordance with the established research objectives¹².

Generally, the Delphi technique is developed in three rounds and, although it can be accomplished in less or more rounds, this is not recommended^6,7. It can be employed in the traditional mode, in which the questionnaires are forwarded to the experts by common mail; or electronically, by e-mail or through an online platform that offers safe and exclusive access to the researchers, on the Internet^6,8,11.

The traditional modality, mainly when applied in geographically dispersed regions, can become costly and time consuming, given possible difficulties to send the material by mail in all phases. On the opposite, the electronic modality, currently used nowadays, has shown to be an effective alternative, without significant differences in the selected participants' adherence percentage. This aspect was addressed in the thesis research that gave rise to this reflection, which used the electronic technique to overcome the spatial barrier deriving from the participants' distribution across different regions of the country. In this study, the selected experts totaled 30 participants, distributed in two groups of 15 - experts in nursing diagnoses and nurse acupuncturists, were located in geographically distinct regions. Thus, the use of the Internet reduced the costs and the time needed to develop the research, without compromising the participants' adherence percentage beyond what was mentioned in the literature¹³.

Noteworthy and significant difficulties were faced to identify the experts, as experienced and mentioned by other researchers in the development of their studies. The researcher individually contacted the participants, through the contact data and e-mails they disseminated in their respective curricula, available on the Lattes platform of the National Council for Scientific and Technological Development (CNPq). Next, as a strategy to permit the location of possible experts, the research directories and groups of institutions and funding entities were investigated. In these bases, the possible participants' academic production and professional activities can be explored and identified by investigating on the topic or activity area the study is based on.

Another limited factor was the selected participants' possible dropout, mainly in those situations without a good time margin for their participation, or without actual interest in the research, which end up compromising the research development. In that sense, in studies developed at the stricto sensu level, whose deadline to complete the Master's or Doctoral program is predefined, this situation can compromise the conclusion of the study or the reliability of its results. That was a constant concern of the authors in the development of the Master's thesis, which required special attention and the use of strategies to guarantee the participants' continuation in the three study rounds, such as frequent contacts by e-mail and permanent availability to solve occasional doubts or any difficulties that may emerge in the course of the data collection.

The production of heuristics and biases should also be carefully considered, so that the resulting data and their analysis are not compromised. In these studies, the heuristics are represented by the production of standardized rules that implicitly drive the judgment process, limiting the experts in their production of answers or exposition of their conceptions and thoughts, while the biases occur when these heuristics are applied inappropriate or erroneously¹⁴. Therefore, in the activities that involve forecasting or subjective analysis tasks, besides large space for the presentation of conceptions and thoughts, all results should be statistically founded in order to demonstrate the achievement of a consensus, which may or may not occur in function of the particularities. The questionnaires elaborated should be clear and objectives, however, without the ability to drive the judgment process to one or another response.

At each concluded round, the results are analyzed in the light of the qualitative and quantitative perspective. The collected data are organized in the simple statistical form and the experts provide the participants with feedback for their knowledge, analysis and reflection^5,6. This movement is displayed in the following figure (Figure 2).

Figure 2
Representative scheme of the Delphi technique. Rio de Janeiro, 2013.

In the final round, considered as the group's forecast, besides the qualitative analysis, the first and third quartile are established, as well as the median and the interquartile interval, in order to provide the process with greater statistical rigor and a quantitative impression of the data^5,6. At the end of the research, a full report is provided to the participants, presenting the consensuses achieved and the recommendations the group produced. The experts' identification is not revealed to the public or the other participants in any of the study phases. Thus, the anonymity not only allows them to freely express their opinions, but also encourages the raising of high-level discussions without concerns with opposing other renowned experts' knowledge⁷.

THE DELPHI TECHNIQUE IN NURSING RESEARCH: LIMITS AND POSSIBILITIES

The use of the Delphi technique in nursing research started in Brazil in the 1990's, when nurses applied it to validate nursing interventions for the rehabilitation of adult patients with bone marrow injury. The Delphi technique furthered reflections on the organizational structures of the rehabilitation centers, as well as knowledge on the nursing proposals aimed at these patients. The technique's flexibility, at the same time as it valued the nurses' opinions, permitted indicating the nursing interventions in this field¹⁵. A further study was undertaken to validate nursing diagnoses and interventions for patients and family members with the same kind of injury⁹.

The use of the technique in other nursing studies shows a trend towards the tertiary level, especially focused on the validation of care practices in hospital institutions. This fact does not disqualify its use in other nursing scenarios or activity areas. Examples are the development of a study in 2004 to adapt an instrument about the dimensioning of daily residential nursing care hours to the Portuguese language. The selected participants were nurses affiliated with the companies located in the city of São Paulo who deliver residential nursing care. The interactive and reflexive nature of the Delphi technique about phenomena that lack new or revisited knowledge furthered the inclusion of new nursing interventions into that instrument, as well as the activity type and the time used to execute them. In addition, it contributed to the planning and systemization of home care, permitting the synchrony between cost containment and the maintenance of care quality in that space¹⁶.

A study undertaken in 2008 investigated nursing competences in public health, contributing through the identification of ten areas that serve as a reference for professional practice in this field¹².

As mentioned, in the context of experiments in nursing studies that applied the Delphi technique, there is the thesis research that originated this paper, which was focused on acupuncture as a nursing intervention proposal. Its use permitted the construction of a cross-paradigmatic dialogue among experts in nursing diagnoses (ND) and acupuncturists about the possible theoretical-philosophical relations between acupuncture, as a technique and branch of traditional Chinese medicine, and nursing.  Given the lack of material to support the proposed research problem, the experts' dialogue served as a foundation for theoretical constructs on acupuncture as care technology. Although nursing's theoretical-methodological framework remains attached to the biomedical model, it was inferred in its conclusion that acupuncture can be incorporated into nursing praxis, as a complementary form of health care, aiming for people's integrality, but without disqualifying the other forms of knowledge previously established in the profession. From the perspective of the nursing diagnoses, the research participants identified and spontaneously mentioned those diagnoses in which acupuncture interventions are feasible, independently of the order and/or taxonomic classifications. Both groups contributed to the dialogue with limits and possibilities to use acupuncture as a proposed nursing intervention. In addition, they produced the consensus that this technology can be applied by nurse experts in acupuncture and indicated by other non-expert nurses, but who acknowledge it as a possibility to intervene in nursing diagnoses.

Based on the development of this research, among other studies, some of which are listed here, no discussions were found about the main challenges for the development of the technique, as well as the forms to cope with them. Thus, to achieve the objective proposed here and contribute to other studies that intend to use it, in the figure 3, an operational model is presented, followed by a discussion, based on the conceptual guidelines. This model was used in our experiment and reflections deriving from that research.

Figure 3
Operation of the Delphi Technique in Nursing Research. Rio de Janeiro, 2013.

Implementation of the electronic Delphi Technique

Phase I - Preparation

This is fundamental to guarantee the successful development of the technique. It consists in the survey of experts and specialists who will participate in the research and in the elaboration of the research platform or the research questionnaire, to be forwarded by e-mail. This is a critical phase that should be well founded to guarantee the success of subsequent phases. Very specific questions and which depend on the opinion of a very select group of professionals do not raise barriers to proceed with the study, even if the location of these experts and specialists demands a broader search.

Selection of Experts and specialists

The selection of the experts and specialists should be in accordance with the research objectives. The selected subjects should be familiar with the proposition one intends to validity, and therefore should have academic, scientific production and/or professional experience in the study area, so as to be considered experts^6,7,10. Hence, for all research situations, clear inclusion and exclusion criteria should be established, which permit configuring the participant's expertise in the study area¹¹.

Thus, to overcome possible limits in the selection of the participants and at the same time comply with the specifications of the technique in terms of the participants' characteristics as experts, it was inferred that a good selection strategy should follow a bibliographic survey, production of the state-of-the-art and systematic or integrative reviews about what one intends to validate or the knowledge area it is directly related to. Thus, the experts can be identified who focus and produce on the theme or area under analysis. Even without acknowledged academic production, professionals who work in the area and have relevant professional experience can also participate as experts and, in these situations, their professional experience will be the main point to consider them as such. Another fundamental aspect for a successful research is the guarantee of anonymity. After identifying the possible participants, the anonymity should be explicitly guaranteed to all, in all research phases. Therefore, since the beginning, any contact should take place by e-mail, preferably through an address created for this purpose only. The experts should be contacted one by one without the creation of an e-mail group, as this can allow one participant to identify the other, compromising the credibility and development of the study.

Upon the first contact, the reasons why each participant was selected should be expressed and what will be investigated in the research, inviting him/her to participate. It is important to verify the availability to participate in the study, as well as the conditions to access electronic media, the frequency of internet and e-mail access, so as not to compromise the deadline of the rounds. If the participant accepts, the Free and Informed Consent Form should be forwarded for signing and return by e-mail. In the course of the study, the number of participants may drop by up to thirty percent (30%)⁶; therefore, during the selection, a technical safety index should be applied that is superior to this margin of losses, so as to guarantee the accomplishment of the study without compromising the quality of the results.

Elaboration of the research platform or research questionnaire

In the electronic modality, one can choose between two ways to develop the research. The first is the elaboration of a research platform or site for the experts to access. In this modality, the expert accesses the platform through a login and password, gaining access to the propositions (s)he is supposed to judge. The advantage of using this tool is the practicality for the researcher or research coordinator and for the expert him/herself. Thus, there is no need to forward e-mails with files that can cause conflicts, difficulties to open the files and the dissemination of electronic viruses. In addition, they summarize the answers of all participants, store them in a virtual database that permits unrestricted access, however, without permitting modifications by the experts or by the researcher or coordinator. Given its complexity, however, it should be elaborated and preferably managed by computer technology companies or professionals, which can increase the research costs, although it tends to enhance the reliability in the research process.

No less effective and certainly cheaper to operate are the traditional questionnaires forwarded by e-mail. One can choose between direct sending, when the propositions and answers are inserted in the e-mail itself; or indirect sending, when a file is forwarded, to be opened in a word processing file in order to insert the answers. This manner demands greater attention from all parties involved in the study, in view of possible difficulties to open the files or the dissemination of electronic viruses. In addition, the deadline to close off the round may be compromise in case of the experts' delay to forward the answer or if they face difficulties that need to be solved. In any of these forms, the propositions should be formulated with answers on assessment, position or attitude scales, like Likert, Guttman and Thurstone scales for example. This scaling with permit the use of quantitative data analysis in the subsequent phases⁶. In the thesis research that led to this article, the Likert scale was chosen because it allows the participants to better understand the presented propositions and facilitates the percentile distribution, with a fixed variation by 25% (twenty-five percent) at each option the participant can choose from. Although it does not represent a step of this phase, in the same research, a single-round pilot study was proposed to check for possible errors, difficulties to transmit the data and obtain the answers, adjustments needed in the texts for the sake of greater clarity, among other necessary corrections. The development of a pilot study is recommended, mainly for newcomers in the use of this technique.

Phase II - Execution

This is the data collection phase. In general, a maximum of three rounds of the questionnaire is defined among the participants. The propositions are launched for validation through the achievement of a consensus. The researcher sets the cut-off point to obtain a consensus, which should be higher than the fiftieth percentile (50%). Percentiles very close to the minimum limit, however, may not produce sufficient credibility to consolidate a proposition. Therefore, although this is considered a high point of consensus, which is difficult to obtain in many cases, in nursing research undertaken in situations of scarce production or to elaborate theoretical constructs, cut-off points for consensus inferior to the seventy-fifth percentile (75%) should not be applied. As a consensus is achieved on the propositions, they are gradually removed from the round, so that only those propositions on which no consensus has been obtained yet remain in the questionnaire for the next round. At the end of the round, the participants receive feedback, when the results of each proposition are presented, as well as the experts' justifications and considerations (anonymously) in the analysis of each proposition. This feedback can include graphs and tables that express the situation each proposition under analysis achieved. Nevertheless, the aptitude of all elements in the group should be considered with a view to the analysis and interpretation of strictly numerical data. This can hamper the group analysis, induce to error or discourage the participant from continuing in the study. Returning to the results of the thesis, in the first round, two propositions researched consensus levels for both participant groups, without the need to return to the groups for further analyses. The remainder returned to the participants in the second round. In some of them, adjustments were needed based on the experts and specialists' position, but without changing the meaning or sense of the propositions. The results of each round returned to the participants in a PDF document by e-mail and were later posted online on the research platform.

Once the consensuses have been achieved, the research proceeds to the third round, when the entire process is usually concluded. In this phase, all results are analyzed, round by round, although only those proposals for which a consensus was obtained are validated. Propositions without a consensus represent a research results. They cannot be neglected or suppressed and should be presented and discussed in the final report. For the quantitative data treatment, the central trend measures should be calculated, which will demonstrate the achievement and regularity of the consensuses^5,6. Therefore, the analysis of the first and third quartile, median and interquartile difference of each proposal should be used, independently of the consensus. The resulting qualitative data can be treated with the help of related techniques, such as content analysis or thematic content analysis, previously defined by the research in the elaboration of the research protocol.

Phase III - Dissemination

This is the presentation of the results obtained in the research. In this phase, the participants receive or access the report on the research results, which presents the consolidated data, consensuses and recommendations related to each proposition launched⁵.

FINAL CONSIDERATIONS

The construction of this article was based on the authors' preliminary experience in the development of a thesis that promoted a dialogue about acupuncture as a nursing intervention among nurses distributed in two distinct groups: nursing diagnosis experts and acupuncture specialists. This experience permitted the construction of a reflexive process on the Delphi technique, its possibilities and usage limits. There was no intent to exhaust this reflection here, but to address the main influences and contributions of the technique, applied particularly to nursing studies, through an operational model. The intent is for the use of this model to contribute to studies that employ the Delphi technique in nursing, so as to consolidate, reject or offer new elements to its use.

This technique is considered as a methodologically sound proposal that is applicable to nursing research, mainly in those situations in which the lack of data and material can represent a limiting factor for the development of nursing research. Based on the analysis of the studies cited here, it is considered that, when well applied, the technique produces consistent results that contribute to support new forms of care and technologies applicable to nursing care. The wealth in the use of the Delphi technique is not only related to the production of data that sustain one or another proposal, but to the production of a dialogue among the participants, with their respective expertise.

REFERÊNCIAS

• ¹
  Carvalho V, Barreira IA. Marco conceitual para o ensino e pesquisa da enfermagem fundamental: um ponto de vista. In: Carvalho V, organizadora. Sobre enfermagem: ensino e perfil profissional. Rio de Janeiro: EEAN/UFRJ; 2006. p.197-217.
• ²
  Júnior JMP, Nóbrega VKM, Miranda FAN. O cuidado de enfermagem na pósmodernidade: um diálogo necessário. Esc Anna Nery. 2012;16(3): 603-6.
• ³
  Minayo, MCS; Deslandes SF; Gomes, R Pesquisa Social: teoria, método e criatividade. Petrópolis (RJ): Vozes; 2010.
• ⁴
  Cuesta, BC. La investigación cualitativa y el desarrollo del conocimiento en enfermería. Texto & contexto enferm. 2010;19(4):762-6.
• ⁵
  Oliveira JSP, Costa MM, Wille MFC. Introdução ao Método Delphi. Curitiba: Mundo Material, 2008.
• ⁶
  Wright JTC, Giovinazzo RA. Delphi: uma ferramenta de apoio ao planejamento prospectivo. Cad. pesqui. adm. 2000;12(1):54-65.
• ⁷
  Silva AM, Rodrigues CDS, Silva SMR, Witt RR. Utilização da técnica Delphi on-line para investigação de competências: relato de experiência. Rev. gauch. enferm. 2009;30(2):348-51.
• ⁸
  Salles LF, Silva MJP. Enfermagem e as práticas complementares em saúde. São Paulo: Yendis; 2011.
• ⁹
  Maciocia G. Os fundamentos da medicina chinesa: um texto abrangente para acupunturistas e fisioterapeutas. São Paulo: Roca; 2005.
• ¹⁰
  Scarparo AF, Laus AM, Azevedo ALCS, Freitas MRI, Gabriel CS, Chaves LDP. Reflexões sobre o uso da técnica Delphi em pesquisas na enfermagem. Rev Rene. 2012;3(1):242-51.
• ¹¹
  Castro AV, Resende M. A técnica Delphi e seu uso na pesquisa de enfermagem: revisão bibliográfica. Reme, Rev. Min. Enferm. 2009;3:429-34.
• ¹²
  Witt RR, Almeida MCP. Identification of nurses' competencies in primary health care through a Delphi study in southern Brazil. Public Health Nurs. 2008;4:335-43.
• ¹³
  Pereira, RDM. Acupuntura como proposta de Intervenção de enfermagem: um diálogo transparadigmático entre experts em diagnósticos de enfermagem e enfermeiros acupunturistas [dissertação]. Rio de Janeiro (RJ): Escola de Enfermagem Anna Nery, Universidade Federal do Rio de Janeiro; 2012.
• ¹⁴
  Kayo, EK, Securato JR, Método Delphi: Fundamentos, Críticas e Vieses. Cad. pesqui. adm. 1997;4:51-61.
• ¹⁵
  Faro, ACM. Ténicna Deplhi na validação de intervenções de enfermagem. Rev. Esc. Enferm. USP. 1997;31(1):259-73.
• ¹⁶
  Dal Bem, LW, Sousa, RMC. Adaptação de instrumento para dimensionar horas diárias de assistência de enfermagem residencial. Rev. Esc. Enferm. 2004;38(1):80-9.

Publication Dates

• Publication in this collection
  Jan-Mar 2015

History

• Received
  14 Mar 2013
• Accepted
  03 Apr 2014