Acessibilidade / Reportar erro

THE DELPHI METHOD AS A METHODOLOGICAL FRAMEWORK FOR RESEARCH IN NURSING1 1 Article taken from the thesis - The teaching of infection control in undergraduate courses in nursing in Brazil, presented to the Postgraduate Program in Nursing (PEN), Universidade Federal de Santa Catarina (UFSC), in 2016.

ABSTRACT

Objective:

to describe the process of collecting and analyzing data based on the use of the Delphi method, and the process of articulating qualitative and quantitative procedures for constructing a proposal for curricular innovation in the area of infection control in undergraduate courses in nursing.

Method:

the Delphi method was used with 39 participants, of whom 31 were nurses and eight were physicians, all experts in the area of infection control. Data collection took place over a period of seven months, being undertaken in four rounds. Content analysis was used for analyzing the qualitative data, and descriptive analysis for the quantitative data.

Results: a total of four interactive and sequential rounds was undertaken. The first was an instrument with open questions. The instrument for the second round was elaborated based on the responses to these (39 respondents), and was made up of items to be evaluated by the participants. After the second round (35 respondents), the items for which consensus was not obtained were re-presented to the participants in the instrument of the third round (30 respondents), which contained the problematization of the item so that each participant could reevaluate it and provide their rationale. In the fourth round, all the data gathered was returned to all the participants. The undertaking of rounds, promoting controlled feedback so that all participants can review their opinions is a striking characteristic of this method.

Conclusion:

the Delphi method evidenced its strength as a possibility for articulating qualitative and quantitative approaches, bringing these characteristics together to respond to a complex research issue in the area of nursing and health.

DESCRIPTORS:
Nursing; Curriculum; Education; Infection control; Mixed methods; Qualitative research; Quantitative research

RESUMO

Objetivo:

descrever o processo de coleta e análise dos dados a partir do uso do método Delphi e da articulação de procedimentos qualitativos e quantitativos para a construção de uma proposta de inovação curricular na área de controle de infecções para os cursos de graduação em enfermagem.

Método:

foi utilizado o método Delphi com 39 participantes, sendo 31 enfermeiros e oito médicos, com expertise na área de controle de infecções. A coleta de dados transcorreu em um período de sete meses, com a realização de quatro rodadas. Para a análise de dados qualitativos foi empregada à análise de conteúdo e para os quantitativos à análise descritiva.

Resultados: foram realizadas quatro rodadas interativas e sequenciais, a primeira rodada foi composta por um instrumento com perguntas abertas, a partir destas respostas (39 respondentes) foi elaborado o instrumento da segunda rodada que foi constituído de itens para avaliação dos participantes. Após a segunda rodada (35 respondentes), os itens que não obtiveram consenso foram reapresentados aos participantes no instrumento da terceira rodada (30 respondentes), contendo a problematização do item para que fosse reavaliado com a justificativa do participante. Na quarta rodada foi realizada a devolutiva dos dados à todos os participantes. O desenvolvimento de rodadas, promovendo feedbacks controlados para que os participantes possam rever suas opiniões, é uma característica marcante desse método.

Conclusão:

o método Delphi comprovou seu potencial como possibilidade de articulação das abordagens qualitativas e quantitativas, somando essas características para tratar de um problema complexo de pesquisa na área da enfermagem e saúde.

DESCRITORES:
Enfermagem; Currículo; Educação; Controle de infecção; Método misto; Pesquisa qualitativa; Pesquisa quantitativa

RESUMEN

Objetivo:

describir el proceso de recolección y análisis de los datos a partir del uso del método Delphi y de la articulación de procedimientos cualitativos y cuantitativos para la construcción de una propuesta de innovación curricular en el área de control de infecciones para los cursos de graduación en enfermería.

Método:

se utilizó el método Delphi con 39 participantes, siendo 31 enfermeros y ocho médicos, con experiencia en el área de control de infecciones. La recolección de datos transcurrió en un período de siete meses, con la realización de cuatro rondas. Para el análisis de datos cualitativos se empleó el análisis de contenido y los cuantitativos al análisis descriptivo.

Resultados:

se realizaron cuatro rondas interactivas y secuenciales, la primera ronda fue compuesta por un instrumento con preguntas abiertas, a partir de estas respuestas (39 respondedores) fue elaborado el instrumento de la segunda ronda que fue constituido de ítems para evaluación de los participantes. Después de la segunda ronda (35 respondedores), los ítems que no obtuvieron consenso fueron presentados a los participantes en el instrumento de la tercera ronda (30 contendientes), conteniendo la problematización del ítem para que fuera reevaluado con la justificación del participante. En la cuarta ronda se realizó la devolución de los datos a todos los participantes. El desarrollo de rondas, promoviendo feedbacks controlados para que los participantes puedan revisar sus opiniones, es una característica resaltante de ese método.

Conclusión:

el método Delphi comprobó su potencial como posibilidad de articulación de los abordajes cualitativos y cuantitativos, sumando esas características para tratar de un problema complejo de investigación en el área de la enfermería y salud.

DESCRIPTORES:
Enfermería; Currículo; Educación; Control de la infección; Método mixto; Investigación cualitativa; Investigación cuantitativa

INTRODUCTION

The description of different methodological frameworks through scientific research has been of great value for the health area, in particular, for nursing. The development of scientific research has been evidenced based on a range of innovative possibilities in methodological terms, and the option for the researcher to choose among these depending on their study object, is integral to this great intellectual work.?11 Faleiros F, Käppler C, Pontes FAR, Silva SSC, Goes FSN, Cucik CD. Use of virtual questionnaire and dissemination as a data collection strategy in scientific studies. Texto Contexto Enferm [Internet]. 2016 [cited 2017 Feb 20]; 25(4):e3880014. Available from: http://www.scielo.br/pdf/tce/v25n4/0104-0707-tce-25-04-3880014.pdf
http://www.scielo.br/pdf/tce/v25n4/0104-...

In this context, studies demonstrating successful experiences in the use of widely varying methodological frameworks have assisted researchers in their choices, contributing to the quality of their production, with a view to greater consistency in the method adopted in relation to the research problem and the area investigated. Accordingly, the present study aims to assist researchers in this process of choice, and presents the Delphi method as one option as a methodological framework for research in nursing.

In order to facilitate the operationalization of this method for the reader, we present here a didactic model of its application, with the stages developed in a study undertaken in the area of education in nursing,?22 Massaroli A. O ensino do controle de infecções nos cursos de graduação em enfermagem no Brasil [tese]. Florianópolis (SC): Universidade Federal de Santa Catarina, Programa de Pós-Graduação em Enfermagem; 2016. as well as the Delphi method with the application of the mixed design, articulating the qualitative and quantitative approaches for data collection and analysis.

The text, therefore, is organized didactically in stages, namely: Delphi method - which presents its history and conceptual aspects; the mixed study design - which also develops the conceptual aspects of qualitative and quantitive research; a model of the application of the Delphi method - which presents the operationalization of the method, with a step-by-step description of the collection and analysis of data in a study undertaken through the use of this methodology; and, finally, the conclusions - which elucidate underlying or highlighted aspects of the present article.

Accordingly, this study's objective was to describe the process of collecting and analyzing data, based on the use of the Delphi method and on the articulation of qualitative and quantitative procedures, for constructing a proposal for curricular innovation in the area of infection control for undergraduate courses in nursing.

THE DELPHI METHOD

The term 'Delphi' was inspired by the ancient Oracle of Delphi, which, thousands of years ago, was considered a source of guidance for responding to crucial questions in the lives of the Greeks and Romans, who consulted the oracle in order to define their futures. As these predictions expanded in popularity, people from various places began to consult the oracle on a series of topics of interest, both public and personal, such as the outcome of wars, or the founding of colonies.?33 Oliveira JSP, Costa MM, Wille MFC. Introdução ao método Delphi. Curitiba (BR): Mundo Material; 2008.-44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.

With the growing popularity of the positive predictions received in consultations with the oracle, these became synonymous with judgment on particular issues. As it alluded to the oracle, the name 'Delphi' was strongly criticized because of its mystical connotations, which were felt not to reflect the rigor of the method elaborated.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.

The first records of the use of the Delphi method date from around 1950, the period of the Cold War, when this method for making predictions was first structured, the aim being to determine the possibilities of enemy attacks, as well as the number of atomic bombs necessary for destroying specified targets.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.-55 Linstone H, Turoff M, editors. The Delphi method: techniques and applications [Internet]. 2002. [cited 2016 Mar 15]. Available from: http://is.njit.edu/pubs/delphibook/
http://is.njit.edu/pubs/delphibook/...

Since the Delphi method was developed, its use has undergone a number of expansions, and it has been frequently used by a wide range of disciplines. As a result, the use of the Delphi method has come to be regarded as favorable and reliable for studies which aim to obtain the consensus of a group of specialists regarding a complex problem, or for planning and forecasting for the future of a specified area.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.

5 Linstone H, Turoff M, editors. The Delphi method: techniques and applications [Internet]. 2002. [cited 2016 Mar 15]. Available from: http://is.njit.edu/pubs/delphibook/
http://is.njit.edu/pubs/delphibook/...
-66 Munaretto LF, Corrêa HL, Cunha JAC. A study on the characteristics of the Delphi method and focus group as techniques to obtain data in exploratory research. Rev Admin UFSM [Internet]. 2013 [cited 2016 Mar 16]; 6(1):9-24. Available from: https://periodicos.ufsm.br/reaufsm/article/view/6243/pdf
https://periodicos.ufsm.br/reaufsm/artic...

This process takes place through a systematic communication structure which, controlled by the researcher, allows the experts to receive feedback about the opinions raised, revising their opinions and responding to the points raised by other participants, making it possible that, at the end of the rounds, consensus may be reached on the problem in question.?66 Munaretto LF, Corrêa HL, Cunha JAC. A study on the characteristics of the Delphi method and focus group as techniques to obtain data in exploratory research. Rev Admin UFSM [Internet]. 2013 [cited 2016 Mar 16]; 6(1):9-24. Available from: https://periodicos.ufsm.br/reaufsm/article/view/6243/pdf
https://periodicos.ufsm.br/reaufsm/artic...

In the classic Delphi technique, it is recommended that four rounds be run among the experts. The first should be a questionnaire with open questions, using the qualitative approach, and allowing the participants to express their opinions on a topic, with the technique of content analysis being used for treating these data.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.

For the second round, a further questionnaire is developed based in the responses to the first. This contains quantitative questions and begins the search for consensus between the participants. Statistical techniques should be used for analyzing the data from this second questionnaire. The third and fourth rounds are structured based on the responses to the previous stages, following the same procedure as in the second round for analyzing the data.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.

In the modified Delphi technique, the first round can be made up of focus groups or face-to-face interviews. The data obtained should be analyzed using content analysis. Alternatively, a structured questionnaire may be used, with quantitative questions based in the literature in previous research. Subsequent rounds follow the same process as the classic Delphi.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.

In the area of health, Delphi came to be used from the 1960s onward in studies for defining priority areas for investigation and financing, decision-making on future actions for education and health policies, and for defining clinical conducts.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.

Reports emphasizing this method's effectiveness can be found since Delphi began to be used, describing its potential for investigating varying topics in various areas of knowledge, pointing to the anonymity it affords, the interaction of different people from places near and far, the possibility for rethinking opinions through controlled feedback, and the possibility of finding a path for resolving a problem or defining a consensus.?55 Linstone H, Turoff M, editors. The Delphi method: techniques and applications [Internet]. 2002. [cited 2016 Mar 15]. Available from: http://is.njit.edu/pubs/delphibook/
http://is.njit.edu/pubs/delphibook/...
-66 Munaretto LF, Corrêa HL, Cunha JAC. A study on the characteristics of the Delphi method and focus group as techniques to obtain data in exploratory research. Rev Admin UFSM [Internet]. 2013 [cited 2016 Mar 16]; 6(1):9-24. Available from: https://periodicos.ufsm.br/reaufsm/article/view/6243/pdf
https://periodicos.ufsm.br/reaufsm/artic...

Others, on the other hand, have criticized the Delphi method, stating that it lacks universal guidance regarding how it should be conducted, that the obtaining of consensus between the participants can be influenced by the conduct of the researcher who administrates the technique, or - furthermore - that the group with the strongest opinions can influence participants with differing opinions to ally themselves with the first group's opinion, which may not necessarily represent their position.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.

Upon analyzing the distinct opinions reported in the literature regarding the Delphi method, it can be seen that the main problems relate to the lack of definition on the parameters and criteria to be used during the undertaking of the study. It is emphasized that this can be a problem capable of compromising the validity and reliability of the data of any research method used - and is not a problem exclusive to this method.

MIXED-METHOD DESIGN

'Mixed-method design' refers to the combined use of elements of qualitative and quantitative studies.?77 Tréz TA. Caracterizando o método misto de pesquisa na educação: um continuum entre a abordagem qualitativa e quantitativa. Atos de Pesquisa em Educação [Internet]. 2012 [cited 2017 Feb 20]; 7(4):1132-57. Available from: https://periodicos.ufsm.br/reaufsm/article/view/6243/pdf
https://periodicos.ufsm.br/reaufsm/artic...
The Delphi method is considered a mixed approach as it allows the use of different research strategies for data collection and analysis.

Studies using the mixed-method design have expanded, to cover various areas of knowledge and scientific periodicals; this is because this approach is surrounded by philosophical suppositions which assert that the concomitant or combined use of the qualitative and quantitative methods confers greater strength on a study, as this brings together the strengths of both methods, eliminating the questions and weaknesses which are highlighted through the use of the qualitative and quantitative methods in isolation.?88 Creswell JW. Projeto de pesquisa: métodos qualitativos, quantitativos e mistos. 3ª ed. Porto Alegre (RS): Artmed; 2010.-99 Dal-Farra RA, Lopes PTC. Mixed methods in education: theoretical assumptions Nuances: estudos sobre Educação [Internet]. 2013 [cited 2017 Feb 20]; 24(3):67-80. Available from: http://revista.fct.unesp.br/index.php/Nuances/article/view/2698/2362
http://revista.fct.unesp.br/index.php/Nu...

This methodological approach has been strengthened through the broad and dense network of relations which are found and which have their influence mainly in the context of the social sciences and health sciences - thus requiring different methodological approaches. These methods provide researchers with a greater number of insights, allowing greater understanding on the subject studied,?88 Creswell JW. Projeto de pesquisa: métodos qualitativos, quantitativos e mistos. 3ª ed. Porto Alegre (RS): Artmed; 2010. allowing one to study and understand the phenomenon from different angles, and allowing one to investigate different parts of the whole.

The strengths of the quantitative approach are related to its capacity to operationalize and measure a body of data, proceeding to comparisons and associations between variables. However, it undertakes these procedures without taking into account the context in which these data are involved.?1010 Castro FG, Kellison JG, Boyd SJ, Kopak A. A methodology for conducting integrative mixed methods research and data analyses. J Mix Methods Res [Internet]. 2010 [cited 2017 Feb 20]; 4(4):342-60. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3235529/
https://www.ncbi.nlm.nih.gov/pmc/article...

The qualitative approach allows one to obtain more detailed and in-depth information on the experiences, allowing one to analyze this set of data based on its original context, considering the entire range of beliefs, emotions and values which influence this information. The main weak points of this approach, however, are related to the reliability of the articulation of the data, and to the construction of results obtained when data is collected from different information sources - as well as the difficulty of generalizing from these findings.?1010 Castro FG, Kellison JG, Boyd SJ, Kopak A. A methodology for conducting integrative mixed methods research and data analyses. J Mix Methods Res [Internet]. 2010 [cited 2017 Feb 20]; 4(4):342-60. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3235529/
https://www.ncbi.nlm.nih.gov/pmc/article...

Due to these approaches' characteristics, it is believed that mixed-method designs have great potential for constructing studies with relevance and impact, if the researchers are able to articulate strengths and limitations of both.?99 Dal-Farra RA, Lopes PTC. Mixed methods in education: theoretical assumptions Nuances: estudos sobre Educação [Internet]. 2013 [cited 2017 Feb 20]; 24(3):67-80. Available from: http://revista.fct.unesp.br/index.php/Nuances/article/view/2698/2362
http://revista.fct.unesp.br/index.php/Nu...

A MODEL OF THE APPLICATION OF THE DELPHI METHOD

One example of the use of the Delphi methodology is presented in a study which aimed to answer the following question: how can the teaching of skills for infection control, related to healthcare, be inserted into the curriculums of undergraduate courses in nursing?

The rationale behind this study was the underlying need reported in the Brazilian and international literature to promote transformations in the teaching on undergraduate courses in the health area, with a view to strengthening and consolidating the teaching of control of healthcare-related infections in these professionals' early training. It was submitted to the Committee for Ethics in Research with Human Beings, of the Federal University of Santa Catarina (CEPSH/UFSC) and was approved under Opinion N. 818,839/2014.

Accordingly, the Delphi method was used with the aim of promoting debate and the possibilities for improving the teaching of this topic in the undergraduate nursing courses in Brazil. With the aim of clarifying the methodological path taken, Figure 1 shows all the steps followed for viabilizing this study. The details of the procedures undertaken in each step will be presented later.

Figure 1
Step-by-step explanation of the methodological path undertaken to viabilize the study through the use of the Delphi method.

Once the research problem had been agreed, the possible participants in this study were defined. For this, invitations were sent to professionals with expertise in the area of infection prevention and control, and the characteristics of the Delphi method made it possible to include participants from the various states of Brazil, favoring the construction of data which encompassed the nuances of the Brazilian scenario, as well as the application of this study's results in this territory.?22 Massaroli A. O ensino do controle de infecções nos cursos de graduação em enfermagem no Brasil [tese]. Florianópolis (SC): Universidade Federal de Santa Catarina, Programa de Pós-Graduação em Enfermagem; 2016.

Two of the main characteristics of the Delphi method are the consulting of experts on a specified topic and the anonymity of those participating in the research. The consulting of experts is a critical point, given the complexity of the outlining of the criteria which define or evidence a person's expertise in the study area in order for them to be part of the group of participants.?55 Linstone H, Turoff M, editors. The Delphi method: techniques and applications [Internet]. 2002. [cited 2016 Mar 15]. Available from: http://is.njit.edu/pubs/delphibook/
http://is.njit.edu/pubs/delphibook/...
-66 Munaretto LF, Corrêa HL, Cunha JAC. A study on the characteristics of the Delphi method and focus group as techniques to obtain data in exploratory research. Rev Admin UFSM [Internet]. 2013 [cited 2016 Mar 16]; 6(1):9-24. Available from: https://periodicos.ufsm.br/reaufsm/article/view/6243/pdf
https://periodicos.ufsm.br/reaufsm/artic...

The literature contains suggestions regarding which criteria are relevant to adopt - however, these must be adapted in accordance with the study objective and the public whom the intention is to reach. It is emphasized that it is necessary rigorously to establish the criteria which will guide the choice of these experts, so as to avoid later difficulties in using the Delphi method or in its results.?55 Linstone H, Turoff M, editors. The Delphi method: techniques and applications [Internet]. 2002. [cited 2016 Mar 15]. Available from: http://is.njit.edu/pubs/delphibook/
http://is.njit.edu/pubs/delphibook/...
-66 Munaretto LF, Corrêa HL, Cunha JAC. A study on the characteristics of the Delphi method and focus group as techniques to obtain data in exploratory research. Rev Admin UFSM [Internet]. 2013 [cited 2016 Mar 16]; 6(1):9-24. Available from: https://periodicos.ufsm.br/reaufsm/article/view/6243/pdf
https://periodicos.ufsm.br/reaufsm/artic...

The participants' anonymity is one of the most striking characteristics of the Delphi method, and is one of the advantages it has over the data collection methods, as a participant's position or status cannot influence the group. It is believed that this confidentiality makes it easier for the participants to give and defend their points of view.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.

Two stages were undertaken to define the participants in this study. Initially, a list of possible participants was elaborated, identified based on the following:

- people who presented at the Brazilian Congress for Infection Control and Hospital Epidemiology in 2010, 2012 and 2014. The rationale is that this event is recognized by professionals in this area as being of great scientific importance.

- researchers from Research Groups registered with the Directory of the Research Groups of Brazil who work with the topic of infection control. A search was undertaken in the online directory, identifying all the research groups which contained the terms "infection control" or "control of infection" in their description or name.

- the members of the executive board of the Brazilian Association of Infection Control and Hospital Epidemiology Professionals, from 2011/2012, 2013/2014, 2015/2016. Information available on the above-mentioned association's website was used

- lecturers on the lato sensu postgraduate courses in the area of infection control. Due to the absence of a national catalog of lato sensu courses run, in June 2015 a search was made on the Internet for courses available in Brazil and for lists of lecturers on these courses.

Following that, the Lattes Curricula?11 Faleiros F, Käppler C, Pontes FAR, Silva SSC, Goes FSN, Cucik CD. Use of virtual questionnaire and dissemination as a data collection strategy in scientific studies. Texto Contexto Enferm [Internet]. 2016 [cited 2017 Feb 20]; 25(4):e3880014. Available from: http://www.scielo.br/pdf/tce/v25n4/0104-0707-tce-25-04-3880014.pdf
http://www.scielo.br/pdf/tce/v25n4/0104-...
of all the professionals who made up the list of possible participants were consulted, applying the following criteria for selecting participants for this study: to be a nurse, physician or pharmacist (professionals mentioned as essential members on infection control committees, according to Ministerial Ordinance 1296/1998); to have published an article on the topic (infection control) in a scientific periodical in the last 10 years; to have published an abstract on the topic at a national or international event in the last 10 years; to have been a lecturer on a lato sensu postgraduate course in the area of infection control for over five years; and to have more than 10 years' professional experience in infection control committees or services.

It was established, furthermore, that if the professional was a nurse, she needed to meet only one of the selection criteria, but that if a physician or pharmacist, she needed to meet at least two of the inclusion criteria. This decision was taken due to understanding that professionals from other categories with broad experience in this area could contribute to the discussion on the teaching of nursing in the area of infection control.

After the procedure deciding the criteria for selecting participants, we arrived at the number of 175 professionals who met this study's inclusion criteria. As contact was to be made using email and through contact via the Lattes curriculum, the decision was made to invite all these professionals to participate. Initially, 61 indicated interest in participating in the study, but only 39 confirmed their participation by responding to the first round - these being 31 nurses and eight physicians.

The size of the group of participants in the Delphi method is a critical point, as it is argued that the larger the number of participants, the greater will be the reliability of the results; equally, there is a counterargument that a large number of participants can hinder the breakdown of the analysis of the data, interfering in the perception of the various opinions which will be raised during the process.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.

As a result, the number of participants for beginning data collection was considered adequate, as it met the advice that between 30 and 50 participants are sufficient to establish the debate on the topic and the variety of opinions.?1111 Coutinho SS, Freitas MA, Pereira MJB, Veiga TB, Ferreira M, Mishima SM. Use of Delphi technique in research in the primary health care: integrative review. Rev Baiana Saúde Pública [Internet]. 2013 [cited 2017 Feb 20]; 37(3):582-96. Available from: http://rbsp.sesab.ba.gov.br/index.php/rbsp/article/viewFile/398/pdf_428
http://rbsp.sesab.ba.gov.br/index.php/rb...

Studies on the Delphi method estimate abstention rates varying between 20% and 50% in each round.?1111 Coutinho SS, Freitas MA, Pereira MJB, Veiga TB, Ferreira M, Mishima SM. Use of Delphi technique in research in the primary health care: integrative review. Rev Baiana Saúde Pública [Internet]. 2013 [cited 2017 Feb 20]; 37(3):582-96. Available from: http://rbsp.sesab.ba.gov.br/index.php/rbsp/article/viewFile/398/pdf_428
http://rbsp.sesab.ba.gov.br/index.php/rb...
However, descriptions can be found of studies which achieved response rates above 90% in all of their rounds,?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011. which seems to be related to the participants' interest in the topic or to the strategy used for searches and maintaining contact while the study is being undertaken.

Among the main characteristics of the Delphi method, one is the application of interactive rounds, promoting controlled feedback such that the participants can review their opinions, reflect on them, and either maintain or alter them according to what is raised by the group.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.,66 Munaretto LF, Corrêa HL, Cunha JAC. A study on the characteristics of the Delphi method and focus group as techniques to obtain data in exploratory research. Rev Admin UFSM [Internet]. 2013 [cited 2016 Mar 16]; 6(1):9-24. Available from: https://periodicos.ufsm.br/reaufsm/article/view/6243/pdf
https://periodicos.ufsm.br/reaufsm/artic...

The classic Delphi approach is undertaken using paper, sending out the questionnaires and collecting them either by post or digitally, using email or specific data collection software,?55 Linstone H, Turoff M, editors. The Delphi method: techniques and applications [Internet]. 2002. [cited 2016 Mar 15]. Available from: http://is.njit.edu/pubs/delphibook/
http://is.njit.edu/pubs/delphibook/...
-66 Munaretto LF, Corrêa HL, Cunha JAC. A study on the characteristics of the Delphi method and focus group as techniques to obtain data in exploratory research. Rev Admin UFSM [Internet]. 2013 [cited 2016 Mar 16]; 6(1):9-24. Available from: https://periodicos.ufsm.br/reaufsm/article/view/6243/pdf
https://periodicos.ufsm.br/reaufsm/artic...
,1111 Coutinho SS, Freitas MA, Pereira MJB, Veiga TB, Ferreira M, Mishima SM. Use of Delphi technique in research in the primary health care: integrative review. Rev Baiana Saúde Pública [Internet]. 2013 [cited 2017 Feb 20]; 37(3):582-96. Available from: http://rbsp.sesab.ba.gov.br/index.php/rbsp/article/viewFile/398/pdf_428
http://rbsp.sesab.ba.gov.br/index.php/rb...
this modality being termed 'e-Delphi' by some authors.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.

In the present study, the decision was made to use email, bearing in mind its practicality and its ready availability for use. Initial contact with the participants was made via email using a message created exclusively for this study, using the 'contact' field from the Lattes curriculum, inviting the recipients to participate in the study and requesting the confirmation of the email address to be used for communicating during the data collection process.

Once the participant had confirmed their email address, an email was sent containing the study's letter of presentation, the terms of free and informed consent, and instrument 01, advising the participant to return the terms of free and informed consent and instrument 01 to the same email address.

Care was taken when developing each questionnaire, with a view to clarity and objectivity, so as to avoid phrases with ambiguous meanings or which could 'lead' the participant to a particular response. The time needed to respond to the instrument was also analyzed, so that it could be completed within 30 to 60 minutes - thus avoiding participants' desistance due to the slowness of responding to the questionnaires.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.

After each questionnaire had been elaborated, the same were submitted for a pilot-test prior to implementation with the research participants, so as to ensure the questionnaire's objectivity and clarity. Between three and five nurses participated in this pilot-test in each round. These nurses had experience in the area of care or lecturing, and previous experience in research, bearing in mind the need to evaluate and criticize the questionnaires.

The instrument for beginning the round with the Delphi method was elaborated with open questions, allowing the participants freely to express their opinions regarding the topic, through discursive responses. This instrument contained questions on the infection control-related skills of generalist nurses and nurses who were specialists in infection control, and regarding the teaching of this topic in the undergraduate nursing courses.

Based on the responses in the first round (39 responses), the questionnaire was elaborated for the second round. This contained closed questions, and sought to investigate the level of agreement between the participants regarding the aspects evidenced by the group in the first round. The responses in the second round (35 responses) supported the development of the third instrument, which aimed to clarify the aspects in which consensus had not been reached, or in which the participants had presented a wide variety of responses.

This instrument firstly showed the points where there was divergence of opinion, and then requested the participant to reassess the question, providing the rationale for her position. It was hoped that, in this way, the participants would learn what the group's position was, and that they would have the opportunity to reflect on, and either modify or maintain, their opinion.

The fourth round was structured based on the 30 responses to the third round. It returned the data to the participants and closed this cycle of data collection, sending a message of thanks to the participants and reasserting the commitment to send the final report on the work after the process of the discussion of the data. It is highlighted that the data was returned to all the participants who had handed in the questionnaire from the first round.

Data collection took place over 07 months, from August 2015 to February 2016, with a mean period of 57 days for each round, as shown in Table 1. During the data collection period, periodical contact was maintained with the participants, emphasizing the importance of their collaboration with the study, and reminding them of the time periods for returning the materials, thus seeking to maintain a high level of return from the participants.

Table 1
Duration and number of participants in each round. Florianópolis, SC, Brazil, 2016. (n=39)

A low level of desistance on the part of participants was observed during all stages of the study, in comparison with other studies which used this methodological design.?1111 Coutinho SS, Freitas MA, Pereira MJB, Veiga TB, Ferreira M, Mishima SM. Use of Delphi technique in research in the primary health care: integrative review. Rev Baiana Saúde Pública [Internet]. 2013 [cited 2017 Feb 20]; 37(3):582-96. Available from: http://rbsp.sesab.ba.gov.br/index.php/rbsp/article/viewFile/398/pdf_428
http://rbsp.sesab.ba.gov.br/index.php/rb...
It is believed that this was largely due to the relationship developed between researcher and participant - but mainly to the importance of the topic, and to the need perceived by the professionals who work in the prevention and control of healthcare-related infection to discuss and construct possibilities for strengthening this topic in the undergraduate courses, and to consolidate these skills among the new professionals.

In the Delphi method, analysis takes place simultaneously with data collection, given that the elaboration of the questionnaires for the subsequent rounds is based on the findings of the previous rounds. When the data collection stage was finalized, the final analysis of all the data was effected in a single set of information.

Content analysis was used to analyze the qualitative data present in the first round. Content analysis aims to put into effect logical and justified deductions based on the messages taken for analysis, with the aim of investigating the content of the messages recorded, in search of other realities through these messages.?1212 Bardin L. Análise de conteúdo. 4ª ed. Lisboa (PT): Edições 70; 2010.

Content analysis is divided into three phases:?1212 Bardin L. Análise de conteúdo. 4ª ed. Lisboa (PT): Edições 70; 2010. pre-analysis; the exploration of the material, and the treatment of the results; inference and interpretation. Pre-analysis consists of the organization of all the material which will be subjected to analysis, seeking to operationalize and systematize the initial ideas in a flexible way, leading to a plan of analysis. This first phase's objective is to choose the documents to be analyzed, formulate hypotheses and objectives, and elaborate indicators which will form a basis for the final interpretation - although these stages do not necessarily follow this developmental sequence.

The exploration of the material encompasses the development of operations for codifying and breaking down the data, through previously-formulated rules. If the pre-analysis activities were concluded thoroughly, this stage consists only of the systematic application of the decisions taken.

In the stage of inference and interpretation, the raw data obtained in the previous stages come to be significant and valid, making it possible to establish tables of results, diagrams and models, which condense and highlight the information provided by the analysis. Based on these results, it was possible to make inferences and suggest interpretations compatible with the objectives proposed at the start of the study, or, moreover, inferences regarding unexpected discoveries, supporting the elaboration of the questionnaire for the second round.

In order to analyze the quantitative data, that is, the data from the second and third rounds, the researchers used descriptive statistical analysis through medians and the coefficient of variation. Descriptive statistics is used for describing the data through use of the mean, median and percentages, and uses tables and graphs for presenting the statistical results.?88 Creswell JW. Projeto de pesquisa: métodos qualitativos, quantitativos e mistos. 3ª ed. Porto Alegre (RS): Artmed; 2010. This process was undertaken with the support of a professional from the area of statistics.

In the literature on this method, there is no specific guidance regarding what level of consensus is to be considered as satisfactory when using the Delphi method.?44 Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011. Because of this, in this study, the decision was made to use the median of 5 with a coefficient of variation of up to 20% as the cut-off point for concluding there to be consensus on a specified question.

Figure 2 illustrates the movement developed in the operationalization of the Delphi method, as well as the articulation of the qualitative and quantitative approaches.

Figure 2
The movement developed in the operationalization of the Delphi method and the articulation of the qualitative and quantitative approaches

Finally, the debate regarding the teaching of infection control and the structuring for a proposal for curricular innovation in this area, through the use of a mixed method, was shown to be viable and productive due to the possibility of using distinct structures of instruments in order to promote interaction within a group of experts from separate places and with differing experiences.

It is emphasized that the use of the Delphi method as a methodological framework becomes important because it promotes a dynamic process of data collection and analysis, and allows - through controlled feedback - the participants to rethink and modify their opinions based on the opinions of the other participants, culminating in achieving consensus on the problem in question.

CONCLUSION

The Delphi method evidences its power as a means of articulating the qualitative and quantitative approaches, bringing the detail and subjectivity of the information of the first approach together with the objectivity and capacity to measure made viable by the second.

Regarding the main characteristics of the mixed method, emphasis may be placed on the methodological and paradigmatic variety, and the focus on the issue of research in choosing the methodological outline. It is ascertained that, based on the Delphi method, it is possible to articulate these characteristics in order to work on a complex research problem in the area of nursing and health.

Since the Delphi method appeared, many versions of structures for its operationalization as a research method have been developed and applied. One of its major advantages is that the structure of the Delphi method, through the interactive rounds with the participants, allows distinct methods to be adopted for collecting and analyzing the data obtained in each round, aiming to gain a greater understanding of the problem in question, and to obtain better findings in order to resolve it.

The possibility of using the Delphi method for promoting the debate within a group of experts in the area of infection control, regarding new possibilities for inserting this topic in the nursing curriculum, is evaluated as a highly productive opportunity, given the possibility for articulating open instruments which allow the free expression of the participants' opinions, followed by a closed instrument which allows the objectification of the study's findings, permitting the concomitant use of the two approaches so as to gain an in-depth understanding of widely-varying issues which are polemical within the group of participants.

  • 1
    Article taken from the thesis - The teaching of infection control in undergraduate courses in nursing in Brazil, presented to the Postgraduate Program in Nursing (PEN), Universidade Federal de Santa Catarina (UFSC), in 2016.

Acknowledgments

To the Brazilian National Council for Scientific and Technological Development (CNPq) for their encouragement in the form of a grant.

To EDEN - Laboratory for Research in Education in Nursing and Health - UFSC - for their support in the undertaking of the present study.

REFERÊNCIAS

  • 1
    Faleiros F, Käppler C, Pontes FAR, Silva SSC, Goes FSN, Cucik CD. Use of virtual questionnaire and dissemination as a data collection strategy in scientific studies. Texto Contexto Enferm [Internet]. 2016 [cited 2017 Feb 20]; 25(4):e3880014. Available from: http://www.scielo.br/pdf/tce/v25n4/0104-0707-tce-25-04-3880014.pdf
    » http://www.scielo.br/pdf/tce/v25n4/0104-0707-tce-25-04-3880014.pdf
  • 2
    Massaroli A. O ensino do controle de infecções nos cursos de graduação em enfermagem no Brasil [tese]. Florianópolis (SC): Universidade Federal de Santa Catarina, Programa de Pós-Graduação em Enfermagem; 2016.
  • 3
    Oliveira JSP, Costa MM, Wille MFC. Introdução ao método Delphi. Curitiba (BR): Mundo Material; 2008.
  • 4
    Keeney S, Hasson F, McKenna H. The Delphi technique in nursing and health research. Oxford (UK): Wiley Blackwell; 2011.
  • 5
    Linstone H, Turoff M, editors. The Delphi method: techniques and applications [Internet]. 2002. [cited 2016 Mar 15]. Available from: http://is.njit.edu/pubs/delphibook/
    » http://is.njit.edu/pubs/delphibook/
  • 6
    Munaretto LF, Corrêa HL, Cunha JAC. A study on the characteristics of the Delphi method and focus group as techniques to obtain data in exploratory research. Rev Admin UFSM [Internet]. 2013 [cited 2016 Mar 16]; 6(1):9-24. Available from: https://periodicos.ufsm.br/reaufsm/article/view/6243/pdf
    » https://periodicos.ufsm.br/reaufsm/article/view/6243/pdf
  • 7
    Tréz TA. Caracterizando o método misto de pesquisa na educação: um continuum entre a abordagem qualitativa e quantitativa. Atos de Pesquisa em Educação [Internet]. 2012 [cited 2017 Feb 20]; 7(4):1132-57. Available from: https://periodicos.ufsm.br/reaufsm/article/view/6243/pdf
    » https://periodicos.ufsm.br/reaufsm/article/view/6243/pdf
  • 8
    Creswell JW. Projeto de pesquisa: métodos qualitativos, quantitativos e mistos. 3ª ed. Porto Alegre (RS): Artmed; 2010.
  • 9
    Dal-Farra RA, Lopes PTC. Mixed methods in education: theoretical assumptions Nuances: estudos sobre Educação [Internet]. 2013 [cited 2017 Feb 20]; 24(3):67-80. Available from: http://revista.fct.unesp.br/index.php/Nuances/article/view/2698/2362
    » http://revista.fct.unesp.br/index.php/Nuances/article/view/2698/2362
  • 10
    Castro FG, Kellison JG, Boyd SJ, Kopak A. A methodology for conducting integrative mixed methods research and data analyses. J Mix Methods Res [Internet]. 2010 [cited 2017 Feb 20]; 4(4):342-60. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3235529/
    » https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3235529/
  • 11
    Coutinho SS, Freitas MA, Pereira MJB, Veiga TB, Ferreira M, Mishima SM. Use of Delphi technique in research in the primary health care: integrative review. Rev Baiana Saúde Pública [Internet]. 2013 [cited 2017 Feb 20]; 37(3):582-96. Available from: http://rbsp.sesab.ba.gov.br/index.php/rbsp/article/viewFile/398/pdf_428
    » http://rbsp.sesab.ba.gov.br/index.php/rbsp/article/viewFile/398/pdf_428
  • 12
    Bardin L. Análise de conteúdo. 4ª ed. Lisboa (PT): Edições 70; 2010.

Publication Dates

  • Publication in this collection
    2017

History

  • Received
    03 Mar 2017
  • Accepted
    28 Aug 2017
Universidade Federal de Santa Catarina, Programa de Pós Graduação em Enfermagem Campus Universitário Trindade, 88040-970 Florianópolis - Santa Catarina - Brasil, Tel.: (55 48) 3721-4915 / (55 48) 3721-9043 - Florianópolis - SC - Brazil
E-mail: textoecontexto@contato.ufsc.br