Contribution of mobile applications to the high-fidelity clinical simulation in nursing: an integrative review

Abstract

Objective

To identify the contributions of mobile applications to the design or conduct of high-fidelity clinical simulation in Nursing in order to promote and facilitate the use of simulation as an educational strategy.

Methods

Integrative review of scientific studies published in indexed databases: BDENF, CINAHL, LILACS, MEDLINE/PubMed, Scopus, Web of Science, and Cochrane and SciELO libraries, guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Studies in English, Portuguese or Spanish were included, available in full and without time restrictions. The search yielded 320 studies and, after reviewing full texts and analyzed data, four studies were selected that met the research objective.

Results

Four mobile applications were found that can contribute to high-fidelity clinical simulation activities in Nursing, however one is still under development. Two applications are available for public access, and only one was submitted to validity, reliability and usability tests in the study. The findings showed that three applications can contribute to simulation facilitation, and only one contributes to the simulation design.

Conclusion

Improving the development of a simulation strategy requires practical and easily accessible tools to assist the faculty. Thus, the research allowed the identification of mobile applications contributions to simulation design and facilitation, however, the limited number of studies found and the lack of methodological rigor description and evaluation tests in the development process, indicates the gap of available and evidence-based mobile applications that directly contribute to the needs of simulation providers, which could guide them to develop an effective learning experience.

Health education; Simulation; Mobile applications; Evidence-based practice; Nursing

Resumo

Objetivo

identificar as contribuições de aplicativos móveis para o design ou condução de simulação clínica de alta fidelidade em enfermagem, a fim de promover e facilitar o uso da simulação como estratégia educacional.

Métodos

revisão integrativa de estudos científicos publicados nas bases de dados indexadas BDENF, CINAHL, LILACS, MEDLINE/PubMed, Scopus, Web of Science e bibliotecas Cochrane e SciELO, guiada pelo Preferred Reporting Items for Systematic Reviews and Meta-Analyses . Foram incluídos estudos em inglês, português ou espanhol, disponíveis na íntegra e sem restrições de tempo. A busca resultou em 320 estudos e, após revisão dos textos completos e análise dos dados, foram selecionados quatro estudos que atenderam ao objetivo da pesquisa.

Resultados

foram encontrados quatro aplicativos móveis que podem contribuir com atividades de simulação clínica de alta fidelidade em enfermagem, porém um ainda está em desenvolvimento. Dois aplicativos estão disponíveis para acesso público, sendo que apenas um foi submetido a testes de validade, confiabilidade e usabilidade no estudo. Os resultados mostraram que três aplicativos podem contribuir para a facilitação da simulação e apenas um contribui para o design da simulação.

Conclusão

aprimorar o desenvolvimento de uma estratégia de simulação requer ferramentas práticas e de fácil acesso para auxiliar o corpo docente. Assim, a pesquisa permitiu a identificação de contribuições de aplicativos móveis para o design de simulação e facilitação, mas o número limitado de estudos encontrados e a falta de descrição de rigor metodológico e testes de avaliação no processo de desenvolvimento indicam a lacuna de aplicativos móveis disponíveis e baseados em evidências que contribuam diretamente com as necessidades dos provedores de simulação, o que poderia orientá-los a desenvolver uma experiência de aprendizado eficaz.

Educação em saúde; Simulação; Aplicativos móveis; Prática clínica baseada em evidências; Enfermagem

Resumen

Objetivo

Identificar las contribuciones de aplicaciones móviles para el diseño o conducción de la simulación clínica de alta fidelidad en enfermería, a fin de promover y facilitar el uso de la simulación como estrategia educativa.

Métodos

Revisión integradora de estudios científicos publicados en las bases de datos indexadas BDENF, CINAHL, LILACS, MEDLINE/PubMed, Scopus, Web of Science y bibliotecas Cochrane y SciELO, guiada por el Preferred Reporting Items for Systematic Reviews and Meta-Analyses . Se incluyeron estudios en inglés, portugués y español, con texto completo disponible y sin restricciones de tiempo. La búsqueda dio como resultado 320 estudios, y luego de la revisión de los textos completos y el análisis de los datos, se seleccionaron cuatro estudios que cumplían el objetivo de la investigación.

Resultados

Se encontraron cuatro aplicaciones móviles que pueden contribuir con actividades de simulación clínica de alta fidelidad en enfermería, pero una todavía está siendo elaborada. Dos aplicaciones están disponibles para acceso público, de las cuales solo una fue sometida a pruebas de validez, fiabilidad y usabilidad en el estudio. Los resultados mostraron que tres aplicaciones pueden contribuir para facilitar la simulación y solo una contribuye al diseño de la simulación.

Conclusión

Mejorar el desarrollo de una estrategia de simulación requiere herramientas prácticas y de fácil acceso para ayudar al cuerpo docente. De esta forma, este estudio permitió identificar las contribuciones de aplicaciones móviles para el diseño de simulación y facilitación, pero el número limitado de estudios encontrados y la falta de descripción de rigor metodológico y pruebas de evaluación en el proceso de elaboración indican el vacío de aplicaciones móviles disponibles y basadas en evidencias que puedan contribuir directamente con las necesidades de proveedores de simulación, o que puedan orientarlos a desarrollar una experiencia eficaz de aprendizaje.

Educación en salud; Simulación; Aplicaciones móviles; Práctica clínica basada en la evidencia; Enfermería

Introduction

The use of simulation as a teaching strategy for different levels of nursing education is strongly recommended by the World Health Organization, as it improves the quality of teaching, health care and patient safety. In this way, it benefits patients, students and professionals. ( ¹ ) Furthermore, it is essential to develop evidence-based practice standards, and identify and share best practices in designing, implementing, conducting, and evaluating simulation activities as well as promoting simulation research to advance its science as a teaching methodology. ( ² )

Simulation activities should be integrated throughout the academic curriculum and match the student’s level to achieve a successful program. Support starts with management and must include all faculty members, who must be properly prepared to conduct the simulations and constantly trained in simulation-based education. The simulation program must be periodically reviewed and evaluated to be effective after its implementation. ( ³ , ⁴ )

Despite international recommendations and evidence of benefits in the use of simulation in education, some institutions have difficulty integrating simulation into the academic curriculum. Among the challenges, in addition to the high cost of simulators and equipment, there is the need for continuous maintenance, replacement of materials, adequate infrastructure and qualified personnel for resource management. ( ¹ , ⁵ )

Another fact that may impact the adoption of simulation is that high-fidelity clinical simulation refers to simulation experiences that are incredibly realistic and provide a high level of interactivity for the learner. ( ⁶ ) The complexity of this simulation modality requires a higher proficiency and understanding by teachers and researchers in the area along with the ability to provide quality simulations, in addition to technical knowledge of equipment and technological resources. ( ⁵ )

One way to facilitate the adoption of simulation by teachers is the use of easily accessible tools which can assist them in developing a simulation strategy in an organized and standardized way, such as the mobile applications. Popularly known as app, it has been standing out worldwide for being accessible and relatively low cost. App is a set of tools designed to perform specific tasks, and it can be accessed by electronic devices such as smartphones and tablets, ( ⁷ ) they enable people to access information and get knowledge without time and space restrictions, creating new communication methods. ( ⁸ )

The rapid dissemination of these apps worldwide has contributed to the democratization of information, with prospects of becoming one of the central consultation guides on health care. Besides, they bring a new way for health professionals to communicate with the population because, unlike other media, it allows greater interactivity with the user. ( ⁹ ) The information generated by apps can be used to understand the determining factors that promote health, leading to the consequent reduction in health risks. ( ¹⁰ )

Apps are also being used at an increasing pace in higher education institutions, showing possibilities for better learning experiences, mainly because of their mobility aspect, enabling students and professors to exchange information anywhere and anytime. As shown in a study ( ¹¹ ) with 212 students, an app developed had a positive influence on the learning outcomes of the students, where the experimental group who used this app secured a higher score in the post-test with a statistically significant difference (p < 0,001) between the control group.

With all this potential of apps to enhance and innovate teaching methodologies, they can also be used to help health educators to implement high-fidelity simulation in the academic curriculum. Hence, the present study aimed to identify the contributions of mobile applications to the design or conduct of high-fidelity clinical simulation in Nursing. This study is relevant, as it allowed to verify whether the available apps are reliable, evidence-based and if they present consistent information. It also allowed an analysis of the knowledge already produced on the topic by searching for international studies in indexed databases.

Methods

The study presented is an integrative review of the literature of scientific studies published in indexed databases guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). ( ¹² ) An integrative review is a method that allows the synthesis of knowledge and the incorporation of the results of significant studies into practice. ( ¹³ )

This review consisted of a protocol with 6 steps: formulation of the research question; definition of inclusion and exclusion criteria; definition of the information extracted from the selected studies; analysis of included studies; interpretation and discussion of results; and synthesis of the results and presentation of the review. ( ¹² , ¹³ )

The PICO strategy ( ¹⁴ ) was used to formulate the research question, where Population (P): Nursing students or professionals; Intervention (I): designing or conducting a high-fidelity clinical simulation; Comparison/Control (C): not applicable; and Outcome (O): mobile apps contributions. Thus, the question for this review was as follows: What are the contributions of mobile applications to the design or conduct of high-fidelity clinical simulation in Nursing?

To meet the research objective, publications were identified from initial searches in the following electronic databases: Base de Dados de Enfermagem (BDENF), Cumulative Index to Nursing and Allied Health Literature (CINAHL), Latin American and Caribbean Health Sciences Literature (LILACS), Medical Literature Analysis and Retrieval System Online (MEDLINE)/PubMed, Scopus, Web of Science, and in the Cochrane and Scientific Electronic Library Online (SciELO) libraries.

Four combined search terms were used from Health Sciences Descriptors in English and Portuguese: Nursing”, “Health”, “Mobile Applications” and “Simulation”. To obtain the largest number of studies, AND and OR Boolean expressions were used, and the search terms in the singular and plural. The search was conducted in January 2020 by a researcher and it was supported by a librarian, considering the following strategy: (Nursing OR Health) AND (Simulation OR Simulations) AND (“Mobile application” OR “Mobile applications”) in MEDLINE/PubMed, Scopus, Web of Science, CINAHL and SciELO; and (Nursing OR Enfermagem OR Health Or Saúde) AND (Simulation OR Simulação OR Simulations OR Simulações) AND (“Mobile application” OR “Aplicativo móvel” OR “Mobile applications” OR “Aplicativos móveis) in BDENF, LILACS, and Cochrane.

The inclusion criteria applied were: studies in English, Portuguese or Spanish, available in full, without time restrictions, and that described mobile applications contributions to the design or conduct of high-fidelity clinical simulation in Nursing for students, professionals, or for other Health areas that may also be suitable for Nursing due to their similar characteristics. The exclusion criteria were: studies that addressed different types of simulation: virtual reality, games, training of technical skills or medical procedures, or studies of learning assessment; and repeated studies.

Figure 1 presents the selection and inclusion of studies in this review according to the PRISMA 2020 flow diagram. ( ¹² ) Initially, the search yielded 320 studies. From that, 105 duplicates were excluded by submitting the material to the Mendeley ^® bibliographic management tool. Thereafter, the inclusion and exclusion criteria were applied by carefully reading the titles and abstracts of the 215 remaining articles. After this approach, four studies were selected. The full-text papers were analyzed by two researchers, independently, and after reaching a consensus among them, these four articles met the research objective and were included as a final result.

Figure 1
Selection and inclusion of studies in the review

The relevant information from the selected articles was included in a data collection instrument created from an electronic spreadsheet in the Microsoft Excel ^® program: journal, author, title, country, year, language, objective, method and result of the study, app description and its contributions to the high-fidelity clinical simulation in Nursing, and the app target audience. The main contributions of the apps were compared and grouped by similarity of content in two categories for analysis: design and facilitation. The level of evidence of the studies was established according to the hierarchical classification of the Oxford Centre for Evidence-based Medicine ( ¹⁵ ): level 1, systematic review of randomized controlled trials, randomized controlled trial with narrow confidence interval, or all or none case series; level 2, systematic review of cohort studies, individual cohort studies, low quality randomized controlled trials, outcomes research, or ecological studies; level 3, systematic review of case-control, or individual case-control studies; level 4, case-series, or poor quality cohort and case-control studies; level 5, opinion of experts. The results found were then discussed in light of the scientific literature.

Results

The selected articles were numbered from 1 to 4, and their main characteristics were described in chart 1 . The publications were from 2018 and 2019. The levels of evidence ranged from level 2 to 5. All studies were in English from international journals: BMJ Paediatr Open (1), Simul Healthc (2), BMJ Simul Technol Enhanc Learn (3) and Stud Health Technol Inform (4), whose countries of origin, respectively, are: Uganda, Canada, Netherlands and Norway; where MEDLINE/PubMed and Scopus databases presented the highest number of selected articles (n = 2), followed by Web of Science (n=1) and CINAHL (n=1).

Chart 1
Synthesis of selected articles with the main contributions of the mobile applications to high-fidelity clinical simulation in Nursing

Article	Objective	Methods	Main results	Level of evidence	Mobile app	App target audience	App contributions to simulation
1 ( 16 )	Develop a mobile application (HBB prompt) to help providers maintain HBB knowledge and skills.	Comparative study protocol in two phases: Phase I of development - data collection for the HBB Prompt prototype through discussions in focus groups of the HBB providers. Usability and functionality tests. Phase II of evaluation - a minimum of 20 health professionals from 2 hospitals will receive HBB training. Only one hospital (intervention) will have access to the app.	The primary result will be to compare the skill retention of the 2 groups (intervention and control) over 12 months after training.	3	HBB Prompt (still being developed)	Medical and nursing professionals who perform neonatal care	Facilitation: helps participants retain training knowledge and skills
2 ( 17 )	Determine whether the NRP Prompt app can help NRP novice training providers to learn more about the NRP algorithm effectively and therefore improve its performance.	Randomized controlled pilot study, where 39 participants of NRP training were randomized into intervention group using NRP Prompt app (n=18) and control using only visual aids (n=21) in two simulated neonatal resuscitation training sessions.	The performance score of the groups in the training showed no significant difference (p =.69), using the app did not improve the performance of the participants.	2	NRP Prompt	Medical, nursing and physical therapy health professionals who perform neonatal care	Facilitation: provides audiovisual aids of the neonatal resuscitation program algorithm to the participants during the simulation
3 ( 18 )	Develop an objective and evidence-based assessment tool for instructional design of simulation-based team training courses.	Validation study that developed an assessment tool composed of a questionnaire and a visual chart with its results (ID-SIM app). Validity and reliability analyses were performed by using the scores from raters (n=10) and comparing them with one expert’s ranking. Usability was assessed by an 11-item survey.	The correlation (Spearman) between the expert-opinion-based ranking and the evaluators’scores was 0.95, and the variance due to subjectivity of raters was 3.5%. The G-coefficient was 0.96 and the inter-rater reliability (intraclass correlation coefficient) was 0.91. ID-SIM app was shown to be a reliable assessment tool.	4	ID-SIM	Educators on simulation-based team training courses	Design: evaluation of the instructional design of a simulation-based team training
4 ( 19 )	Introduce healthcare scenarios and technology-supported research methods for young students. Present an app used to guide the flow of tasks in the simulation of eHealth services.	The study presented the description of the simulation of eHealth scenarios in a clinical research laboratory with participation of high school students (n=40) who carried out eHealth scenarios as a role-play together with experienced researchers (n=7); and how the app was developed and used to guide the role-play.	The simulation approach of eHealth services in a clinical laboratory by high school students and researchers provided students with experience in real-life situations and showed how technology can be used, through the development of an interactive smartphone application to guide the task flow, which was developed as a basic web app that can be accessed on any device with a web browser.	5	eHealth role-play	Students and health professional	Facilitation: guides actors and standardized patients regarding the performance of their roles during the simulation scenario

• HBB - Helping Babies Breathe; NRP - neonatal resuscitation training

With the analysis of these publications, four apps were found that met the objective of this review, however one is still in the elaboration phase. Only two apps are available for public access: NRP Prompt and ID-SIM. The latter was the only app referred to in the study that was submitted to validity, reliability and usability tests. As described in chart 1 , the main contributions of the apps to simulation have been classified into two categories: facilitation and design, where three apps (75%) contribute to simulation facilitation, and only one (25%) contributes to the simulation design. Regarding the mobile apps attributes, article 1 sought to develop an app to help providers maintain the knowledge and skills of Helping Babies Breathe (HBB), a simulation-based neonatal resuscitation training program designed by the American Academy of Pediatrics for resource-poor settings, which helps to reduce early neonatal mortality. The HBB Prompt app is still being developed and will undergo usability and feasibility tests before being deployed. ( ¹⁶ )

Article 2 described the NRP Prompt app created by experienced Neonatal Resuscitation Program (NRP) providers to help novice providers learn more about the NRP algorithm. This interactive app from iOS Apple ^® provides audiovisual prompts based on user responses at decision points in the NRP algorithm with audio narration of each step, displaying the time in minutes and seconds since birth and a 30-second countdown timer for the current stage. This app is not intended for use in clinical practice; it is for education purposes only. No usability tests were performed. ( ¹⁷ ) Article 3 developed an objective assessment tool for the instructional design of simulation-based team training courses. This free mobile app ID-SIM is evidence-based and reliable, it consists of two parts: a questionnaire and a visual chart. The answers to the questionnaire are converted to the visual chart that clearly identifies the strengths and weaknesses of the applied instructional design, through scores attributed to each element, in a total of ten items evaluated. Tests of validity, reliability and usability were performed. ( ¹⁸ ) As a part of the leading project that aims to introduce technology-supported health care scenarios and research methods to young students in the Southern region of Norway, article 4 presented the eHealth role-play app; an interactive app developed and used to inform the roles, the tasks and their associated actions in order to guide the scenarios. The user can select a scenario and its role on a screen that displays information about the functions that will be performed as the simulation progresses. The study does not provide information on feasibility or usability tests. ( ¹⁹ )

Discussion

Nowadays, students have a set of characteristics, expectations and particular preferences in their learning, as they are used to easy access to communication technologies. Therefore, developing interactive teaching strategies that use technology is described as the most appropriate for this generation. In this way, a simulation curriculum enhanced with mobile technology helps transfer learned skills to clinical practice. ( ²⁰ )

The use of mobile technology in simulation offers flexibility in accessing teaching materials, encouraging students to pursue their learning. The portable devices allow the teacher to provide immediate feedback during the simulation, guiding the participants in meeting the learning objectives. However, one of the most significant challenges for a simulation educator is to be prepared to meet the student’s expectations regarding the incorporation of technology in the simulation. ( ²⁰ )

Recognizing the complexity of this challenge, this literature review sought to find tools that could help educators in this crucial task, specifically to assist them in their direct role as facilitators of a simulation experience, creating and conducting simulation scenarios to achieve the learning objectives. All simulation-based experiences require a systematic, flexible and cyclical planning. The standardized simulation design provides a framework for developing compelling simulation-based experiences. Besides, it is recommended to perform a pilot test before the implementation of each simulation and constant evaluations of the activities to ensure its success. ( ²¹ ) The ID-SIM app identified in this search meets this recommendation, as it allows the assessment of the instructional design of the simulation.

Facilitation requires an effective facilitator who can guide and support participants to achieve the expected results, assuming the role of educator in the simulation. Facilitation methods involve creating and maintaining scenario fidelity, providing tips to guide participants, managing unexpected events, targeting actors or standard patients in their roles, and using technological resources. ( ²² , ²³ ) In this way, HBB Prompt, NRP Prompt, and eHealth role-play apps have functions that can contribute to facilitation.

These three apps found were used to help, provide tips and instruct participants and actors during the simulation, and also to assist the participant in retaining knowledge. Although these applications contribute to the facilitation, none intended to guide or assist the facilitator in conducting the simulation directly. And the application that contributes to the design provides only an evaluation of the instructional design of the simulation already created, and the facilitator is still responsible for analyzing this information to improve future simulations.

The study by Gambo et al. ( ²⁰ ) reiterates that although the use of mobile devices facilitates students’ access to relevant information on pathophysiology and nursing practice and also provides audiovisual cues and tips to participants in the simulation, evidence is still lacking in the literature to support the integration of mobile technology for other phases of the simulation, including simulation preparation, prebriefing, simulation scenario and debriefing. Even though the analysis of the publications showed that the apps contribute to simulation design and facilitation, this gap was also identified in this literature review.

Among the countries of origin, Brazilian studies were not identified in this review, although Brazil is currently the country with the highest amount of daily time spent in apps according to a survey carried out by the App Annie agency focused on analyzing the mobile market. The report was based on the results of the second quarter of 2021 and according to the data, the average use of apps by Brazilians is 5.4 hours per day, followed by Indonesia with 5.3 hours. Japan comes in 7th in the ranking with 4.4 hours, Canada in 8th with 4.1 hours and United States of America in 9th with 3.9 hours, while China comes in 16th with 3.1 hours. ( ²⁴ )

The evaluation is another relevant aspect regarding apps development, as the successful implementation of an information system depends on its acceptance by users. There are several metrics, but a well-known predictor for use of a technology it is the usability. The users’ needs must be constantly monitored and validated by trained professionals, and the development and application of these media must be supported by scientific research and rigorous methodologies. ( ²⁵ - ²⁷ )

From the four apps analyzed, during their development process, two reported a comparative study with intervention and control groups to evaluate the applications, however, only one has already completed this stage. One app has had usability test applied, and another will undergo usability and feasibility testing before being deployed. Only one study stated that validity and reliability analyzes were performed. There was one study that did not provide any information on whether tests were performed.

This lack of scientific basis and methodological rigor in the development of apps is a topic that has already been discussed by researchers in the area, especially for mobile Health applications due to their rapid dissemination, even though they are not yet regulated in many countries and without validation of their contents. ( ⁹ ) This problem was also evidenced in other review studies on available mobile Health apps, which indicated gaps that need to be filled in terms of information quality, usability, acceptability and effectiveness, and whether the construction of apps was based on scientific literature, and which suggest that apps development should offer users the possibility to evaluate its usability. ( ²⁸ , ²⁹ )

As a limitation of the present study, although apps are available from other search sources, it was decided to only search for scientific studies published in indexed databases to ensure quality and evidence-based results.

Conclusion

In this digital age, the use of mobile electronic devices is increasing and innovating teaching methodologies, and making access to information faster and easier. Besides, improving the development of a simulation strategy in an standardized way requires practical and easily accessible tools to assist the faculty. For this reason, it is necessary to seek apps that help health educators to implement the simulation. Therefore, this integrative review allowed the identification of the contributions of three available mobile applications and one under development for the design and facilitation of high-fidelity clinical simulation in Nursing. The year of publication of the scientific production analyzed shows the recent approach of the topic by the scientific community. The limited number of studies found in this review and the lack of methodological rigor and evaluation tests described in the development of apps, indicates the gap of available and evidence-based mobile apps that directly contribute to the needs of simulation providers, as well as for other simulation phases such as preparation, prebriefing, simulation scenario and debriefing. This shows the demand for app development studies with a straightforward protocol that can guide professors step by step to create and conduct a high-fidelity clinical simulation, leading to a practical learning experience. In this way, educators and simulation researchers need to take the lead in implementing new mobile applications and technologies and support their use in nursing education and simulation strategies, and thereby allow for future comparisons and researches on the topic.

Referências

• ¹
  Martins JC , Mazzo A , Baptista RC , Coutinho VR , Fernandes MI , Fernandes AM . Simulation in nursing and midwifery education Simulation in nursing and midwifery education . Copenhagen : WHO Regional Office for Europe ; 2018 [ cited 2020 Mar 3 ]. Available from http://www.euro.who.int/__data/assets/pdf_file/0011/383807/snme-report-eng.pdf?ua=1
  » http://www.euro.who.int/__data/assets/pdf_file/0011/383807/snme-report-eng.pdf?ua=1
• ²
  McMahon E , Jimenez FA , Lawrence K , Victor J ; INACSL Standards Committee . Healthcare Simulation Standards of Best PracticeTM . Clin Simul Nurs . 2021 ; 58 : 66 .
• ³
  Hallmark B , Brown M , Peterson DT , Decker S , Wells-Beede E , Britt T , Hardie L , Shum C , Arantes HP , Charnetski M , Morse C ; INACSL Standards Committee . Healthcare Simulation Standards of Best PracticeTM Professional Development . Clin Simul Nurs . 2021 ; 58 : 5 – 8 .
• ⁴
  Moran V , Wunderlich R , Rubbelk C . Simulation: best practices in nursing education . Cham : Springer ; 2018 .
• ⁵
  Campbell SH , Nye C , Hébert SH , Short C , Thomas MH . Simulation as a disruptive innovation in advanced practice nursing programs: a report from a qualitative examination . Clin Simul Nurs . 2021 ; 61 : 79 – 85 .
• ⁶
  Lioce L ; Society for Simulation in Healthcare . Healthcare Simulation Dictionary . 2 nd ed. Rockville : AHRQ ; 2020 [ cited 2022 Nov 1 ]. Available from: https://doi.org/10.23970/simulationv2
  » https://doi.org/10.23970/simulationv2
• ⁷
  Barra DC , Paim SM , Dal Sasso GT , Colla GW . Methods for developing mobile apps in health: an integrative review of the literature . Texto Contexto Enferm . 2017 ; 26 ( 4 ): 1 – 12 . Review .
• ⁸
  Keengwe J , Bhargava M . Mobile learning and integration of mobile technologies in education . Educ Inf Technol . 2014 ; 19 ( 4 ): 737 – 46 .
• ⁹
  Grundy QH , Wang Z , Bero LA . Challenges in Assessing Mobile Health App Quality: a Systematic Review of Prevalent and Innovative Methods . Am J Prev Med . 2016 ; 51 ( 6 ): 1051 - 9 . Review .
• ¹⁰
  Marcano Belisario JS , Jamsek J , Huckvale K , O’Donoghue J , Morrison CP , Car J . Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods . Cochrane Database Syst Rev . 2015 ; 2015 ( 7 ): MR000042 . Review .
• ¹¹
  Arain AA , Hussain Z , Rizvi WH , Vighio MS . An analysis of the influence of a mobile learning application on the learning outcomes of higher education students . Univers Access Inf Soc . 2018 ; 17 ( 2 ): 325 – 34 .
• ¹²
  Page MJ , McKenzie JE , Bossuyt PM , Boutron I , Hoffmann TC , Mulrow CD , et al . The PRISMA 2020 statement: an updated guideline for reporting systematic reviews . BMJ . 2021 ; 372 : n71 . Review .
• ¹³
  Sousa LM , Marques-Vieira CM , Severino SS , Antunes AV . Metodologia de revisão integrativa da literatura em enfermagem . Rev Investigação Enfermagem . 2017 ; 2 ( 21 ): 17 – 26 .
• ¹⁴
  da Costa Santos CM , de Mattos Pimenta CA , Nobre MR . The PICO strategy for the research question construction and evidence search . Rev Lat Am Enfermagem . 2007 ; 15 ( 3 ): 508 - 11 .
• ¹⁵
  University of Oxford . Centre for Evidence-Based Medicine (CEBM) . Oxford Centre for Evidence-Based Medicine. Levels of Evidence . Oxford : University of Oxford ; 2009 [ cited 2020 Mar 16 ]. Available from: https://www.cebm.net/2009/06/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/
  » https://www.cebm.net/2009/06/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/
• ¹⁶
  Merali HS , Chan NH , Mistry N , Kealey R , Campbell D , Morris SK , et al . Designing and evaluating a novel mobile application for Helping Babies Breathe skills retention in Uganda: comparative study protocol . BMJ Paediatr Open . 2019 ; 3 ( 1 ): e000561 .
• ¹⁷
  Chan NH , Mistry N , Campbell DM . A Simulation-based pilot study of a mobile application (NRP Prompt) as a cognitive aid for neonatal resuscitation training . Simul Healthc . 2019 ; 14 ( 3 ): 146 – 56 .
• ¹⁸
  Fransen AF , van der Hout-van der Jagt MB , Gardner R , Capelle M , Oei SP , van Runnard Heimel PJ , et al . Assessment tool for the instructional design of simulation-based team training courses: the ID-SIM . BMJ Simul Technol Enhanc Learn . 2018 ; 4 ( 2 ): 59 - 64 .
• ¹⁹
  Torblå Olsen J , Prinz A , Smaradottir B . Simulation of eHealth Scenarios with Role-Play Supported by an Interactive Smartphone Application . Stud Health Technol Inform . 2019 ; 257 : 418 - 23 .
• ²⁰
  Gambo JM , Bahreman NT , Watties-Daniels D , Neal M , Swoboda SM . Can Mobile Technology Enhance Learning and Change Educational Practice? Comput Inform Nurs . 2017 ; 35 ( 8 ): 375 – 80 .
• ²¹
  Watts PI , McDermott DS , Alinier G , Charnetski M , Ludlow J , Horsley E , Meakim C , Nawathe PA ; INACSL Standards Committee . Healthcare Simulation Standards of Best PracticeTM Simulation Design . Clin Simul Nurs . 2021 ; 58 : 14 – 21 .
• ²²
  Persico L , Belle A , DiGregorio H , Wilson-Keates B , Shelton C , Persico L , Belle A , DiGregorio H , Wilson-Keates B , Shelton C ; INACSL Standards Committee . Healthcare Simulation Standards of Best PracticeTM Facilitation . Clin Simul Nurs . 2021 ; 58 : 22 – 6 .
• ²³
  Nestel D , Kelly M , Jolly B , Watson M , editors . Healthcare simulation education: evidence, theory and practice . Chichester : John Wiley & Sons ; 2018 . 214 p.
• ²⁴
  Koetsier J . Top 10 Apps By Downloads And Revenue Q2 2021: Report. Forbes Magazine; 2021 July 15 [ cited 2022 Jan 25 ]. Available from: https://www.forbes.com/sites/johnkoetsier/2021/07/15/top-10-apps-by-downloads-and-revenue-q2-2021-report/?utm_campaign=forbes&utm_source=twitter&utm_medium=social&utm_term=Carrie&sh=2d53d4132959
  » https://www.forbes.com/sites/johnkoetsier/2021/07/15/top-10-apps-by-downloads-and-revenue-q2-2021-report/?utm_campaign=forbes&utm_source=twitter&utm_medium=social&utm_term=Carrie&sh=2d53d4132959
• ²⁵
  Lewis JR . The System Usability Scale: Past, Present, and Future . Int J Hum Comput Interact . 2018 ; 34 ( 7 ): 577 - 90 .
• ²⁶
  Østensen E , Hardiker NR , Hellesø R . Facilitating the Implementation of Standardized Care Plans in Municipal Healthcare . Comput Inform Nurs . 2021 ; 40 ( 2 ): 104 – 12 .
• ²⁷
  Maia JS , Marin HF . Mobile apps for disadvantaged societies . Acta Paul Enferm . 2021 ; 34 : eAPE002214 . Review .
• ²⁸
  Diniz CM , Leal LP , Guedes TG , Linhares FM , Pontes CM . Contributions of mobile applications on the breastfeeding practice: integrative review . Acta Paul Enferm . 2019 ; 32 ( 5 ): 571 – 7 . Review .
• ²⁹
  Pereira SA , Cecilio SG , Lima KC , Pagano AS , Reis IA , Torres HC . Mobile applications for sickle cell disease management: an integrative review . Acta Paul Enferm . 2018 ; 31 ( 2 ): 224 - 32 . Review .

Edited by

Associate Editor (Peer review process): Bartira de Aguiar Roza (https://orcid.org/0000-0002-6445-6846) Escola Paulista de Enfermagem, Universidade Federal de São Paulo, São Paulo, SP, Brazil

Publication Dates

• Publication in this collection
  08 May 2023
• Date of issue
  2023

History

• Received
  14 Aug 2021
• Accepted
  30 Nov 2022