Development and Validation of a Smartphone Application for Orthopedic Residency Education

Desenvolvimento e Validação de um Aplicativo para Smartphone para Ensino de Residência em Ortopedia

Abstract:

Introduction:

Mobile learning offers several benefits, such as new learning environments. We developed and validated a smartphone application for orthopedics and traumatology residents in an attempt to assist their study and, consequently, pass the Board of Orthopedics and Traumatology Exam.

Methods:

quantitative study, aimed at the validation of a mobile application. It was developed for the iOS® and Android® platforms, in the Portuguese language, and free of charge. 132 participants, divided into three groups, used the tool. A validated questionnaire - System SUS Scale (SUS) - and a questionnaire created by the authors (properly validated) were used to assess the practicality and viability of the application as a learning tool. The mode, the absolute frequency and the percentage of the studied variables were crossed by Fisher's test or Pearson's chi-square test.

Results:

of the 132 participants, 55.3% have iOS®. All participants said they had already used an application on their smartphones, whereas 6 (4.5%) mentioned never having used applications for academic purposes (p value <0.001). 100% said it was a useful technology in the resident's theoretical development. 124 (93.9%) agreed it was an auxiliary learning method for orthopedic doctors in general. An average SUS score of 84.2 (SD 10.8) was obtained with a margin of error of 1.9. The SUS score varied between 82.4 and 86.1 (95% CI). Cronbach's alpha coefficient was 0.797.

Conclusion:

the developed application was successful in the tests performed and can be an alternative in medical education in the orthopedic area.

Keywords:
Education in Orthopedics; Software; Mobile Applications; Orthopedics; Simulation Training

Resumo:

Introdução:

Além de proporcionar vários benefícios, o aprendizado móvel oferece novos ambientes de ensino. Desenvolvemos e validamos um aplicativo de smartphone para residentes de ortopedia e traumatologia, a fim de ajudá-los no estudo e, consequentemente, na aprovação do exame do Conselho de Ortopedia e Traumatologia.

Métodos:

Trata-se de um estudo quantitativo para validar a aplicação móvel. O instrumento foi desenvolvido para as plataformas iOS® e Android®, em português e gratuito. Os 132 participantes do estudo, divididos em três grupos, usaram a ferramenta. Adotaram-se dois questionários: a já validada System Usability Scale (SUS) e um outro criado pelos autores (devidamente validado) para avaliar a praticidade e viabilidade do aplicativo como ferramenta de aprendizagem. O modo, a frequência absoluta e o percentual das variáveis estudadas foram cruzados pelo teste de Fisher ou qui-quadrado de Pearson.

Resultados:

Dos 132 participantes, 55,3% possuem iOS®. Todos os participantes afirmaram que já haviam usado um aplicativo em seus smartphones; seis (4,5%) mencionaram que nunca utilizaram aplicativos para fins acadêmicos (valor de p < 0,001); 100% destacaram a utilidade dessa tecnologia no desenvolvimento teórico do residente; e 124 (93,9%) concordaram que se trata de um método auxiliar de aprendizagem para médicos ortopedistas em geral. Obteve-se um escore médio da SUS de 84,2 (DP 10,8) com margem de erro de 1,9. O escore da SUS variou de 82,4 a 86,1 (IC95%). O coeficiente alfa de Cronbach foi de 0,797.

Conclusão:

O aplicativo desenvolvido foi bem-sucedido nos exames realizados e pode ser uma alternativa no ensino médico na área ortopédica.

Palavras-chave:
Educação em Ortopedia; Software; Aplicativos Móveis; Ortopedia; Treinamento por Simulação

INTRODUCTION

Communication and Information Technologies (ICTs) started being used to help the acquisition of knowledge and facilitate the teaching-learning process¹. The use of technology through mobile devices or mobile learning (m-learning) is known to offer many benefits, such as access to content anywhere and at any time, the ability to perform tasks and activities and note down ideas or queries on the Internet. Teaching through m-learning enables the simulation of complex systems, along with the benefit of promoting a large number of perspectives with a high quality of visualization and interaction². In this context, ICTs, according to each modality, can enable the creation of new learning environments, allowing remote and/or online access to training platforms³, increasing communication and distance learning. Thus, these support tools incorporated into an electronic health management system exponentially increase the growth of medical information acquisition⁴.

There are several practical training methods for orthopedic surgeons, including hands-on training with synthetic bones or cadavers, and the use of software and computer simulators for planning and simulating situations in 3D environments⁵. This reduces financial costs and brings about improvements in skills development, the handling of surgical instruments, competence to reach the learning curve earlier, better assessment of the acquired techniques, and perhaps most importantly, the maximization of the patient’s safety⁶.

Within the ICT-based learning modalities, cognitive simulation is one of the most recent examples of medical training innovation. It is the process by which medical residents or students evaluate and rehearse actions within their minds without physical movement and, among them, is the training related to solving questions for theoretical-practical training. It is hypothesized that residents may improve their intraoperative performance and surgical skills through appropriate preoperative cognitive simulation, with or without appropriate preceptors to help them⁷.

The use of smartphone applications for the orthopedic surgeon training is a powerful tool for improving the quality of these professionals’ training⁸. Currently, 67.1% of medical students choose an application to help them with their daily academic doubts, and 42.2% use it to assist them in their professional practice⁹.

Among the several approaches to the teaching-learning process using mobile applications, the knowledge acquired through the resolution of various issues, also called retrieval learning, can be highlighted, which has proved to be more effective than just reading the content to be studied¹⁰. A search was conducted in the virtual stores Applestore^® and GooglePlay^® and it was found that most of the applications available for the orthopedic student based on retrieval learning were not properly scientifically validated. Therefore, the aim of this research was to develop and validate an application for the improvement and training of orthopedics and traumatology residents, which allows the resolution of preparatory questions for the Brazilian Board of Orthopedics and Traumatology (TEOT) exam.

MATERIALS AND METHODS

App development

The development of this application involved the following steps: review of the available education and training applications in orthopedics and comparison with evidence-based practices; creation of an initial database of questions based on the bibliography proposed by the Teaching and Training Committee (TTC) of the Brazilian Society of Orthopedics and Traumatology (SBOT); selection of strategies to change behavior and guidelines to be used in the application; development of the interfaces and designs of the application; and finally, selection of participants for the application of the validation test.

The application was developed in both Android^® and iOS^® platforms. Regarding the development, the following tools were used: IDE^® (Integrated Development Environment) Android Studio; Android^® Software Development Kit (SDK); the Android^® Emulator System with Google's Application Programming Interface (APIs); and the OpenCV (Open Source Computer Vision Library) library for the development of the image processing functions present in the application. The application allows the customization of the content of the simulated questions, the visualization of the elapsed time and right or wrong answers with the respective bibliographic references. Also, the user’s results progression can be followed.

The participants were randomly selected using Microsoft Excel^® software. They were then divided into three groups: group I, 4-year medical students of the Christus University Center (Unichristus) (33 participants) and the Federal University of Ceará (UFC) (33 participants); group II, residents of orthopedics and traumatology from the state of Ceará (33 participants); group III, consisting of orthopedists and traumatologists who are members of SBOT and medical residency staff (33 participants). This sample size was calculated based on the study conducted in Leipzig, Germany⁹, which showed the preference of applications for health training, when evaluating two groups of students, divided among those who want to use an application (69.9%) versus those who do not (28.0%); it was considered necessary to evaluate a sample of 33 individuals per study group in order to obtain a sample that represented 90% of power and 95% confidence of the null hypothesis in this study (Fleiss method with continuity correction). Calculations were performed using the Open Epi Stat Calc^® (http://www.openepi.com/Menu/OE_Menu.htm).

The inclusion criteria were: age above 18 years old and participants who signed the Free and Informed Consent Form. Those participants who wished to discontinue their participation in the study were withdrawn from the study.

Regarding the ethical aspects, the basic principles of ethics in human research were considered, such as autonomy, justice, beneficence, and nonmaleficence. The study was submitted to the Human Research Ethics Committee (CEP) of Unichristus, protocol number 65487617.5.0000.5049. There are no conflicts of interest to be disclosed by the authors.

Data Collection

The questionnaire (Appendix A) was organized into three sessions: 1) related to the user’s authorization and experience with applications; 2) related to the SUS^® (System Usability Scale) questionnaire¹¹, validated in Portuguese to quantify the applicability of the developed application¹²; and 3) related to the importance of the application in learning, being developed based on Davis Technology Acceptance Model¹³ and the Likert Scale, which measures the system practicability level perceived by users during the resolution of questions and evaluates the usefulness of specific questions about orthopedics and traumatology, as well as learning.

The data were tabulated in Microsoft Excel® and exported to the Statistical Package for Social Sciences (SPSS) software, version 17.0 for Windows®, where the analyses were performed with a confidence interval of 95%. The mode, absolute frequency and percentage of the variables studied were cross-checked using Fisher's exact test or Pearson's chi-squared test.

RESULTS

The developed application was named Quiz Ortopedia ^®, and its patent was registered in the National Institute of the Industrial Property (INPI) for iOS^® (registration number BR 512018000006-1) and Android^® (registration number BR 512018000005-3).

The application allows the user’s registration with name, e-mail, year of residence, a 6-digit password and password confirmation.

On the initial screen, it is possible to choose between a customized test, a simulation according to the suggestion by SBOT, a summary of the user's profile and information about the development and version of the application.

Regarding personalized questions, the user has access to main topics (Orthopedics, Trauma, and Basic) and subtopics divided mainly by body joints (hip, knee, etc.). It is also possible to choose several simultaneous topics and then one can choose the number of questions and the timing (per question or total time). Throughout the resolution of the questions, the user will be able to see the elapsed time, the total number of questions, as well as the number of correct or incorrect questions up to that time. After that, a result in percentage should be displayed and the user is given the option to access the comments about both right and wrong questions. In the user’s profile, there is a screen with the results, so that the user can check study progression up to that point.

The images of some representative screens of the application are depicted in Figure 1.

Figure 1
Quiz screens: Topic selection, questions, and test results from Quiz Ortopedia^®.

Regarding the profile of the study participants and their previous experiences with the use of applications, 55.3% of the participants (73/132) have iOS^® as their operating system, while 44.7% (59/132) use Android^®. All 132 participants reported that they had already used some type of application on their smartphones, and only 6 (4.5%) mentioned never having used applications for academic purposes (Table 1).

Table 1.Statistical
analysis among students, residents, and orthopedists regarding the operating system and the use of applications for academic purposes.

	Occupation
	Student	Resident	Orthopedist	p-Value
Operating System
Android	32 (48.5%)	18 (54.5%)	9 (27.3%)	0.057
IOs	34 (51.5%)	15 (45.5%)	24 (72.7%)
Used application for academic purposes
No	3 (4.5%)	0 (.0%)	3 (9.1%)	0.208
Yes	63 (95.5%)	33 (100.0%)	30 (90.9%)

• * p <0.05, chi-square. Data expressed as absolute and percentage frequencies.

In group I, 63 participants (95.5%) had already used mobile applications for academic purposes. In group II, all of them had already used the mobile phone with the same purpose, and in group I, 30 (90.9%) used academic applications (Table 1). The type of operating system (iOS^® or Android^®) showed a significant statistical difference between the groups (p-value <0.001). The use of applications for academic purposes did not show a statistical difference between the groups.

When we analyzed the usability of the application, evaluated according to the SUS^® questionnaire, we observed that it showed an average SUS score of 84.2 (standard deviation of 10.8) with a margin of error of 1.9. To verify the reliability of the obtained results, Cronbach's alpha coefficient was used¹⁴. The value obtained for this coefficient was 0.797, showing the sample had a good level of reliability.

The analysis of the answer to each question in the SUS questionnaire (part II) is found in Appendix B. We observed a statistical difference between groups regarding the following statements: 1) I think I would like to use this system frequently, in which the resident group entirely agreed with this statement, while the other groups only agreed with the statement (p-value = 0.018); 2) I found that the various functions in this system were well integrated, in which the resident and the orthopedic groups entirely agreed with this statement, whereas the academic group only agreed (p-value = 0.024); and 3) I was very comfortable using this system, in which the resident and the orthopedic groups fully agreed with this statement, whereas the academic group only agreed (p-value = 0.027).

The analysis of part III, regarding the importance of the application in learning, evidenced differences between the groups with statistical significance for all questions (0.001-0.032), except for question 6 (p-value = 0.177) (Table 2).

Table 2
Absolute frequencies and percentages of the answers to the questions of part III of the evaluation tool among medical students, orthopedic residents, and orthopedists with specialist qualifications.

	Occupation
	Student	Resident	Orthopedist	p-Value
1) Is problem-solving training part of learning?
Neutral	1 (1.5%)	0 (.0%)	0 (.0%)	0.003
Agree	26* (39.4%)	3 (9.1%)	4 (12.1%)
Strongly agree	39 (59.1%)	30* (90.9%)	29* (87.9%)
2) Can the resolution of questions help in the preparation for the Board of Orthopedics and Traumatology (BOT) exam?
Neutral	7* (10.6%)	0 (.0%)	0 (.0%)	0.001
Agree	23* (34.8%)	3 (9.1%)	5 (15.2%)
Strongly agree	36 (54.5%)	30* (90.9%)	28* (84.8%)
3) Do I believe that the practicality of a mobile application on problem-solving in orthopedics and traumatology could help in preparing residents for exams?
Neutral	1 (1.5%)	0 (.0%)	0 (.0%)	0.005
Agree	25* (37.9%)	3 (9.1%)	4 (12.1%)
Strongly agree	40 (60.6%)	30* (90.9%)	29* (87.9%)
4) Does the application seem to be a useful technology in the theoretical development of the orthopedic resident during their training?
Agree	26* (39.4%)	6 (18.2%)	6 (18.2%)	0.027
Strongly agree	40 (60.6%)	27* (81.8%)	27* (81.8%)
5) Can the application be used as an auxiliary method of learning for medical students studying Orthopedics and Traumatology?
Disagree	2 (3.0%)	4 (12.1%)	7 (21.2%)	0.032
Neutral	1 (1.5%)	3* (9.1%)	2 (6.1%)
Agree	32 (48.5%)	17* (51.5%)	14 (42.4%)
Strongly agree	31* (47.0%)	9 (27.3%)	10* (30.3%)
6) Can the application be useful as an auxiliary learning method for general orthopedic physicians?
Disagree	1 (1.5%)	0 (.0%)	3 (9.1%)	0.177
Neutral	2 (3.0%)	2 (6.1%)	0 (.0%)
Agree	34 (51.5%)	15 (45.5%)	19 (57.6%)
Strongly agree	29 (43.9%)	16 (48.5%)	11 (33.3%)

• * p <0.05, chi-square. Data expressed as absolute and percentage frequencies.

DISCUSSION

A substantial increase in the availability and use of mobile applications for smartphones and tablets for medical education has been reported in recent years, including the handling of work tools and care in the learning mode¹⁵. They are of great importance for enabling remote and online access to the information, allowing greater flexibility in learning. Students are more comfortable regarding when, how and where to study, making it possible for them to escape the traditional method of teaching conferences in the classrooms¹⁶. This teaching alternative can bring safety to the patient treated by the future doctors, since it challenges students regarding decision-making⁴.

The advantages for the medical sciences include the encouragement of and engagement in independent and group learning, as well as flexibility in the method of study¹⁷. The use of ICTs in the medical area also includes improved patient care and multi-professional communication¹⁸.

Among the disadvantages that can be mentioned, in addition to the cost of developing the technology, are the failures of the mobile device method, such as access to multiple contents, offering distraction through other tools, or even superficial learning, inadequate screen size and insufficient memory, the lack of validation and, sometimes, lack of scientific evidence related to some application categories⁷. However, with the improvements in science, it is believed that many of these difficulties can be solved¹⁹.

In recent years, the growth of ICTs and their involvement in medical education has become significant, revolutionizing the access to information and learning methods. In previous literature review studies, it was observed that smartphone application technology can be integrated into successful orthopedic practice¹⁸. However, currently, the market does not have a large availability of applications targeting this specialty. Among users of these technology devices, approximately 50% use the Android^® 22.5% use Blackberry^® and 18.5% use iOS / Apple^®(20 operating system. Among orthopedic surgeons, 84% use smartphones, mostly iPhone^® (55%) and 53% use it in clinical practice. At the time of the survey, there were 61 orthopedic applications for the iOS / iPhone^® platform and 13 applications for the Android^® platform²¹.

Moreover, there are concerns about the content available in this tool, since published information validation tools are pointed out as a challenge to be put into practice²¹. In this perspective, a study evaluated the main applications used by orthopedic specialists in hand surgery. Among the educational purposes, the reading of scientific journals and recent publications were the main modes of teaching and medical training. As a result, there was no application for direct resident training regarding the resolution of issues or clinical situations²².

As for other studies, it could be inferred that, concerning orthopedic applications for teaching and medical training, the great majority consisted of guidelines, training of surgical techniques and videos^3),(20, as well as for anatomy, medical calculators, laboratory results and drugs and medicines²³. A pilot study of a simulation application aimed at the management of septic arthritis for orthopedic residents showed positive results when comparing the percentage of correct answers in the tests versus the correctness of the simulations, concluding that the subject training application has great potential to educate and evaluate the user in an efficient and cost-effective way²⁴.

Faced with this brief bibliographic review, the Quiz Ortopedia ^® is a revolutionary application for teaching and medical training, related to an innovative and promising improvement in medical education.

The developed application followed certain characteristics to meet the objectives of proposing an auxiliary tool in the preparation of residents of orthopedics and traumatology for the TEOT exam, based on the principle of retrieval learning. In addition, the method used to create the training application for orthopedic residents was based on some pillars of education by simulation: positive feedback, repetitive practice, difficulty scale, variable clinical catch, individual or group learning and validation of the simulator²⁵. All these pillars were taken into account during the creation of the application assessed in this study. It has been observed that witnessing the development and improvement of their skills throughout the training increases the residents' confidence, as well as receiving constructive, direct and specific feedback²⁶. The developed application allows the user to follow their evolution throughout the test, in addition to receiving the comments related to each question answered by them. The other applications on the market do not have comments related to the questions, do not have the bibliographic reference from which the question was taken, do not allow the comparison of the user's own results and do not allow a comparison of results with the other users.

In the context of the questionnaire applied to the participants, we can infer the high acceptance of the practicality of the application by the sample group, since 96.2% (127) of the users agreed that it is easy to use, whereas 90.7% of the participants affirmed they would like to use the app frequently. The lowest agreement regarding the other evaluations (85.6%, corresponding to 113 users) was related to whether the use of the application was considered as an auxiliary study method for students; this can be justified by the answers of the group of students, since the purpose of the developed application is not to assist, at this moment, the study of the discipline of Orthopedics and Traumatology, but help the residents who will take the Board Exam.

When asked about the usability of the application (part II and III of the questionnaire), the response of the majority of users was positive among the different sample groups.

Regarding the usability, 100% (132) stated that it was a useful technology for the resident’s theoretical learning, in addition to the 93.4% (124) that agreed it is an auxiliary method of learning for general orthopedic physicians. These data indicate the users’ recognition regarding the importance of alternative ways to support the teaching-learning process, as well as references from previous studies, which identified the increase in the number of residents and the limited opportunities to develop surgical and technical skills. Many different solutions have been proposed in education centers around the world, including e-training programs, simulation, and mandatory training to maximize learning opportunities within existing resource constraints²⁷. In this context, the use of simulation applications allows residents to practice skills in a safe and controlled environment, demonstrating their improved confidence and minimizing patient risk.

For comparative purposes, a study related to the development of a teaching and training application in anesthesiology, analyzed 20 participants (medical students and residents) based on the SUS evaluation score of 90.6²⁸, which is referred to as a high-value usability index²⁸. Although the SUS^® score of our study was slightly lower (84.2), the size of our sample denotes a more significant impact and scientific representativeness, as well as greater variability, being translated by a similar sample reliability between the two studies (0.79). These data were measured taking into account Cronbach's alpha coefficient, which represents the best method to evaluate variation between samples¹⁴. These data demonstrate the importance of the SUS^® score obtained by the application developed by this study.

Another study assessing a medical education application for residents in the area of surgical simulation was able to conclude that this type of tool is a good complementary alternative for the training of these students’ profile by increasing cognitive performance and thus ensuring greater patient safety²⁹. The Quiz Ortopedia^®, albeit through a different proposal, can achieve the same goal by helping residents take the Board Exam after having developed more knowledge of the covered content.

Among the limitations of the study, we can mention the lack of a comparative efficiency test between residents who used the application and the residents who studied using only the traditional methods. Another limitation is that it included residents from only one federation unit, not covering all the other states.

More comparative studies are required, in the near future, to consolidate the efficiency of the Quiz Ortopedia^® application as a support teaching and training method aimed at the education of medical residents of orthopedics and traumatology.

CONCLUSION

This study demonstrated the development, use, and evaluation, through a questionnaire validated in the study, of a mobile application created for the iOS^® and Android^® platforms aimed at the training of residents in orthopedics and traumatology based on the resolution of questions.

The application showed high usability with good sample variability when validated by specialists in Orthopedics and Traumatology, as well as by medical students and orthopedic residents. The developed application was successful in the tests performed and may be an alternative in medical education in the orthopedic area.

REFERENCES

• ¹
  Pereira FGF, Silva DVD, Sousa LMOD, Frota NM. Construção de um aplicativo digital para o ensino de sinais vitais. Rev. Gaúcha Enferm. 2016;37(2):e59015.
• ²
  Marçal E, Viana W, Andrade RMC, Rodrigues D. A mobile learning system to enhance field trips in geology. 2014 IEEE Frontiers in Education Conference (FIE) Proceedings. 2014;1-8.
• ³
  Kulendran M, Lim M, Laws G, Chow A, Nehme J, Darzi A, et al. Surgical smartphone applications across different platforms: their evolution, uses, and users. Surgical Innovation. 2014;21(4):427-440.
• ⁴
  Cook DA, Blachman MJ, Price DW, West CP, Thomas BLB, Berger RA, et al. Educational Technologies for Physician Continuous Professional Development. Acad Med. 2018;93(1)1:104-112.
• ⁵
  Akhtar K, Sugand K, Sperrin M, Cobb J, Standfield N, Gupte C. Training safer orthopedic surgeons. Acta Orthop. 2015;86(5):616-621.
• ⁶
  Atesok K, Mabrey JD, Jazrawi LM, Egol KA. Surgical Simulation in Orthopaedic Skills Training. J Am Acad Orthop Surg. 2012;20(7):410-422.
• ⁷
  Masalimova AR, Levina EY, Platonova RI, Yakubenko KY, Mamitova NV, Arzumanova LL, et al. Cognitive Simulation as Integrated Innovative Technology in Teaching of Social and Humanitarian Disciplines. EURASIA Journal of Mathematics, Science and Technology Education. 2017;13(8):4915-4928.
• ⁸
  Shaw CM, Tan SA. Integration of Mobile Technology in Educational Materials Improves Participation: Creation of a Novel Smartphone Application for Resident Education. J Surg Educ. 2015;72(4):670-673.
• ⁹
  Sandholzer M, Deutsch T, Frese T, Winter A. Medical students’ attitudes and wishes towards extending an educational general practice app to be suitable for practice: A cross-sectional survey from Leipzig, Germany. Eur J Gen Pract. 2016;22(2):141-146.
• ¹⁰
  Karpicke JD, Roediger HL. The Critical Importance of Retrieval for Learning. Science. 2008;319(5865):966-968.
• ¹¹
  Brooke J. SUS-A quick and dirty usability scale. Usabil Eval Ind. 1996;189(194):4-7.
• ¹²
  Tenório JM, Cohrs FM, Sdepanian VL, Pisa IT, Marin HF. Desenvolvimento e Avaliação de um Protocolo Eletrônico para Atendimento e Monitoramento do Paciente com Doença Celíaca. Revista de Informática teórica e aplicada. 2010;17(2):210-220.
• ¹³
  Davis FD. Perceived usefulness, perceived ease of use and user acepptance of information technology. MIS Quarterly. 1989;13(3):319-340.
• ¹⁴
  Bonett DG, Wright TA. Cronbach’s alpha reliability: Interval estimation, hypothesis testing, and sample size planning. Journal of Internet Banking and Commerce. 2015;36(1):3-15.
• ¹⁵
  Singler K, Roth T, Beck S, Cunningham M, Gosch M. Development and initial evaluation of a point-of-care educational app on medical topics in orthogeriatrics. Archives Of Orthopaedic And Trauma Surgery. 2016;136(1):65-73.
• ¹⁶
  Shaw CM, Tan SA. Integration of Mobile Technology in Educational Materials Improves Participation: Creation of a Novel Smartphone Application for Resident Education. J Surg Educ . 2015;72(4):670-673.
• ¹⁷
  Juanes JA, Ruisoto P. Computer Applications in Health Science Education. J Med Syst. 2015;39(9):97.
• ¹⁸
  Ozdalga E, Ozdalga A, Ahuja N. The smartphone in medicine: a review of current and potential use among physicians and students. J Med Internet Res. 2012;14(5):e128.
• ¹⁹
  Walsh K. Mobile Learning in Medical Education: Review. Ethiop J Health Sci. 2015;25(4):363-366.
• ²⁰
  Al-Hadithy N, Gikas PD, Al-Nammari SS. Smartphones in orthopaedics. Int Orthop. 2012;36(8):1543-1547.
• ²¹
  Franko OI. Smartphone Apps for Orthopaedic Surgeons. Clin Orthop Relat Res. 2011;469(7):2042-2048.
• ²²
  Franko OI. Mobile software applications for hand surgeons. J Hand Surg, 2012;37(6):1273-1275.
• ²³
  Duncan SF, Hendawi TK, Sperling J, Kakinoki R, Hartsock L. iPhone and iPad use in orthopedic surgery. Ochsner J. 2015;15(1):52-57.
• ²⁴
  Shore BJ, Miller PE, Noonan KJ, Bae DS. Predictability of Clinical Knowledge Through Mobile App-based Simulation for the Treatment of Pediatric Septic Arthritis: A Pilot Study. J Pediatr Orthop. 2018;38(9):e541-e545.
• ²⁵
  Guze PA. Using technology to meet the challenges of medical education. Trans Am Clin Climatol Assoc,2015;126:260.
• ²⁶
  Lees MC, Zheng B, Daniels LM, White JS. Factors Affecting the Development of Confidence Among Surgical Trainees. J Surg Educ . 2019;76(3):674-683.
• ²⁷
  Stirling ER, Lewis TL, Ferran NA. Surgical skills simulation in trauma and orthopaedic training. Journal of orthopaedic surgery and research. 2014;9(1):126.
• ²⁸
  Pereira RVS, Kubrusly M, Marçal E. Desenvolvimento, utilização e avaliação de uma aplicação móvel para educação médica: Um estudo de caso em Anestesiologia. RENOTE. 2017;15(1):1-10.
• ²⁹
  Sugand K, Mawkin M, Gupte C. Training effect of using Touch Surgery™ for intramedullary femoral nailing. Injury. 2016;47(2):448-452.

Publication Dates

• Publication in this collection
  28 Oct 2020
• Date of issue
  2020

History

• Received
  03 July 2020
• Accepted
  16 Sept 2020