Nurse educators’ satisfaction with online Objective Structured Clinical Examination scoring system Objective the paper-based OSCE (Pa-OSCE) scoring system has several drawbacks, including significant paper waste and being time-consuming. This study aims to assess examiner satisfaction with the On-OSCE scoring management system, identify and weigh the potential benefits of the innovation, and promote its use.
Method using a cross-sectional study design, we developed satisfaction inventory consisting of four domains: time-saving, user-friendliness, prospective application, and objectivity. This inventory was used to compare satisfaction scores between two examiner groups (a total of 67 subjects).
Results the 20-item satisfaction inventory demonstrated high validity and reliability (0.98 and 0.97, respectively). The average scoring time was significantly shorter in the online scoring management system group (p<0.001). Both the total satisfaction score and the scores for each domain were significantly higher in the online scoring management system group compared to the paper-based scoring management system group (p<0.001).
Conclusion a high proportion of the examiners provided positive feedback on the online scoring management system. The online scoring management system saves the time to score, and is more objective, easier to use, and is recommended for implementation. Online scoring management systems can be implemented in nursing education institutions globally to improve scoring efficiency. Additionally, the 20-item satisfaction inventory can serve as a benchmarking tool to assess educators worldwide.
Descriptors:
Nursing Education; Educational Technology; Internet; Participant Satisfaction; Questionnaires; Developing Nations
Highlights:
(1) Valid and reliable 20-item questions on the nursing scoring system.
(2) A high proportion of the examiners (lecturers) provided positive feedback on the online OSCE.
(3) Nursing education institutions may adopt the On-OSCE scoring system to improve scoring.
(4) The utilization of On-OSCE can save time, objective, and simplify the scoring process.
(5) The On-OSCE scoring system is highly recommended for worldwide implementation.
Objetivo: el sistema de puntuación del OSCE en papel (Pa-OSCE) tiene varias desventajas, incluyendo un desperdicio significativo de papel y el hecho de que consume mucho tiempo. Este estudio tiene como objetivo evaluar la satisfacción de los examinadores con el sistema de gestión de puntuación del On-OSCE, identificar y evaluar los posibles beneficios de la innovación y promover su uso.
Método: usando un diseño de estudio transversal, desarrollamos un inventario de satisfacción que consiste en cuatro dominios: ahorro de tiempo, facilidad de uso, aplicación prospectiva y objetividad. Este inventario fue utilizado para comparar las puntuaciones de satisfacción entre dos grupos de examinadores (un total de 67 individuos).
Resultados: el inventario de satisfacción de 20 ítems demostró alta validez y confiabilidad (0,98 y 0,97, respectivamente). El tiempo medio de puntuación fue significativamente menor en el grupo del sistema de gestión de puntuación en línea (p<0,001). Tanto la puntuación total de satisfacción como las puntuaciones de cada dominio fueron significativamente mayores en el grupo del sistema de gestión de puntuación en línea en comparación con el grupo del sistema de gestión de puntuación en papel (p<0,001).
Conclusión: una gran proporción de los examinadores proporcionó feedback positivo sobre el sistema de gestión de puntuación en línea. El sistema de gestión de puntuación en línea ahorra tiempo en la puntuación, es más objetivo, más fácil de usar y se recomienda para su implementación. Los sistemas de gestión de puntuación en línea pueden ser implementados en instituciones de enseñanza de enfermería en todo el mundo para mejorar la eficiencia en la puntuación. Además, el inventario de satisfacción de 20 ítems puede servir como una herramienta de referencia para evaluar a los educadores en todo el mundo.
Descriptores:
Educación en Enfermería; Tecnología Educacional; Internet; Satisfacción del Participante; Cuestionarios; Países en Desarollo
Destacados:
(1) Preguntas de 20 ítems válidas y confiables sobre el sistema de puntuación de enfermería.
(2) Una alta proporción de examinadores (profesores) proporcionó feedback positivo sobre el OSCE en línea.
(3) Las instituciones de enseñanza de enfermería pueden adoptar el sistema de puntuación del On-OSCE para mejorar la evaluación.
(4) La utilización del On-OSCE puede ahorrar tiempo, ser objetiva y simplificar el proceso de puntuación.
(5) El sistema de puntuación del On-OSCE es altamente recomendado para su implementación a nivel mundial.
Objetivo o sistema de pontuação do OSCE em papel (Pa-OSCE) tem várias desvantagens, incluindo o desperdício significativo de papel e o fato de consumir muito tempo. Este estudo tem como objetivo avaliar a satisfação dos examinadores com o sistema de gerenciamento de pontuação do On-OSCE, identificar e avaliar os possíveis benefícios da inovação e promover seu uso.
Método usando um projeto de estudo transversal, desenvolvemos um inventário de satisfação que consiste em quatro domínios: economia de tempo, facilidade de uso, aplicação prospectiva e objetividade. Esse inventário foi usado para comparar os escores de satisfação entre dois grupos de examinadores (total de 67 indivíduos).
Resultados o inventário de satisfação de 20 itens demonstrou alta validade e confiabilidade (0,98 e 0,97, respectivamente). O tempo médio de pontuação foi significativamente menor no grupo do sistema de gerenciamento de pontuação online (p<0,001). Tanto a pontuação total de satisfação quanto as pontuações de cada domínio foram significativamente maiores no grupo do sistema de gerenciamento de pontuação online em comparação com o grupo do sistema de gerenciamento de pontuação em papel (p<0,001).
Conclusão uma grande proporção dos examinadores forneceu feedback positivo sobre o sistema de gerenciamento de pontuação online . O sistema de gerenciamento de pontuação online economiza o tempo de pontuação, é mais objetivo, mais fácil de usar e é recomendado para implementação. Os sistemas de gerenciamento de pontuação online podem ser implementados em instituições de ensino de enfermagem em todo o mundo para melhorar a eficiência da pontuação. Além disso, o inventário de satisfação de 20 itens pode servir como uma ferramenta de benchmarking para avaliar os educadores em todo o mundo.
Descritores:
Educação em Enfermagem; Tecnologia Educacional; Internet; Satisfação do Participante; Questionários; Países em Desenvolvimento
Destaques:
(1) Perguntas de 20 itens válidas e confiáveis sobre o sistema de pontuação de enfermagem.
(2) Uma alta proporção de examinadores (professores) forneceu feedback positivo sobre o OSCE online .
(3) As instituições de ensino de enfermagem podem adotar o sistema de pontuação do On-OSCE para melhorar a pontuação.
(4) A utilização do On-OSCE pode economizar tempo, ser objetiva e simplificar o processo de pontuação.
(5) O sistema de pontuação do On-OSCE é altamente recomendado para implementação em todo o mundo.
Introduction
The objective structured clinical examination (OSCE) was developed in the United Kingdom to assess medical students’ clinical competencies in a controlled, simulated environment ( 1 ) . The OSCE has been introduced and used to assess students’ clinical competencies in various health professional education programs, including nursing ( 2 - 4 ) . There are several clinical competencies assessment methods in health professional education, such as the mini clinical evaluation exercise (mini-CEX) and direct observation of procedural skills (DOPS); however, OSCE remains the most widely used method for health professional qualification examinations, including in Asian countries like Indonesia ( 5 - 7 ) .
The OSCE examination has several test stations with simulated environments ( 7 - 9 ) , and during the exam, students are monitored and evaluated by an examiner using predetermined marking criteria ( 10 ) . The OSCE reduces examiner bias, standardizes the exam procedure, and provides objectivity to evaluate clinical competencies ( 1 ) . Moreover, the OSCE highlights the students’ strengths and weaknesses ( 11 - 13 ) , assesses their knowledge and attitudes towards clinical practice ( 14 ) , provides profound learning ( 4 ) , and builds students’ self-confidence ( 3 , 5 ) . Both students and examiners have acknowledged the satisfaction and benefits of the OSCE experience in their education ( 2 , 15 - 17 ) . Furthermore, the OSCE has received excellent feedback as an assessment tool for evaluating clinical competence and addressing student diversity in education ( 12 , 18 ) . Interestingly, there are significant limitations associated with OSCEs. Some students find them stressful, and they require considerable resources, including clinical skills laboratories, equipment, and examiners ( 13 ) . Despite the challenges, authors believe that the educational benefits of OSCEs significantly outweigh the concerns related to resource allocation ( 16 ) .
OSCE scoring management system in health professions and nursing education has traditionally been conducted using a paper-based method (Pa-OSCE). However, examiners have reported several issues with the Pa-OSCE system, including lost assessment sheets, missing details such as students numbers and names, and illegible handwriting ( 19 ) . Additionally, OSCE scores are not immediately available, which creates time pressure and reduces efficiency in resource and management of OSCE examiners ( 20 ) . Our previous study confirmed the concerns associated with Pa-OSCE, including significant paper waste, excessive time spent by examiners manually computing the score based on the checklist, and the checklist’s absence of an appropriate feedback column for evaluating examinee performance ( 21 ) . Therefore, developing an online OSCE (On-OSCE) scoring management system is essential, particularly in Indonesian nursing education institutions, to address these gaps in the Pa-OSCE ( 22 ) .
Several studies have evaluated assessors’ satisfaction with the implementation of the online OSCE scoring management system ( 13 , 16 , 23 ) . These studies have explored examiners’ perceptions and attitudes toward the online scoring management system in nursing education in Ireland ( 23 ) . The findings indicated high levels of examiner satisfaction. The online OSCE scoring management system reduced the missing data and improved the provision of timely feedback to students ( 23 ) . Similarly, a study on the online assessment of OSCE in pharmacist education institutions in Taiwan reported that examiners demonstrated high recognition, acceptance, and satisfaction with the online assessment system ( 24 ) . Additionally, the online OSCE scoring management system facilitates data storage and enables detailed analysis of overall group or individual students performance, which supports further teaching and learning evaluation ( 23 ) .
Although the online OSCE scoring management system is well-established in developed countries ( 3 , 6 , 23 ) , its implementation in developing countries such as Indonesia represents a novel approach, particularly in nursing education. Our previous study revealed that the common OSCE scoring management system in Indonesia relied on traditional paper-based scoring management system (Pa-OSCE) ( 25 ) . To address issues such as excessive paper waste and time consumption, our nursing department has developed an online scoring system (On-OSCE) ( 21 ) . To ensure the effectiveness and acceptance of the On-OSCE among administrators and examiners accustomed to paper-based assessment techniques, it is crucial to evaluate its practical use. Thus, this study aims to assess examiner satisfaction with the On-OSCE scoring management system, identify, and weigh the potential benefits of this innovation, and promote its adoption.
Method
Study design
This study utilized a cross-sectional approach, wherein the investigator simultaneously measures both the outcomes and the participants’ exposures. The study was conducted at the School of Nursing, Universitas Muhammadiyah Yogyakarta, Indonesia, from June 21, 2019 to February 28, 2021. Data collection spanned three years to support the transition from a paper-based OSCE to an online OSCE scoring management system, and to enable data capture from both groups (paper-based vs. online).
Participants
We used a total sampling technique in this study. A total of 30 examiners using a paper-based OSCE scoring management system (Pa-OSCE) and 37 examiners using an online OSCE scoring management system (On-OSCE) participated in the study. The examiners were nursing lecturers or lecturer assistants from the School of Nursing, Universitas Muhammadiyah Yogyakarta, Indonesia, who regularly conduct OSCE assessments at the nursing school.
Instruments
Before collecting the data, we developed a satisfaction inventory containing 20 items spanning four domains: time-saving (6 items), user-friendliness (5 items), prospective application (4 items), and objectivity (5 items), as detailed in Table 1 . The subjects were asked to rate their satisfaction with each item on an 11-point Likert-type scale ranging from unsatisfied (0) to highly satisfied (10) ( 26 ) . The instrument was developed by researchers based on the literature review on this topic. We categorized subjects’ satisfaction into three groups based on their total score; 0-70 indicated low satisfaction , 71-140 indicated medium satisfaction , and 141-200 indicated high satisfaction .
We recruited five faculty members with experience in Pa-OSCE and On-OSCE to conduct an inventory content validity index (CVI). The CVI consists of content suitable (CVIs) and word precise (CVIw) ( 27 ) . The CVI scores of five experts were 0.98. Three or more experts with an average CVI of 0.78 or higher are considered to have good content validity ( 28 ) . We also examined the item-total correlation value across the four domains (a total of 20 questions) to assess examiners’ satisfaction using the Pa-OSCE and the On-OSCE scoring management system. Items with a correlation value greater than 0.3 were retained. The Composite Reliability (CR) for the extreme was statistically significant ( p < 0.05).
The reliability (Cronbach’s α) and internal consistency (comparisons of extreme groups and corrected item-total correlation) were analyzed using the data from 15 examiners who conducted the Pa-OSCE scoring management system and 15 examiners who conducted the On-OSCE scoring management system. Cronbach’s α score was 0.97. With an alpha > 0.60, the questionnaire can be considered reliable ( 26 , 29 ) .
Data collection
Research invitations were sent via WhatsApp application to all eligible participants. Participants who expressed willingness to participate in the study could contact the researchers if further information related to the research were needed. The survey of examiners’ satisfaction was administered online using Google Forms. The Google Forms included 20 questions covering four variables (time-saving, user-friendliness, prospective application, and objectivity). Completing and submitting the Google forms indicated participants’ consent to participate in this study.
Data analysis
The quantitative data obtained from examiner performance at OSCE stations and the questionnaire survey were analyzed descriptively using Microsoft Excel 365 for Windows 10 (Microsoft Corporation, Redmond, WA, USA) and IBM SPSS statistics version 22 statistical software (IBM Corporation, Armonk, NY, USA). Descriptive data were presented in frequency and percentage for categorical data such as participants’ gender, education, and teaching experience, or mean for continuous variables such as participants’ age. Mean rank was used to describe data in minutes to finish the scoring and satisfaction scores in four domains (time-saving, user-friendliness, prospective application, and objectivity).
Further analysis was conducted to ascertain differences between the Pa-OSCE and On-OSCE groups in demographics, the time required to complete students’ scores, and satisfaction from examiners’ perspectives. Appropriate tests, such as Fisher’s Exact, Chi-square, or Mann-Whitney U tests, were used to investigate mean differences between the paper-based and the online OSCE scoring groups on the four examiners’ satisfaction domains. Data for total examiners’ satisfaction scores were initially categorized into three categories (low, medium, and high). However, due to the very few responses in the low satisfaction category, this group was merged with the medium satisfaction category to enable meaningful analysis. Statistical significance was defined as p < 0.05.
Ethical considerations
Ethical clearance was approved by the Ethical Committee of the Faculty of Medicine and Health Sciences, Universitas Muhammadiyah Yogyakarta (no. 051/EC-KEPK FKIK UMY/II/2019). Participants were informed about the study’s goal and provided verbal consent to participate after being fully informed. They were assured that their data remained confidential. Responses were kept anonymous, and a non-disclosure agreement was established, giving the respondents the right to withdraw from the study at any time.
Results
Demographic data of the participants
Table 2 shows statistical differences between groups based on the categories of age, gender, educational level, and teaching experience. The result shows that participants’ characteristics in both groups were quite similar, except for age. The category of age was significantly different among these four variables, while gender, education level, and teaching experience were not significant ( p > 0.05). The mean age of respondents was 34.10 ± 5.58 in the Pa-OSCE group and 30.70 ± 5.78 in the On-OSCE group ( p = 0.011). Most examiners were female, in both the Pa-OSCE group (90%) or the On-OSCE group (89.19%). Most examiners held graduate degree levels, with 66.67% in the Pa-OSCE group and 56.76% in the On-OSCE group. The respondents’ teaching experience was categorized as less than or equal to one year experienced and less than ten years experienced.
Examiners spent less time scoring with On-OSCE
There was a statistically significant difference in time required to finish the scoring OSCE in Pa-OSCE and On-OSCE. The mean rank of the On-OSCE group was lower than Pa-OSCE group, as shown in Table 3 . Pa-OSCE examiners spent more time calculating the score, whereas the scores in the On-OSCE were calculated automatically by the system. The data reveal that examiners saved more time in assigning the final score to the student in the On-OSCE group.
The examiners had positive feedback on the On-OSCE scoring management system
Satisfaction scores in four domains of the inventory are shown in Table 4 . A statistically significant difference was reported between the groups. The results indicated that the On-OSCE offered time-saving, user-friendliness, prospective applications, and objectivity compared to the Pa-OSCE. On-OSCE improved scoring efficiency (time-saving) and objectivity and was accepted (user-friendliness) for future use (prospective applications) by the majority of examiners.
Examiners’ satisfaction was higher in the On-OSCE group compared to the Pa-OSCE group
We combined the low and medium total satisfaction scores categories for statistical reasons. Examiners’ satisfaction with Pa-OSCE and On-OSCE is shown in Table 5 . A high proportion of On-OSCE examiners (97.30%) were highly satisfied with scoring students’ skills, while a high proportion of Pa-OSCE examiners (86.67%) reported low satisfaction levels. Tables 3 and 4 demonstrate that On-OSCE improved scoring efficiency and objectivity, with a high proportion of the examiners expressing satisfaction with the new scoring management system.
Discussion
This study aims to comprehensively assess examiners’ satisfaction with the On-OSCE scoring management system, leveraging validated tools developed through factor analysis. The evaluation was conducted across four key aspects (timesaving, user-friendliness, prospective application, and objective) with a high validity and reliability inventory. Moreover, this study conclusively demonstrated that examiners were able to save significant time when assigning final scores to students within the On-OSCE scoring management system ( Table 3 ). Among four key aspects, the On-OSCE group showed high satisfaction ( Table 4 ). The results showed a high proportion (97.30%) of the examiners were satisfied with the efficiency (timesaving) and objectiveness of the scoring and accepted for future application (user-friendliness and prospective application) compared to the Pa-OSCE ( p < 0.001) ( Table 5 ).
One of the most heartbreaking findings in our context was the dramatic improvement in time finishing score; to calculate the final score and to decide whether the examinee passed or failed on the skill being examined. The study showed that by employing the On-OSCE scoring management system, examiners required less time to complete the scoring process than the Pa-OSCE scoring management system ( Table 3 ). Our hypothesis was coherent with several studies indicating that the online OSCE scoring management system reduced the official result faster than the traditional OSCE scoring management system ( 22 , 30 ) . There are four reasons why the On-OSCE scoring management system was faster than the Pa-OSCE scoring management system. First, the On-OSCE scoring management system calculated students’ scores automatically, ensured a non-missing checklist was submitted, marked easily, and had no handwriting needed. Second, the online OSCE scoring management system (tablet or PC) allows examiners to mark the checklist easily, effortlessly, and continuously ( 10 ) . Additionally, the marked checklist by the examiners is processed automatically by the system, causing students’ scores to appear immediately ( 24 ) . Third, the On-OSCE scoring management system prompts against missing marks by the systems. As with the alarm system, the examiner receives an alert on the tablet or PC screen if a mark is missing ( 24 ) . Fourth, the On-OSCE scoring management system grants examiners a straightforward spot on the checklist ( 7 , 22 ) . The examiners easily “click” the On-OSCE checklist instead of marking it by pen on the Pa-OSCE scoring management system. Examiners merely use the On-OSCE scoring management system by flipping over the checklist sheet by dragging or scrolling the checklist on the screen ( 30 - 31 ) . Thus, the On-OSCE scoring management system was superior to the Pa-OSCE scoring management system in completing student assessments. These findings believe that the application would be beneficial for nursing students and nursing institutions as our prior studies stated that students were satisfied with being assessed by the On-OSCE scoring management system, as they retrieved their scores immediately and their scores were incredibly accurate ( 15 ) . Additionally, we assessed the readiness of nursing education in Indonesia for implementing the On-OSCE scoring management system and hence acknowledged that most of them were ready to realize this application in their institution ( 25 ) .
Assessing examiners’ satisfaction with the implementation of the On-OSCE scoring management system is vital to gain an understanding, particularly of its feasibility, and provide fundamental information for further improvement programs. Our results showed that examiners were highly satisfied with giving students’ scores using the On-OSCE scoring management system (97.30%) compared to the Pa-OSCE scoring management system, which conveyed low satisfaction (86.67%) ( Table 5 ). Regarding the utilization of online OSCE scoring management systems, examiners assessed their satisfaction level in four domains: time savings, user-friendliness, prospective applications, and objectives ( Table 4 ). Other studies discovered that electronic tools facilitated the analysis of aggregated results, resulting in significant time savings ( 13 , 30 ) . Similarly, utilizing our unique electronic system significantly reduced the time required for data analysis, providing additional time for data interpretation to improve curriculum development and the evaluation of the learning process which might be one of the reasons why this scoring management system satisfied examiners. Furthermore, the On-OSCE scoring management system became an effortless future application for examiners since they were quickly recognized and familiar with the system; examiners reported being highly satisfied in all four domains. However, the authors stated that examiners must obtain adequate training and practice to be confident in using the computerized assessment system ( 24 ) .
According to the findings of this study, it was realized that there were three major reasons for the examiners’ satisfaction with utilizing the On-OSCE scoring management system. First, they might sign or create comments on the assessment form at any moment, which they could not do with the paper-based scoring management system. The online OSCE scoring management system was designed so examiners could effectively provide their feedback adequately. This finding aligns with previous research indicating that online OSCE offers a viable means to evaluate students’ competencies and provide immediate feedback on assessed abilities ( 11 , 22 ) . Second, the On-OSCE scoring management system disabled the add/subtract score function, making it easier for the examiners to objectively give the students’ overall scores according to their performance. Adding or subtracting the students’ scores is usually done by examiners in the Pa-OSCE scoring management system, decreasing the scoring objectivity. Third, Table 2 shows that there is a statistically significant difference in age between examiners. Comparing the ages, the On-OSCE group is younger than the Pa-OSCE group. It might be because young adult examiners have experience in digital competencies such as computer use, internet services ( 32 ) , and being exposed to better technological developments compared to older people ( 33 ) . Therefore, future work should be considered, especially to analyze factors that affect examiners’ decisions in the electronic OSCE assessment system.
During the COVID-19 pandemic, the On-OSCE scoring management system allows examiners to examine students’ skills without face-to-face assessment ( 31 ) . Moreover, a study acknowledged that students’ favorable response to the modified online OSCE scoring management system supports an online assessment to enhance the traditional OSCE scoring management system ( 7 ) . However, students believed that the On-OSCE scoring management system makes examiners overlook their behaviors because they focus on laptops or tablets, which may increase students’ anxiety ( 15 ) . Furthermore, the On-OSCE scoring management system can be utilized in conjunction with the traditional OSCE scoring management system as long as institutions have an adequate internet connection, appropriate hardware (tablet or PC), legitimate skill checklists, and appropriately trained examiners ( 21 ) .
Clinical proficiency is examined consistently, comprehensively, and organized on a wide spectrum. The examination highly demands an objective process in which the examiner’s bias must be diminished, and the discrimination between students’ performance levels must be shown ( 22 ) . The hypothesis was coherent with studies showing that the online OSCE scoring management system has reduced the administrative issue ( 6 , 30 ) . Moreover, our results, consistent with the study, revealed that the electronic software saved time and eliminated the possibility of missing data ( 23 ) . In addition, the potential benefits of electronic software include the ability to store, and analyze aggregate and individual findings, and provide students with immediate objective feedback ( 23 ) .
This study still has limitations. First, as a single-site study, the findings may not generalize to other institutions, nations, or healthcare specialties. Second, we did not assess examiners’ perceptions of the amount of mental effort necessary. The experienced examiners’ mental effort may affect their performance while observing and evaluating the students, which might affect the examiners’ satisfaction level. Thus, future work may be arranged to investigate the examiners’ mental effort on Pa-OSCE and On-OSCE. Third, our study variables and sample size are limited and do not meet the statistical requirements or assumptions required to run the logistic regression test ( 34 ) . Therefore, future research needs to be conducted to assess variables that may contribute to students’ and examiners’ satisfaction comprehensively and improve sample sizes to enable regression tests.
Conclusion
The On-OSCE scoring management system can alleviate the challenges encountered during the rigorous OSCE process. In comparison to the traditional OSCE scoring management system, the On-OSCE scoring management system offers the advantages of improved time-saving, user-friendliness, prospective application, and objectivity in assessing students’ performance. Examiners who are familiar with the system’s functionality demonstrated higher satisfaction with overall function and system utilization than the Pa-OSCE scoring management system. The On-OSCE scoring management system enhances the effectiveness of planning and implementation of the OSCE. It has tremendous advantages and is viable for clinical practice testing, which could be widely employed in practical learning in connected medical domains in the future. Moreover, further studies may use the recent 20-item satisfaction inventory to assess nurse educators worldwide.
Acknowledgments
The authors acknowledge contributions from the Research and Innovation Bureau, Universitas Muhammadiyah Yogyakarta that provided funding for this study.
References
-
Harden RM, Stevenson M, Downie WW, Wilson G. Assessment of clinical competence using objective structured examination. Br Med J. 1975;1(5955):447-51. https://doi.org/10.1136/bmj.1.5955.447
» https://doi.org/10.1136/bmj.1.5955.447 -
Vincent SC, Arulappan J, Amirtharaj A, Matua GA, Al Hashmi I. Objective structured clinical examination vs traditional clinical examination to evaluate students’ clinical competence: A systematic review of nursing faculty and students’ perceptions and experiences. Nurse Educ Today. 2022;108:105170. https://doi.org/10.1016/j.nedt.2021.105170
» https://doi.org/10.1016/j.nedt.2021.105170 -
Cristobal E, Perkins K, Kang C, Chen S. Impact of a Mock OSCE on Student Confidence in Applying the Pharmacists’ Patient Care Process. Pharmacy (Basel). 2024;12(2):54. https://doi.org/10.3390/pharmacy12020054
» https://doi.org/10.3390/pharmacy12020054 -
Shimizu I, Mori J, Yamauchi A, Kato S, Masuda Y, Nakazawa Y, et al. Progress testing of an objective structured clinical examination during undergraduate clinical clerkship: a mixed-methods pilot study. BMC Med Educ. 2023;23(1):958. https://doi.org/10.1186/s12909-023-04940-8
» https://doi.org/10.1186/s12909-023-04940-8 -
Kassabry MF. Evaluation of simulation using objective structured clinical examination (OSCE) among undergraduate nursing students: A systematic review. Int J Africa Nurs Sci. 2023;18:100553. https://doi.org/10.1016/j.ijans.2023.100553
» https://doi.org/10.1016/j.ijans.2023.100553 -
Tseng WY, Lai EHH, Wang TM, Wang CY, Lin TF, Lin LD, et al. The multi-factor analysis of computer-based test scores in objective structured clinical examination of clerkship in school of dentistry, National Taiwan University. J Dent Sci. 2023;18(2):754-60. https://doi.org/10.1016/j.jds.2022.12.012
» https://doi.org/10.1016/j.jds.2022.12.012 -
Indarwati F, Primanda Y, Haris F, Sutrisno RY. Content validity and inter-rater reliability of procedural skill checklists used in the online OSCE scoring management system. Bali Med J. 2023;12(1):456-61. https://doi.org/10.15562/bmj.v12i1.3760
» https://doi.org/10.15562/bmj.v12i1.3760 -
Guerrero JG, Rosales NS, Castro GMT. Impact of high-fidelity simulation exposure of nursing students with their objective structured clinical examination: A quasi-experimental study. Nurs Open. 2023;10(2): 765-72. https://doi.org/10.1002/nop2.1343
» https://doi.org/10.1002/nop2.1343 -
Abbasi MA, Imannezhad S, Movahedinia S, Shojaei H, Fazli B. Different Assessment Tools for Evaluating Objective Structured Clinical Examinations in Medical Education: A Rapid Review. Med Edu Bull. 2023;4(4):14. https://doi.org/10.22034/meb.2023.433369.1087
» https://doi.org/10.22034/meb.2023.433369.1087 -
Kharaba Z, Al Ahmed M, Elnour AA, AbouHajal A, Abumweis S, Ghattas MA. Are we ready yet for digital transformation? Virtual versus On-Campus Objective Structured Clinical Examination (OSCE) as an assessment tools in pharmacy education. A randomized controlled head-to-head comparative assessment [Preprint]. Research Square. 2022. https://doi.org/10.21203/rs.3.rs-1893972/v1
» https://doi.org/10.21203/rs.3.rs-1893972/v1 -
Chang O, Holbrook AM, Lohit S, Deng J, Xu J, Lee M, et al. Comparability of Objective Structured Clinical Examinations (OSCEs) and written tests for assessing medical school students’ competencies: a scoping review. Eval Health Prof. 2023;46(3):213-24. https://doi.org/10.1177/01632787231165797
» https://doi.org/10.1177/01632787231165797 -
Chan SCC, Choa G, Kelly J, Maru D, Rashid MA. Implementation of virtual OSCE in health professions education: A systematic review. Med Educ. 2023;57(9):833-43. https://doi.org/10.1111/medu.15089
» https://doi.org/10.1111/medu.15089 -
Berendes S, Schaper E, Tipold A, Wissing S. Evaluation of the eOSCE for testing clinical skills. Front Vet Sci. 2023;10:1196311. https://doi.org/10.3389/fvets.2023.1196311
» https://doi.org/10.3389/fvets.2023.1196311 -
Alamri S, Al Hashmi I, Shruba K, Jamaan S, Alrahbi Z, Al Kaabi T. Nursing Students’ Perception and Attitude towards Objective Structured Clinical Examination in Oman. Sultan Qaboos University Med J. 2022;22(3):343. https://doi.org/10.18295%2Fsqumj.2.2022.012
» https://doi.org/10.18295%2Fsqumj.2.2022.012 -
Sutrisno RY, Primanda Y, Haris F. Students’ Satisfaction on Online Nursing OSCE (ON-OSCE) Assessment Application. Indonesian J Nurs Pract. 2020;4(2). https://doi.org/10.18196/ijnp.v4i2.10142
» https://doi.org/10.18196/ijnp.v4i2.10142 -
Manuaba IBAP, Yani MVW. Evaluation of the Implementation of Objective Structural Clinical Examination (OSCE) in the Faculty of Medicine: a Literature Review. Bali Med J. 2023;12(1):1163-6. https://doi.org/10.15562/bmj.v12i1.4383
» https://doi.org/10.15562/bmj.v12i1.4383 -
Shariff A, Kumar TMP, Siddalingegowda SM, Wani SUD, Elsobky Y. PharmD students experiences on end year clerkship evaluation using objective structured clinical exam (OSCE) method at JSS College of Pharmacy Mysuru India. PEC Innov. 2024;4:100271. https://doi.org/10.1016/j.pecinn.2024.100271
» https://doi.org/10.1016/j.pecinn.2024.100271 -
Barnes KN, Hardinger KL, Graham MR. Review of Objective Structured Clinical Examination Practices within Pharmacy Programs throughout the United States. Am J Pharm Educ. 2024;88(4):100686. https://doi.org/10.1016/j.ajpe.2024.100686
» https://doi.org/10.1016/j.ajpe.2024.100686 -
Snodgrass SJ, Ashby SE, Onyango L, Russell T, Rivett DA. Electronic practical skills assessments in the health professions: a review. Internet J Allied Health Sci Pract. 2014;12(1):8. https://doi.org/10.46743/1540-580X/2014.1471
» https://doi.org/10.46743/1540-580X/2014.1471 -
Natarajan J, Thomas D. Integrative Review Literature on Objective Structured Clinical Examination and its implications in Nursing Education. IOSR J Nurs Health Sci [Internet]. 2014 [cited 2023 Apr 14];3(4):23-30. Available from: https://www.iosrjournals.org/iosr-jnhs/papers/vol3-issue4/Version-1/F03412330.pdf
» https://www.iosrjournals.org/iosr-jnhs/papers/vol3-issue4/Version-1/F03412330.pdf -
Primanda Y, Sutrisno R, Haris F. The Development of Online OSCE Prototype for OSCE in School of Nursing: Lesson Learned. In: Proceedings of the Third International Conference on Sustainable Innovation 2019–Health Science and Nursing (IcoSIHSN 2019) [Internet]; 2019 July 30-31; Yogyakarta, Indonesia. Amsterdam: Atlantis Press; 2019 [cited 2023 Apr 14]. Available from: https://doi.org/10.2991/icosihsn-19.2019.47
» https://doi.org/10.2991/icosihsn-19.2019.47 -
Wagner FL, Feller S, Schmitz FM, Zimmermann PG, Krings R, Guttormsen S, et al. Usability and preference of electronic vs. paper and pencil OSCE checklists by examiners and influence of checklist type on missed ratings in the Swiss Federal Licensing Exam. GMS J Med Educ. 2022;39(2):Doc24. https://doi.org/10.3205/zma001545
» https://doi.org/10.3205/zma001545 -
Meskell P, Burke E, Kropmans TJ, Byrne E, Setyonugroho W, Kennedy KM. Back to the future: An online OSCE Management Information System for nursing OSCEs. Nurse Educ Today. 2015;35(11):1091-6. https://doi.org/10.1016/j.nedt.2015.06.010
» https://doi.org/10.1016/j.nedt.2015.06.010 -
Chuo WH, Lee CY, Wang TS, Huang PS, Lin HH, Wen MC, et al. Evaluate the Feasibility of the Implementation of E-Assessment in Objective Structured Clinical Examination (OSCE) in Pharmacy Education from the Examiner’s Perspectives. Educ Sci. 2021;11(5):194. https://doi.org/10.3390/educsci11050194
» https://doi.org/10.3390/educsci11050194 -
Primanda Y, Indarwati F, Haris F, Sutrisno RY. Nursing education institutions’ readiness to use online objective structured clinical examination scoring management system. Bali Med J. 2023;12(2):2185-90. https://doi.org/10.15562/bmj.v12i2.3700
» https://doi.org/10.15562/bmj.v12i2.3700 -
Koustelios AD, Bagiatis K. The Employee Satisfaction Inventory (ESI): Development of a scale to measure satisfaction of Greek employees. Educ Psychol Meas. 1997;57(3):469-76. https://doi.org/10.1177/0013164497057003008
» https://doi.org/10.1177/0013164497057003008 -
Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nursing Health. 2006;29(5):489-97. https://doi.org/10.1002/nur.20147
» https://doi.org/10.1002/nur.20147 -
Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459-67. https://doi.org/10.1002/nur.20199
» https://doi.org/10.1002/nur.20199 -
Kregel J, Vuijk PJ, Descheemaeker F, Keizer D, van der Noord R, Nijs J, et al. The Dutch Central Sensitization Inventory (CSI): factor analysis, discriminative power, and test-retest reliability. Clin J Pain. 2016;32(7):624-30. https://doi.org/10.1097/AJP.0000000000000306
» https://doi.org/10.1097/AJP.0000000000000306 -
Yousef AA, Awary BH, AlQurashi FO, Albuali WH, Al-Qahtani MH, Husain SI, et al. Innovative method to digitize a web-based OSCE evaluation system for medical students: A cross-sectional study in university hospital in Saudi Arabia. Int J Gen Med. 2022:1085-95. https://doi.org/10.2147/IJGM.S351052
» https://doi.org/10.2147/IJGM.S351052 -
Bhuiyan AAM, Islam AM. E-assessment during COVID-19 pandemic at a Saudi university: impact on assessment methods and course learning outcomes. J Lang Cult Educ. 2023;11(3):1-11. https://doi.org/10.2478/jolace-2023-0021
» https://doi.org/10.2478/jolace-2023-0021 -
Heponiemi T, Kaihlanen AM, Kouvonen A, Leemann L, Taipale S, Gluschkoff K. The role of age and digital competence on the use of online health and social care services: a cross-sectional population-based survey. Digit Health. 2022;8:20552076221074485. https://doi.org/10.1177/20552076221074485
» https://doi.org/10.1177/20552076221074485 -
Haase M, Bosse M, Sackmann S, Meyer G. IT Skills of Young and Older People: A Qualitative Study. Int J Hum-Comput Int. 2024:1-11. https://doi.org/10.1080/10447318.2024.2319916
» https://doi.org/10.1080/10447318.2024.2319916 -
Harris JK. Primer on binary logistic regression. Fam Med Community Health. 2021;9(Suppl 1):e001290. https://doi.org/10.1136/fmch-2021-001290
» https://doi.org/10.1136/fmch-2021-001290
-
All authors approved the final version of the text.
-
*
Supported by Ministry of Science and Technology Taiwan, Grant # MOST 109-2314-B-468-006-MY3, Taiwan, and by Research and Innovation Bureau, Universitas Muhammadiyah Yogyakarta, Grant # 550/PEN-LP3M/II/2020, Indonesia.
-
How to cite this article
Haris F, Indarwati F, Primanda Y, Sutrisno RY, Irawati K, Shih YH. Nurse educators’ satisfaction with online Objective Structured Clinical Examination scoring system. Rev. Latino-Am. Enfermagem. 2024;32:e4344 [cited]. Available from: . https://doi.org/10.1590/1518-8345.6816.4344
Edited by
-
Associate Editor:
Andrea Bernardes
