Objective structured clinical examination , OSCEs : an advance in the teaching and learning process in the student ’ s perception

Introduction: The OSCE is a method of clinical competencies evaluation that has gained international popularity in medical and dental education. Objective: The purpose of the present study was to describe the stages of development of the OSCE for the undergraduate course in Dentistry and to report the students' perception about this method of evaluation, regarding the degree of difficulty, time for each stage, importance of each station, number of stations, organization of the exam, as well as the total time for the OSCE. Material and method: This research was an observational and cross-sectional study, composed of the carry out of an OSCE and later application of an evaluative questionnaire to the students who were in the fourth semester of the Dentistry course. These students had basic knowledge and an intermediate level of competences, compatible with their stage of education. Result: As regards the number of stations, 43(97.7%) of the students responded that this was appropriate, OSCE process as a whole was very well organized (n=25; 56.8%), organized (n=17; 38.6%) or not very Objective structured clinical... Rev Odontol UNESP. 2019;48:e20190027. DOI: https://doi.org/10.1590/1807-2577.02719 2/10 organized (n=1; 2.27%) and about the total time of the OSCE process, 29(65.9%) reported that it was appropriate; 10 (2.27%) said that it was short; 4 (9.09%), that it was long. Conclusion: The student’s perception was positive especially regarding to organization and the time attributed to each station. Furthermore, the students considered that the topics and questions applied in each station were relevant. Descriptors: Dental education; dental students; perception.


INTRODUCTION
The clinical competence of dental students is usually assessed in terms of the number of case observations and/or assistance with a professor's treatment and their performance of procedures in the student dental clinic 1 .Traditional methods of assessment in dental education often concentrated on student knowledge and memorization abilities rather than on the cognitive skills needed for clinical practice.Traditional examinations also cannot assess how effectively higher level cognition is applied when performance-based criteria are involved 2 .Comprehensive assessment of students' academic performance plays an important role in educational planning 3 .
Unlike the traditional methods, OSCEs (Objective Structured Clinical Examination) is a method for assessment of clinical competences 4 .The OSCE was introduced by Harden in 1975 5,6 and has attained international popularity in medical and health care education 7,8 .One of the main strengths of the OSCE examination is its inherent objectivity whereby the aim is to remove patient and examiner variation so that the only variable being examined is the ability of the candidate.These examinations involve students rotating around a series of pre-determined, specifically designed stations which assess clinical and communication competence.Examiners assess student performance objectively against pre-determined, structured criteria 5 .This means that it is possible to examine a range of skills and disciplines and even to incorporate more than one skill or discipline simultaneously in the design of a particular station 9 .Examples of competences applicable to a range of disciplines include communication skills, aspects of anamnesis and biosafety.The disadvantages of the method are: greater exam preparation time; demanding in nature for both examiners and patients, and an OSCE is considered more cost-effective than other assessment methods 10 .
Assessment of student's perception must be explored to understand innovative strategies for the assessment of dental practice.Analysis of perceptions specifically about the educational value of an exam allows educators to know whether a method is very successful in promoting important competences, such as the application of knowledge and solution of problems 11 .Studies have found evidence that dental students perceived OSCE as an efficient and significant method of assessment, as well as a positive learning experience 12-14 .The Dentistry Course at the School of Medical and Health Sciences of Juiz de Fora (SUPREMA) has been using OSCE since 2014, with the intention of adopting an evaluative approach with focus on the clinical competence.As OSCE is a form of evaluating the competences in all their domains, it is fundamental to analyze the perception of students about this practice.Therefore, the aim of the present study was to describe the stages of development of the OSCE for the undergraduate course in Dentistry and to assess the Dental students' perception of the OSCE regarding to the degree of difficulty, time for each stage, importance of each station, number of stations, organization of the exam, as well as the total time for the OSCE.

METHOD
The study involved the implementation of the OSCE and the subsequent application of an evaluation questionnaire to the students who were in the fourth semester of the Dentistry course of the School of Medical and Health Sciences of Juiz de Fora (SUPREMA), Minas Gerais, Brazil.These students (n=44) had basic knowledge and an intermediate level of competences, compatible with their stage of education.The Dentistry course is a four-year long program.All the students were invited and agreed to answer the questionnaire.The inclusion criteria were to be in the fourth period and to have carried out the OSCE.This was an observational and crosssectional study and it was perfomed in 2018.

OSCE Stations
Six interdisciplinary stations were developed, approaching the contents of anatomy, biosafety, dentistry, periodontics, dental materials and radiology.The stations were developed to contain their respective evaluation criteria.For each station, a check-list was cognitive, affective and psychomotor capacities for performing specific tasks were examined.The test was conducted in a standardized manner, with a structured method divided into stations according to each discipline.Each student (n= 44) completed a six-station circuit (Table 1).Six students entered at the same time, and each of them went to a different station.They took turns until they completed a circuit.

Duration of Stations
Each station lasted 6 minutes: one minute to read the purpose of the station, and five minutes to perform the task proposed.The students had to remain in the station for a total of 5 minutes, even if the completed the task in a shorter time.A sound alert went off to mark these time intervals and for continuity of the circuit.

Check-List and Data Analysis
In each station there were two examiners, who had a check-list as reference, to observe and evaluate each student.Scoring was calculated to obtain a final grade on a numerical scale of achievement.The results of the two examiner's assessments of each station were quantified, and the final score obtained was the arithmetic mean that was used as one of the assessment criteria of a curricular discipline of the mentioned course.

Questionnaire
At the end of the circuit, the examiners asked the students (n= 44) to fill in an anonymous questionnaire that contained questions relative to the degree of difficulty, time of performing the tasks, and the importance of the stations, impressions relative to the number of stations, organization and the total time of the OSCE.The questionnaire was adapted from Näpänkangas et al. 15 .
Regarding to each station, the following questions were done: 1) Regarding to the difficult degree, this station was:

Feed-back
After the end of the circuit by all students, there was a time of feed-back in which the teachersevaluators explained the criteria assessed in each station.The students were also encouraged to report their doubts and anxieties regarding to the process.This study was approved by the Research Ethics Committee of the School of Medical and Health Sciences of Juiz de Fora (SUPREMA); Protocol No. 58497516.4/0000.5103.The results are presented as percentage distribution.

RESULT
The students (n=44; 35 women and 9 men with a mean age of 20 years) filled out the questionnaire.They pointed out their impressions relative to the number of stations, organization and total time of duration of OSCE.
As regards the number of stations, 43(97.7%) of the students responded that this was appropriate.To the students, the OSCE process as a whole was very well organized (n=25; 56.8%), organized (n=17; 38.6%) or not very organized (n=1; 2.27%).One questionnaire was excluded because the student scored several alternatives.About the total time of the OSCE process, 29(65.9%)reported that it was appropriate; 10 (2.27%) said that it was short; 4 (9.09%), that it was long.
The students also filled out a questionnaire that contained questions relative to the degree of difficulty, time for performing the tasks and the importance of the content approached in each of the proposed stations (Figures 1, 2 and 3).The time attributed to each station was considered appropriate (65.9%).Furthermore, the students considered that the topics and questions applied in each station were relevant.Exclusion refers to some questions that presented more than one response marked.

DISCUSSION
The purpose of this study was to describe the stages of developing an OSCE for the Dentistry course and to find out the perception of students about this new assessment format, in comparison with the models previously used.OSCE is a summative assessment of students' graduation outcomes.The OSCE results are thought to predict the extent to which students had learned the related course material and their performance as general dentists 16 .
The specificities of the health area may be observed in the decisions to be taken, which involve the articulation of various fields of knowledge arising from the general education (with emphasis on scientific knowledge), professional education (with emphasis on technical knowledge) and work and social experiences (tacit qualifications) that are mediated by the ethical-political dimension 17 .Studies have shown that a positive correlation has been identified between the performance of undergraduate students in Dentistry in OSCE and their clinical and didactic performance, which corroborates the value of OSCE as an efficient and complete evaluative instrument 18 .However, OSCE has limitations in assessing students' capacity to perform clinical procedures, in large part because it is not feasible to ask students to perform invasive and nonreversible procedures on patients.This may be real or simulated yet students have a limited amount of time at each OSCE station to perform tasks, usually less than would be available in a real clinical setting for invasive or more complex procedures 19,20 .
Studies that have examined dental OSCEs from a student's perspective have similarly found that the examination is generally perceived favorably by students and seen to be a good test of clinical skills 12,13 .When dental OSCEs assess focused aspects of clinical skills rather than being comprehensive, they have also been viewed positively by students 14,21 .In this study, students' perception was positive, especially with regard to organization (56.8% considered the OSCE very organized and 38.6% organized); and the time attributed to each station (65.9% of the students considered the time appropriate).Furthermore, the students considered that the topics and questions applied in each station were relevant.Although, another study 15 considered the time allocated too short (63%).
Regarding to the degree of difficulty of each station, two among the 6 stations -that of Periodontics and Anatomy -were considered difficult by the majority of the students, whereas, the other stations were considered easy.Similar results were found in the School of Dentistry of the University of Jordan 14 with regard to organization of the OSCE, because 75% of the students judged the OSCE of Jordan organized; the time of stations appropriate (80.2%) and considered a good portion of the stations easy (54.1%).Another study 15 evaluated the validity of the OSCE by measuring the attitude of examiners and dental students towards the OSCE and it was considered easy for 87% of the students.
Planning, elaboration and application of the OSCE is far more complex and work-intensive than the conventional models of assessment 22 .It demands greater and integrated involvement of the staff members among the different disciplines, and more time dedicated from the time of planning through to application of the OSCE.To carry out the OSCE in a group of 44 students with a circuit of 6 stations, the involvement of 12 examiners in the morning shift, and 12 examiners in the afternoon shift was required.It was also necessary to consider the questions of logistics, physical area with the necessary space to mount the stations, as well and the material and instruments necessary, according to each station planned.For standardization and reliability of the exam, all the material was provided by the Institution.In relation to the quantity of stations, the Jordanian student considered the 67 stations a very high number 14 , differently from the students of this study, who judged the 6 station to be appropriate.Another study, considered 8 stations a good number with credible assessment results, and indices such as mean and discrimination coefficient of test scores from each station appropriated 4 .Therefore, the proposed protocol was found to be reliable and suitable.
Traditional teaching (written, oral assessments including essay and multiple-choice question [MCQ] examinations) has been criticized for focussing simply on abilities to memorize, ignoring manual clinical skills, hypothesis formation and decision-making as well as patient interaction 23 .Reorientation of education has been fostered by the Brazilian Public Policies of Education and Health, since the DCN, and in the case of graduation, passing through the Program of Reorientation of Professional Education in Health (Pró Saúde) and through the Program of Education for Working in the Area of Health (PET Saúde), with wide scope, in the undergraduate courses in health throughout Brazil 24 .Changes in student's evaluation generate many conflicts between administrators, teachers and students, but OSCE is set up to provide an objective evaluation of a student's performance by giving every student the same questions or scenarios from evaluators who rate student responses according to a standardized grading system, thus minimizing interexaminer error.Recently our understanding of dental education, including the OSCE, has been changing gradually 14 .
The strength of this study was the high student response rate.Students were asked to answer the questionnaire anonymously immediately after their last task.Only one questionnaire was excluded from the study due to student response being an array of alternatives.In addition to the limited number of stations available, the main limitation of our study was the small sample size, due to the number of students attending the same dental period (fourth) at our institution.New evaluations regarding the students' perception will be made and new data will be presented subsequently.

CONCLUSION
For the success of OSCE, it is primordial to plan stations that are relevant to clinical practice of the student; with a group of teachers involved in the teaching-learning process, and particularly, to appreciate the students' perception, thereby establishing the formative nature of this evaluation.The student's perception was positive especially regarding to organization and the time attributed to each station.Furthermore, the students considered that the topics and questions applied in each station were relevant.

2 ) 1 )
Regarding to time to carry out the activity of the station, the time was: Regarding to the number of stations, OSCE presents:

Figure 1 .
Figure 1.Quantification of the responses relative to the degree of difficulty of each station.Exclusion refers to some questions that presented more than one response marked.

Figure 2 .
Figure 2. Quantification of the responses relative to the time available for each station.Exclusion refers to some questions that presented more than one response marked.

Figure 3 .
Figure 3. Quantification of the responses relative to the importance of the content approached in each station.Exclusion refers to some questions that presented more than one response marked.

Table 1 .
Description of the discipline, activity and check-list of each station corresponding to these structures.Your task will be to identify in the macro model the anatomical structures and paste the labels with the number corresponding to the form in the macro template 1) Identification of the vertex of the cusp 2) Identification of longitudinal edge 3) Transverse edge identification 4) Identification of the vestibular sulcus 5) Identification of the mesial marginal ridge 6) Identification of crushing strand 7) Flat strand identification 8) Identification of the enamel bridge Periodontology At this station you should prepare the ultrasound device and then perform a supragingival scraping on the first molar (left side).1)Use of individual protection equipment (mask, hat, procedure gloves, goggles, white clothing) 2) Setting the power of the appliance 3) Correct insertion of the "TIP" 4) Identification of the correct tooth 5) Angle of use of the tip 6) Positioning the operator 7) Application on all faces of the tooth 8) Correct removal of the "TIP" 3.3 -Triple syringe 4 -Positioned Cloth / Bench Paper 5. Opened the clinical kit without contaminating it under the clinic table.Discipline Activity Check-listDental MaterialsAt this station you will find some dental plasters for modeling dental arches.Your activity will be to select type IV gypsum and properly provide the powder and liquid of this to different degrees.After this step, manipulate it.1 -Selected materials suitable for the task (Spatula and gral for alginate, powder meter and alginate water, Algvinate) 2 -Dose the amount of powder determined using the meter 3 -Dosed the amount of water using the meter 4 -He poured the powder into the water; 5 -Used the handling time correctly; 6 -It resulted in a homogeneous and adequate