Objective structured clinical examination in physiotherapy teaching: a systematic review

Exame clínico objetivo estruturado no ensino da fisioterapia: revisão sistemática

Examen clínico objetivo estructurado en la enseñanza de la fisioterapia: revisión sistemática

Abstract

Introduction:

Problems related to the clinical abilities of physiotherapy students are not always identified throughout the educational process and might only be observed when these future professionals have to treat patients. The Objective Structured Clinical Examination (OSCE) includes a problematization approach and can be used in Health Sciences teaching to help this identification before internship practices. However, there are few studies on its use in Physiotherapy.

Objective:

To gather evidence of the OSCE use to evaluate clinical abilities in Physiotherapy teaching.

Method:

Articles published from 2000 to 2016 were surveyed in the Biblioteca Virtual em Saúde (BVS) (Virtual Health Library), Centro Latino-Americano e do Caribe de Informação em Ciências da Saúde (BIREME) (Latin-American and Caribbean Health Sciences Information Center), PubMed, Scielo and Web of Science, using the descriptors “educational assessment”, “assessment methods”, “objective structured clinical examination”, “clinical competence”, “professional competence”, “clinical skills”, “student competence”, “student skills”, “physiotherapy” and the Booleans “OR” and “AND”.

Results:

The initial number of identified publications was 3,242. From these, seven were included in this review. Two studies were developed in Brazil, four in Australia and one in Canada. The studies were scored 7 to 12 regarding methodologic quality, and 1B and 2B regarding scientific evidence.

Conclusion:

Students’ clinical abilities were grouped into three classes: cognitive, psychomotor and affective, and four studies described their use. There is very little evidence of the use of OSCE, but the instrument can be applied to evaluate skills and competences in Physiotherapy teaching.

Keywords:
Educational Evaluation; Education Based on Competences; Physiotherapy

Resumo

Introdução:

No processo educacional, nem sempre se identificam problemas relacionados às habilidades clínicas dos estudantes, sendo estas observadas quando o futuro profissional está à frente do paciente. Para realizar essa identificação antes das práticas de estágio, foi desenvolvido o Objective Structured Clinical Examination (OSCE). Incluindo uma abordagem problematizadora, o OSCE pode ser usado no ensino de Ciências da Saúde. Contudo, há poucos estudos sobre seu uso na Fisioterapia.

Objetivo:

Sintetizar evidências acerca do uso do OSCE para avaliação de habilidades clínicas no ensino da Fisioterapia.

Método:

Foram selecionados artigos publicados no período de 2000 a 2016 e consultadas as bases de dados Biblioteca Virtual em Saúde (BVS), Centro Latino-Americano e do Caribe de Informação em Ciências da Saúde (BIREME), PubMed, Scielo e Web of Science, utilizando os descritores educational assessment, assessment methods, objective structured clinical examination, clinical competence, professional competence, clinical skills, student competence, student skills, physiotherapy, e os booleanos OR e AND.

Resultados:

O número de publicações identificadas foi de 3.242. Destas, sete foram incluídas nesta revisão. Dois estudos foram desenvolvidos no Brasil, quatro na Austrália e um no Canadá. Os estudos foram pontuados com notas de 7 a 12, relativas à qualidade metodológica, e classificados em 1b e 2b em relação à evidência científica.

Conclusão:

As habilidades clínicas dos estudantes foram agrupadas em três classes - cognitiva, psicomotora e afetiva -, e quatro estudos descreveram o uso das mesmas categorias. Há uma escassez de evidências sobre o OSCE, e o instrumento colabora na avaliação de habilidades e competências no ensino em Fisioterapia.

Palavras-chave:
Avaliação Educacional; Educação Baseada em Competências; Fisioterapia

Resumen

Introducción:

En el proceso educativo, ni siempre se identifican problemas relacionados con las habilidades clínicas de los estudiantes. Estas son observadas cuando el futuro profesional está delante del paciente. Para realizar esta identificación antes de las prácticas, se desarrolló la Objective Structured Clinical Examination (OSCE). Incluyendo un abordaje problematizador, OSCE puede usarse en la enseñanza de las ciencias de la salud, pero hay pocos estudios sobre su uso en la Fisioterapia.

Objetivos:

Sintetizar evidencias acerca del uso del OSCE para evaluación de habilidades clínicas en la enseñanza de la Fisioterapia.

Método:

Se seleccionaron artículos publicados em el periodo de 2000 a 2016 y consultadas las bases de datos Biblioteca Virtual en Salud (BVS), Centro Latinoamericano y del Caribe de Información en Ciencias de la Salud (BIREME), PubMed, Scielo y Web of Science, utilizando los descriptores educational assessment, assessment methods, objective structured clinical examination, clinical competence, professional competence, clinical skills, student competence, student skills, physiotherapy, y los booleanos OR y AND.

Resultados:

El número de publicaciones identificadas fue de 3.242. De estos, siete fueron incluidos en esta revisión. Se desarrollaron dos estudios en Brasil, cuatro en Australia y uno en Canadá. Los estudios se calificaron de 7 a 12 con respecto a la calidad metodológica y se clasificaron 1b y 2b en relación con la evidencia científica.

Conclusión:

Las habilidades clínicas de los estudiantes fueron agrupadas en tres clases, a saber, cognitivo, psicomotor y afectivo, y cuatro estudios describieron el uso de las mismas. Hay una escasez de evidencias sobre el OSCE y el instrumento colabora en la evaluación de habilidades y competencias en la enseñanza en Fisioterapia.

Palabras clave:
Evaluación Educacional; Educación Basada en Competencias; Fisioterapia

Introduction

The evaluation of university students plays an important role in their initial education, either as a guide of the individual formative trajectory or as an adjustment of formative options. It should focus not only on the knowledge achieved (cognitive skills), but also on the psychomotor and affective areas, which jointly configure the necessary professional skills of working as a physiotherapist. Thus, planning an effective evaluation has to consider specific educational objectives of each phase of the qualification, according to the desired profile of competences and skills ^1)-(3, to perform in the different jobs that are included in the definitions of legal norms and pedagogical projects of the course. Competence allows the individual to mobilize knowledge in order to face certain situations promptly and in the necessary manner. Competence is a set of knowledge and abilities, a know-how related to work practice ^{2), (4), (5}.

The education of health professionals requires that the students master clinical abilities such as communication with patients, physical examination, clinical reasoning and proposition of functional evaluation measures along with the therapeutic approach ^{5), (6}. There are several alternatives to evaluate these skills, which frequently approach the practice from questioning. The most reliable alternative, according to Newble ⁷, is the in loco assessment. However, this is not always possible and problems related to skills and attitudes are often only identified when the future professional has to deal with a patient ^8)-(10. Given this context, questioning the evaluation of teaching and learning in the perspective of professional action reinforces the necessary transformation of university education, as pointed out by health education policies.

Aiming at carrying out this identification prior to the start of the mandatory curricular internship, the simulation of attendance evaluation was designed, also known as OSCE - Objective Structured Clinical Examination ^{11), (12}. This method provides to the clinical evaluations the advantages equivalent to objective written tests, in which students are evaluated for the accomplishment of tasks related to the collection of clinical background, physical examination, clinical reasoning and procedure ^{12), (13}.

Nowadays, OSCE is considered one of the most reliable and effective methods to evaluate skills, mainly in the health area ^14)-(16. Most of the studies available in literature refer to the evaluation of students in medical schools ¹⁰. In Brazil, its use outstands in two large medical colleges in the state of São Paulo: the Faculdade de Medicina de Ribeirão Preto da Universidade de São Paulo (FMRP-USP) (Ribeirão Preto Medical School - University of São Paulo) and the Faculdade de Medicina de Marília (FAMEMA) (Marilia Medical School). Other few studies have been developed in Nursing ^{16), (17} and Pharmacy ^{11), (18), (19} courses, for example. The low adhesion to the use of this instrument in universities might be due to lack of knowledge on the tool, lack of professor training or the controversies regarding its evidence in teaching.

Although OSCE is a useful tool, which can also be used in Physiotherapy teaching since it includes this questioning focus, it is not a systematic practice in Higher Education Institutions. Therefore, evidencing positive and negative effects of a tool to investigate students’ clinical skills might interest professors, coordinators and administrators of physiotherapy undergraduate courses to broaden the scope of the student-professor-patient relationship throughout academic education. In this sense, a literature review might provide information about the areas to which it can be applied and on the perspectives of this instrument that might become an effective intervention from its use in higher education. OSCE has not yet received a structured review for implementation in institutions that qualify physiotherapists. Therefore, the objective of this review was to synthesize evidence about the use of OSCE to evaluate clinical skills in Physiotherapy teaching.

Methods

Design

Publications were selected from 2000 to 2016 that discussed OSCE and its use in the education of physiotherapists. There were no restrictions regarding the methodology of the studies, however, publications that were editorial, interviews, projects, pilot studies and preliminary or conceptual data were excluded. Likewise, articles whose theme addressed other educational evaluation methods were excluded, as well as those investigating physiotherapeutic treatment techniques or treatments in other health areas. This strategy was important to identify studies that evidenced the use of this tool in a study as a method of evaluation among other investigations available in literature.

Sources of information

The included studies were published in English, Spanish or Portuguese, and were available at the Biblioteca Virtual em Saúde (BVS) (Health Virtual Library), Centro Latino-Americano e do Caribe de Informação em Ciências da Saúde (BIREME) (Latin-American and Caribbean Health Sciences Information Center), PubMed and Web of Science databases. The search period ended in May 2016. Three search themes were combined using the Boolean operators “AND” and “OR”. The first search included instruments to evaluate students’ clinical skills, combined in the title/abstract from the keywords “educational assessment, assessment methods” or “clinical competence, professional competence, clinical skills, student competence, student skills”. Next, studies that analyzed OSCE were identified using “objective structured clinical examination”. And, finally, a third search was carried out, in which “physiotherapy” was included. The articles should contain the terms listed above, not necessarily all of them, however, the term “objective structured clinical examination” was the most relevant factor for inclusion.

Selection phases

Publications containing the searched keywords in the title or abstract were tracked. Studies with abstracts available and whose complete text could be accessed for free were selected, then read for data extraction. With the previously defined sample, an exploratory reading was carried out, using a proper form, specifically devised for data analysis and extraction, and the recommendations by the Cochrane Library ²⁰ were adjusted according to the authors’ interest. The items considered were: objectives, methodological design, and place where the study was developed; sample size, main results, description of students’ clinical skills and performance.

Evaluation of methodological quality and level of scientific evidence of the studies

The assessment of the methodological quality of the studies was carried out by two independent evaluators (AMR and JSCA). This assessment was based on the evaluation instrument by the McMaster Occupational Therapy Evidence-Based Practice Research Group (http://srs-mcmaster.ca) - Table 1 ^21)-(23. The studies were scored in each one of the 14 items, scoring one point if the methodological rigor was suitable or zero if the methodology was not suitable. The final score could range from 0 to 14 points. The studies were also categorized regarding levels of evidence according to the Oxford Centre for Evidence-based Medicine - Levels of Evidence ²⁴.

Table 1
Evaluation of the methodological quality of the studies

Score	Quantitative studies
0 or 1	Study objective clearly indicated
0 or 1	Relevant literature surveyed
0 or 1	Detailed description of the sample
0 or 1	Justified sample size
0 or 1	Reliable outcome measurements
0 or 1	Valid outcome measurements
0 or 1	Detailed description of intervention
0 or 1	Avoided Data contamination
0 or 1	Avoided Co-intervention
0 or 1	Results reported in terms of statistical significance
0 or 1	Suitable methods of analysis
0 or 1	Reported educational relevance
0 or 1	Reported drop outs or failure to conclude
0 or 1	Proper conclusion

Data analysis

After collection, the data were consolidated in a database for analysis. Due to the methodological heterogeneity and the different outcomes sought with the use of OSCE, a descriptive analysis of the results was carried out. The analysis sought to identify the main reflections, discussions, limitations and potentialities of the use of this tool. Data presentation followed strictly the terminology used by the authors, from a proper form, specifically developed to analyze those data, adjusted according to the authors’ interest and following recommendations by the Cochrane Library ²⁰.

Results

2,766 publications were identified in PubMed, 14 in Bireme and BVS and 462 in Web of Science, totaling 3,242 studies. After applying the exclusion criteria, seven studies were selected for review (Figure 1), they were all published between 2011 and 2016. The use of the instrument was reported in detail in four studies (57.1%) and the most prevalent investigation design was the cross-sectional study (n = 3; 42.9%), followed by the randomized clinical test (n = 2; 28,6%). From these, four were developed in Australia, two in Brazil and one in Canada. The characteristics of the studies included regarding objectives and level of evidence are shown in Table 1.

Figure 1
Flowchart with the number of identified studies, included and excluded from the literature review.

None of the studies was scored for all validity criteria from the evaluation of the methodological quality (Table 1), with scores ranging from 7 to 12. Regarding the level of scientific evidence, the studies showed levels 1B and 2B. The general features of the studies regarding objectives are described in Table 2.

Table 2
General features of the studies included regarding objectives and level of evidence

Author, year	Country	Design	Sample	Objectives	Level of evidence
Cacho et al., (2016)14	Brazil	Reliability study	29 physiotherapy students	To report the experience of applying OSCE in a physiotherapy undergraduate course and to demonstrate its inter-examiner reliability.	9 / 2B
			(7º period of the course)
Davies et al., 2015)26	Canada	Cross-sectional study	9 physiotherapy courses were included in the study.	To identify professional conduct measured using OSCE in physiotherapy undergraduate courses.	8 / 2B
Edgar et al., (2014)27	Australia	Retrospective observational study	141 physiotherapy students	To determine the existence of association between the score in the course entrance interview and the students’ subsequent academic and clinical performance.	10 / 2B
			(2 first years of the course)
Maloney et al., (2013)28	Australia	Controlled randomized clinical trial	60 physiotherapy undergraduate students	To examine the impact of the students’ self-video* to obtain clinical skills.	12 / 1B
			(3rd year)
Silva et al., (2011)29	Brazil	Cross-sectional	47 physiotherapy undergraduate students	To evaluate the OSCE use as a tool to assess students’ skills in the pulmonary physiotherapy area and to verify the internal consistency of the instrument.	11 / 2B
			(3rd year)
Maloney et al., (2013)32	Australia	Controlled randomized clinical trial	40 physiotherapy undergraduate students	To compare the effectiveness of two innovative teaching methods of practical skills against a traditional teaching method.	12 / 1B
			(3rd year)
Snodgrass et al., (2014)38	Australia	Cross-sectional study	206 physiotherapy and occupational therapy students and 25 examiner tutors	To describe the electronic OSCE (eOSCE) implementation and evaluation using iPad in two universities.	7 / 2B

• Note: *The self-video was a university study in which the students recorded their performance in a certain task assigned during the study.

Sample and OSCE use in Physiotherapy teaching

The general features of the included articles are listed in Table 2. The total sample size was 632 physiotherapy students (variation 29-206) and 25 examiner tutors. Table 3 shows data of the OSCE used and the main results obtained.

Table 3
Characteristics of the OSCE used and results evidenced from the selected publications

Author, year	Methodology	Main results
Cacho et al., (2016)14	Professors met prior to the examination to devise clinical cases. The concepts “yes”, “no” and “insufficient” were ascribed to the task accomplishment, with a final score of 10 points. The second phase referred to the use of OSCE. Students experienced the practice in stations (attendance simulated environment) where two examiners applied the instrument.	Excellent reliability was seen in the stations 1 (CCI=0,89), 2 (CCI=0,99) and 3 (CCI=0,99), and satisfactory (CCI=0,73) in station 4, being shown as a useful method for the evaluation process of professional education. However, it requires pedagogical preparation and examiners’ previous experience. It is preferable to use narrow response, e.g. correct or incorrect.
Davies et al., (2015)26	Telephone interview to determine which physiotherapy courses used OSCE and to collect data about the methodology employed: number of stations, time spent in each station, type of examiner, and evaluation methods/scales. It also included questions regarding professional behavior, criteria and classification components.	The number of stations varied from 1 to 10 stations (mean 3-4), with duration from 5 to 15 minutes. All programs reported evaluating only practical skills during the OSCE. All programs reported the use of different classes of examiners, both full-time professors and assistant professors. All used evaluation forms like checklists to evaluate the components: communication, respect, patient safety and work characteristics.
Edgar et al., (2014)27	Predictive variables (course entrance interview) and outcome variables (students’ marks during the course and in the OSCE) were analyzed. There was no description of the way OSCE was carried out.	The course entrance interview scores significant association with performance in three of the six clinical internships of the course. The authors highlighted the role of entrance measurements in the selection of physiotherapy undergraduate students.
Maloney et al., (2013)28	Students were divided into two groups: one group was assigned the task of learning an ability through the self-video; to the other, the self-video showed an ability less relevant to the clinic case. At the end of the semester, both groups took OSCE based on the following abilities: introduction to the patient, interpretation of the clinical examination result, conduct execution, giving feedback to the patient, reformulating conduct or evaluation technique, proper handling, communication with the patient, etc.	The students got a significantly higher score in the OSCE when the clinical ability had been analyzed with a self-video on how to perform the task (p=0.048). The analysis of students’ perception identified that the self-video contributed to the improvement of the clinical performance and self-confidence in future clinical practice. The clinical ability with highest score was reached when the traditional teaching methods were complemented with the self-videos.
Silva et al., (2011)29	Two types of examinations were evaluated: traditional and OSCE. The traditional test consisted in four theoretical and one practical question. OSCE included five stations (0-2 points/each). The students had a minute to read a task description and five to accomplish it. The abilities evaluated were divided into classes, cognitive, psychomotor and behavioral skills.	The OSCE mean score oscillated between 4.4 and 9.6, with good internal consistency between stations (0.7). The agreement between examinations determined that they are not comparable. Low agreement was also observed between the two examinations (r=-0.1, p=0.9). OSCE evaluates distinctly different abilities when compared to the traditional examination, which suggests that OSCE complements the evaluation of skills and competences that traditional exams fail in evaluating.
Maloney et al., (2013)32	Three teaching methods were compared: “traditional”, pre-recorded lessons (tutorial videos) and students’ self-performance videos. The students received a clinical case that required the use of their skills (evaluated by OSCE). Performance was scored according to their communication abilities, clinical reasoning, safety and risk management and patient handling, and the results were classified as “concluded”, “partially concluded” or “insufficient”.	No significant differences were found in the clinical performance between the three teaching practices. There was significant difference between the methods regarding the education value noticed by the students, with teaching approaches using tutorial videos and students’ self-performance videos, in which the tutorial videos were considered superior. Alternative teaching methods might produce equivalent learning results when applied to the development of practical skills. They might be applied instead of live lessons, rather than being a mere complement or a supplementary method to traditional approaches.
Snodgrass et al., (2014)38	The electronic OSCE (eOSCE) was devised using an iPad to improve students’ feedback. The eOSCE was tested in two universities regarding user friendliness, preference (electronic or printed) and performance in the feedback given to the students. The authors did not describe how the OSCE was applied.	Most of the examiners (68%) preferred eOSCE, mainly due to the consistency and speed of the students’ feedback. The advantages listed were automation, individual and immediate feedback and time saved in the OSCE applications. The disadvantages were the necessary pre-exam preparation and challenges to the examiners that were not comfortable using the technology.

• Note: OSCE: Objective Structured Clinical Examination; eOSCE: electronic OSCE.

Clinical skills assessed in the studies

The main clinical skills identified in the studies were grouped in different classes, namely cognitive, psychomotor and affective. Four studies described the evaluation method of such abilities ^{14), (26), (28}. Table 2 shows that the cognitive skills were defined as the ability to integrate intellectual knowledge and mastering of the content to accomplish a task or devise a suitable therapeutic procedure in a clinical case ²⁸. The execution of evaluation components was mentioned and, whenever necessary, changes were performed in the evaluation technique according to each case; the result interpretation of the clinical evaluation ²⁸, the use of suitable equipment for the therapy and whether the selected procedure achieved the proposed objective ¹⁴.

As regards psychomotor abilities, the students’ skills in executing a physiotherapeutic technique or a physical examination were tested ²⁸. The authors reported the correct application of the evaluation techniques, proper manual handling of the patient ²⁷; handling patients’ transfers and suitable care with exoskeleton devices ¹⁴.

Finally, affective or behavioral skills were evaluated through the students’ reactions, their speech, as well as verbal command and intonation when interacting with the patients along with their initiative (proactivity) ²⁸. The considered aspects were: whether students introduced themselves when interacting with patients ¹⁴; whether it was a suitable introduction; whether they adjusted their communication according to the patients’ needs; whether they confirmed patients’ understanding on the evaluation techniques and treatment using suitable terms; whether they obtained patients’ informed consent; whether they used proper non-verbal communication (eye contact) ²⁶; whether they provided patients with feedback at the right time (language and clarity) ¹⁴; whether they considered patient comfort, dignity and respect, including proper undressing for evaluation or treatment; whether they presented a professional posture ²⁶; and whether they considered risks and took measures to minimize them ²⁷.

Discussion

This study aimed to synthesize data about OSCE and to investigate its use as an evaluation tool to assess clinical skills in physiotherapy teaching. The results evidenced a gap in literature regarding the instrument and its use in physiotherapy, maybe as a result of lack of knowledge on the tool ¹¹, methodological difficulties when performing scientific investigations, added to the challenge of implementing this evaluation tool in undergraduate courses due to its complex organization and the time spent in the selection/training of patients ¹⁵, along with coordinators’ and professors’ preferences. All these factors determinate the difficulty to develop scientific studies, in addition to demonstrating different strategies of evaluation among the different higher education institutions.

The identified study designs varied and reflected limitations to establish relations between learning and the use of OSCE, with different levels of evidence, a wide variety of investigation purposes, restriction in sample size and low number of publications, which prevent data systematization with robust statistics. The characteristics of the identified studies in this review include the use of the OSCE in research as an evaluation of learning in physiotherapy undergraduate courses ^{14), (28}, as a proficiency examination ²⁵ in convenience sampling ²⁸, and the fact that there were only two Brazilian studies ^{14), (28}. These results did not allow a conclusion about the use of the instrument to evaluate students’ abilities and competences by the professor, due to the difficulty of visualizing results from the evidence, but also for the lack of an accurate method of diagnosis able to reach these elements and because evaluation is an issue that involves the kind of education experienced by the professor ^{2) (29), (30}.

The included studies revealed that very often the appreciation of students’ performance was based on the reference norm, with the use of marks ^{10), (25), (26), (28), (31} while the use of OSCE might reveal formative aspects that go beyond the summative. Literature points out a few recent initiatives that started to use Learning Based on Problems as a strategy to reorganize the curriculum, and students’ evaluation became criterion-referenced, using concepts that qualify the students know-how in relation to profiles of professional competence defined for different undergraduate courses ^{14), (30}. The traditional examination focuses on cognitive and psychomotor skills, while the OSCE addresses attitude and psychomotor skills that are needed to develop procedures in clinical practice. Students’ evaluation based on cognitive skills tends to promote a learning strategy of information memorization, while evaluations based on the ability of integrating knowledge tend to develop an attitudinal learning strategy.

One of the studies included ³¹ in this systematic review aimed at comparing the efficacy of two different methods of teaching practical skills in relation to traditional teaching. Those authors did not find significant differences in clinical performance among the three modalities, and they emphasized that the allocation of teaching resources in alternative methods to train skills is recommended, however, it is relevant to consider the complementarity and interdependence between them, and one cannot be considered a priori better than the other. In another study, Silva et al. ²⁸ evidenced that the lack of association between the results of student performance in a traditional examination and in OSCE might be explained by the specificity of each exam, which may suggest that both exams are complementary but not mutually excluding. In addition, confirming the studies by Brasford ³², we observed the predominance of the discussion about the objective of a learning evaluation by privileging the development of critical-reflexive thinking, intellectual independence and creativity, built through the development of metacognition ability.

Even if traditional evaluations provide relevant data about students’ academic and clinical competence, objectivity and standardization are still incipient. The results obtained regarding students’ skills are strongly influenced by the profile of patients available for certain examination, the type of service they are involved with, and the whole evaluation depends on the examiner’s subjective concepts. In addition, students are tested in very different conditions, which limits peer comparison. These particularities contribute to the low reliability of these methods ³³.

OSCE appears as an opportunity to deal with a particularly suitable instrument to evaluate many, certainly not all, the components of clinical competence. For example, attitudinal and behavioral aspects are probably better overcome with the use of multiple classifications collected along a period of time during internship, in different modalities of in-service teaching and during interaction with different professionals of the team ². At the other end of the instrument, the relevant knowledge needed to be competent, including aspects of diagnosis, investigation and treatment, can be more efficiently and cheaply tested in written form, differentiating from the traditional clinical examination ³⁴.

The main benefit of using OSCE is the possibility of linking a set of essential competences to the configuration of simulated clinical situations, which allow the measurement of certain structured competences. This implies that OSCE can be considered a “gold standard” to evaluate the ordinary education of health professionals. Some of the OSCE positive points that can be highlighted are the possibility of selecting the type of skill to be evaluated, the level of complexity required and the evaluation circumstances, without depending on the availability of cases, environmental factors or circumstances ³³. The use of real patients (not simulated) is usually considered more suitable, since they have realistic problems, needs, demands and conditions with a degree of complexity which is common in the everyday professional practice, particularly in outpatient care. However, direct observation is more detailed and provides a closer and integrated view of the students’ clinical skills, making it easier to identify the learning areas to be reinforced and including the interaction that qualifying professionals have with the service in which they are inserted and the teams they belong to. The use of structured tools, such as a check-list, increases the accuracy of failure detection and offers objective elements to provide students with relevant feedback ^35)-(38.

The current scenario of physiotherapists’ initial education requires a certain vision of the profile of competences desired in higher education institutions in our country ^{39), (40}. For Reis and Monteiro ⁴¹, the teaching-learning process has emphasized the acquisition of technical competences rather than the understanding and analysis of problems and application of knowledge based on evidence, which reveals a certain unsuitability in physiotherapy initial education, as pointed out by Bispo Júnior ⁴². The institutions still privilege the technical model based on the biological paradigm, stimulation to specialization and emphasis on biological research, without paying much attention to the aspects of students’ behavioral education. A fast increase in the number of newly graduate physiotherapists has also been observed, who are usually inserted in the labor market with very little experience in clinical environments ⁴³. In a complementary way, Davies et al. ⁴⁴ pointed out that students in the rehabilitation area reported that many times they are not really sure of what is expected from them concerning their clinical skills and competences. Regulating agencies such as the Physiotherapy and Occupational Therapy Federal and Regional Councils, in their Ethics and Deontology Codes, provide basic norms, regulations and conducts linked to the fundamental beliefs or values of the profession, which are undertaken through the professional socialization that one student experiences when taking part in an environment of practice ^{44), (45}. Thus, the evaluation of skills and competences, both in the academic and clinical environments, should emphasize the estimated value of these behaviors ⁴⁴. The information provided before and the feedback of the clinical training should allow the students to recognize subjectively their clinical skills and to develop learning plans to improve their knowledge, behaviors and skills step by step ^{43), (45}.

However, the lack of uniformity when professors give marks/concepts, failure to detect mistakes made by the students and the high level of stress experienced by them, as well as the short time available to carry out the tasks, reducing the professional-patient interaction, are limitations presented in the use of OSCE according to the studies selected for review ^{25), (27), (28}. Another inconvenience is the need of a complex organization process, including time, cost and guarantee of confidentiality ³⁶. Snodgrass et al. ³⁷ devised an online version of an electronic OSCE (eOSCE), and, when comparing it to the printed version, they found significant improvement regarding accuracy, financial viability and time-cost relation ³⁸. Despite these advantages, the challenge of implementing the eOSCE is to guarantee that all examiners and course coordinators feel comfortable with this technology.

Systematically reviewing clinical research on the use of OSCE in physiotherapy courses emphasized its relevance in relation to the scenario of improvement of the courses offered by higher education institutions, which might result in changes in the Brazilian curricular guidelines, according to the profile required by the current health system and the education policies in the health area. As a result, the instrumentation of the teaching job, based on the qualification of the students’ evaluation becomes relevant and might, in addition, stimulate the teaching-learning process and help in the efficiency of the subjective evaluation. However, the low number of researches included in this systematic review shows the scarcity of studies related to the use of OSCE in physiotherapy, revealing a gap in scientific literature. These data suggest the need for reflection upon how the development of skills and competences in physiotherapy undergraduate students has been evaluated ³², mainly considering the context of changes in the initial education, as pointed out by the current policies of the fields of education and health, expanding the professional attributes considered necessary for their work in teams and different jobs in the health system ^{39), (40}.

Regarding future investigations, it seems necessary to analyze the ability of the OSCE examination to predict the students’ performance in the practical training of different modalities of learning, in service or in their well-succeeded entrance in the job market. In relation to the professors’ practice, some caution seems necessary when making decisions, especially in relation to the study profiles, the methodology adopted and the outcome measurements. Regarding the implications to the research, due to the absence of homogeneity in the methodological approaches and outcomes found, it is not possible to determine systematic positive effects able to justify the effects on the clinical skills of physiotherapy students. The findings regarding the use of OSCE to evaluate students seem to be promising. However, in order to make broader statements on the development of teamwork, services and health systems concrete, adjustments are still needed in future studies.

Conclusion

The results of this systematic review indicate the need for further studies on the use of OSCE, mainly with high methodological quality, in which the main objective is to investigate the role of this instrument in the evaluation of clinical skills in physiotherapy undergraduate courses. The use of this tool, either to analyze students’ clinical skills or to reflect upon the learning conditions offered by the higher education institutions seems to be relevant. This study offers an international panorama regarding the use of OSCE in physiotherapy, revealing that its use in Brazil is still limited.

Such scenario also brings about the relevance of great depth to the discussion and practice of evaluating professional skills throughout the undergraduate education, to face the different initiatives of changes in the professional profile required by the current health system and the education policies in the health area, as well as the development of teamwork, services and health systems.

References

• ¹
  Panúncio-Pinto MP, Troncon LEA. Avaliação do estudante - aspectos gerais. Medicina (Ribeirão Preto). 2014;47(3):314-23.
• ²
  Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(Suppl 9):63-7.
• ³
  Townsend AH, McLlvenny S, Miller CJ, Dunn EV. The use of an objective structured clinical examination (OSCE) for formative and summative assessment in a general practice clinical attachment and its relationship to final medical school examination performance. Med Educ. 2001;35(9):841-6.
• ⁴
  Garcia MAA. Saber, agir e educar: o ensino-aprendizagem em serviços de saúde. Interface Comunic Saude Educ. 2001;5(8):89-100.
• ⁵
  Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002;287(2):226-35.
• ⁶
  Troncon LEA. Utilização de pacientes simulados no ensino e na avaliação de habilidades clínicas. Medicina (Ribeirão Preto). 2007;40(2):180-91.
• ⁷
  Newble D. Assessment of clinical competence. BJA. 2000;84(4):432-3.
• ⁸
  Megale L, Gontijo ED, Mota JAC. Competências clínicas essenciais em pediatria: estão os estudantes aptos a executá-las? Rev Bras Educ Med. 2012;36(4):478-88.
• ⁹
  Nunes SOV, Muraguchi EMO, Ferreira Filho OF, Pontes RMA, Cardoso LTQ, Grion CMC, et al. O Ensino de habilidades e atitudes: um relato de experiências. Rev Bras Educ Med. 2013;37(1):126-31.
• ¹⁰
  Störmann S, Stankiewicz M, Raes P, Berchtold C, Kosanke Y, Illes G, et al. How well do final year undergraduate medical students master practical clinical skills? GMS J Med Educ. 2016;33(4):1-16.
• ¹¹
  Galato D, Alano GM, Trauthman SC, França TF. Exame Clínico Objetivo Estruturado (ECOE): uma experiência de ensino por meio de simulação do atendimento farmacêutico. Interface Comunic Saude Educ. 2011;15(36):309-19.
• ¹²
  Carraccio C, Englander R. The objective structured clinical examination: a step in the direction of competency-based evaluation. Arch Pediatr Adolesc Med. 2000;154(7):736-41.
• ¹³
  Sampaio AMB, Pricinote SCMN, Pereira ERS. Avaliação clínica estruturada. Rev Eletr Gest Saud. 2014;5(2):410-26.
• ¹⁴
  Cacho RDO, Baroni MP, Ruaro JA, Lopes JM, Britto HMJDS, Ferreira TB, et al. Active methods in physical therapy: intereliability study of the OSCE method. Rev Bras Educ Med. 2016;40(1):128-37.
• ¹⁵
  Troncon LEA. Clincal skill assessment: limitations to the introduction of an "OSCE" (Objective Structured Clinical Examination) in a traditional Brazilian medical school. São Paulo Med J. 2004;122(1):12-7.
• ¹⁶
  Rushforth HE. Objective structured clinical examination (OSCE): review of literature and implications for nursing education. Nurse Educ Today. 2007;27(5):481-90.
• ¹⁷
  Araújo JNM, Fernandes APNL, Costa RA, Júnior MAF, Pinto DPDSR, Carvalho AFV. Avaliação de estudantes de enfermagem sobre o exame clínico objetivamente estruturado. Rev Eletr Enf. 2015;17(3):1-8.
• ¹⁸
  Galato D, Alano GM, França TF, Vieira AC. Exame clínico objetivo estruturado (ECOE): uma experiência de ensino por meio de simulação do atendimento farmacêutico. Interface Comunic Saude Educ. 2010;15:309-20.
• ¹⁹
  Medina MS, Stark JE, Vesta KS, Lockhart SM. Evaluating the impact of a pre-rotation workshop on student preparation for clinical advanced pharmacy practice experiences. Pharm Pract. 2008;6(4):219-23.
• ²⁰
  Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 4.2.6. Chichester (UK): John Wiley & Sons, Ltd.; 2006. 257 p.
• ²¹
  Law M, Pollock N, Letts L, Bosch J, Westmorland M. Critical review form: quantitative studies. Ontario: McMaster University; 1998 [cited 2016 May 9]. Available from: http://bit.ly/2YtITta
  » http://bit.ly/2YtITta
• ²²
  Letts L, Wilkins S, Law M, Stewart D, Bosch J, Westmorland M. Critical Review form: qualitative studies. Ontario: McMaster University; 1998 [cited 2016 May 9]. Available from: http://bit.ly/2YAcmll
  » http://bit.ly/2YAcmll
• ²³
  Lekkas P, Larsen T, Kumar S, Grimmer K, Nyland L, Chipchase L, et al. No model of clinical education for physiotherapy students is superior to another: a systematic review. Aust J Physiother. 2007;53(1):19-28.
• ²⁴
  Phillips B, Ball C, Sackett D, Badenoch D, Straus S, Haynes B, et al. Oxford Centre for Evidence-based Medicine - Levels of Evidence: Grades of Recommendation. Oxford: CEBM; 2001 [cited 2016 May 10]. Available from: http://bit.ly/2E9N5X6
  » http://bit.ly/2E9N5X6
• ²⁵
  Davies R, Ellerton C, Evans C. Measuring professional behaviour in Canadian physical therapy students' objective structured clinical examinations: an environmental scan. Physiother Can. 2015;67(1);69-75.
• ²⁶
  Edgar S, Mercer A, Hamer P. Admission interview scores are associated with clinical performance in an undergraduate physiotherapy course: an observational study. Physiotherapy. 2014;100(4):331-5.
• ²⁷
  Maloney S, Storr M, Morgan P, Ilic D. The effect of student self-video of performance on clinical skill competency: a randomised controlled trial. Adv Health Sci Educ. 2013;18:81-9.
• ²⁸
  Silva CC, Lunardi AC, Mendes FA, Souza FF, Carvalho CR. Objective structured clinical evaluation as an assessment method for undergraduate chest physical therapy students: a cross-sectional study. Rev Bras Fisioter. 2011;15(6):481-6.
• ²⁹
  Norcini JJ, Mckinley DW. Assessment methods in medical education. Teach Teach Educ. 2007;23:239-50.
• ³⁰
  Cyrino EG, Toralles-Pereira ML. Trabalhando com estratégias de ensino-aprendizado por descoberta na área da saúde: a problematização e a aprendizagem baseada em problemas. Cad Saude Publica. 2004;20(3):780-8.
• ³¹
  Maloney S, Storr M, Paynter S, Morgan P, Ilic D. Investigating the efficacy of practical skill teaching: a pilot-study comparing three educational methods. Adv Health Sci Educ. 2013;18:71-80.
• ³²
  Bransford JD, Brown AL, Cocking RR. Como as pessoas aprendem: cérebro, mente, experiência e escola. São Paulo: Senac São Paulo; 2007.
• ³³
  Troncon LEA. Avaliação do estudante de Medicina. Medicina (Ribeirão Preto). 1996;29(4):429-39.
• ³⁴
  Newble D. Techniques for measuring clinical competence: objective structured clinical examinations. Med Educ. 2004;38(5):199-203.
• ³⁵
  Amaral E, Domingues RCL, Bicudo-Zeferino AM. Avaliando competência clínica: o método de avaliação estruturada observacional. Rev Bras Educ Med. 2007;31(3):287-90.
• ³⁶
  Moonen-van Loon JM, Overeem K, Donkers HH, van der Vleuten CP, Driessen EW. Composite reliability of a workplace-based assessment toolbox for postgraduate medical education. Adv Health Sci Educ Theory Prac. 2013;8(5):1087-102.
• ³⁷
  Snodgrass SJ, Ashby SE, Rivett DA. Implementation of an electronic objective structured clinical exam for assessing practical skills in pre-professional physiotherapy and occupational therapy programs: examiner and course coordinator perspectives. Austr J Educ Technol. 2014;30(2):152-66.
• ³⁸
  Kropmans TJB, O'Donovan BGG, Cunningham D, Murphy AW, Flaherty G, Nestel D, et al. An online management information system for objective structured clinical examinations. Comp Inform Scien. 2012;5(1):31-48.
• ³⁹
  Ferla AA. Clínica em movimento: cartografias do cuidado em saúde. Caxias do Sul: EDUCS; 2007.
• ⁴⁰
  Funghetto SS, Silveira M, Silvino AM, Karnikowski MGO. Perfil profissional tendo o SUS como base das Diretrizes Curriculares da Área da Saúde no processo avaliativo. Saude Redes. 2015;1(3):103-20.
• ⁴¹
  Reis FJJ, Monteiro MGM. O ensino na Fisioterapia: momento de revermos a prática? Fisio Pesq. 2015;22(4):340-1.
• ⁴²
  Bispo Júnior JP. Formação em fisioterapia no Brasil: reflexões sobre a expansão do ensino e os modelos de formação. Hist Cienc Saude (Manguinhos). 2009;16(3):655-68.
• ⁴³
  Kanada Y, Sakurai H, Sugiura Y. Difficulty levels of OSCE items related to examination and measurement skills. J Phys Ther Sci. 2015;27(3):715-8.
• ⁴⁴
  Davies R, Ellerton C, Evans C. Reaching consensus on measuring professional behaviour in physical therapy objective structured clinical examinations. Physiother Can. 2017;69(1):65-72.
• ⁴⁵
  Pugh D, Touchie C, Humphrey-Murto S, Wood TJ. The OSCE progress test - measuring clinical skill development over residency training. Med Teach. 2016;38(2):168-73.

Publication Dates

• Publication in this collection
  30 May 2019
• Date of issue
  2019

History

• Received
  09 Jan 2018
• Accepted
  09 Jan 2019