Comparing and measuring the practice-based performance and competence of occupational therapy students between traditional and role emerging placements: a retrospective cohort study

Comparação entre o desempenho e competência de estudantes de terapia ocupacional em estágios tradicionais e não-tradicionais: um estudo retrospectivo

Abstract

Introduction

Increasingly occupational therapy programmes have complemented traditional practice placements with Role Emerging Placements (REPs) in settings with no existing occupational therapy provision. Previous research has predominantly explored the student experience of such placements with largely favorable findings. However, there is a lack of understanding regarding the professional practice-based skills students develop within REPs.

Objective

To measure and compare students’ competencies for occupational therapy practice developed within Role Emerging Placements (REP) to those in ‘traditional’ practice placements.

Method

A retrospective cohort study design was used to analyze data extracted from the Competency Based Fieldwork Evaluation for Occupational Therapists (CBFE-OT) of MSc pre-registration occupational therapy UK students (n=181). Descriptive and inferential statistical analysis measured and compared student fieldwork competency scores between four placements, one of which was a Role Emerging Placement (REP).

Results

Students scored significantly higher in ‘Communication’ and ‘Professional Interactions’ compared to all other practice competencies but scored significantly less well in ‘Clinical Reasoning’ and ‘Practice Knowledge’ regardless of placement model. However, in a REP, students scored significantly higher in ‘Performance Management’ compared to traditional placements (z=-2.222 p=0.026).

Conclusion

Students can develop similar skills of competence to practise in a REP as traditional placements and may better develop leadership and time management skills. These additional skills may advantage students in developing their careers and scoping the profession.

Keywords:
Professional Practice; Students; Occupational Therapy; Clinical Competence

Resumo

Introdução

Os cursos de graduação em terapia ocupacional têm crescentemente complementado os campos de prática tradicional dos estágios com modelos alternativos, nos quais não existem serviços de terapia ocupacional. Pesquisas anteriores exploraram predominantemente a experiência do aluno em tais estágios, com resultados amplamente favoráveis. No entanto, há uma lacuna de conhecimento sobre as habilidades profissionais que tais alunos desenvolvem a partir deste modelo alternativo de estágio.

Objetivo

Medir e comparar as competências dos alunos para a prática de terapia ocupacional, desenvolvida em modelos de estágios alternativos onde não existem serviços de terapia ocupacional, com os estágios tradicionais.

Método

Trata-se de um estudo retrospectivo no qual se analisou dados extraídos da avaliação “Competency Based Fieldwork Evaluation for Occupational Therapists (CBFE-OT)”, utilizada para avaliar o desempenho de 181 alunos de terapia ocupacional de um curso de mestrado em uma universidade pública do Reino Unido. A análise estatística descritiva e inferencial mediu e comparou as pontuações de competência dos alunos em quatro estágios, em que um era o modelo alternativo, onde não havia oferta de serviços de terapia ocupacional.

Resultados

Os alunos obtiveram pontuações significativamente mais altas em 'Comunicação' e 'Interações profissionais', em comparação com todas as outras competências práticas, mas pontuaram significativamente menor em 'Raciocínio clínico' e 'Conhecimento prático', independentemente do modelo de estágio. No entanto, nos modelos de estágios alternativos, os alunos obtiveram pontuações significativamente mais altas em 'Gestão de desempenho' em comparação com os campos de estágio tradicionais (z = -2,222 p = 0,026).

Conclusão

Os alunos podem desenvolver habilidades de competência semelhantes para a prática a partir de modelos alternativos de estágio quando comparados com os estágios tradicionais, com melhor desenvolvimento nas habilidades de liderança e gerenciamento de tempo, favorável ao estágio alternativo. Essas habilidades adicionais podem beneficiar os alunos no desenvolvimento de suas carreiras e no escopo da profissão.

Palavras-chave:
Prática Profissional; Estudantes; Terapia Ocupacional; Competência Clínica

Introduction

Globally, innovation and research within practice-based pedagogy has highlighted that placement models need to ensure that they are a good ‘fit’ between evidence-based theory and competent professional practice (Roberts et al., 2015; World Federation of Occupational Therapists, 2016). Students value learning from participation in practice contexts, where profession-specific knowledge and skills are consolidated through the integration of evidence-based theory within a workplace setting (Holmes et al., 2010; Cantero-Garlito et al., 2020). Shifts in contemporary occupational therapy practice towards community and health promotion settings have prompted the emergence of new roles and career paths for occupational therapists (Syed & Duncan, 2019; Thew et al., 2018) reflecting a need for student practice-based placements to follow suit.

Role Emerging Placements (REP) (s) typically involves placing two or more students into an ‘emerging’ setting with no occupational therapy provision to carry out an occupation focused project facilitated by an offsite occupational therapist via long-arm supervision, with an on-site member of staff overseeing the students’ day to day learning (Thew et al., 2008; Edwards & Thew, 2011).

Research suggests that students who have attended a REP experienced increased levels of confidence, a sense of professional identity, and enhanced employability (Golos & Tekuzener, 2019; Tyminski, 2018; Thew et al., 2018). Further, REPs have been noted to address educational placement shortages in statutory sectors (traditional settings) and reflect the emerging landscape of health and social care towards non-statutory, community-based agencies which expands the scope of occupational therapy practice (Lau & Ravenek, 2019; Linnane & Warren, 2017; Roberts et al., 2015).

The nature of the REP model, however, poses a challenge to both student and supervisor in that they can require complex models of supervision and without an occupational therapist ‘in situ’ the learning of professional unique or ‘core’ skills could be inhibited (Price & Whiteside, 2016). It is therefore important to consider whether the REP can provide students with the opportunity to learn the essential occupational therapy skills for competent practice compared to a traditional placement (Holmes et al., 2010; World Federation of Occupational Therapists, 2016).

Literature Review

Much of the literature appears to have explored (usually by qualitative research methods), the inward effects of REPs on students subjective personal and professional development (Clarke et al., 2014; Mattila et al., 2018; Thew et al., 2018; Syed & Duncan, 2019). Specifically, evidence suggests students benefit from a REP experience in the development of their entrepreneurship and organisational skills (Thew et al., 2018); interprofessional knowledge and clinical reasoning (Tyminski, 2018); self-efficacy and professional identity (Clarke et al., 2015; Mattila et al., 2018). Further, studies suggest that students and have a competitive edge in terms of employability (Clarke et al., 2015; Thew et al., 2018; Dancza et al., 2019). Literature also indicates that involvement of students with populations who have not previously received occupational therapy services, amplifies the scope of the profession (Linnane & Warren, 2017; Tyminski, 2018).

Some studies have suggested that students particularly value the opportunity to explore occupation focused interventions that often a REP can offer above that of a traditional placement, which can be restricted by the medical model environment or generic roles within mental health settings (Dancza et al., 2013; Thew et al., 2018). Furthermore, research demonstrates that the structure of role-emerging placements with no determined condition specific occupational therapy procedures or strategies, helps students to develop their own critical thinking in deciding what to do and when (Dancza et al., 2019).

A review of the literature on REPs by Clarke et al. (2015) asserts that there are benefits of REPs to the profession through examples of expansion in the occupational therapists’ role and an increase in posts within third sector organisations. However, their review revealed several issues, particularly the paucity of evidence measuring student skills and competence developed within the REP and whether they are relevant for professional practice. Further, there is a considerable gap in the occupational therapy literature concerning how the professional skills that students can potentially attain in a REP when compared to the more traditional based placements or fieldwork (Clarke et al., 2015). This may be because few universities use the same assessment criteria to compare and measure student’s performance in practice-based competencies for all placements regardless of setting or genre of placement (Holmes et al., 2010; Thew & Harkness, 2018).

In light of the lack of evidence of the measurement and comparison of specific students' competencies for practice and to what extent these are developed in a REP, this study offers an opportunity to quantitively measure the professional competencies for practice and consider whether these skills are developed similarly or are comparable to a traditional placement experience. To this end, this study addresses the following research question: To what extent does attendance on a REP develop the competencies and skills for occupational therapy practice and how do these compare to those developed in traditional placements? With an aim to: measure and compare MSc students’ level of competence in occupational therapy practice-based skills developed within REPs to those in traditional practice placements.

Methods

Study design

This study employed an observational retrospective cohort design of a single population (n=181) of previous student placement scores collected from 2007 to 2020 of an MSc occupational therapy pre-registration program based within the United Kingdom.

All students within the sample received x4 practice/fieldwork placements, x3 ‘traditional’ placements in settings such as: in-patient medical, trauma, rheumatology, neurorehabilitation; community based social care, equipment provision, community stroke team; and mental health settings such as: acute inpatient, early psychosis intervention teams. dementia/memory services, forensic mental health and learning disabilities teams.

The REPs adopted a ‘project based’ model in settings with no occupational therapy provision in situ (Edwards & Thew, 2011) with daily supervision from an onsite (usually non-occupational therapist) and long arm supervision from a qualified occupational therapist usually from outside of the university team. Both supervisors' training and support were provided, and the students were prepared for placement with additional sessions from tutors regarding the occupation-focused project, as well as attending the workshop. The ways in which the placements differ is explained to both students and educators. Table 1 describes how placements differ in the following ways:

Table 1
A comparison between Role Emerging Placement and Traditional Placement.

● Role Emerging Placement	● Traditional Placement
● Setting ‘expert’ provides some leadership and supervision	● Experienced ‘expert’ OT in situ
● There is usually no OT in situ.	● One-to-one student supervision-the norm.
● Long arm OT supervision to more than one student-the norm.	● Student as apprentice learner of condition specific skills
● Students work on a project that demonstrates a potential for occupational therapy.	● Student shadows and repeats practice of the OT
● Students considers the needs of the people in the setting as a whole	● Students learns the role of OT within a health or social care existing team.
● Growth in the OT role	● Student practices core skills of OT process with service users.
● Emphasis on peer support
● Students takes leadership role

The REPs were attended by two (or three) students at a time for peer support and learning. A wide variety of REPs setting had been developed over the years, and students choose which setting they wish to attend. Examples of settings have included, community allotments, domestic violence refuges, mainstream and specialist schools, homeless shelters, social enterprises, charity support groups (socially prescribed), vocational settings, community libraries, GP practices etc.

Preparation of the educators for using the student placement assessment tool

Each educator received he same training in the assessment tool irrespective of model of placement. Assessment training is provided to all educators via a practice placement educators course, this one day (mandatory) involves understanding the role of supervision, understanding student learning styles, reasonable adjustments, how to complete the assessment tool (CBFE), including how to support students if they are failing. In addition, the REP educators (both the onsite and Occupational Therapy educator) received a separate workshop/ training session and added paperwork, which included describing a REP, how the long arm supervision should be provided and how students should be supported in a project-based placement model. The long arm supervisor would retain contact with the students via phone call/ texting/ emails to support the students in developing the project and remain within the scope of professional practice. They would meet the students in the setting once a week for formal supervision and meet with the one site supervisor to ensure that the students and their project were on track with learning outcomes.

Instrument of data collection

The university measures the performance of all students in all four placements using the 'Competency Based Fieldwork Education' (CBFE) tool (Bossers et al., 2007) which is a validated, internationally recognized measurement of occupational therapy student competence in field work professional practice-based skills (Holmes et al., 2010). This tool provides validity for comparison of specific skills both inter and intra placement (Miller et al., 2001; Bossers et al., 2007).

The tool uses a 9-point anchored- Likert scale from U (‘Unacceptable-below that of entry level student) to E (‘Exceptional’- which denotes competence above that of entry level clinician), ‘8’ denotes ‘entry level clinician’, this scale is used to measure student performance in seven separate competencies which are:

'Practice Knowledge' (Setting/condition/discipline specific theory, technical knowledge) ; 'Clinical Reasoning' (analytical and conceptual thinking, intervention delivery, decision making); 'Facilitating Change with a practice process' (applying the occupational therapy process, e.g., assessment, intervention, and evaluation); 'Professional Interactions and Responsibility' (legal and ethical practice, relationships with clients and colleagues); 'Communication' (formal, written, professional, rapport, client centered, nonverbal, verbal); 'Professional Development' (professional behaviors); 'Performance Management' (personal and resource time management, leadership). (Bossers et al., 2007, p. 7).

Data collection

Each student’s performance on practice placement was measured by qualified occupational therapists using the CBFE Likert scale in seven competencies in four consecutive placements at the full-time stage. All Practice Educators (PE) of fieldwork completed identical training in how to score the student performance using the scale within the CBFE assessment tool (Bossers et al., 2007). The educators had guidance regarding the expectations of the learning outcomes and level of placement and were also given a guide as to the 'expected’ range of score* the students should attain per placement and related to the stage of training (See Table 2). They are encouraged to be ‘student focused’ and score based on students' actual competence, if the student demonstrates skills above the expected range, then a higher score can be awarded, thus allowing the full range to be taken into account.

Table 2
Practice Placements details including the expected range of scores for the level of placement.

Placement	Months into Program	Duration	CBFE expected Range*
1. Practice Placement 1 (PP1)	5 months	6 weeks	1-3
2. Practice Placement 2 (PP2)	12 months	6 weeks	3-7
3. Role Practice Placement (REP)	15 months	6 weeks	3-7
4. Practice Placement (PP4)	19 months	10 weeks	6-8

• ^*
  Note CBFE Scale is 9-point Likert Scale from U=0 (Unacceptable) to E=9 (Exceptional).

Failure of the placement is scored when a student performs below the expected range* in more than 4 out of 7 competencies, they are then permitted one further attempt to pass the placement. Within the study period no students failed the second attempt of the placement and in line with academic regulations where there are no academic credits attached to the placement modules, the student is allowed the full range of marks on repeat attendance as there is no ‘capping’ of performance by the student. It would not have been possible to enter data for the same two placements against one student as this would have misrepresented progression of the student (i.e. no student can progress to the next level of placement if they fail).

Data collection was performed by accessing the university student placement Logbooks where the CBFE had been completed by the supervising occupational therapy practice Educators, this data per competency, within each placement, per student, were entered directly into a data analytical statistical package- SPSS version 27 (International Business Machines, 2020).

Data analysis

Descriptive and inferential statistics were used to analyse and interpret the results intra and inter placement, representation of the results is illustrated verbatim. All scores for each of the seven competencies and the final ‘overall’ score were compared within all four fieldwork placements to measure the relationship between each competency within each placement, then compare the difference in the ranked competency scores to elucidate whether any particular practice competencies were enhanced or were lacking within a REP compared with the traditional placements.

All analyses of data were completed using the Statistical Package for the Social Sciences version SPSS version 27 (International Business Machines, 2020), Multiple imputation (MI) of missing data was conducted to reduce bias and followed guidelines as advocated by Eekhout et al. (2014). This occurred in 8 cases where there were some missing placement logbooks, but the final score for that particular placement was recorded (as it was digitally recorded for the exam board), but the individual competencies scores for one of the placements were missing. The subjectively measured assessment tool within the CBFE implies ordinal data, however, there is an argument that such scales with an absolute ‘0’ value at the farthest bottom of the scale (‘U’ =unacceptable) and ‘9’ (E=Exceptional) the absolute top of the scale, can provide more meaningful ratio data, which allows for greater validity (Dexter & Chestnut, 1995), This 0-9 CBFE ‘anchored’ Likert scale is in keeping with previous research studies which measured student performance on practice (fieldwork) placement (Holmes et al., 2010; Miller et al., 2001) adding further validity to the results.

An initial exploration using descriptive statistics, including the Mean, Standard Deviation and Median of the data was conducted, where relevant, the percentage that under-performed or over performed at each competency in relation to the expected range was examined and compared inter-placement for any relationship. Initial analysis suggested that the central tendency was skewed within each placement, this was confirmed by applying a Kolmogorov - Smirnov (KS) test (Altman & Bland, 1996). Therefore, to measure the data for any significant difference between the 7 competencies, a non-parametric test was conducted on paired competency data using the Wilcoxon Signed Ranks test. This test is focused on the differences between the paired rank of each competence and accounts for outliers that may skew the data. Statistical significance of any difference in scores was met at a P value of 0.05 or less.

Ethics

This study was ethically approved via the University ethics process. Ethical consideration was given to ensuring that all data was anonymous and access to the data set was limited to the staff imputing the data and the researchers. The student identifying criteria was coded and kept separate from the main data set.

Results

Final scores for all placements

The ‘step-up’ progression of student competency through the level of placements is clearly illustrated (see Table 3), all students moved up incrementally with each placement over time, as demonstrated by the final ’Overall’ Mean score from the first placement (M=3.51) to the final placement (PP4: M= 8.30). Indeed, students’ final performance overall scores were skewed above the top of the range expected in all placements, apart from in the second placement (PP2) where the mean score was still skewed towards the top (M=6.01) but remained within the expected performance range (CBFE score 3-7):

Table 3
Summary of descriptive data for all placements (n=181).

Practice Placement-PP	Practice Knowledge	Clinical Reasoning	Facilitating Change	Professional Interaction	Communication	Professional Development	Performance Management	Overall
PP1 Mean (Rank)	3.10 (7)	3.28 (6)	3.47 (3)	3.51 (2)	3.66 (1)	3.43 (4)	3.43 (4)	3.51
PP1 Median	3.00	3.00	3.00	3.00	4.00	3.00	3.00	3.00
PP1 Mode	3	3	3	3	4	3	3	3
PP1 Std. Dev	1.008	1.044	1.098	1.138	1.184	1.150	1.151	1.114
PP1 Range	8	8	8	8	8	8	8	8
PP1 Min	1	1	1	1	1	1	1	1
PP1 Max	9	9	9	9	9	9	9	9
PP2 Mean (Rank)	5.62 (7)	5.72 (6)	5.84 (4)	6.10 (2)	6.18 (1)	5.98 (3)	5.98 (3)	6.01
PP2 Median	6.00	6.00	6.00	6.00	6.00	6.00	6.00	6.00
PP2 Mode	6	6	6	6	6	6	6	6
PP2 Std. Dev	.962	1.056	1.023	1.162	1.181	1.128	1.067	1.093
PP2 Range	5	5	6	6	6	6	6	6
PP2 Min	3	3	3	3	3	3	3	3
PP2 Max	8	8	9	9	9	9	9	9
REP Mean (Rank)	7.00 (7)	7.06 (6)	7.12 (5)	7.33 (1)	7.33 (1)	7.19 (4)	7.29 (3)	7.24
REP Median	7.00	7.00	7.00	7.00	7.00	7.00	7.00	7.00
REP Mode	7	7	7	7	7	7	7	7
REP Std. Dev	1.130	1.010	1.071	1.070	1.048	1.074	1.139	1.061
REP Range	6	5	6	5	5	5	6	5
REP Min	3	4	3	4	4	4	3	4
REP Max	9	9	9	9	9	9	9	9
PP4 Mean (Rank)	8.08 (7)	8.17 (6)	8.19 (5)	8.41 (1)	8.39 (2)	8.28 (3)	8.22 (4)	8.30
PP4 Median	8.00	8.00	8.00	9.00	8.00	8.00	8.00	8.00
PP4 Mode	8	8	8	9	9	8	8	8
PP4 Std. Dev	.657	.744	.759	.706	.703	.725	.727	.641
PP4 Range	4	4	4	4	4	4	4	4
PP4 Min	5	5	5	5	5	5	5	5
PP4 Max	9	9	9	9	9	9	9	9

The relative ‘jump’ in final scores from the first placement to the second can be explained in that the first placement was only 5 months into the program whereas PP2 is halfway through the 24-month long MSc course. Table 3 reveals another relevant trend, a very high percentage (90.06%) of final ‘Overall’ scores were at the top or higher than the expected range in PP1. This percentage drops dramatically within PP2 to 28.73%, then increases over threefold at the top or over the expected range in the REP (78.45%); despite this range being identical to the PP2 and taking place only 3 months later. There is then an uplift again to 93.92% of students' scores at the top or above in their final placement (PP4), and a remarkable 36.6% of scores were classed as ‘Exceptional’ (above entry level clinician). This suggests that the REP as the penultimate placement does not hinder students' progress in achieving clinical competence to practise by the end of their training.

Notably within this study, there are very few students scoring beneath the scores expected, (Table 4), this in the REP is even more starkly observed with 0% of students scoring below what is expected of them within ‘Clinical Reasoning’ ‘Professional Interactions’ and ‘Professional Development’.

Table 4
Percentage of scores at top/above and low/below the expected scale per placement (n=181).

COMPETENCY	Placement and Scores at the top/above or lowest/below than the Expected Range of Scores (%)
	PP1		PP2		REP		PP4
	CBFE 1-3		CBFE 3-7		CBFE 3-7		CBFE 6-8
	Top & above	Low & Below	Top & Higher	Lowe & Below	Top & above	Low & Below	Top & above	Low & Below
Practice Knowledge	78.45	1.66	17.67	1.66	69.61	0.55	87.29	1.66
Clinical Reasoning	84.53	1.66	19.89	2.76	72.37	0	87.84	2.76
Facilitating Change	89.50	1.66	27.07	1.66	75.69	1.10	88.39	3.87
Professional Interactions	90.60	0.55	35.63	1.10	82.87	0	91.16	1.10
Communication	86.74	0.55	35.91	1.10	76.80	2.21	92.82	1.10
Professional Development	87.29	2.21	30.39	1.66	75.14	0	90.60	1.66
Performance Management	83.43	1.66	27.62	1.10	79.01	0.55	91.16	1.10
Overall	90.06	1.66	28.73	1.66	78.45	0	93.92	1.10

Comparison of individual competencies between placements.

The Mean, Median, Standard Deviation, and ranking of the scores for each of the competencies within each placement reveals some distinct patterns in the data, principally, there is variability between competency scores intra-placement, but these are consistent inter-placement. The range of student scores within all competencies and across all placements except PP4 do vary, for example, in ‘Communication’ -the top scoring competency for most placements- the standard deviation is high in PP1 (SD=1.184); PP2 (SD=1.181) and REP (SD=1.048), compared to PP4 (SD=.703). This can partially be explained by the limited range of the expected scores (6-8), for PP4 (final placement) at the top of the whole CBFE Likert scale, with only one other higher point (9-exceptional’) which denotes competence that is considered above that of an entry level clinician.

In terms of the ranking of the competencies, 'Practice Knowledge' is the lowest scored competency across all placements (PP1:M=3.10; PP2 M=5.62; REP: M=7.00; PP4: M=8.08). Despite this, the Mean and Median for this competency is at the top or above the expected range in all placements except for PP2. 'Clinical Reasoning' is (consistently again) the second lowest scoring competency (PP1:M=3.28; PP2:M=5.72; REP:M=7.06; PP4:M=8.17). Although these two competencies were ranked at the bottom of all the competencies, a Wilcoxon signed-rank test indicated that the difference between these two competencies in PP1 were statistically significant (z=-3.580 p <.001). This test applied to all other placements such as PP2, shows that although the ranking is the same, the difference between the two lowest competency scores is not significantly different (z= -1.814 p=0.07); this is a similar result in the REP (z= -1.361 p=0.174) and PP4 (z=-1.912 p= .056).

‘Communication’ is the highest scored competence in PP1 which is statistically higher than the next highest competence ‘Professional Interactions’ (z=-2.834 p=0.005), ‘Communication’ scores the highest again in PP2, however, this is not significantly higher than the next ranked competence of ‘Professional Interactions’ (PP2: z=-1.786 p=0.74). Within the REP, there is a close relationship between the two top competencies, and this is reflected in the Wilcoxon Signed Ranks test result (REP: z=-.128 p=0.898), interestingly, the ranking between the two top competencies swap round by PP4 so that ‘Professional Interactions’ is the top competency but not statistically significantly so (z=-.521 p=0.603).

In terms of the ranking of the remaining three other competencies, namely, ‘Facilitating Change in a Practice Process,’ ‘Professional Development’ and ‘Performance Management’ there are very few variations within the traditional placements in terms of rankings, with some competencies sharing identical mean values (in PP1 and PP2- see Table 2). However, the notable exception is ‘Performance Management’ in a REP (third highest) which is not significantly (z= -750 p=0.453) lower than ‘Communication’ (second highest); yet was found to be significantly higher than the next ranked ‘Professional Development’ (z=-2.222 p=0.026). Therefore, ‘Performance Management' is significantly the third top ranked competence in the REP which contrasts with other traditional placements, for example in PP4 and PP1 it was ranked fourth and in PP2 it was ranked at the identical Mean and therefore, the same level as ‘Professional Development’.

It can be observed that the REP develops competence for professional practice in all domains very similarly to traditional placements, with one exception, ‘Performance Management’ which was scored third highest and significantly so compared to all other competencies except the higher ‘Communication’ and ‘Professional Interactions.’ In terms of competence in applying the occupational therapy process (‘Change within a Practice Process’), this was third lowest in terms of ranking within the REP, however, 75.69% of scores were at the top of the range or higher than expected for the level of placement. This contrasts with PP2, where only 27.07% of scores were at the top or above the range, despite both placements sharing the same expected grade for the placements and the two placements occurring within a few months of each other.

Discussion

This study aimed to compare and measure professional and practice-based competencies of MSc occupational therapy students to identify differences between two pedagogical models of placement (traditional and REPs). The discussion of the findings is presented under the following four categories: 1) REPs are fit for purpose and may enhance ‘Performance Management’ skills; 2) Students perform particularly well in ‘Communication’, ‘Professional Interaction’ regardless of model of placement; 3) The REP placement experience was able to develop students’ ability to develop 'core' skills such as applying the Occupational Therapy process; 4) The same tool can be used to measure student core competencies for practice irrespective of genre of placement.

REPs are fit for purpose and may enhance skills in ‘Performance Management’

Despite supervisors of students on placement being guided by a standardized 'expected range' which encourages scoring relative to the stage of training (Bossers et al., 2007; Holmes et al., 2010; Miller et al., 2001), the sizable percentage of students performing at or above the expected range in all competencies in all placements; particularly in the first, last and including the REP, suggests that REPs are ‘fit for purpose’. It is noteworthy that the elevated levels of achievement within the REP were maintained and increased on completion of the subsequent final placement (PP4), suggesting that the REP did not disadvantage the students in attaining high outcomes in the ensuing final traditional placement.

The students were taught to assess population or community occupational needs and present these with supporting evidence, utilising a service development strategy and therefore a high-level skill set. This level of skills arguably is lost when working in traditional settings, due to the ‘shadowing 'of existing practice with the therapist in situ, or as Crabtree et al. (2012) assert, could be in part due to educators in the field not devoting sufficient time to explore evidence-based strategies. Further, allowing student autonomy and responsibility to contribute to the placement setting in a meaningful and valued way which is considered an important element of supporting students transition to practice (Patterson & D'Amico, 2020).

Further, this study has measured competence for practice separately from academic performance, and as the REP was a mandatory placement for all students, (ergo: not selecting the highest academic achievers), the findings suggest that academic performance is not necessarily associated with practice-based competence reflect that of other studies, (Thew & Harkness, 2018). Although these factors need to be more specifically explored with a correlational study of competence for practice with level of degree (e.g., Bachelors) using standardised measures.

Although there was consistency regarding the ranking of competencies inter-placement. The REP appears to offer the opportunity to expand on skills in performance management (i.e, time management being a self-starter, leadership etc), this was the third highest competency and significantly so in the REP vs the traditional placements. This “ability to be self-directed” (Tyminski, 2018, p.8) increases student confidence in being client centred. This enhanced skill derived from the self-directed nature of the pedagogical model of placement is beneficial to students within a more challenging environment and can contribute to their overall development (Lau & Ravenek, 2019). Further, Linnane & Warren (2017) found that the majority of students and occupational therapists who were surveyed identified that role-emerging placements offer possibilities of creativity, development of evidence-based practice and understanding of occupational therapy roles. Indeed, the enhancement of service development skills demonstrate that students are not only capable of such placements but that they are willing to take on the entrepreneurial roles required to expand the profession (Baptiste & Molineux, 2011), particularly via occupation focused projects which can influence the directions of organisations (Hunter & Volkert, 2017).

Students perform particularly well in communication and professional interaction skills regardless of placement model

The students proportionally out-perform in all the competencies than that expected in all placements apart from in PP2. Despite high scores in all competencies, certain competencies such as ‘Communication,’ and ‘Professional Interactions’ were significantly higher compared to ‘Clinical Reasoning' and ‘Practice Knowledge’ and this was consistent across all placements. This is in keeping with the findings of a larger scale study of fieldwork placements by Holmes et al. (2010). The fact that the top two and two lowest competencies were the same regardless of placement model shows that students are able to develop similar skills and attributes regardless of genre of placement. This reflects the findings of other studies such as that by Golos & Tekuzener (2019) who found a lack of significant differences in perceptions, expectations, and satisfaction of 155 undergraduate students between role emergent and role established placements. The factors at play here are potentially many-fold. This elevated level of achievement could reflect the fact that all students in the study were at MSc level, which is relevant in that many countries have mandated Masters’ level training for occupational therapy (Lemez and Jimenez, 2022), however, maturity does not necessarily predict practice placement outcome (Thew & Harkness, 2018). Another feature worth considering is the fact that within this cohort study students were educated utilizing a Problem Based Learning pedagogical curriculum, which has been argued to develop clinical reasoning skills in practice placements (Spalding & Killett, 2010).

The REP placement experience was able to develop students’ ability to develop 'core' occupational therapy process skills

Although a REP has been considered a challenging placement experience for students in that they are not shadowing experienced clinicians and are often expected to work entrepreneurially without direct daily occupational therapy supervision (Hunter & Volkert, 2017; Rodger et al., 2011). The results of this study do not support these concerns. The REP followed directly on from the PP2 in which the previous high performance of the students in competence for practice had levelled out. The high scores in the subsequent REP placement demonstrated that students were able to develop 'core' skills such as applying the Occupational Therapy process ('Facilitating Change' in the CBFE). Indeed, a high percentage of students scored above the expected range in the REP for this competency; despite other studies (of traditional practice placements) suggesting that skills in: “Practice Knowledge,’ ‘Clinical Reasoning,’ and ‘Facilitating Change’ may need special attention to reach entry level” (Holmes et al., 2010, p.42).

Further, most students excelled in their competence to practise within the REP despite the fact in traditional settings they are more likely to have daily input from a qualified occupational therapist and receive more direct supervision (Dancza et al., 2013; Short et al., 2017). This could be explained by the preparation for the REP which is more instructive and involved as compared to the more traditional placement (Edwards & Thew, 2011; Thomas & Rodger, 2011). Additionally, the pedagogical project-based model forces the students to take a lead and base their learning on evidence-based strategies and not rely on directly observed skills, which may enhance their self-efficacy, autonomy and confidence, reflecting previous studies (Lau & Ravenek, 2019; Short et al., 2017).

The same placement assessment tool can be used to measure student core competencies for practice- irrespective of genre of placement.

Occupational therapy curricula must provide a suitable measure of student practice-based learning that considers the theoretical skills from the classroom translated into practice and vice-versa (Patterson & D’Amico, 2020). An efficacious strategy for assessing students on placement is to have the same assessment tool that educators/supervisors are taught how to use in all genres of placement settings (Rodger et al., 2011). This study (although not a central aim) does evidence that REPs do not need to be measured differently to traditional placements. Further, having the same assessment tool for a REP as for traditional placements could be an added incentive for practice placement educators, as they do not need further training to learn how to assess students within non-traditional settings.

However, given that placement experience and the area of practice that graduates choose to pursue on graduation is related (Thomas et al., 2007), the more entrepreneurial and innovative/visionary practice skills which students can express in more role emerging placements (Thew et al., 2018; Lumague et al., 2006) are arguably now a necessity in new and emerging markets within contemporary health and social care (Lau & Ravenek, 2019; Linnane & Warren, 2017). Perhaps (and beyond the scope of this study) and that these skills should be recognised as a professional necessity for students to develop. It is perhaps pertinent for the addition of entrepreneurship, health promotion and service development to be an important skill to measure within student development for practice.

Limitations and further research

This study has only considered students at Masters Level, although whether this is an advantage within practice-based performance due to maturity or academic attainment remains to be comprehensively established within Allied Health Professional programs (Thew & Harkness, 2018; Watson, 2013). In addition, a limitation is the relatively small sample size and representative from only one specific program within the UK. There is a need to compare scores of students’ performance from other programs before any form of generalizability could be considered. The lack of a control group to compare the students’ ability at the same point in the curriculum between students attending a REP is also a limitation as the REP is a mandatory placement for all students within this study. The inter-rater reliability was not controlled for, as practice placements must be assessed by external occupational therapists in the field and not researchers.

Conclusion

This cohort study suggests that REPs offer a suitable learning environment for students to develop all professional practice or fieldwork-based skills in occupational therapy. There is a progression of skills in all competencies over time that is not interrupted by the REP being the penultimate placement. Students proportionally and significantly out-perform in communication, and professional interaction skills compared to other professional competencies such as clinical reasoning and specialist knowledge and that this trend is irrespective of model of placement (whether traditional or REP). A REP, however, appears to significantly develop skills in leadership and self-directed performance management, which is not a consistent finding within traditional placements. While it difficult to break down which aspect of the REP model promotes the differing skills, they are reflected in (or a function of) the autonomous nature of the occupation project-based pedagogical model adopted in a REP.

There are several factors that may have influenced overall student performance in their practice placement, such as the fact that all students in this study were mature and studying on a Masters’ level problem-based curriculum, it is important therefore, that this study should be repeated with data from under-graduate students.

Finally, this study provides valuable evidence that core or key skills of occupational therapy practice can indeed be developed within a REP just as effectively as in a traditional placement. Significantly, however, as the nature of REPs focus on the potential role of occupational therapists and offers students an added dimension to their learning of how to enhance the scope of occupation focused practice.

• How to cite: Thew, M., Cruz, D. C., & Thomas, Y. (2023). Comparing and measuring the practice-based performance and competence of occupational therapy students between traditional and role emerging placements: a retrospective cohort study. Cadernos Brasileiros de Terapia Ocupacional, 31, e3467. https://doi.org/10.1590/2526-8910.ctoAO265734672

References

• Altman, D. G., & Bland, J. M. (1996). Detecting skewness from summary information. BMJ (Clinical Research Ed.), 313(7066), 1200.
• Baptiste, S., & Molineux, M. (2011). The way forward? In M. Thew, M. Edwards, S. Baptiste & M. Molineux (Eds.), Role emerging occupational therapy: maximising occupational focused practice (pp. 149-155). Chichester: Wiley Blackwell.
• Bossers, A., Miller, L. T., Polatajko, H. J., & Hartley, M. (2007). Competency Based Fieldwork Evaluation for Occupational Therapists - CBFE-OT. Toronto: Nelson Education.
• Cantero-Garlito, P., Rodríguez-Hernández, M., & López-Martín, O. (2020). Descripción de los factores que determinan la elección de prácticas clínicas de los estudiantes en el grado en terapia ocupacional. Cadernos Brasileiros de Terapia Ocupacional, 28(3), 900-916. http://dx.doi.org/10.4322/2526-8910.ctoAO2021.
  » https://doi.org/10.4322/2526-8910.ctoAO2021
• Clarke, C., de Visser, R., Martin, M., & Sadlo, G. (2014). Role-emerging placements: a useful model for occupational therapy practice education? A review of the literature. International Journal of Practice-Based Learning in Health and Social Care, 2(2), 14-26.
• Clarke, C., Martin, M., de Visser, R., & Sadlo, G. (2015). Sustaining professional identity in practice following role-emerging placements: opportunities and challenges for occupational therapists. British Journal of Occupational Therapy, 78(1), 42-50.
• Crabtree, J. L., Justiss, M., & Swinehart, S. (2012). Occupational therapy master-level students’ evidence-based practice knowledge and skills before and after fieldwork. Occupational Therapy in Health Care, 26(2-3), 138-149.
• Dancza, K., Copley, J., & Moran, M. (2019). Occupational therapy student learning on role-emerging placements in schools. British Journal of Occupational Therapy, 82(9), 567-577.
• Dancza, K., Warren, A., Copley, J., Rodger, S., Moran, M., McKay, E., & Taylor, A. (2013). Learning experiences on role-emerging placements: an exploration from the students’ perspective. Australian Occupational Therapy Journal, 60(6), 427-435.
• Dexter, F., & Chestnut, D. H. (1995). Analysis of statistical tests to compare visual analog scale measurements among groups. Anesthesiology, 82(4), 896-902.
• Edwards, M., & Thew, M. (2011). Models of role emerging placements. In M. Thew, M. Edwards, S. Baptiste & M. Molineux (Eds.), Role Emerging Occupational Therapy: Maximising occupational focused practice (pp. 15-35). Oxford: Wiley-Blackwell.
• Eekhout, I., de Vet, H. C., Twisk, J. W., Brand, J. P., de Boer, M. R., & Heymans, M. W. (2014). Missing data in a multi-item instrument were best handled by multiple imputation at the item score level. Journal of Clinical Epidemiology, 67(3), 335-342.
• Golos, A., & Tekuzener, E. (2019). Perceptions, expectations and satisfaction levels of occupational therapy students prior to and after practice placement and comparison of practice placement models. BMC Medical Education, 19(1), 1-8.
• Holmes, J. D., Bossers, A. M., Polatajko, H. J., Drynan, D. P., Gallagher, M., O’Sullivan, C. M., Slade, A. L., Stier, J. J., Storr, C. A., & Denney, J. L. (2010). 1000 fieldwork hours: analysis of multi-site evidence. Canadian Journal of Occupational Therapy, 77(3), 135-143.
• Hunter, H., & Volkert, A. (2017). Issues and challenges of role-emerging placements. World Federation of Occupational Therapists Bulletin, 73(1), 57-65.
• International Business Machines ­- IBM Corp. (2020). IBM SPSS Statistics for Windows, Version 27.0. Armonk: IBM Corp.
• Lau, M., & Ravenek, M. (2019). The Student perspective on role-emerging fieldwork placements in occupational therapy: a review of the literature. The Open Journal of Occupational Therapy, 7(3), 1-21.
• Lemez, S., & Jimenez, D. (2022). Occupational therapy education and entry-level practice: a systematic review. Education Sciences, 12(7), 431.
• Linnane, E., & Warren, A. (2017). Apprehension and interest: therapist and student views of the role emerging placement model in the Republic of Ireland. Irish Journal of Occupational Therapy, 45(1), 40-51.
• Lumague, M., Morgan, A., Mak, D., Hanna, M., Kwong, J., Cameron, C., Zener, D., & Sinclair, L. (2006). Interprofessional education: the student perspective. Journal of Interprofessional Care, 20(3), 246-253.
• Mattila, A., DeIuliis, E., & Cook, A. (2018). Increasing self-efficacy through role emerging placements: implications for occupational therapy experiential learning. Journal of Occupational Therapy Education, 2(3), 1-20.
• Miller, L. T., Bossers, A. M., Polatajko, H. J., & Hartley, M. (2001). Development of the Competency Based Fieldwork Evaluation (CBFE). Occupational Therapy International, 8(4), 244-262.
• Patterson, B., & D’Amico, M. (2020). What does the evidence say about student, fieldwork educator, and new occupational therapy practitioner perceptions of successful Level II fieldwork and transition to practice? A scoping review. Journal of Occupational Therapy Education, 4(2), 1-22.
• Price, D., & Whiteside, M. (2016), Implementing the 2:1 student placement model in occupational therapy: Strategies for practice. Australian Occupational Therapy Journal, 63, 123-129.https://doi.org/10.1111/1440-1630.12257
  » https://doi.org/10.1111/1440-1630.12257
• Roberts, M. E., Hooper, B. R., Wood, W. H., & King, R. M. (2015). An international systematic mapping review of fieldwork education in occupational therapy. Canadian Journal of Occupational Therapy, 82(2), 106-118.
• Rodger, S., Fitzgerald, C., Davila, W., Millar, F., & Allison, H. (2011). What makes a quality occupational therapy practice placement? Students’ and practice educators’ perspectives. Australian Occupational Therapy Journal, 58(3), 195-202.
• Short, M., Letham, C., Currie, L., & Drynan, D. P. (2017). Comparing occupational therapy students’ competency scores: 2:1 versus 1:1 fieldwork. International Journal of Practice-Based Learning in Health and Social Care, 5(1), 1-11.
• Spalding, N. J., & Killett, A. (2010). An evaluation of a problem-based learning experience in an occupational therapy curriculum in the UK. Occupational Therapy International, 17(2), 64-73.
• Syed, S., & Duncan, A. (2019). Role emerging placements: skills development, postgraduate employment, and career pathways. The Open Journal of Occupational Therapy, 7(1), 1-18.
• Thew, M., & Harkness, D. (2018). Predictors of practice placement and academic outcomes in master’s-level pre-registration occupational therapy students. British Journal of Occupational Therapy, 81(4), 234-242.
• Thew, M., Hargreaves, A., & Cronin-Davis, J. (2008). An evaluation of a role-emerging practice placement model for a full cohort of occupational therapy students. British Journal of Occupational Therapy, 71(8), 348-353.
• Thew, M., Thomas, Y., & Briggs, M. (2018). The impact of a Role Emerging Placement while a student occupational therapist, on subsequent qualified employability, practice and career path. Australian Occupational Therapy Journal, 65(3), 198-207.
• Thomas, Y., & Rodger, S. (2011). Successful role emerging placements: it is all in the preparation. In M. Thew, M. Edwards, S. Baptiste & M. Molineux (Eds.), Role emerging occupational therapy: maximising occupational focused practice. Chichester: Wiley Blackwell.
• Thomas, Y., Dickson, D., Broadbridge, J., Hopper, L., Hawkins, R., Edwards, A., & McBryde, C. (2007). Benefits and challenges of supervising occupational therapy fieldwork students: Supervisors’ perspectives. Australian Occupational Therapy Journal, 54(Suppl. 1), S2-S12.
• Tyminski, Q. (2018). The development of a role-emerging fieldwork placement in a homeless shelter. Journal of Occupational Therapy Education, 2(2), 1-13.
• Watson, J. (2013). Progression routes and attainment in occupational therapy education: the impact of background characteristics. British Journal of Occupational Therapy, 76(12), 520-527.
• World Federation of Occupational Therapists - WFOT. (2016). Minimum standards for the education of occupational therapists. Retrieved in 2022, November 30, from https://wfot.org/resources/new-minimum-standards-for-the-education-of-occupational-therapists-2016-e-copy
  » https://wfot.org/resources/new-minimum-standards-for-the-education-of-occupational-therapists-2016-e-copy

Edited by

Section editor

Profa. Dra. Ana Luiza Alegretti

Publication Dates

• Publication in this collection
  11 Aug 2023
• Date of issue
  2023

History

• Received
  30 Nov 2022
• Reviewed
  12 Dec 2022
• Reviewed
  30 Jan 2023
• Accepted
  22 May 2023