Reliability of a script agreement test for undergraduate speech-language therapy students

ABSTRACT

Purpose

To estimate the reliability of scripts designed for undergraduate Speech-Language Therapy students.

Methods

A descriptive cross-sectional study was carried out. Qualitative variables were summarized by frequency or proportion and quantitative through means (CI 95%). Reliability was estimated through Cronbach's α coefficient, and inter-rater agreement was determined using Fleiss’s Kappa index. The analytical tests considered a significance level of p<0.05.

Results

80 scripts organized in four areas of speech-language therapy were validated by 41 speech-language pathologists. The average experience of the professionals was 17.1 years. The reliability of the corpus was α: 0.67 (min= 0.34; max: 0.84), and the inter-rater agreement κ: 0.29 (min: 0.07; max: 0.45).

Conclusion

The corpus's reliability scores were similar to those reported by previous studies in different health professions. Having validated strategies aimed at developing proficiency and supporting classic training actions in undergraduate courses will contribute to increasing the quality of future health professionals.

Keywords:
Reliability; Reproducibility of Results; Higher Education; Speech-Language Pathology; Professional Competence