Acessibilidade / Reportar erro

Translation and Transcultural Adaptation of the Milestones Instrument to Assess Teaching in Medical Residency Services on Orthopedics and Traumatology* Study developed at Hospital das Clínicas, Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brazil, and Centro de Ciências Médicas, Universidade Federal da Paraíba, João Pessoa, PB, Brazil.

Abstract

Objective

Orthopedics is not very common in many Brazilian medical schools, and there is no questionnaire to assess the teaching of musculoskeletal disorders during medical training. The Orthopedic Surgery Milestone Project is an assessment tool for orthopedic residents in programs or fellowships recognized by the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Orthopedic Surgery (ABOS). This study aims to translate the Orthopedic Surgery Milestone Project into Portuguese and to perform its transcultural adaptation.

Methods

The translation and transcultural adaptation consisted of the initial translation into Portuguese, back-translation into English, preparation of a pretest consensual text, and the subsequent elaboration of a final text.

Results

The final text was deemed adequate and equivalent to the original one for the evaluation of orthopedics residents throughout their program.

Conclusions

Given the lack of instruments for the assessment of orthopedic residents, the translation and transcultural adaptation of the Orthopedic Surgery Milestone Project were compatible; this can be an instrument for improved medical education.

Keywords
translating; transcultural adaptation; medical education; surveys and questionnaires

Resumo

Objetivo

A ortopedia é uma especialidade pouco presente na formação de muitas escolas médicas brasileiras, sendo que não foi identificado nenhum questionário que avalie o ensino das desordens musculoesqueléticas durante a formação médica. O Orthopedic Surgery Milestone Project constitui um instrumento de avaliação de residentes de ortopedia nos programas de residência ou bolsas credenciados pela ACGME (The Accreditation Council for Graduate Medical Education) e pelo ABOS (The American Board of Orthopedic Surgery). O objetivo é realizar a tradução para a língua portuguesa e adaptação transcultural do Orthopedic Surgery Milestone Project.

Métodos

A tradução e adaptação transcultural consistiram na tradução inicial para o português, retro tradução para o inglês, análise para a obtenção de uma versão consensual pré-teste e posterior versão final.

Resultados

A versão final foi considerada adequada e equivalente à original para a avaliação dos residentes de ortopedia ao longo do programa de residência médica.

Conclusões

Diante da falta de instrumentos para a avaliação dos residentes em Ortopedia, a tradução e a adaptação transcultural do Orthopedic Surgery Milestone Project foi compatível, podendo ser um instrumento para uma formação médica mais adequada.

Palavras-chave
tradução; adaptação transcultural; ensino médico; pesquisas e questionários

Introduction

Musculoskeletal disorders, including lumbalgia, lumbosciatalgia, and arthrosis, are frequent complaints in primary clinics, accounting for up to 25% of the reasons to seek medical care. The World Health Organization declared the period from 2000 to 2010 as the Bone and Joints Decade, thus confirming the importance of this topic in medicine.

Therefore, it is essential to train adequately the professionals who will meet this high demand. The development of skills in multiple care dimensions during the training of orthopedists and traumatologists is paramount to assure the proper treatment of patients. To do so, tools assessing the development of these skills are protagonists both to encourage individuals to acquire abilities they still do not possess and to provide data to improve the educational quality of residency programs.11 Ames SE, Ponce BA, Marsh JL, Hamstra SJ. Orthopaedic Surgery ResidencyMilestones: Initial Formulation and Future Directions. J Am Acad Orthop Surg 2020;28(01):e1-e8

No instrument in Portuguese evaluates the teaching of musculoskeletal disorders throughout medical training. However, similar questionnaires, available for several medical specialties, have been created in other countries, including the Orthopedic Surgery Milestone Project. This instrument evaluates physicians from residency or fellowship programs recognized by the Accreditation Council for Graduate Medical Education (ACGME), a non-profit private institution that assesses and endorses all graduate medical training programs, including internships, residencies, fellowships, and subspecialty programs, in the United States.22 Accreditation Council for Graduate Medical Education (ACGME). Accreditation Council for Graduate Medical Education (ACGME) About Us [access in 2021 Mar 19]. Available in: http://www.acgme.org/About-Us/Overview
http://www.acgme.org/About-Us/Overview...

The Orthopedic Surgery Milestone Project started in 2013 to evaluate the skills and knowledge of residents from a specialty within the same set of competencies. It does not replace existing assessments in residency programs but provides a common framework to analyze acquired skills.33 Hobgood CD, Riviello RJ, Jouriles N, Hamilton G. Assessment of communication and interpersonal skills competencies. Acad Emerg Med 2002;9(11):1257-1269 The Milestones instrument generates detailed information on the strengths and weaknesses of each residency service and identifies syllabus gaps to be addressed, improving the educational quality of residency programs.

The ACGME requires an annual evaluation, including systematic analysis of curriculum, resident performance, faculty development, and program quality. It also suggests that assessment instruments include written examinations, global ratings, and case/procedure records. The teaching analysis model recommended by the ACGME is based on the skills deemed fundamental for professional practice.

Residents are evaluated through milestones, that is, medical knowledge, patient care, professionalism, communication skills, and system practice, in addition to learning based on such practices and improvements. During the assessment, a preceptor selects the milestone that best describes the resident's current performance, from levels 1 to 5. Level 1 represents the milestones expected from a 1st-year resident, and level 5 refers to the performance goals defined for the residency, potentially describing the performance of a professional practicing for several years.44 Guillemin F, Bomba6dier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 1993;46(12):1417-1432 However, it is worth mentioning that these levels do not correspond to the year of residency; moreover, the selection of a certain level implies that the resident substantially demonstrates the characteristics related to it, as well as those from lower levels. All residents, from the 1st to the last year of medical residency, will be evaluated using the Milestones instrument.

Thus, the translation and transcultural adaptation of the Orthopedic Surgery Milestone Project will help in the evaluation and, consequently, training of professionals specializing in orthopedics.

Methods

After the ACGME authorized the translation of the questionnaire, two Brazilian translators fluent in English performed the first translation into Portuguese, and both texts were compared to formulate a consensus text. This text was back translated into English by two bilingual translators whose main language is English and who had no knowledge of the original text of the instrument. This translation was evaluated and compared to the original text by a review committee to correct potential discrepancies and enable the preparation of a pretest version of the questionnaire.

The last step consisted of evaluating the clarity and understanding of the questionnaire regarding its intended use. Since the pretest version had the desired characteristics, it was confirmed as the final text.

Results

Figures 1 to 5 describe the translation into Portuguese and transcultural adaptation of the Milestones 2.0 instrument. It is noteworthy that the back translation process showed that the initial Portuguese text corresponded to the English text, with no semantic divergence between translators.

Fig. 1
Diagram of the protocol used for translation and transcultural adaptation of the Milestones evaluation instrument.

Fig. 2
Translation of the Milestones instrument. Clinical case 1-original text.

Fig. 3
Translation of the Milestones instrument. Clinical case 2-original text.

Fig. 4
Translation of the Milestones instrument. Clinical case 3-original text.

Fig. 5
Translation of the Milestones instrument. Clinical case 4-original text.

The first two texts were the initial translations into Portuguese. Both presented semantic similarity, with no modification of the analyzed proposition. However, sometimes, different words with similar meanings were used to translate the same term (e.g., basic imaging exams versus basic imaging studies). A specialized committee reviewed the two translations and reconciled them into a single, easier-to-understand text. The final text consisted of the back translation of the reviewed versions.

The original Milestones instrument has 54 pages and 20 clinical situations from different areas of orthopedics; its questions encompass the doctor-patient relationship, in addition to theoretical and practical medical knowledge (Table 1 and Figures 12).

Table 1
Consensus for final text in Brazilian Portuguese

Discussion

There are no instruments to assess medical skills regarding musculoskeletal disorders in Portuguese, but the Orthopedic Surgery Milestone Project fulfills this purpose in English. The method of translation and transcultural adaptation used here was recommended by Guillemin et al.44 Guillemin F, Bomba6dier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 1993;46(12):1417-1432 It consists of four stages: initial translation, back translation, elaboration of a consensus text, and preparation of a commented pretest version and the final text.

The present study followed the guidelines recommended by Guillemin et al.44 Guillemin F, Bomba6dier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 1993;46(12):1417-1432 to minimize bias and biased results. This methodology makes the translated Milestones questionnaire suitable for application in Brazilian orthopedic residency services to assess the skills and abilities of these professionals.

Translation and adaptation of this instrument occurred after ACGME approval. The entire process was carried out properly, following the standards of cultural adaptation, and there were no difficulties in understanding any question. The Milestones instrument is widely recognized and used in the United States in different medical services and subspecialties, including clinical and surgical areas (e.g., Milestones for Plastic Surgery, Internal Medicine, Urology). It is applied easily through clinical situations that assess professionalism, communication skills, and medical knowledge. Residents are analyzed and graded on competency scales and developmental milestones throughout the year. It is a way for the student to recognize their limitations and deficiencies and improve them. The goal is to make specialist training programs more adequate to the current demand so that the specialized physician can work in public and private sectors, either in large or small centers.55 Fernandes CR, Farias Filho A, Gomes JMA, Pinto Filho WA, Cunha GKF, Maia FL. Currículo baseado em competências na residência médica. Rev Bras Educ Med 2012;36(01):129-136

A study based on competencies and milestones aims to identify and analyze any deficiency in the training process of resident physicians, allowing preceptors to reinforce the teaching/learning of that competency.66 Stodel EJ, Wyand A, Crooks S, Moffett S, Chiu M, Hudson CC. Designing and Implementing a Competency-Based Training Program for Anesthesiology Residents at the University of Ottawa. Anesthesiol Res Pract 2015;2015:713038 In addition, competency-based training allows for more uniform curricular development in medical residency schools.

The adaptation of this assessment instrument to the Brazilian context required minor grammatical and cultural adjustments. In preparing the pretest version and the final text, we tried to maintain the form that most closely resembled the original to achieve equivalent semantics between them.

It is also worth mentioning that no items from the original questionnaire were included or excluded throughout the development of the final text, preventing further changes in psychometric properties to allow texts comparison.

The potential use of Milestones as a method for evaluation and improvement of the quality of teaching in medical residency services will result in physicians better prepared to exercise their profession. The translation of this instrument and its application will allow the assessment of the main deficiencies in Brazilian Orthopedic Services, which may improve medical education.

The Portuguese version of the Milestones instrument, adapted to the Brazilian culture, is applied easily.

This instrument should be used once a year in residents of the 3 years, similar to the Assessment Test of Residents in Orthopedics (TARO) by the Brazilian Society of Orthopedics (SBOT). We initially propose the assessment of competency 1 in the 1st year of residency, competencies 2 and 3 in the 2nd year, and competencies 4 and 5 in the 3rd year of residency. However, instrument application will eventually determine the equivalence. The final text of this questionnaire was the result of the corrections made by all translators and review teams involved. Although there is no evidence against the adequate translation and transcultural adaptation of the Milestones instrument to Brazil, future studies will establish the accuracy and reliability of this tool in Brazilian orthopedic and traumatology residency programs.

Conclusion

The translation into Portuguese and the transcultural adaptation of The Orthopedic Surgery Milestone Project provide an instrument for assessing competencies suited to the Brazilian reality. This instrument can evaluate different aspects of knowledge and practice over the 3 years of specialization, seeking a better medical education. The translation of the Milestones assessment instrument into Portuguese was consistent with the original texts.

References

  • 1
    Ames SE, Ponce BA, Marsh JL, Hamstra SJ. Orthopaedic Surgery ResidencyMilestones: Initial Formulation and Future Directions. J Am Acad Orthop Surg 2020;28(01):e1-e8
  • 2
    Accreditation Council for Graduate Medical Education (ACGME). Accreditation Council for Graduate Medical Education (ACGME) About Us [access in 2021 Mar 19]. Available in: http://www.acgme.org/About-Us/Overview
    » http://www.acgme.org/About-Us/Overview
  • 3
    Hobgood CD, Riviello RJ, Jouriles N, Hamilton G. Assessment of communication and interpersonal skills competencies. Acad Emerg Med 2002;9(11):1257-1269
  • 4
    Guillemin F, Bomba6dier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 1993;46(12):1417-1432
  • 5
    Fernandes CR, Farias Filho A, Gomes JMA, Pinto Filho WA, Cunha GKF, Maia FL. Currículo baseado em competências na residência médica. Rev Bras Educ Med 2012;36(01):129-136
  • 6
    Stodel EJ, Wyand A, Crooks S, Moffett S, Chiu M, Hudson CC. Designing and Implementing a Competency-Based Training Program for Anesthesiology Residents at the University of Ottawa. Anesthesiol Res Pract 2015;2015:713038

Publication Dates

  • Publication in this collection
    21 Nov 2022
  • Date of issue
    Sep-Oct 2022

History

  • Received
    12 Aug 2021
  • Accepted
    14 Mar 2022
Sociedade Brasileira de Ortopedia e Traumatologia Al. Lorena, 427 14º andar, 01424-000 São Paulo - SP - Brasil, Tel.: 55 11 2137-5400 - São Paulo - SP - Brazil
E-mail: rbo@sbot.org.br