Reflections on the use of the Progress Test in the programmatic student assessment

Abstract:

Introduction:

The Progress Test (PT) is a well-established and mostly successful modality of student knowledge assessment in the health professions, mainly those in the medical area, with the potential to contribute substantially to the formative and informative purposes (quality control and indication of improvement in the teaching-learning processes). Additionally, the PT has characteristics that are adequate for its inclusion in institutional evaluation systems that facilitate the formative purpose, such as programmatic assessment (PA), but that also meet the summative purpose. In schools that have defined actions aimed at introducing PA in their undergraduate courses, it is necessary to reflect on the strengths and limitations of using PT in the evaluation system.

Development:

based on the considerations of a working group representative of the entire institution, tasked with proposing means of introducing PA in a new curriculum for the medical course, with international advice with experience in both PT and PA, we generated a reflection on this topic, based on the authors’ experience and data from the literature. It is proposed that, within the longitudinal perspective of the PA, the PT constitutes one of the pillars in the assessment of knowledge. The PT can be used as a basis for monitoring the students, in the context of their class (cohort), and its results should be discussed with the mentors who accompanies and supports them. The PT must also play a central role in management, as a source of information for eventual review and qualification of the curriculum and its teaching-learning activities. It is predictable that the use of the PT in PA will bring different challenges and barriers, which will be more easily overcome if the institution has already consolidated experiences in the application of institutional exams and in faculty development for the production of good quality objective questions.

Conclusion:

the effectiveness of the PT within the institutional PA system will depend on measures aimed at increasing its effectiveness in the assessment and that encourage the student’s active participation, reflecting on their performance in the PT, with the support of their mentor, aiming to engage in actions that encourage learning self-regulation.

Keywords:
Educational Measurement; Student Assessment; Progress Test; Undergraduate Medical Education

Resumo:

Introdução:

O Teste de Progresso (TP) constitui modalidade estabelecida e bem-sucedida de avaliação de conhecimentos do estudante das profissões da saúde, principalmente os de Medicina, com potencial de contribuir substancialmente para as finalidades formativa e informativa (controle de qualidade e indicação de melhoria nos processos de ensino e aprendizagem). Adicionalmente, o TP apresenta características adequadas à sua inclusão em sistemas institucionais de avaliação que privilegiem a finalidade formativa, como a avaliação programática (AP), mas que cumprem também a somativa. Nas escolas que vêm definindo ações visando à introdução da AP em seus cursos de graduação, é necessária a reflexão sobre as fortalezas e limitações da utilização do TP no sistema de avaliação.

Desenvolvimento:

A partir das considerações de um grupo de trabalho representativo de toda a instituição, incumbido de propor meios de introdução da AP em um novo currículo para o curso de Medicina, contando com assessoria internacional com experiência tanto no TP como na AP, elaborou-se reflexão sobre esse tema, baseada na experiência dos autores e em dados da literatura. Propõe-se que, dentro da perspectiva longitudinal da AP, o TP constitua um dos pilares na avaliação de conhecimentos. O TP pode servir de base para acompanhamento do estudante, no contexto da sua turma (coorte), e seus resultados devem ser discutidos com o mentor que o acompanha e lhe dá suporte. O TP deve ter também papel central na gestão, como fonte de informações para eventual revisão e qualificação do currículo e das suas atividades de ensino e aprendizagem. É previsível que a utilização do TP na AP traga diferentes desafios e barreiras, que serão mais facilmente superados se houver na instituição experiências já consolidadas de aplicação de exames institucionais e de desenvolvimento docente para a elaboração de questões objetivas de boa qualidade.

Conclusão:

A efetividade do TP dentro do sistema institucional de AP vai depender de medidas que visem aumentar a sua efetividade na avaliação e que estimulem a participação ativa do estudante, refletindo sobre seu desempenho no TP, com o apoio do seu mentor, de modo a se engajar em ações que fomentem a autorregulação da aprendizagem.

Palavras-chave:
Avaliação Educacional; Teste de Progresso; Educação Médica

INTRODUCTION

The Progress Test (PT) is a type of knowledge assessment applied institutionally to all classes of a course, at the same time. It consists of contextualized objective questions, with content and level of complexity that are adjusted to the student soon to be graduated^1)-(5.

The PT is an important tool for evaluating and controlling the quality of the curriculum teaching-learning activities, allowing the verification of the students’ level of knowledge in each class, especially those who are going to be graduated^1)-(5. It also allows checking, through its successive applications, the progress in the acquisition of knowledge, at the individual level and in the different cohorts^1)-(5. This information helps students and curriculum managers to identify learning strengths and weaknesses in the cognitive domain and also in clinical reasoning, decision-making, diagnostic and therapeutic management, health promotion, protection and rehabilitation domains. The results of the sequential tests help students to perform a self-assessment and monitor the evolution of their performance, contributing to the self-regulation of learning and to the formative purpose of the assessment^1)-(5.

Recently, there has been a growing interest from medical schools in implementing student programmatic assessment (PA) in their courses. PA is an innovative way of practicing student assessment, aiming to better meet its purposes, but strongly favoring the formative scope^6)-(8. PA is carried out using different instruments, following the recommendations of good practices, under centralized coordination and well-defined institutional responsibility⁸.

Many of the pioneering schools in the implementation of the PA were also pioneers in the introduction of the PT^1,3 and use it as an important source to obtain information on student performance in the cognitive domain. The Ribeirão Preto Medical School at the Universidade de São Paulo (FMRP-USP) has invested heavily over the last decade in improving educational assessment in its seven undergraduate courses. This investment took the form of the foundation, in 2014, of its undergraduate education assessment center, which institutionally directs the evaluation of the quality of the disciplines and internships in the courses and student assessment. In 2016, the faculty development center for teaching was founded, which, among its different activities, has those that fall into a specific line of work, aimed at training teachers with a view to the future implementation of PA. In 2018, the goal of introducing PA in undergraduate courses was included in the institution academic project within the scope of the multi-year strategic planning.

The institutional movement aimed at restructuring the curriculum of the FMRP-USP undergraduate medical course, which started in 2020, provided a unique opportunity for changes in student assessment within the principles of PA. In the discussions on this topic, it was proposed to include the PT among the main instruments to be used in obtaining information about the student’s performance. The acceptance of this proposal originated the demand for reflection on advantages, limitations and challenges related to the role of PT in PA.

This article aims to disseminate these reflections, which originated from the discussions involving several teachers, constituting a representative group of the entire institution, with the advice of teachers from a pioneering school both in the introduction of PT¹ and PA⁶ and with the support of international literature.

Brief history and dissemination of the Progress Test

The PT has been used worldwide, since its pioneering use in the 1980s, at the Kansas City Medical School (United States of America)⁹, and at the current University of Maastricht (Netherlands)¹, in curricula based on problem-based learning.

In Brazil, PT started being used in the last century, at the School of Medicine of Universidade de São Paulo and in the medical course of Universidade Estadual de Londrina, Paraná^4),(5. In 2005, thanks to the efforts of Professor Angélica Maria Bicudo, from the School of Medical Sciences of Universidade Estadual de Campinas (UNICAMP), the first Brazilian consortium was created for the application of PT to students from nine medical schools in the Southeast and South regions^4),(5, which FMRP-USP soon joined. The successful experience of this consortium inspired initiatives in other regions of the country, until emerged the proposal to carry out the PT in all medical schools in Brazil, made by the Brazilian Association of Medical Education (ABEM, Associação Brasileira de Educação Médica)⁴. In 2021, in a recent nationwide administration of the PT carried out by ABEM, 48,496 students from 130 medical schools participated, resulting in 18 PT consortia in different regions of Brazil, bringing together 182 medical schools¹⁰.

Currently, these 18 consortia remain active, bringing together 220 schools, representing a universe of around 50,000 students, and characterizing a very successful strategy¹¹.

This dissemination has contributed to improve the evaluative culture within the scope of Brazilian medical education, as well as to increase knowledge about the PT and the meaning of its findings, in addition to giving rise to other innovative exam uses, as demonstrated by the articles in the collection recently published by the Brazilian Journal of Medical Education (RBEM, Revista Brasileira de Educação Médica)¹².

Abroad, there are countless countries, on several continents, in which the introduction of PT was successful in medical schools, in addition to records of its use in training programs for medical specialists and in other courses in the health area, such as Nursing, Pharmacy, Veterinary Medicine and Dentistry¹³.

Characteristics and operation of the Programmatic Assessment

In PA, the articulated and organized use of several instruments, under centralized supervision, predominantly meets the formative purpose of student assessment, but also provides data on their performance in the various domains of learning and in relation to the different skills that the curricular objectives determine^6)-(8. These data can be “triangulated”, providing robust information that support decision-making of summative nature^6),(7. These should privilege remediation and not simply the approval-failure binomial. These decisions are the responsibility of the institutional instance and not only of the curricular units (disciplines, modules and internships)^6)-(8.

The practice of PA contributes to changes in paradigms that rule student assessment. Figure 1 shows the main points of this change, going from the model of assessment of learning towards the assessment for learning, one of the goals of PA^6)-(8.

Figure 1
Comparison of the main characteristics of the traditional student assessment (current) and the proposed programmatic approach (future).

In the more traditional assessment of medical students, exams and tests are more focused on the cognitive domain within the scope of the curricular units, usually at the end of their activities, with the sole purpose of making decisions about passing or failing. Some of these events have a strong impact on the student, being called “high stakes”¹⁴. In PA, on the contrary, more comprehensive assessments should predominate, with greater frequency and continuity, focusing not only on knowledge, but, preferably, on clinical and procedural skills and competences in the affective-behavioral domain^6)-(8. The formative purpose of the assessments should also predominate in PA, aiming to contribute to the student’s learning and development, through good quality feedback, which should involve and engage the student^6)-(8. The information obtained in these assessments also contribute to decisions of summative nature, but with reduced weight and less impact on the student’s life than those of the “high stakes” type, being therefore called “low stakes”¹⁴.

Students actively participate in the PA, being the target of the several feedbacks they receive, as well as in the management of some evaluative events, especially in clinical activities, and, mainly, making records of their activities and exercising reflection and self-assessment in their individual portfolio, supported by their mentors^6)-(8.

A proposal for the introduction of PA in which PT is an important component

For the effective introduction of PA in the curriculum of an undergraduate course in the health professions, it is necessary to produce a student evaluation program that, among other objectives, describes in detail its main components. Chart 1 depicts, as an example, measures proposed to introduce the main components in the implementation of the PA at FMRP-USP.

Chart 1
Example of measures to introduce the main components of an assessment program based on the principles of Programmatic Assessment for a medical course curriculum, including the Progress Test.

PROPOSALS
- Organizing evaluation events and, in particular, making decisions associated with the summative purpose become the responsibility of the institution and not just of the coordinators of the curricular units (disciplines, modules, internships);
- Establish institutional evaluation centers in the different phases of the course, as centralized instances for the management of evaluation events and for integrated analysis of their results;
- Establish decision-making committees for the different phases of the course, with a recommendation to prioritize the remedying of deficient aspects and not simply pass-and-fail outcomes.
- Adopt, as a reference, in the organization of assessment events, the analysis of obtained data and decision-making, the three main domains of education and training (cognitive, clinical-procedural and affective-behavioral);
- Plan the frequent occurrence of “low stakes” assessment events with training purposes, in all semesters, covering the three main domains of education and training;
- Invest in the uniform use of established assessment methods in the different education and training areas, following the recommendations of good practices in their use;
- Organize a calendar with predefined moments for carrying out the summative assessment of the curricular units, to avoid competition between them and the teaching-learning activities and to reduce the effect of fleeting retention in learning;
- Organize a smaller number of “high stakes” summative exams focused on knowledge and clinical and procedural skills in two strategic moments of the course (middle and end of the course);
- Include the PT among the main formal evaluation events to be considered in the program, for the cognitive domain;
- Coupling the assessment program with a mentoring program, aiming to develop the student’s capacity for reflection and self-assessment, using a reflective portfolio and interaction with a mentor.

For greater effectiveness of the student assessment in the new paradigm, it is essential that the actions of the assessment program be articulated with activities of a curricular axis of Personal and Professional Development, containing regular activities typical of a mentoring program, with extensive use of a reflective electronic portfolio, which mediates the interaction of students with their mentors^6)-(8. This portfolio should include the student’s self-assessment and their reflections at different times, also serving as a source of quantitative and qualitative information relevant to the assessment of each student, recorded throughout the entire course^6)-(8.

The role of the PT in an assessment program based on PA principles

The presented proposal predicts that, unlike what occurs in most Brazilian schools that apply the PT^4),(5),(10, this will constitute an important element of the assessment program (Chart 1), with the compulsory participation of all students and the recording or the obtained results in the individual student portfolio. This will make it easier for each student to follow the progression of their performance. The results will be analyzed in general and in each area of training, thus stimulating the self-regulation of learning. This will be enhanced by the student’s interaction with their mentor, in the processes of reflection and self-assessment of performance.

The data obtained in the PT, for each student, will be taken into account, together with other information on the performance and attitudinal assessment, in decisions regarding progression or indication of remediation. Each PT application will allow obtaining a relatively large and varied set of relevant information (Chart 2).

Chart 2
Information on each student that can be obtained by including the Progress Test in a Programmatic Assessment system

INFORMATION
- Overall test score
- Scores in each of the tested areas
- Position of the student in relation to their cohort
- Student position in relation to the set of participants from the same period of the other schools included in the test
- Magnitude of the student’s individual progression in relation to their previous test score
- Magnitude of the student’s individual progression in relation to the average progress of their cohort, since the previous test
- Magnitude of the student’s individual progression, in relation to the average progress of participants from other schools, enrolled in the same period, since the previous test

Thus, the successive applications of the PT will provide, for each student, several relevant “low-stakes” elements in decision-making. In these, the triangulation of the PT results with the assessments carried out in the several curricular units will help to identify students in need of remediation and the areas where this will be necessary. The student’s global scores in the PT performed in the middle and at the end of the course will be used as “high stakes” elements, in triangulation with the institutional exams of knowledge devised at these times.

In the informative purpose (evaluation of the quality of the curriculum and its activities), the TP will continue to be important. Additionally, as the proposal is part of the curriculum restructuring, the TP may be an important tool for comparing the new curriculum with the previous (current) one.

Expected challenges and limitations

The formal inclusion of the PT in the new student assessment program, according to the PA principles, should bring positive contributions, but it offers some challenges and barriers to be considered, among which there are those shown and briefly discussed in Chart 3.

Chart 3
Some of the possible challenges and limitations to the introduction the Test of Progress among the components of Programmatic Evaluation

CHALLENGE OR LIMITATION	COMMENTS
Assessment culture (values, habits and practices that favor fragmented summative assessment)	Resistance to the inclusion of PT among the formal events of the evaluation program may arise both among teachers and among students, accustomed to considering only the summative evaluations of each curricular unit as relevant.
Need for additional investment in production of questions and logistics	In the context of PA, it would be desirable to have a greater number of PT applications each semester or year. Giving greater institutional relevance to the PT also implies greater care with the quality of the questions and with the logistics of the applications.
Higher financial cost	The inclusion of the PT in the PA could have repercussions on the financial cost for the institution, with the need for greater investment, whether in greater participation of teachers or with an eventual increase in the annual number of test applications.
Alignment of the contents of the questions to be produced with the instructional objectives of the local curricular units	The PT will be more valid if there is evidence that the assessment matrix on which it is based, as used by the consortium responsible for the test, is in line with the instructional objectives of the curricular units or modules of the new curriculum of the medical course.
Production of questions	With the inclusion of PT as an important component of PA, it will be necessary to engage a greater number of teachers in the production of good quality questions, which will require greater availability of time, either for the participation in faculty development, or in the production of questions itself.

Currently, the PT is used in our institution for education and information purposes. The first requires the students’ voluntary participation and engagement when reflecting on their results and investing in improving them, with their degree of involvement in these processes being uncertain. As for the second purpose, the PT is seen as being of interest only to the management of the courses and not to the group of teachers. Thus, the PT tends to be considered as a minor initiative by teachers and students and the proposal to include the results for the summative purpose, contributing to decisions related to student progression, who must participate compulsorily in the applications, can generate tensions. These will be particularly relevant if the use of the PT for the summative purpose takes the student away from the reflection on their performance.

The application of the PT only once a year constitutes an important limitation to its use as a source of relevant information on the acquisition of knowledge by students. However, expanding the use of this test brings organizational, logistical and cost challenges. Even with a single annual administration, institutional efforts and investment demand in the logistics of student participation on a single day are considerable and will predictably increase with the desirable increase in the number of PT administrations in the year. This also implies engaging more teachers in the production of a greater number of good quality questions, which will require greater availability of time, either for participation in faculty development, or in the production of the questions itself.

Finally, there remain challenges related to the validity and reliability of the PT, which were very well discussed in a recent review⁵. These challenges are related, among other aspects, to the degree of alignment between the local curriculum and the assessment matrix on which the PT is based, prepared by the consortium responsible for the test, which will need to be determined. Along the same lines, the concern with validity and reliability is related to the frequency of annual administrations and also to the number of questions per exam, which, in Brazil, are lower than those of other institutions abroad and, mainly, than what would ideally be desirable^15)-(17. For instance, in Dutch and German schools, the alignment between the instructional objectives of the curriculum and the PT (validity) is sought with exams that have up to 200 questions per administration and an attempt is made to guarantee the reproducibility and stability of the results indicative of the student performance profile (reliability or consistency) with examinations at a frequency of four per year¹⁶.

DISCUSSION

The reflections presented herein are the result of ongoing work, whose progress may reveal other aspects to be considered. The success in the introduction of the PT as an important element of PA will depend on overcoming the expected challenges and others that may arise. It is also worth considering the existence of facilitating factors for the intended changes. Among them are the good reputation that the PT have in the institution, especially among those involved in curriculum management and the student body, which recognizes its contribution to the promotion of learning. Another fact to be considered is that the institution has already been running supra-disciplinary exams to assess knowledge and skills at specific moments of the course, with formative, summative and informative purposes, depending on the exam and the time of application. For this purpose, there has been a large investment in the institution’s faculty development center, which has qualified personnel and established experience in preparing teachers for student assessment and, specifically, for the task of preparing objective questions of good quality^18),(19.

In developing the proposal for the institutional student assessment program, we tried to characterize it as a collective construction, as recommended for the construction of the political-pedagogical project of a course²⁰ and, especially, for the introduction of programmatic assessment²¹. It is uncertain whether this intention became true but, in its search, an attempt was also made to invest in the educational and evaluation culture of the institution, especially that of the faculty, which may reduce future resistance.

Propositions for the new assessment program were based on the basic principles and concepts of the PA^6)-(8, on general international recommendations for student assessment^22)-(25 and on those for the use of specific methods^14),(23. The experience of foreign consultants and that of one of the authors of this article, lived in another institution²⁶ additionally contributed to the design of the current propositions and reflections.

Although the concepts and practice of PA are relatively recent^6)-(8, an integrative review of 27 quantitative and qualitative studies in courses and programs of the health area in which this modality of student assessment was implemented showed its effectiveness in promoting student learning and development, for whom decision-making on more solid bases plays an important role²⁷. However, many of the studies covered in this review point out that the introduction of the PA can generate tensions within the local educational community²⁷.

The inclusion of the PT in the evaluation program is justified by its indisputable advantages, which were recently reaffirmed in important literature reviews published in Brazil⁵ and abroad²⁸, which are consistent with the institution’s clearly positive experience, arising from its active participation, from approximately 15 years ago, in the first Brazilian consortium to apply the PT.

In particular, the systematic use of PT in medical schools can bring clear benefits to students, who can learn more and better with the feedback and become informed about their degree of cognitive mastery of the topics addressed in each exam, in comparison with both their previous results and those of their peers, in serial examinations⁵. The course management, in turn, can, based on student performance, obtain inferences about the effectiveness of the curriculum in promoting learning, thus detecting its strengths and vulnerabilities²⁸.

In the context of wide-ranging changes in student assessment practices, which are intended with the introduction of a PA-based program in which PT plays a relevant role, it is important to highlight that, in addition to providing relevant information for curriculum management and for the student, who can identify areas in which self-directed investment in study and learning is greater (formative assessment), regular participation in successive PT applications can have other educational impacts. Among them is the possibility of helping students abandon the habit of only studying for tests and then forgetting what they learned (“cramming-and-dumping”)^29),(30, as well as facing the assessment of knowledge with a lower level of stress³⁰.

Another likely contribution of the inclusion of PT in the institutional student assessment program is the encouragement of deeper and more meaningful learning, resulting from the use of contextualized questions in concrete situations and problems, which discourages the study focused on memorization and superficial learning^5),(28. In this assessment program, students should be constantly encouraged to reflect on their performance and, together with their mentor, establish maintenance or improvement strategies.

As for the management, it is important to consider that records of PT participation and performance have shown to be good indicators of student performance in future exams of the “high stakes” type^5),(28),(31.

The exclusively summative use of PT has limitations previously mentioned in the literature⁵ and, additionally, it seems to be less effective for this purpose, resulting in lower scores and lower student engagement, as shown by a study specifically designed to evaluate the inclusion of PT in an assessment program organized along the lines of the PA³². In this study, two medical courses were compared, with the exclusively summative use of the test in one of them and with the inclusion of PT in the PA, together with other measures, similar to what is proposed herein, in the other course³². Thus, even with the summative use of the test, it will be important to take measures that encourage students to reflect on their results and engage in the improvement of their performance. Coupling the assessment program with a mentoring schedule and encouraging students to use a reflective portfolio can contribute to overcoming this challenge.

CONCLUSION

The proposal to include the PT among the formal procedures of a student assessment program in the new curriculum of the medical course, based on the concepts and practices of PA, considers that the characteristics of this test are suitable for institutional assessment systems and programs and can facilitate its implementation. However, challenges, barriers and limitations of a practical nature are expected, especially those linked to the more traditional “assessment culture” prevalent among teachers and students, who tend to consider as relevant only summative assessments within the scope of curricular units (disciplines, modules and internships). These barriers will be overcome due to the facilitating factors present in the institution, which has been discussing the educational evaluation topic for several years, reorganizing its management instances to introduce the PA and the already consolidated experiences of participation in the PT, of the administration of institutional exams and faculty development for student assessment. Anyway, the effectiveness of the PT within the institutional PA system will depend on measures that encourage the active participation of the students, which will require recording their results and being able to reflect on their performance in the PT in order to engage in actions that encourage learning.

REFERÊNCIAS

• ¹
  van der Vleuten CPM, Verwijnen GM, Wijnen WHFW. Fifteen years of experience with progress testing in a problem-based learning curriculum. Med Teach. 1996;18(2):103-10.
• ²
  Freeman A, van der Vleuten CPM, Nouns Z, Ricketts C. Progress testing internationally. Med Teach . 2010;32(6):451-5.
• ³
  Schurwirth L, Bosman G, Henning R, Rinkel R, Wenink A. Collaboration on progress testing in medical schools in the Netherlands. Med Teach . 2010;32(6):476-9.
• ⁴
  Bicudo AM, Hamamoto Filho PT, Abbade JF, Hafner MLMB, Maffei CML. Consortia of cross-institutional progress testing for all medical schools in Brazil. Rev Bras Educ Med. 2019;43(4):151-6.
• ⁵
  Cecílio-Fernandes D, Bicudo AM, Hamamoto-Filho PT. Teste de Progresso como padrão de excelência para avaliação de conhecimento dos alunos de medicina - conceitos, história e perspectivas. Medicina (Ribeirão Preto). 2021;54(1):e-173770.
• ⁶
  Schuwirth LWT, van der Vleuten CPM. Programmatic assessment: from assessment of learning to assessment for learning. Med Teach . 2011;33(6):478-85.
• ⁷
  van der Vleuten CPM, Schuwirth LW, Driessen EW, Dijkstra J, Tigelaar D, Baartman LK, et al. A model for programmatic assessment fit for purpose. Med Teach . 2012;34 (3): 205-14.
• ⁸
  Troncon LEA. Estruturação de sistemas para avaliação programática do estudante de Medicina. Rev Bras Educ Med . 2016;40(1):30-42.
• ⁹
  Arnold L, Willoughby TL. The quarterly profile examination. Acad Med. 1990;65(8): 515-6.
• ¹⁰
  Oliveira SS de, Postal EA, Afonso DH, Merss CE, Cyrino EG, Abreu Júnior A, et al. Brazilian assessment with ABEM 2021 National Progress Test. Oral presentation at the Ottawa Conference, Lyon, France; 2022.
• ¹¹
  Oliveira SS de, Postal EA, Afonso DH, Merss CE, Cyrino EG, de Abreu Junior AF, et al. Teste de Progresso da Abem: consolidando uma estratégia de avaliação para o ensino médico. Rev Bras Educ Med . 2022;46(1):e061.
• ¹²
  Palhares Neto AA, Batista NA, Diniz RVZ, Oliveira SS de, Bollela VR. Teste de Progresso: avanços e perspectivas. Rev Bras Educ Med . 2022;46(supl 1): e160. doi: https://doi.org/10.1590/1981-5271v46.supl.1
  » https://doi.org/10.1590/1981-5271v46.supl.1
• ¹³
  Green DJ, Heales CJ. Progress testing: an educational perspective exploring the rationale for progress testing and its introduction into a diagnostic radiography curriculum. J Med Imaging Radiat Sci. 2023;54:35-42.
• ¹⁴
  Norcini J, McKinley DW. Assessment methods in medical education. Teach Teach Educ. 2007;23:239-50.
• ¹⁵
  Rademakers J, Ten Cate T, Bär P. Progress testing with short answer questions. Med Teach . 2005;27(7):578-82.
• ¹⁶
  Wrigley W, van de Vleuten CP, Freeman A, Muijtjens A. A systemic framework for the progress test: strengths, constraints and issues. Med Teach . 2012;34(9):683-97.
• ¹⁷
  Albanese M, Case S. Progress testing: critical analysis and suggested practices. Adv in Health Sci Educ. 2016;21(1):221-34.
• ¹⁸
  Bollela VR, Borges MC, Troncon LEA. Avaliação somativa de habilidades cognitivas: experiência envolvendo boas práticas para a elaboração de testes de múltipla escolha e a composição de exames. Rev Bras Educ Med . 2018;42(4):74-85.
• ¹⁹
  Feliciano CS, Elias LLK, Osako MK, Guimarães FS, Troncon LEA, Bollela VR. Oficina para elaboração de testes de múltipla escolha de ciências básicas aplicadas: relato de experiência. Rev Bras Educ Med . 2023; 47 (2): e067 .
• ²⁰
  Veiga IPA. Projeto político-pedagógico da escola: uma construção coletiva. In: Veiga IPA, organizadora. Projeto político-pedagógico da escola: uma construção possível. 17a ed. Campinas: Papirus; 2004. p. 11-35.
• ²¹
  van der Vleuten CPM, Schuwirth LWT, Driessen EW, Govaerts MJB, Heeneman S. 12 tips for programmatic assessment. Med Teach . 2015:37(7):641-6.
• ²²
  Wass V, van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet. 2001;357(9260):945-9.
• ²³
  Norcini J, Anderson B, Bollela V, Burch V, Costa MJ, Duvivier R, et al. Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach . 2011;33(3):206-14.
• ²⁴
  Norcini J, Brownell-Anderson M, Bollela V, Burch V, Costa MJ, Duvivier R, et al. 2018 consensus framework for good assessment. Med Teach . 2018,40(11):1102-9.
• ²⁵
  World Federation for Medical Education. Basic Medical Education WFME Global Standards for Quality Improvement. The 2020 Revision. Assessment. WFME; 2020 [acesso em 27 de outubro de 2022]. Disponível em: Disponível em: http://www.wfme.org
  » http://www.wfme.org
• ²⁶
  Troncon LEA. Efetividade da avaliação programática do estudante de medicina: estudo de caso baseado nas impressões de estudantes e professores de uma escola médica britânica. Rev Bras Educ Med . 2018;42(3):153-61.
• ²⁷
  Schut S, Maggio LA, Heeneman S, van Tartwijk J, van der Vleuten C, Driessen E. Where the rubber meets the road: an integrative review of programmatic assessment in health care professions education. Perspect Med Educ 2021;10:6-13.
• ²⁸
  Neeley S. The value of progress testing in undergraduate medical education: a systematic review of the literature. Med Sci Educ. 2016;26:617-22.
• ²⁹
  Coombes L, Ricketts C, Freeman A, Stratford J. Beyond assessment: feedback for individuals and institutions based on the progress test. Med Teach . 2010;32(6):486-90.
• ³⁰
  Chen Y, Henning M, Yielder J, Jones R, Wearn A, Weller J. Progress testing in the medical curriculum: students’ approaches to learning and perceived stress. BMC Med Educ. 2015;15:e147.
• ³¹
  Hamamoto-Filho PT, Lourenção PLTA, Valle AP do, Abbade JF, Bicudo AM. The correlation between students’ Progress Testing scores and their performance in a residency selection process. Med Sci Educ . 2019;29:1071-5.
• ³²
  Heeneman S, Schut S, Donkers J, van der Vleuten C, Muijtjens A. Embedding of the progress test in an assessment program designed according to the principles of programmatic assessment. Med Teach . 2017;39(1):44-52.

• 6
  Evaluated by double blind review process.

• SOURCES OF FUNDING:

  FAEPA - Fundação de Apoio ao Ensino, Pesquisa e Assistência do Hospital das Clínicas da Faculdade de Medicina de Ribeirão Preto da Universidade de São Paulo.

Chief Editor: Rosiane Viana Zuza Diniz. Associate Editor: Maria Helena Senger.

Publication Dates

• Publication in this collection
  17 July 2023
• Date of issue
  2023

History

• Received
  08 Dec 2022
• Accepted
  31 Mar 2023