Perception and performance of nursing undergraduates in evaluation of active methodologies

Abstract

Objective:

To identify the performance and perception of nursing students on the progression test, and to verify the existence of a relationship between performance and the current series, and between the performance and the degree of difficulty of the test.

Methods:

This was a descriptive study with a quantitative approach. An instrument was administered with socio-demographic information, classification of the degree of the test difficulty, and two questions about the advantages and disadvantages of the progression test.

Results:

The population consisted of 78 students. The mean age was 24.4 years, with a predominance of females (89.7%). The students, who classified the test as easy, showed higher performance (p = 0.036), the second-year students stood out in relation to the first-one (p = 0.014). The advantage stated was, test content 32.0%, progression and performance 25.7%, multiple choice questions 23.1%, and preparation for competitive entrance examinations 7.7%. The disadvantages were: 39.7% were very extensive, 26.9% lack of knowledge, 15.3% stated inadequate thematic content and structure, and insufficient time to complete the test for 6.4%.

Conclusion:

The progression test evaluates, longitudinally, the student's performance, identifies curricular strengths and weaknesses, and evaluates the institution. As one of the assessment instruments in use of active teaching methodologies, its applicability should be encouraged in undergraduate courses.

Keywords
Education, nursing, baccalaureate; Educational measurement; Problem-based learning; Competency-based education; Educational technology

Resumo

Objetivo:

Conhecer o desempenho e a percepção dos estudantes de enfermagem na prova de progressão e verificar a existência de relação entre o desempenho e a série em curso e entre o desempenho e o grau de dificuldade da prova.

Métodos:

Trata-se de um estudo descritivo de abordagem quantitativa, utilizou-se um instrumento constituído por questões sócio demográficas, classificação do grau de dificuldade da prova e duas perguntas sobre vantagens e desvantagens da prova de progressão.

Resultados:

A população constituiu-se de 78 estudantes. Apresentaram idade média de 24,4 anos e predomínio do sexo feminino, 89,7%. Os estudantes que consideraram a prova fácil tiveram desempenho maior (p=0.036), os estudantes do segundo ano destacaram-se em relação ao primeiro (p=0,014). Consideraram vantagem: conteúdo da prova 32,0%, progressão e desempenho 25,7%, questões de múltipla escolha 23,1%, e preparo para concursos 7,7%. As desvantagens, 39,7% prova muito extensa, 26,9% ausência de conhecimento, 15,3% temática e estrutura inadequadas e 6,4%pouco tempo para realização da prova.

Conclusão:

A Prova de Progressão avalia o desempenho do estudante longitudinalmente, identifica potencialidades e fragilidades curriculares, além de avaliar a instituição. Compreende um dos instrumentos avaliativos no uso de metodologias ativas de ensino e deve ser incentivada quanto à sua aplicabilidade nos cursos de graduação.

Descritores
Bacharelado em enfermagem; Avaliação educacional; Aprendizagem baseada em problemas; Educação baseada em competências; Tecnologia educacional

Resumen

Objetivo:

Conocer desempeño y percepción de estudiantes de enfermería en la prueba de progresión, y verificar existencia de relación entre desempeño y la serie en curso, y entre desempeño y grado de dificultad de la prueba.

Métodos:

Estudio descriptivo, de abordaje cuantitativo, utilizando instrumento constituido por cuestiones sociodemográficas, clasificación de grado de dificultad de la prueba y dos preguntas sobre ventajas y desventajas de la prueba de progresión.

Resultados:

Población constituida por 78 estudiantes. Su media etaria era de 24,4 años, con predominio de sexo femenino (89,7%). Los estudiantes que consideraron la prueba como fácil tuvieron mejor desempeño (p=0,036), los estudiantes de segundo año se destacaron respecto de los de primero (p=0,014). Mencionaron como ventajas: contenido de la prueba (32,0%), progresión y desempeño (25,7%), preguntas de elección múltiple (23,1%) y preparación para concursos (7,7%). Las desventajas, prueba muy extensa (39,7%), falta de conocimientos (26,9%), temática y estructura inadecuadas (15,3%) y tiempo insuficiente para realizar la prueba (6,4%).

Conclusión:

La Prueba de Progresión evalúa el desempeño del estudiante longitudinalmente, identifica potencialidades y debilidades curriculares, además de evaluar la institución. Constituye uno de los elementos evaluativos para uso de metodologías activas de enseñanza, debiendo incentivarse su aplicabilidad en los cursos de grado.

Descriptores
Bachillerato en enfermería; Evaluación educacional; Aprendizaje basado en problemas; Educación basada en competencias; Tecnología educacional

Introduction

The contemporary world demands that educational institutions present quality results, and document efficiency and effectiveness in the process of their students' education. Thus, a more reflective education is required, that seeks answers to health population challenges. In addition to the technical competences, the national guidelines of Brazilian education indicate the need for university education that meets the needs of the labor market, which requires competent professionals capable of acting with social responsibility, commitment to citizenship, and who exercise the role of promoting integral health of the human being.

The teacher can adopt different methodological strategies, with special attention to the social and political context of the students, integrated in the macro scenario. The adoption of active methodologies in higher education contributes to the formation of this student's profile, especially when using problematization and Problem-Based Learning (PBL) as learning strategies.

The PBL is a pedagogical proposal, developed in the late 1960's at McMaster University (Canada) and at the University of Maastrich (The Netherlands). It is a student-centered method, aiming at autonomous and independent learning. It seeks to meet the needs of knowledge and acquisition of skills to achieve learning objectives in the most diverse situations of vocational education.

The PBL is designed to enable students to develop conceptual, procedural, and attitudinal learning and prepare them for the job market.⁽¹⁾ This method provides autonomous learners, critical students, and those with the ability to lead and work as a team, as compared to nursing students who experience a traditional educational curriculum.⁽²⁾

In addition to the knowledge learned, the PBL and problematization allow the student to connect the community and the outside world to the classroom, providing access to community resources, promoting citizenship and the social-political exercise.⁽³⁾ For cognitive-learning, knowledge retention and knowledge transfer are stimulated and developed in problem-based teaching methodologies.⁽⁴⁾

The teaching grounded on active methodologies requires the use of several methods of evaluation, such as the Progression Test (PT), also known as progressive assessment or progress test. This assessment is characterized by multiple-choice tests organized by the content of each of the areas of which the nursing curriculum consists.

This type of assessment enables the student the real knowledge of his/her performance and progression in the series of courses, by the number of correct and erroneous answers on the test. For academic management, it is a valuable diagnostic tool for learning related to the retention of knowledge on all areas of the student's education curriculum.^(5,6)

The progress test is considered a longitudinal assessment, with objective questions of the total pedagogical content, administered to all students, independently of their year.⁽⁷⁾ Wrigley et al.⁽⁸⁾ de-scribe the progression test as a test of 100 to 200 questions, varying by course and university.

The nursing education of the Pontifical Catholic University of São Paulo (PUCSP) has been in existence for 67 years, with 10 years of implementation of a curriculum based on Active Learning Methodologies, standing out as one of the pioneer schools in the country, using this methodology. An integrated curriculum is the main characteristic, and uses problematization and PBL as a learning strategy. It is based on the construction of cognitive, affective, and behavioral knowledge of undergraduate nursing students.

The process of evaluating undergraduate nursing students is quite complex, and involves continuous teacher and student learning. As in traditional methods, in PBL and in problematization, evaluating is one of the major problems of curricular implementation and maintenance. The literature indicates concerns and explanations regarding the procedures and characteristics of that methodology; few studies analyze and indicate the trajectories of evaluative processes.

Therefore, we understand that the pedagogical dimensions of evaluating should be understood as part of the process of teaching-learning, in a conscious and systematic manner, focused on the process of student education.

In addition to the PT, other evaluating methods are used in our university, the following are the formative ones: self-assessment, peer assessment, assessment by a tutor, portfolio, and conceptual map. The summative assessment is composed of written, practical evaluation, and the PT.

In the studied institution, the progression tests have, on average, 60 questions related to the content taught and described in the Pedagogical Project of the Course (PPC), is conducted annually in the second semester, requiring the participation of all students regularly enrolled.

Considering the complexity of the PT as a global formative assessment tool, and the scarcity of scientific literature and practical evidence, we limited this research, as a first study initiative, to the following questions: How is the undergraduate student's performance according to each collegiate year? Is there a relationship between the performance and the grades, as well as the expected evolution? How does the student assess PT as to the level of difficulty? What advantages and disadvantages do students perceive from the PT?

In this sense, this study aims to deepen the knowledge on the PT, to improve its practical application, with the objectives: to identify the performance on and perception of undergraduate nursing students in the progression test, and to verify the existence of a relationship between the performance and the year in progress and between the performance and the level of the test difficulty.

Methods

This was a descriptive study, with a quantitative approach, conducted at PUCSP.

Among the 106 students enrolled in 2015, in the four years of the University's nursing undergraduate major, 78 (73.5%) were part of this study, 21 of the first year (26.9%), 19 of the second (24.3%), 20 from the third (25.7%), and 18 from the fourth year (23.1%). The data were collected in October of 2015, by one of the researchers, after the progression test, which was pre-scheduled and published in the academic calendar.

A questionnaire was developed for data collection, comprised of: socio-demographic characterization, which corresponded to age, sex, grades in progress, paid enrollment in study/promotion programs; the student's perception of the test, how they classified the degree of difficulty of the test - very easy, easy, medium, difficult and very difficult - and their answers to two structured questions, “What are the advantages of the progression test?” and “What are the disadvantages of the progression test? “

The student performance data on the tests were taken from the University's online system, copied to a data spreadsheet from Excel for Windows (version 2013), and entered into the System for Windows, Version 9.2 (SAS) program. Descriptive statistics were used with dispersion measures to characterize the population, performance, and perception of the students on the PT.

The Pearson's Chi-Square test or Fisher's exact test was used for comparison of the categorical variables. The Mann-Whitney test was used for comparison of the numerical variables between two groups, and between three or more groups, the Kruskal-Wallis test was used, due to the absence of normal distribution of the variables. The significance level adopted for the statistical tests was p <0.05.

The characterization of the population and the classification of the test questions were treated by means of descriptive statistics, also used for the two questions about the advantages and disadvantages, in addition to the dispersion measures. The answers to each question were grouped by frequency, forming the groups described below.

First question - advantages of the progression test, four groups: 1.1 PT content - answers related to the question issue; Group 1.2 Performance and progression - characterized by students' perception of performance and evolution; 1.3 Type of test - related to multiple choice questions; 1.4 Preparation for competitive entrance examinations represented by the possible training for selective processes.

Second question - disadvantages of the progression test, three groups: 2.1 PT structure - attributes of the formulation of questions; 2.2 Duration of the PT - time dedicated to the test; 2.3 Absence of knowledge - related to the current year of progress.

At the presentation of these results, examples were inserted that discriminated each group; the answers were coded with the letter E, and numbered from 1 to 78.

The study was conducted after approval by the Research Ethics Committee of PUCSP, CAAE number 43861115.5.0000.5373, according to the provisions of Resolution No. 466/2012 and Operational Norm No. 001/2013. All participants who agreed to respond to the questionnaire signed the Terms of Free and Informed Consent Form, with the guarantee of confidentiality and anonymity of their participation.

Results

Among the 78 participating students, 70 (89.7%) were female and 8 (10.3%) were male; the ages ranged from 17 to 51 years, with a mean of 24.4 years, standard deviation 6.6.

The majority of the students, 60.3% had an employment relationship with remuneration; 70.5% had a scholarship from the programs: University for All Program (PROUNI), São, Paulo Foundation—(FUNDASP) and Student Financing Fund (FIES), and 12.8% were in research promotion programs: National Council for Scientific and Technological Development (CNPq) and the São Paulo Research Foundation (FAPESP).

The student performance related to the number of correct scores in percentile (%) in the PT is shown in table 1.

Table 1
Undergraduate student performance on the PT, per year

Year	n	Mean	SD	Minimum	Maximum	Median	p-value *
1st year	21	42.06	8.67	25.00	58.33	43.33	P=0.014 -> 1≠2
2nd year	19	50.00	5.58	43.33	65.00	50.00
3rd year	20	47.33	9.39	31.67	63.33	47.50
4th year	18	49.07	7.57	36.67	65.00	48.33

• ^*
  p-value for the Kruskal-Wallis test to compare the variables between three or more groups

The first year undergraduate students presented better performance when compared to the other grades, with statistical significance when compared to the second (p = 0.014).

Regarding the participants' perceptions of the level of difficulty of the progression test, four students (5.1%) classified it as easy, two students of the first year and two of the third; 50 (64.2%), students considered the PT as medium difficulty, 15 (7.5%) of the first year, 12 (6.0%) of the second, 12 (6.0%) of the third, and 11 (5.5%) of the fourth year; 22 students (28.2%) considered the test difficult, two (0.4%) of the first year, seven (1.5%) the second; six (1.3%) of the third, and seven (1.5%) of the fourth year; two students (2.5%) of the first year mentioned that the test was very difficult.

When comparing the number of correct answers and the classification of questions as easy, medium, difficult or very difficult, there was a statistically significant difference (p = 0.036), and the students who considered the test easy showed a better performance than those who found it to be a very difficult test; the mean of correct answers was 54.0% and 32.0%, respectively.

When comparing students' performances in the four years, statistically significant differences were found only between the first and second years (p = 0.014), such that the second year students presented a better performance, mean scores of 50.0%, while the first year had a mean of 42.1% correct answers.]

Among the participants, nine (11.6%) did not answer the questions related to the qualitative variables, the other 69 (88.4%) stated their perception on the advantages and disadvantages of the PT.

• First question - Advantages of the progression test - 69 students - 100%

According to the PT content, 25 (36.2%) students identified it as relevant and comprehensive, demonstrating the need for articulating knowledge.

• –

  “The opportunity we have to use the knowledge we learned during the past modules to answer the questions, as we use new knowledge and reinforce our learning, and can resume subjects from the first year” (E 37).

• –

  “The great majority of the content of the questions is related to the everyday content of nursing, regarding primary, secondary and tertiary care” (E 66).

• –

  “The greatest advantage is to have, in the tests, questions that are part of our curriculum” (E 74).

With regard to performance and progression, 20 (28.9%) undergraduate students showed the perspective of a progressive student, method and course evaluation.

• –

  “It enables the evaluation of student progression during the nursing education according to each year” (E 27).

• –

  Very good test for evaluation of the major, and performance of the students and their progress during the four years; also enables evaluation of the method “(E 28).

• –

  “I believe that the test itself is a facilitator, and the student can know his parameters in the four years of the nursing education, evaluating if he has progressed, regressed, or remained the same in relation to acquired knowledge” (E 69).

• –

  Regarding the type of test, 18 (26.1%) students stated the advantage of the multiple-choice test.

• –

  “For me, multiple choice questions accelerate completing the test, and we can train for future competitive entrance examinations” (E 52).

• –

  “I think that the aspect facilitates the fact that the exam is a test, because we have more time to answer the questions” (E 17).

• –

  “The fact that the test has alternative questions helps me to evaluate the options and, in case I do not know the answer, I can get closer to the answer that I think is the most correct” (E 75).

Regarding the preparation for competitive entrance examinations, six (8.7%) students stated that the PT simulates these tests, by addressing all the content of the undergraduate program.

• –

  “I think it is good to have the opportunity to do a different test than the ones we use to participate during the modules, as it includes all the content, simulating competitive examinations” (E 78).

• –

  “The test allows training for competitive examinations, checks how we find ourselves in relation to the studies and learning. I think the fourth year is the best grade for a good performance on the test, because, theoretically, students of fourth year are already prepared” (E 25).

• –

  “This test provides training for a test in future selective processes, because it helps us to remember subjects that we have already seen in previous semesters, and helps us to know what we will still see in the future” (E 26).

• –

  Second question - Disadvantages of the progression test - 69 students - 100%

Regarding the structure of the PT, they found it to be an extensive, tiring, and laborious test - 43 (62.3%) students.

• –

  “The greatest disadvantage for me, lies in the fact that the texts that compose the test are very extensive” (E 48).

• –

  “I think that very extensive questions cause a lack of concentration at the end of the test” (E 65).

• –

  “I found the questions very long and I could not focus for a long time” (E 13).

• –

  “Difficulty was found in the elaboration of the questions, because there were some extensive alternatives” (E 17).

• –

  “The formulation of some questions, which have divergences in the answers” (E 37).

• –

  “The issues are sometimes out of date, ambiguous, and the information provided is inconsistent, the issues are dull” (E 38).

In addition, five (7.3%) students considered the duration of PT, insufficient to accomplish.

• –

  “Test time is short for us to complete, and the content is very long, so it gets tiresome” (E 43).

• –

  “Short test time for long text questions” (E 44).

• –

  “I have difficulty in understanding many questions in a short time” (E 63).

As expected, 21 (30.4%) students commented on the absence of knowledge, depending on their current year.

• –

  “I believe that, due to the current year and to the fact that some questions correspond to the next semesters, I did not know how to answer. That was the greatest difficulty I have” (E 1).

• –

  “Not having enough knowledge from the future semesters to answer the test” (E 6).

• –

  “The most difficulty was to answer questions related to the next semesters, as the fact that I do not know the contents, make me unsatisfied with my result” (E 7).

• –

  “I found difficulties in issues that I had not yet known, even so, I tried to answer by deduction, which, many times, may have hindered my result” (E 9).

Discussion

The predominance of females and the mean age of the students were similar to those found in the Brazilian literature, which shows that 85.1% of the nursing professional contingent is female; 40% are ages between 36 and 50 years; 38% between 26 and 35 years, and 2% above 61 years.⁽⁹⁾

The high number of students who have remunerated employment and support programs show a part of the Brazilian reality, experienced by the working student. Knowledge of the student profile, such as age, sex and work activity, provides the development of targeted learning strategies, critical -reflexive practice activities and innovative teaching procedures that are interrelated with the socio – political, economic, and institutional reality.

The shortage of publications related to the PT performed by nursing students, the diversity of definitions, the characteristics of the test, and the particularities of the pedagogical projects of the undergraduate health majors make it difficult to compare with the literature. The studies found point to the area of medical education.⁽¹⁰⁾

The PT is an evaluation instrument with the purpose of measuring the final competences of the nursing education; the measurement must be done by means of the evaluation of the student's learning regarding the aspects essential to formation. Although it is based on memorization, the individual scores of the students obtained in the PT should be used as formative and non-summative evaluation, therefore, the evaluation should be performed through several instruments, and not in a single test.⁽¹¹⁾

Regarding the degree of the test difficulty, this questioning has not yet been satisfactorily answered by the literature, as the students' perceptions are characterized by the subjectivity of the response. However, there is a need for studies related to the perception and performance of students' successively monitored, with variable metric statistics, and analytical procedures such as Item Response Theory, so that the results can be considered a consensus.

The development of the PT should be made to enable the measurement of students' performance in a longitudinal and progressive manner.^(7,8,10)

Thus, and because of the volume and complexity of increasing content over the years, it is expected that the performance of the second year students will be better than the first year, and so on, successively, finishing in the fourth year, which should be better than all the previous ones.

In this study, this fact was only observed in the performance of the students of second year in relation to the first one, which suggests the need to deepen the investigation regarding the efficiency of PT to differentiate the progressive evolution of learning and the possibility of correct answers by chance - ‘to guess’; the distribution of the content by the questions, as mentioned by the year in progress, also needs an evaluation. In addition, we note that the number of respondents may have been insufficient for this type of statistical analyses.

Knowing the students' perceptions regarding the PT makes us reflect on the learning behaviors and the evaluation of teaching. Content, directly related to learning, has been considered to be an advantage; it refers to the idea that the student is relating the theme of the questions addressed in the test to their ability to respond to their learning.

From this perspective, studies show the students' perception about the PT is that it is a useful test, which enables an evaluation of memorization and repetition, leading to better learning, guided by feedback.^(11,12)

The PBL teaching methodology, however, is not restricted to memorization, as it provides the freedom for students to pursue their own learning, focusing on personal and individual choices and needs for study and learning.^(12–14)

Students are encouraged to study the main contents of each module that make up the curriculum, facilitating the acquisition and the retention of knowledge in the long term.

The opportunity to demonstrate progression and performance was another advantage stated by the students, which is consonant with the primary purpose of PT, although this result was not observed in the comparative performance between the grades.

The PT was considered to be useful for measuring knowledge acquired during the nursing education, based on the curricular formulation,⁽¹⁵⁾ and establishes an important relationship between the student's level of knowledge and his performance on this evaluative model.⁽¹⁶⁾

Another study⁽¹⁷⁾ on academic perception for the PT showed that the test evaluates academic learning, which is a fair and valid test, and students would like to have more time for the tests. Feedback has led these students to devote more time for this type of assessment.

The importance of cognitive development in the education of health professionals is considered a continuous process of acquiring and consolidating a set of components necessary for the knowledge domain in one or more areas of performance. It demonstrates, in addition to cognitive growth, a greater participation of students in the tests.⁽⁷⁾

The PT is considered an orientation instrument for studies, especially when related to content evaluation and retention of this knowledge in the long term.⁽¹³⁾ The students perceived the test as a kind of training for future tests and competitive examinations, mainly because this test consists of multiple choice questions. All the written evaluations that compose the PPC of the PUCSP consist of-ended open questions; the only exception is the PT. As found in this study, we observed that in the area of medicine, the PT is considered to be a “training” so that students, after completing their undergraduate course, were better prepared to take the medical qualification tests.⁽¹⁸⁾

The effective development of these questions, however, is so laborious that a software was developed⁽¹⁹⁾ able to measure cognitive knowledge in tests with multiple choice questions.

With regard to the content and form of the structure of the questions, the development of the tests is initially done by professors in Brazilian universities, and later these questions are grouped and revised by a commission indicated for this purpose.^(20,21)

In comparison to international studies, this is quite different, and discussed. The tests are written either by a database (computer program), such as in MacMaster,⁽²²⁾ which has a database of questions with 3500 items selected and reviewed by a responsible teacher. After the test, students evaluate each item of the test, which is then maintained in or removed from the question bank.

As the Brazilian experiences in Maastricht,⁽²³⁾ the process of creating the tests is laborious and involves a large number of teachers who write wording and alternatives, review, and approve the final organization of the tests.

In addition, it is essential to verify evidence of validity to measure students' performance on certain knowledge.^(24,25)

The structuring of assertions and alternatives in multiple-choice tests for use in the PT is also an important aspect for reliability of the results. In this sense, the assertions of the tests should not only require the student to memorize, but mainly to address higher taxonomic categories, such as synthesis or evaluation.⁽⁵⁾

The PT is designed to test final skills, addressed by complex exercises that demonstrate problem-solving skills. Such items may be difficult to be developed, and require teachers' training and skills as well as a good understanding of the objectives.⁽¹¹⁾ This analysis has not yet been performed in the PT of the studied university.

The characterization of the test as extensive and with insufficient time for its completion may be associated to the perception of it being exhausting.

In spite of this, the analyzed test had 60 questions, a number below the recommendations of international studies. MacMaster University recommends the application of 180 multiple-choice questions. The University of Missouri - Kansas recommends the application of 400 items, and the University of Maastricht the use of 250 questions of true or false type.^(11,22,23) Brazilian studies recommend PT with multiple choice structure, varying from 120 to 150 questions.^(20,21)

Proposing practical suggestions for the application of PT, some authors describe that, although the PTs have the purpose of evaluating the general curricular competencies, one of the disadvantages is related to the development, administration, correction and feedback, a high-cost evaluation process for educational institutions.⁽¹¹⁾

As for the time, in PUCSP, the students can take the test over a period of up to four hours, and cannot leave before the first hour. On average, it takes four minutes to complete each question, which is opposed to the MacMaster and Missouri Universities, where the student has, on average, one minute to solve each question. In only one Brazilian university, there is a time record with an average of 80 seconds.^(20,21) The minimum time to answer the PT questions is approximately 75 - 85 seconds.⁽⁸⁾

The lack of knowledge and the inadequate structure and thematic content may be related to the fact that they did not know all the content to answer the test, as only a quarter of the students were in the fourth year, and therefore only that portion of students had had all the content of the nursing program.

A disadvantage reported by the students was related to the development of the questions, because they have characteristics of assertions with extensive alternatives, questions with divergences in the answers, and had ambiguous information provided that was inconsistent and exhausting. These aspects are not mentioned in the literature.

The application of PT as an evaluation process of knowledge is integrated in the methodology of teaching PBL. The construction of the knowledge and learning advocated in this methodology is based on previous knowledge and experience, considering the importance of common sense questioned by the science in a constant dialogue in the transformation of learning. The PBL methodology is supported by meaningful learning, in which all knowledge requires the interrelation of the students' prior knowledge to be meaningful. Reflective learning considers the importance of cognitive responses, however, thoughts and feelings should be revealed because they are involved in the learning process.⁽²⁶⁾ Participants in this study have realized that they depend on relevant concepts to respond to the PT questions.

We can understand these findings, as well, through the cultural dimension and understand that “our students” need to get accustomed to this type of assessment. In international studies, for example, the application of PT in Maastricht and the University of Missouri Kansas City (UMKC) occurs four times a year, and at MacMaster and Utrecht it occurs three times annually.^(11,27)

Studies consider that the PT provides stability in the evaluation procedures, helps with changes and curricular adjustments, promotes content change, and must be analyzed under these dimensions, both by students and teachers, not only aiming at performance and progression.⁽⁸⁾

Given the favorable aspects and problems identified, we believe that the formation of a teaching committee for the elaboration, administration, and improvement of the PT, and the feedback to the students, can contribute as an effective academic management tool in the evaluation processes of the education. The development of the PT in an objective way, interrelated with the curricular matrices, can provide the reduction of institutional costs and greater credibility of the PT as an evaluation tool.

Conclusion

As recommended by the literature, the PT provided the student with reflections on his performance and progression in the years. Although problems were pointed out, we affirm its importance as a tool for diagnostic and formative evaluation in the student education. The students' statements clarify formative aspects of the PT, showing perceptions on the advantages and disadvantages from the optics of students. It is necessary to articulate knowledge of the questions' content, as well as the opportunity to test the overall contents of the nursing program. These aspects, in turn, support the teacher in the planning and decision-making process of teaching, learning, and evaluation. Problematization and the PBL, with the support of constructivism, refute learning restricted to the transmission and memorization of knowledge, and in this way, it is essential to deepen the investigation regarding the TP so that, in fact, it fulfills its evaluation role in the construction of a critical and active professional. To gain insight into this process, evidence of in-depth research is needed and should focus on the needs analysis, learning monitoring, and assessment goals. We assume, as limiting factors of this research, the regionalization and small size of the population that was not stratified and, therefore, may have skewed the results, in addition to preventing the performance of more refined statistical tests, which impair the generalization of the findings.

Despite these limitations, we consider as contributions of the study the identification of important aspects to be investigated, not only for the university field of study, but also for higher education institutions that already use it to review their practice, and for those that intend to implement the PT as a formative and longitudinal evaluation tool. Moreover, the dissemination of an analysis of the PT, although incipient, is essential, as no studies are available in the nursing literature, and all the studies cited derive from the medical area. Finally, we emphasize that this study motivated the researchers to deepen the research, resulting in a doctoral thesis.

Acknowledgements

Isabel Cristina Campos Feitosa. Librarian of the Faculty of Health and Medical Sciences (FCMS), Pontifical Catholic University of São Paulo (PUCSP) Campus Sorocaba.

Referências

• ¹
  Borochovicius E, Tortella JC. Aprendizagem Baseada em Problemas: um método de ensino-aprendizagem e suas práticas educativas. Ensaio: Aval Pol Públ Educ. 2014; 22(83):263–94.
• ²
  Kong L-N, Qin B, Zhou Y, Mou S, Gao H-M. The effectiveness of problem-based learning on development of nursing students' critical thinking: a systematic review and meta-analysis. Int J Nurs Stud. 2014; 51(3):458–69.
• ³
  Sousa SO. Aprendizagem baseada em problemas como estratégia para promover a inserção transformadora na sociedade. Acta Scientiarum Educ. 2010; 32(2):237–45.
• ⁴
  Yew EH, Goh K. Problem-based learning: an overview of its process and impact on learning. Health Prof Educ. 2016; 2(2):1–5.
• ⁵
  Pinheiro OL, Spadella MA, Moreira HM, Ribeiro ZM, Guimarães AP, Almeida Filho OM, et al. Teste de progresso: uma ferramenta avaliativa para a gestão acadêmica. Rev Bras Educ Med. 2015; 39(1):68–78.
• ⁶
  Finucane P, Flannery D, Keane D, Norman G. Cross-institutional progress testing: feasibility and value to a new medical school. Med Educ. 2010; 44(2):184–6.
• ⁷
  Sakai MH, Ferreira Filho OF, Matsuo T. Avaliação do crescimento cognitivo do estudante de medicina: aplicação do teste de equalização no teste de progresso. Rev Bras Educ Med. 2011; 35(4):493–501.
• ⁸
  Wrigley W, Van Der Vleuten CP, Freeman A, Muijtjens A. A systemic framework for the progress test: strengths, constraints and issues: AMEE Guide No. 71. Med Teach. 2012; 34(9):683–97.
• ⁹
  Machado MH, Aguiar Filho W, Lacerda WF, Oliveira E, Lemos W, Wermelinger M, et al. Características gerais da enfermagem: o perfil sócio demográfico. Enferm Foco. 2016; 7(esp 686.):9–14.
• ¹⁰
  Neeley SM, Ulman CA, Sydelko BS, Borges NJ. The value of progress testing in undergraduate medical education: a systematic review of the literature. Med Sci Educ. 2016; 26(4):617–22.
• ¹¹
  Albanese M, Case SM. Progress testing: critical analysis and suggested practices. Adv Health Sci Educ Theory Pract. 2015; 21(1):221–34.
• ¹²
  Matsuyama Y, Muijtjens AMM, Kikukawa M, Stalmeijer R, Murakami R, Ishikawa S, et al. A first report of East Asian students' perception of progress testing: a focus group study. BMC Med Educ. 2016; 16(1):245.
• ¹³
  Heijne-Penninga M, Kuks JBM, Hofman WHA, Muijtjens AMM, Cohen-Schotanus J. Influence of PBL with open-book tests on knowledge retention measured with progress tests. Adv Health Sci Educ Theory Pract. 2013; 18(3):485–95.
• ¹⁴
  Cónsul-Giribet M, Medina-Moya JL. Strengths and weaknesses of problem based learning from the professional perspective of registered nurses. Rev Lat Am Enfermagem. 2014; 22(5):724–30.
• ¹⁵
  Muijtjens AMM, Schuwirth LWT, Cohen-Schotanus J, Van Der Vleuten CPM. Differences in knowledge development exposed by multi-curricular progress test data. Adv Health Sci Educ Theory Pract. 2008; 13(5):593–605.
• ¹⁶
  Verhoeven BH, Verwijnen GM, Scherpbier AJJ, Holdrinet RSG, Oeseburg B, Bulte JA, et al. An analysis of progress test results of PBL and nonPBL students. Med Teach. 1998; 20(4):310–6.
• ¹⁷
  Wade L, Harrison C, Hollands J, Mattick K, Ricketts C, Wass V. Student perceptions of the progress test in two settings and the implications for test deployment. Adv Health Sci Educ Theory Pract. 2012; 17(4):573– 83.
• ¹⁸
  Norman G, Neville A, Blake JM, Mueller B. Assessment steers learning down the right road: impact of progress testing on licensing examination performance. Med Teach. 2010; 32:496–9.
• ¹⁹
  Pugh D, Bhanji F, Cole G, Dupre J, Hatala R, Humphrey-Murto S, et al. Do OSCE progress test scores predict performance in a national high-stakes examination? Med Educ. 2016; 50(3):351–8.
• ²⁰
  Sakai MH, Ferreira Filho OF, Almeida MJ, Mashima DA, Marchese MC. Teste de progresso e avaliação do curso: dez anos de experiência da medicina da Universidade Estadual de Londrina. Rev Bras Educ Med. 2008; 32(2):254–63.
• ²¹
  Tomic ER, Martins MA, Lotufo PA, Benseñor IM. Progress testing: evaluation of four years of application in the school of medicine, University of São Paulo. Clinics (Sao Paulo). 2005; 60(5): 389–96.
• ²²
  Blake JM, Norman GR, Keane DR, Mueller CB, Cunnington J, Didyk N. Introducing progress testing in McMaster University's problem-based medical curriculum: psychometric properties and effect on learning. Acad Med. 1996; 71(9):1002–7.
• ²³
  Van der Vleuten CPM, Verwijnen WH, Wijnen WHFW, Verwijnen GM, Wijnen WHFW. Fifteen years of experience with progress testing in a problem-based learning curriculum. Med Teach. 1996; 18(2):103–9.
• ²⁴
  Findyartini A, Werdhani RA, Iryani D, Rini EA, Kusumawati R, Poncorini E, et al. Collaborative Progress Test (cPT) in three medical schools in Indonesia: the validity, reliability and its use as a curriculum evaluation tool. Med Teach. 2015; 37(4):366–73.
• ²⁵
  De Rijdt C, Stes A, van der Vleuten C, Dochy F. Influencing variables and moderators of transfer of learning to the workplace within the area of staff development in higher education: research review. Rev Educ Res. 2013; 8:48–74.
• ²⁶
  Campos LR, Ribeiro MR, Depes VB. Autonomia do graduando em enfermagem na (re)construção do conhecimento mediado pela aprendizagem baseada em problemas. Rev Bras Enferm. 2014; 67(5):818–24.
• ²⁷
  Freeman A, Van Der Vleuten C, Nouns Z, Ricketts C. Progress testing internationally. Med Teach. 2010; 32(6):451–5.

Publication Dates

• Publication in this collection
  Jan-Feb 2018

History

• Received
  07 Nov 2017
• Accepted
  15 Feb 2018