College Program evaluation according to graduates

Abstracts

O egresso da graduação enfrenta, no seu cotidiano, situações complexas que o levam a confrontar as competências desenvolvidas durante o curso com as requeridas no exercício profissional. Tal vivência permite-lhe avaliar a adequação da estrutura pedagógica do curso e resgatar aspectos intervenientes desse processo. Este texto aborda a Avaliação de Cursos de Graduação, resgatando conceitos no campo da Avaliação Educacional; descreve as práticas desenvolvidas no Brasil dirigidas à Avaliação Institucional no âmbito da graduação e apresenta pesquisas relativas à avaliação de cursos, tendo a perspectiva do egresso como indicador. Foram utilizadas as seguintes palavras-chave: egressos, indicadores, avaliação e graduação, disponíveis nas bases de dados: PeriEnf, BDEnf, LILACS e no banco de dados DEDALUS. A análise das pesquisas encontradas ressalta a importância de se considerar a opinião dos egressos na adequação de propostas curriculares com vistas à concretização de um processo educacional com características emancipatórias e transformadoras.

Educação em enfermagem; Estudantes de enfermagem; Avaliação

CRITICAL REVIEW

College Program evaluation according to graduates

Evaluación del Curso de Graduación según los egresados

Maria Dyrce Dias Meira^I; Paulina Kurcgant^II

^INursing doctorate graduate at School of Nursing at University of São Paulo. Professor of the Nursing Course at Centro Universitário Adventista de São Paulo. São Paulo, SP, Brazil. [e-mail]

^IIDoctor Professor. Professor of the Nursing Graduate Program of the School of Nursing at University of São Paulo. São Paulo, SP, Brazil. [e-mail]

Correspondence addressed to

ABSTRACT

Every day, college graduates face complex situations that make them check skills developed during the course in comparison with the skills needed at the professional level. This experience helps them evaluate the adequacy of the course's teaching structure and recover intervening aspects of this process. This text discusses the Evaluation of College Courses, recovering concepts in the field of Education Evaluation; describes the practices developed in Brazil, directed towards course evaluation, based on the graduate's perspective. The following descriptors were used: college graduates, indicators, evaluation and graduation available in the databases: PeriEnf, BDEnf, LILACS and DEDALUS. The analysis of the surveyed studies highlights the need to consider the opinion of college graduates in the curriculum proposals with a view to implementing an educational process with emancipating and transforming characteristics.

Key words: Education, nursing. Students, nursing. Evaluation.

RESUMEN

El egresado de los cursos universitarios de enfermería enfrenta, en lo cotidiano, situaciones complejas que lo llevan a confrontar las competencias obtenidas durante el curso con las requeridas en el ejercicio profesional. Esta vivencia le permite evaluar la adecuación de la estructura pedagógica del curso y rescatar aspectos que intervienen, en ese proceso. Este texto aborda la Evaluación de Cursos Universitarios rescatando conceptos en el campo de la Evaluación Educacional; describe las prácticas desarrolladas en Brasil, dirigidas a Evaluar la Institución en el ámbito de la formación y presenta investigaciones relacionadas a la evaluación de cursos teniendo, como indicador, la perspectiva del egresado. Fueron utilizadas las palabras claves: egresados, indicadores, evaluación y graduación, disponibles en las bases de datos: PeriEnf, BDEnf, LILACS y en el banco de datos DEDALUS. el análisis de las investigaciones encontradas destaca la importancia de considerar la opinión de los egresados en la adecuación de propuestas curriculares con el objetivo de concretizar un proceso educacional con características emancipadoras y transformadoras.

Descriptores: Educación en enfermería. Estudiantes de enfermería. Evaluación.

INTRODUCTION

The educational evaluation is currently an expanding field in Brazil, considering the growing interest in systematizing evaluation as a means to monitor and implement policies for higher education. It is also considered to be the practice that allows for the transparence of institutional achievements, both for the university community and for society. Furthermore, an evaluation proposal must be related to the object to which it aims; the evaluation takes place depending on the values and interests of the subjects involved, and

it must enable the academic community to analyze the contradictory processes that occur during academic life and try to give some dynamics to the evaluation process, which covers a complete view of the educational problem⁽¹⁾.

Among the different possibilities for evaluating graduate courses, there is the alternative of verifying the opinion of graduates who are already performing professional activities. This is one of the dimensions that allow one to see the transformations that happen in the students due to the influence exercised by the curriculum.

Graduates face complex situations in their daily work, which lead them to compare the competencies developed during their coursework with those required for the professional activity they are pursuing. It is then possible to evaluate the accuracy of the pedagogic structure of the experienced course, as well as the intervening aspects in the academic development process.

Therefore this text aims: to present a synthesis of the concepts approached in the educational evaluation field ⁽²⁾; to describe the practices developed in Brazil for institutional evaluation in the scope of graduate courses; and to present studies related to the evaluation of courses, using the perspectives of graduates, and attempting to gain an understanding of the aspects they consider important for their academic process as an indicator.

The bibliographical review was the method used, and since the standard descriptors (Evaluation, Nursing Education, Nursing Students) restricted the search results, the following key-words have been used: graduates, indicators, evaluation and graduation, in the following databases: PeriEnf, BDEnf, LILACS and DEDALUS. Since the studies found regarding evaluation from the perspective of nursing graduates were too few to subsidize the analysis, studies that examined other graduate areas were presented.

HISTORIC CONCEPTUALIZATION

The educational evaluation field has been the object of study by several authors, who have been searching for new conceptualizations of evaluation practices that are consistent with the pedagogical conceptions present since the 1950's. Tyler was one of the first theoreticians to try to develop this field, contributing with studies about evaluation by purpose, and with the conceptual differentiation of evaluation and measurement. His theory was based on the concept that educating consists of changing behavior patterns. Tyler assigned the responsibility of verifying accuracy, through evaluation, to the school. He also considered that the evaluation provides fertile ground for a critical analysis, which allows the reformulation and improvement of the curricular program. Tyler was concerned with the judgment of the curriculum efficiency, identifying the evaluation as an investigation process of values that aimed to improve the educational system⁽²⁾.

In the conception of Cronbach, the purpose of the evaluation is not only to make a final judgment, but to offer means for the improvement of the curriculum. He presents four aspects considered important: the association between evaluation and the decision making process; the different roles of the educational evaluation; the student's performance as a criterion of course evaluation and measurement techniques available to the educational evaluator⁽²⁾.

Stake defends the idea that the evaluation shall not have a merely episodic character, since it must be traced to the comprehension of the whole process concerning the educational program, and be based on the collection of data established in reality. His ideas originated the responsive evaluation and highlight the formal and informal aspects of the evaluation, bringing an important contribution to the case study methodology by considering that it must be based on a reflexive and interpretative observation⁽²⁾.

According to Scriven, the educational evaluation plays several roles, but has the sole purpose of determining the value or merit of what is being evaluated. The purpose is the response to the problems and the roles are the several possibilities for using this response. For this author, there is no difference between formative and summative evaluation, because both aim to determine the value of a particular entity. He also stresses that the difference between them lies in the timing in which they are performed, in their target and in the use of their results. Everything depends on the perspective of those who make the analysis⁽²⁾.

Stufflebeam, as of the 1970's, defines evaluation as a process to describe, obtain and process useful information to judge alternative decisions. He highlights three essential aspects for understanding his model: the evaluation as a systematic and continuous process; the evaluation as a basis to create questions and search for information that answers them in a relevant way, and, finally, he proposes that the evaluation must favor the decision making of the administrators. Stufflebeam presents the model CIPP (Context, Input, Process and Product), whose central point is to improve, instead of approving⁽²⁾.

These concepts were integrated into the evaluative processes and had an influence on the implementation of educational evaluation systems applied in Brazil over the years, aimed at higher education.

THE INSTITUTIONAL EVALUATION IN HIGHER EDUCATION IN BRAZIL

Considering that the evaluation must diagnose critical aspects of the educational program to allow the necessary corrections to be made to the educative proposal, some initiatives in the evaluation field of higher education in Brazil are important, as follows.

In the 1990's, the Institutional Support Program of Brazilian Universities (PAIUB - Programa de Apoio Institucional das Universidades Brasileiras) was created. It conceived the evaluation as a continuous process of academic improvement for the university administration planning process, for rendering an account to society and as a creative self-criticism process^(3).

The PAIUB, despite being highly influenced by international models, projected the evaluation into a rather progressist and emancipatory perspective, suitable to the Brazilian educational system, aimed at improving the administrative, scientific and pedagogical quality, without giving up the autonomy. Practiced from the social reality, this proposal would allow the visualization of the peculiarities of each institution, and could be considered a reference for an evaluation policy of Institutions of Higher Education (IHE).

After the publication of the law no. 9.131/95, which broaches the creation of the National Committee of Education and institutes the National Exam of Courses and, mainly, of the law no. 9394/96, which establishes the guidelines and basis of the National Education⁽⁴⁾, the institutional evaluation became a priority for the Culture and Education Department (MEC - Ministério da Educação e Cultura). MEC then assumed a controlling role, with the State on one side and the educational institutions on the other, having the search for quality as their common meeting ground. Therefore, the role of the schools was to carry out the social and scientific mission, promote self-knowledge and decision making, whereas the role of the State was to guarantee that the guidelines, legal requisites, goals, ranking and accreditation were observed.

The evaluation, according to this proposal, should happen on four levels: in the global performance of the educational system, using indicators by region according to the knowledge areas and the type of institution; in the individual performance of the IHE at all levels and categories of education, research and continuing education; in the evaluation of the courses through structural conditions and results of the National Exam of Courses; and in the post-graduation evaluation by knowledge area⁽⁴⁾. Some authors state that the application of these exams reduced the evaluative activity to a measurement procedure for the quantification of results, with criticism arising towards this evaluation system, citing, for instance the encouragement of competition caused by the establishment of a ranking of institutions and the use of exam results as a criterion for the approval of financial resources⁽⁵⁾.

In opposition to this evaluative model, Dias Sobrinho suggested an educative evaluation that should be informative, democratic, participative, global and continuous, with a qualitative emphasis, and that should have: flexibility, credibility and legitimacy, institutionalism and continuity. He believes the evaluation should encompass the national and international levels, and should be considered in three essential dimensions: self-evaluation, external evaluation with re-evaluation and target evaluation⁽⁶⁾. This author's contributions have been essential for leading the proposals toward a new evaluation system.

As of 2004, the MEC established the National System of Higher Education Evaluation (SINAES - Sistema Nacional de Avaliação da Educação Superior), aimed at broadening the frontiers of the educational evaluation. Based on a predominantly formative conception⁽⁷⁾, this system undertakes , in an integrated way, the evaluation of institutions, courses and the performance of students, which allows the institution to examine its own quality and social responsibility.

The comprehension of this proposal gives a systemic character to the evaluation, by integrating spaces, moments and different evaluation instruments around a unique global conception⁽⁸⁾.

Among the ten dimensions that the SINAES evaluates, the ninth dimension refers to the service policy for students and graduates. The orientation guide for the use of the IHE self-evaluation, regarding this dimension, presents the professional insertion of graduates and their participation in the institution life as a basic and common goal. This guide instructs the IHE to observe whether there are indicators and data to evaluate this dimension, such as research or studies regarding graduates and/or their employers⁽⁹⁾.

GRADUATES AS AN INDICATOR FOR INSTITUCIONAL EVALUATION

The analyzed literature presented few references that evidenced the perception of graduates as an indicator for institutional evaluation. The performed study aims to comprehend the aspects graduates considered important during their education process. The results obtained by schools that have undertaken research in this area will be presented as follows.

The Federal University of Ceará developed research with students who graduated in 2003 and 2004, aimed at analyzing the treatment towards undergraduate students and graduates, and at examining the contribution of the course curriculum to the fulfillment of their professional needs. The study involved the participation of one hundred and one (101) subjects from several areas, including nursing. A structured questionnaire was given, which allowed the research to state that 72% of the graduates were working in their study area and 88% needed more knowledge to enrich their education. As for the contribution of the course curriculum to the fulfillment of their professional needs, 60.4% stated they considered it insufficient in the exercise of their profession⁽¹⁰⁾.

Another research study regarding the Institutional Evaluation practice, observing the 9^th dimension of SINAES, was undertaken by the Federal Center of Technological Education of Bahia. It studied 17 graduates from Administration and Engineering courses, and the applied instrument searched for the representation of the graduates in a broadening way. Results were presented through tables and graphs, with percentage analyses of the main indicators, such as: satisfaction level of graduates regarding the course, research activities, internships, development as citizens, infrastructure and image of the school⁽¹¹⁾.

The initiatives of both universities considered the institutional evaluation criteria and evidenced the importance of an evaluative process from the perspective of graduates.

A study performed by the Nursing Graduation Center - Medical School Triângulo Mineiro (CGE-FMTM - Centro de Graduação em Enfermagem da Faculdade de Medicina do Triângulo Mineiro) used a qualitative-quantitative approach and researched 108 students who graduated from 1992 to 1999, aimed at studying the educational process in nursing according to the opinion of graduates, studying their insertion in the workplace and obtaining suggestions to improve the course curriculum⁽¹²⁾. The study results showed that 37% of the graduates were inserted into the hospital care area and 24% were in public health. Regarding their choice of profession, 76.4% of the graduates felt they had made the right choice. As for the educational process, 83.8% would choose to study in the same institution again, since they recognized that the course effectively contributed to their education, and that the practical experiences helped them choose their favored working area. Graduates presented suggestions for the program's quality improvement; for instance, teaching disciplines with integrated contents; making the internship longer and the residency mandatory; focusing on family health programs; investing more in nursing research; promoting the academic exchange with the population, as well as encouraging the participation of graduates in lectures and conferences⁽¹²⁾.

Nursing professors from the Federal University of Santa Catarina (UFSC) have undertaken a study that qualitatively and quantitatively analyzes the perception of thirteen graduates employed in several health institutions in Florianópolis. Results indicated problems regarding the curriculum of the courses: insufficient time for practical experience; lack of integration among course stages; divergence between the nurse's role as focused on by the course and the one expected by employers; and graduates' satisfaction in being a nurse and dissatisfaction regarding the expected role, salary and market recognition. They also show contradictions between speech and practice (what is said and what is actually done), and between the holistic view of the human being and the non-contextualized education. From the authors' point-of-view, this study contributed to the deliberation of the academic world and the professional reality⁽¹³⁾.

Professors from the Medical School of São José do Rio Preto (FAMERP) carried out a study with graduates from the first three nursing groups graduated up to 1996, aimed at studying their entry into the workplace as a parameter for the evaluation of the quality of their education. A questionnaire with open and closed questions was applied to 59 graduates. The authors believe the study allowed the elaboration of proposals to establish a better connection between practice and theory, thus enriching the graduate course development⁽¹⁴⁾.

Another study, carried out by professors from the State University of West Paraná (UNIOESTE), presented results that led to the construction of a new Pedagogic Political Project (PPP), which came into force in 2003. The authors analyzed the process and joined two working groups: one group was in charge of conceptualizing values that guide the nursing practice, whereas the other group had to create a diagnosis from the perspective of academics, professors, graduates and professionals from the internship programs. The study report does not mention the instrument used, nor the results obtained from the graduates' point of view, but the authors admit the study allowed the reformulation of the course PPP in terms of complying with the national curricular guidelines, as well as with the demands raised by the diagnosis⁽¹⁵⁾.

It is worth mentioning, due to its comprehension and broadness, a study project that is being developed by professors from the University of São Paulo (USP) Education School, regarding the professional future of USP graduates⁽¹⁶⁾. This study's purpose is to subsidize the enrichment of the academic management and to improve the graduate courses' effectiveness and efficiency. It also aims at: developing studies regarding the profile of students who graduate from USP; learning the current professional activity and occupational journey of graduates; identifying their opinion about the education provided by the university; and understanding possible variables affecting their occupational choices and destinies.

This project proposes a survey of three courses from the Human Sciences, Exact Sciences and Biological areas, aimed at implementing a system for monitoring ex-students, as a usual and continuing practice. It includes the design and implementation of an evaluation methodology that incorporates the university's social role in the discussion, regarding both the internal and external public, and the professional destiny of its graduates. The proposed study with graduates may provide valuable contributions for the IHE, by examining relevant aspects both for the evaluative and the educational processes.

FINAL CONSIDERATIONS

The concepts presented allowed an understanding of the relevance of the educational evaluation, systematized in order to improve the educational process and to incorporate the dialogue among the leaders of this process, including the participation of graduates.

The journey of the organizers of evaluation policies for higher education in Brazil characterizes an increasing movement to guarantee quality education, connected to an evaluation system capable of bringing about the transformations demanded by society. It also underscores the importance of considering the graduates' perception, as an effective indicator for institutional evaluation.

In spite of their restricted quantitative data, the presented nursing studies show the important contribution of graduates towards evaluating the educational process they experienced. Highlights indicated were: the suitability of the curriculum to the professional education needs; greater investment in research; academic exchange with the community, including continuing education projects; greater integration between theory and practice, and the need to increase the time dedicated to internship programs.

This study allowed an understanding that evaluative processes carry a potential to make it happen and to motivate change, and that the opinion of graduates regarding their education is fundamental for the construction of an evaluative system to consolidate the educational process with emancipatory and transforming characteristics.

REFERENCES

• 1. Amorim A, Sousa SMZL. Avaliação institucional: definições e posicionamentos. Est Aval Educ. 1990;(1):9-12.
• 2. Vianna HM. Avaliação educacional. São Paulo: Ibrasa; 2000.
• 3. Ristoff DI. Algumas definições sobre avaliação. In: Dias Sobrinho J, Ristoff DI, organizadores. Avaliação e compromisso público: a educação superior em debate. Florianópolis: Insular; 2003. p. 21-33.
• ⁴
  Brasil. Lei n. 9.394, de 20 de dezembro de 1996. Estabelece as Diretrizes e Bases da Educação Nacional (LDB). Diário Oficial da União, Brasília, 23 dez. 1996. Seção 1, p. 27833-41.
• 5. Sousa SMZ, Freitas DNT. Políticas de avaliação e gestão educacional: Brasil, década de 1990 aos dias atuais. Educ Rev. 2004;(40):165-86.
• 6. Dias Sobrinho J. Avaliação da educação superior, regulação e emancipação. In: Dias Sobrinho J, Ristoff DI, organizadores. Avaliação e compromisso público: a educação superior em debate. Florianópolis: Insular; 2003. p. 42-52.
• ⁷
  Brasil. Ministério da Educação. Comissão Nacional de Avaliação da Educação Superior (CONAES). Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira (INEP). Avaliação externa de instituições de educação superior: diretrizes e instrumento. Brasília; 2005.
• 8. Brasil. Ministério da Educação. Sistema Nacional de Avaliação da Educação Superior (SINAES). Da concepção à regulamentação. 2Ş ed. Brasília: INEP; 2004.
• ⁹
  Brasil. Ministério da Educação. Comissão Nacional de Avaliação da Educação Superior (CONAES). Sistema Nacional de Avaliação da Educação Superior (SINAES). Orientações gerais para o roteiro da auto-avaliação das instituições. Brasília: INEP; 2004.
• ¹⁰
  Universidade Federal do Ceará (UFC). Dimensão nove: políticas de atendimento dos estudantes e de acompanhamento dos egressos da UFC [texto na Internet]. Fortaleza; 2006. [acesso 2006 jul. 25]. Disponível em: http://www.ufc.br/_files/conhecaaufc/avaliacao/d9_egressos.pdf
• ¹¹
  Centro Federal de Educação Tecnológica da Bahia (CEFET-BA). Relatório de auto-avaliação institucional do CEFET-BA: etapa educação superior [texto na Internet]. Salvador: CPA; 2006. [citado 2006 jul. 25]. Disponível em: http://www.cefetba.br/cpa/downloads/Rel_Avaliacao_CEFET-BA_versao_final.pdf
• 12. Cardoso RJ. Egressos do Centro de Graduação em Enfermagem da Faculdade de Medicina do Triângulo Mineiro: sua formação e trajetória profissional [tese]. Ribeirão Preto: Escola de Enfermagem de Ribeirão Preto, Universidade de São Paulo; 2002.
• 13. Ghiorzi AR, Ribeiro EM, Fenilli RM. O currículo, o curso e a profissão, na visão de egressos da graduação em enfermagem da UFSC. Texto Contexto Enferm. 1995; 4(n.esp):69-91.
• 14. Soler ZASG, Perroca MG, Santos MLSG, Santos MR. Inserção dos egressos do curso de graduação em enfermagem da Faculdade de Medicina de São José do Rio Preto no mercado de trabalho: parâmetros para a qualidade do ensino. Acta Paul Enferm. 2001;14(1):37-47.
• 15. Oliveira BRG, Schneider JF, Rizzotto MLF, Rodrigues RM. Avaliação e construção de um projeto político pedagógico para a graduação em enfermagem. Rev Bras Enferm. 2003;56(4):369-73.
• 16. Oliveira RP, Souza SZ. Programa de estudo sobre o destino profissional de graduados pela USP [projeto de pesquisa]. São Paulo: Centro de Estudos e Pesquisas em Políticas Públicas de Educação, Faculdade de Educação, Universidade de São Paulo; 2004.

Correspondência:

Maria Dyrce Dias Meira

Av. Giovanne Gronchi, 6675 - Bl. 10 - Apto. 155

CEP 05724-005 - São Paulo, SP, Brasil

Publication Dates

• Publication in this collection
  30 June 2009
• Date of issue
  June 2009

History

• Accepted
  21 Aug 2008
• Received
  21 Dec 2006