Acessibilidade / Reportar erro

Higher education policy: A case study on quality assessment towards a model of university management

Política de ensino superior: um estudo de caso sobre a avaliação da qualidade para um modelo de gestão universitária

Abstract:

The institutional self-assessment is considered herein in the context of higher education assessment as a university management tool. The paper aims to present and discuss the impact of the results of an institutional self-assessment committee in the management of pedagogical and administrative issues of a Brazilian university in accordance with contemporary education policy. Four consecutive annual reports were reviewed to demonstrate the impacts of institutional self-evaluation and to raise questions about the academic community. Content analysis principles were used to validate the assumptions of the committee’s annual routines regarding the importance of the institutional assessment framework as the basis for the qualification of academic procedures. It was possible to observe the positive effects of the committee on the university’s practices: the effective participation of the academic community in the evaluation processes as well as validation of the information for university management. Another aspect that the evaluation allowed was the identification of areas of greater or lesser impacts on the scientific, social, pedagogical and technological aspects. The role of the committee was essential in identifying strengths and weaknesses, thus contributing towards performing actions to improve the institutional quality of the University of Uberaba under the panorama of Brazilian higher education assessment.

Key words:
Higher education; Management; Policy; Quality; Self-assessment; University

Resumo

A auto-avaliação institucional é considerada aqui no contexto da avaliação do ensino superior como uma ferramenta de gestão universitária. O artigo tem como objetivo apresentar e discutir o impacto dos resultados de um comitê de auto-avaliação institucional na gestão de questões pedagógicas e administrativas de uma universidade brasileira de acordo com a política educacional contemporânea. Quatro relatórios anuais consecutivos foram revisados ​​para demonstrar os impactos da auto-avaliação institucional e levantar questões sobre a comunidade acadêmica. Os princípios de análise de conteúdo foram utilizados para validar os pressupostos das rotinas anuais do comitê quanto à importância do quadro de avaliação institucional como base para a qualificação dos procedimentos acadêmicos. Foi possível observar os efeitos positivos do comitê sobre as práticas da universidade: a efetiva participação da comunidade acadêmica nos processos de avaliação, bem como a validação da informação para a gestão universitária. Outro aspecto que a avaliação permitiu foi a identificação de áreas de maior ou menor impacto nos aspectos científicos, sociais, pedagógicos e tecnológicos. O papel do comitê foi essencial na identificação de pontos fortes e fracos, contribuindo assim para a realização de ações de melhoria da qualidade institucional da Universidade de Uberaba sob o panorama da avaliação do ensino superior brasileiro.

Palavras-chave:
Educação superior; Administração; Política; Qualidade; Auto-avaliação institucional; Universidade

1 Introduction

The recent history of Brazilian higher education assessment had many influences and contexts, in search of a more formative comparative analysis under expected political and social bias. Like other countries such as the United Kingdom, the higher education assessment and accreditation of institutions in Brazil are under aegis of the State (BRAZIL, 1996BRAZIL. Framework, Guidelines and Bases of Education. Presidency of the Republic Law No. 9394/1996. 1996 Disponível em: <Disponível em: http://www.planalto.gov.br/ccivil_03/leis/l9394.htm >. Acesso em: 30 jul. 2014.
http://www.planalto.gov.br/ccivil_03/lei...
), despite accrediting bodies are publicly funded, there are indications that independent or private actors are gaining ground in this area and recent groups of higher education institutions are forming their own evaluation schemes (STENSAKER, 2011STENSAKER, Björn. Accreditation of higher education in Europe-moving towards the US model? Journal of Education Policy, London, v. 26, n. 6, p. 757-769, 2011.). Historically in Brazil, Mello’s term (1990-1992) designed an education assessment articulated to the demands of state reform, guided by a centralized, authoritarian and controlling vision. This contradictory approach was refused by academia and the government's intention to increase the efficiency and productivity of higher education without an appropriated dialogue with institutions.

After the impeachment of Mello, Franco’s government (1992-1994) heard the demand from the academic community to establish the Secretary of Higher Education and the National Commission of Assessment (BRAZIL, 1993BRAZIL. Higher Education Secretariat. Ministry of Education. Regulatory act No. 130/1993. 1993. Disponível em: <Disponível em: http://pesquisa.in.gov.br/imprensa/jsp/visualiza/index.jsp?jornal=1&pagina=73&data=15/07/1993 >. Acesso em 30 jul. 2014.
http://pesquisa.in.gov.br/imprensa/jsp/v...
) aiming to establish guidelines and enable the implementation of the institutional review of the Brazilian universities’ routines. Subsequently, Cardoso’s neoliberalism (1995-2002) strengthened the role of the university's compatibility to markets and to production, oriented to personal success, technical training, and in pursuit of the functional efficiency of the students (OLSEN; PETERS 2005OLSSEN, Mark; PETERS, Michael. Neoliberalism, higher education and the knowledge economy: From the free market to knowledge capitalism. Journal of Education Policy, London, v. 20, n. 3, p. 313-345, 2005.) in a National Exam. Silva’s government (2003-2010) and Rousseff’s first term (2011-2014) expanded the notion of broader access to university and provided the highest funding for scholarships and robust resources for the expansion of the federal universities framework. They also established the National Evaluation System of Higher Education - SINAES in portuguese) (BRAZIL, 2003BRAZIL. Ministry of Education. Basis for a new proposal for evaluation of Brazilian higher education. 2003Disponível em:< Disponível em:http://portal.mec.gov.br/arquivos/pdf/sinaes.pdf >. Acesso em: 17 jul. 2014.
http://portal.mec.gov.br/arquivos/pdf/si...
). It provides the higher education assessment procedures as being continuous and permanent, neither episodic, nor fragmented. Such continuous processes should create a culture of educational assessment incorporated in the daily routines of universities (BRAZIL, 2009BRAZIL. Higher Education Secretariat. Ministry of Education. Regulatory act No. 02/2009. 2009. Disponível em: <Disponível em: http://download.inep.gov.br/download/superior/condicoesdeensino/Extrato_reconhecimento_licenciatura.pdf >. Acesso em: 30 jul. 2014.
http://download.inep.gov.br/download/sup...
).

One of the first steps of Brazilian universities in order to attend SINAES was to create the Institutional Self-Assessment Committee to follow the guidelines for the Evaluation of Institutions of Higher Education (BRAZIL, 2004BRAZIL. National Evaluation System of Higher Education. Ministry of Education. Law No 10861/2004.2004. Disponível em: <Disponível em: http://www.planalto.gov.br/ccivil_03/_ato2004-006/2004/lei/l10.861.htm >. Acesso em: 17 jul. 2014.
http://www.planalto.gov.br/ccivil_03/_at...
). It described therein the self-assessments for professors, students and technicians, in addition to the policy recommendations about the principles of SINAES. Its major instruments are: Institutional Data Assessment, Institutional Self-Assessment; External Assessment Commission and National Examination of Student’s Performance. Thus, each Institutional Self-Assessment Committee is a part of SINAES and is composed of parity between professors, students, administrative staff, and community representatives. In general terms, Institutional Self-Assessment Committee represents the Ministry of Education in the universities’ staff in order to assure the goals of educational policies.

From the viewpoint of Buarque (2005, p. 44BUARQUE, Cristovam. A refundação da universidade. Brasília: ABMES, 2005.) in his university reform proposal, two basic goals for Brazilian universities should emerge in the context of higher education assessment: one based on student’s personal success and the other on nation-building social transformation. Nonetheless, until now, Brazilian private and public universities have been faced with the demand of a new and challenging paradigm - how to teach in a context of social and economic inequality to achieve the high quality standards set by SINAES.

The University of Uberaba (Uniube) is a vibrant private university with 68 years of uninterrupted and growing activities in the richest region of Triângulo Mineiro in Minas Gerais State which has 90.545 km² and a gross domestic product of US$ 18 billion and a per capita annual income of US$ 8.000,00 (IBGE, 2010IBGE - Brazilian Institute of Geography and Statistics . Archives of national census. 2000. Disponível em: <Disponível em: http://www.sidra.ibge.gov.br/bda/territorio/infounit.asp?codunit=267&codunitibge=3105&nomeunit=Tri%E2ngulo+Mineiro%2FAlto+Parana%EDba+-+MG&n=8&nomenivel=Mesorregi%E3o+Geogr%E1fica&z=t&o=4 >. Acesso em: 12 jan. 2014.
http://www.sidra.ibge.gov.br/bda/territo...
). Its political influence reaches almost two million people living in a strategic agribusiness region. It has almost thirty thousand people including undergraduate, graduate, and professional students. It offers graduate, post graduate and short courses in the areas of education, science, human and animal health, management, engineering, architecture and technology. Also applies part of its efforts in research within three master programs. Uniube is supported in part by its funding society grants to maintain its social responsibilities which include free of charge access to hospital and dentistry services open for the community, scholarship programs, and the development of new enterprises and agribusinesses.

Since 2004 and, in particular 2008-2011, Uniube supported Institutional Self-Assessment Committee to embrace the SINAES guidelines. The first challenge was to introduce a model of higher education assessment feasible for fifty thousand people. Software was designed to create a Institutional Self-Assessment Committee interface with each academic representative group presented structured questionnaires. The responses were automatically processed and analyzed by the software and peer reviewed to Institutional Self-Assessment Committee standards.

According to SINAES, it was possible to analyze ten dimensions of higher education assessment which were: institutional mission and development plan; educational policies; social responsibility; social communication; staff resources; organization and management; infrastructure; academic planning and evaluation; student’s assistance; and sustainability. It is in the context of these dimensions, with special regard to pedagogical, administrative and academic networks, which are described herein.

This article presents the results of the Institutional Self-Assessment Committee of University of Uberaba and the managing improvements thereof and discusses the politics related to the higher education assessment in Brazil.

2 Development

2.1 Subjects and ethical implications

The present paper is based on public databases of the Institutional Self-Assessment Committee (UNIUBE, 2014UNIUBE.Archives From Self Institutional Committee. 2009. Disponível em: <Disponível em: http://www.uniube.br/cpa/relatorios.php >. 2014.Acesso em: 10 jan. 2014.
http://www.uniube.br/cpa/relatorios.php...
) from University of Uberaba obtained from unidentified subjects by using web secured access. Despite having no ethical implications, the study complies with ethical issues.

2.2 Databases

Annual reports from Institutional Self-Assessment Committee of Uniube covering the period of 2009-2012 were downloaded through the accession of open public databases (UNIUBE, 2009). The data was analysed by the former committee’s coordinator who is one of the authors of the present article. Most of the pedagogical and administrative data were accomplished due to academic management system which presented as support tool for academic and management prerogatives.

2.3 Content Analysis

The proposal of Minayo (2001MINAYO, M. C. S. Pesquisa social: teoria, método e criatividade. Rio de Janeiro: Vozes, 2001.) was used in the content analysis of the four institutional self-assessment annual reports from Institutional Self-Assessment Committee. Briefly, it consisted to divide in pre-analytical (qualitative interpretation of text according to key terms) analytical (statistical analyses of quantitative data, figures and tables) and post-analytical parameters (review of original interpretations of the reports). In other to cover all dimensions suggested in the SINAES, ten categories of higher education assessment were used in the evaluation: (1) the mission and the institutional development plan; (2) pedagogical perspective and policies of incentives for teaching, research, post-graduate and extension, (3) social responsibility; (4) communication with the community; (5) human resources, career improvement and working conditions; (6) organization and management; (7) physical infrastructure; (8) academic planning and evaluation; (9) the attendance policy of students and graduates, and (10) financial sustainability.

2.4 Statistical analysis

The chi-square test was used to evaluate the significance of the report’s qualitative results. The equitable apportionment among all the alternatives (X2) was considered as the expected response and it was compared to the amount of answers observed in reports (X2 R). The significance level was 5%. Differences between median and the standard deviation of values from two or more groups were analyzed by Student t or Tukey test respectively with significance levels of 5%. Data was analyzed by using GraphPad Prism 4.

2.5 Major Findings

The Institutional Self-Assessment Committee covered a large number of individuals within Uniube and these in turn were divided into different categories established in accordance to the focus of the results to be disclosed. Figure 1 gives three quantitative panoramas of (a) students, (b) professors and (c) undergraduate courses that were submitted to assessment by (d) hundreds of questionnaires.

It can be noted that Institutional Self-Assessment Committee was effective in the augmentation of the students’ participation; the strategy behind it was to disclose the questionnaires in the first access of an academic web-page. The students could refuse to answer the questionnaire in three consecutive attempts but on the next access, the web-page would require the participation of student in order to complete the questionnaire.

Also, free-contribution on evaluation processes was allowed by personal communication to the Institutional Self-Assessment Committee web-site or in scheduled meetings between students and teachers’ representatives. Unfortunately, the results came mostly from undergraduates because Pro-Rectory of Post Graduate, Research and Extension continually refused the Institutional Self-Assessment Committee initiative. On the contrary, the Rectory and Pro-Rectory of Undergraduate Courses supported and discussed with Committee in order to perform a more realistic assessment.

The participation of professors was a constructive but continuous goal (Figure 1b). In the beginning, according to the transparency demanded by SINAES, the result of each professor’s evaluation was published openly to the academic community which caused embarrassment and legal implications mostly of the professors having the lowest performances. Since 2010, the results of the professors’ evaluations became encrypted to avoid personal identification, but broadly disclosed in both Institutional Self-Assessment Committee and academic web-pages to access the evaluation in order to define pedagogical and administrative improvements. As can be seen in Figure 1b, the professors felt more comfortable to join the Committee initiatives in 2011, giving important and unexpected feedback in the results.

The contribution of the Committee, mainly in the period of 2008-2011, reflected in the pedagogical and administrative arrangements which resulted in a better quality assessment. According to SINAES the quality of undergraduate courses is established by an index (1 to 5) which represents the three dimensions of academic development: (i) infrastructure (ii) learning experience and (iii) professor’s qualifications.

In Figure 2a it is possible to note that index varied from 2 to 4 (3.55 ± 0.88) in most of Uniube's graduate courses. Nonetheless, the efforts of the Institutional Self-Assessment Committee contributed to augment the level of quality of courses. In Figure 2b it is possible to note the contribution on another index of quality, called IGC that represents the component of graduation investment (pedagogical projects, specific laboratories, equipment, hardware, software and general classroom infrastructure, libraries, auditoriums and also the professor’s qualifications) on global assessment of a Brazilian university. However, Uniube despite having improved its graduate component of IGC, its post graduate programs stagnated a better university’s assessment in the period of 2008-2011(Figure 2c).

Thus, it is possible to note in Figure 2d that the Institutional Self-Assessment Committee helped Uniube to improve these aspects in its worst performing graduate courses (2008-2009) with several interventions which included the consulting of directors and supervisors to discuss pedagogical projects; to offer more suitable regular exams and to assist inexperienced professors, unfamiliar with institutional policies and pedagogical documents. In later CPC indexes (2011-2012) it was possible to note a lower dispersion due to better students’ performance in the National Exam. Such results were obtained by universities with similar characteristics to Uniube (not shown). When the CPC was broken down, it was possible to note the major contribution of the performance by students in the National Exam, rather than the efforts of investments in the pedagogical or infrastructural issues (Figure 2e). The overall analysis of the results pointed to the need of improvements in the learning experience offered to students by Uniube in order to effect better achievement in the National Exam.

Figure 1:
Quantitative aspects of self-evaluation at the University of Uberaba. Participation of students in the (a) CPA questionnaires to evaluate (b) teachers; of the various (c) undergraduate courses; in each of the (d) ten dimensions of SINAES. * p <0.001 chi-square test.

Fig,1b

Fig,1c

Fig,1d

Figure 2
Impacts at the University of Uberaba by the Institutional Self-Evaluation Committee. a) Quality of courses after internal evaluations; b) Increasing influence of component graduation on IGC into the c) General concept of institutional quality; d) Influences of Committee in quality indicators of different undergraduate courses (CPC); (e) breaking down of the most important components of quality assessment at the University of Uberaba.

Fig,2b

Fig. 2c

Fig. 2d

Fig.2e

As can be seen in Table 1, the Institutional Self-Assessment Committee contributed to a variety of institutional, pedagogical and administrative dimensions. Most of the weaknesses appeared in the results obtained in the questionnaires. Uniube was asked to propose suitable solutions to solve the demands, in particular regard to the infrastructure of academic facilities (classrooms, laboratories and libraries). On the other hand, due to the massive investments needed to accomplish quality parameters from SINAES, the strategy behind e-learning expansion should be carefully rethought mainly as an expectation of the growth observed elsewhere (ALONSO, 2010ALONSO, Katia Morosov. The expansion of higher education in Brazil and distance learning: Dynamics and places. Educação e Sociedade, Campinas, v. 31, n. 113, p. 1319-1335, 2010. ). Another relevant aspect pointed out by the Institutional Self-Assessment Committee is the low adeptness of students in the National Exam, which has been targeted by the pedagogical initiatives of the Pro-Rectory of Undergraduate Courses.

To solve the problem of high professor turnover, the Committee stated two proposals: (i) enhancement of salary benefits, as predicted in the Institutional Plan of Career Development and Salaries and (ii) continuing education of professors focusing the pedagogical aspects of the institutional guidelines. Unfortunately, the Committee deeply regrets that despite repeated efforts, it was not possible to obtain data for the necessary skills to compose the institutional self-assessment of Pro-Rectory of Research, Post graduate and Extension, because the absence of information available about these important aspects of Uniube in four consecutive annual reports. The general opinion of the person in charge is that the post graduate programs are under supervision of another government body (CAPES, 2010CAPES. Coordenação de Aperfeiçoamento de Pessoal de Nível Superior. Plano Nacional de Pós-graduação 2011-2020. Brasilia: CAPES, 2010.) other than SINAES.

Nonetheless, the Institutional Self-Assessment Committee pointed out the urgent need of Uniube to suit the minimum requirements to maintain the quality of higher education supported by research, post graduation and extension as a mandatory prerequisite foreseen by law (CAPES, 2010CAPES. Coordenação de Aperfeiçoamento de Pessoal de Nível Superior. Plano Nacional de Pós-graduação 2011-2020. Brasilia: CAPES, 2010.). New stricto sensu - master and doctoral degree programs - must be considered as primary goals of Uniube's policy strategies. The authors were glad to inform that simultaneously to the publication of this article in 2016, Uniube has approved one doctoral and two new master degree programs (UNIUBE, 2016UNIUBE. Acontece na UNIUBE - Universidade de Uberaba. Disponível em: <Disponível em: http://www.uniube.br/conteudo2.php?p=4&m=&c=412 >2016. Acesso em: 18 maio 2016.
http://www.uniube.br/conteudo2.php?p=4&m...
).

Beyond the legal implications of regarding the prerogatives of SINAES, the most important area of interest for higher education assessment is the student. Institutional Self-Assessment Committee attempted to increase the confidence of students in the self-assessment processes and Table 2 presents some results. It is possible to note that despite the massive participation in results; around 20% of students were unaware of the Institutional Self-Assessment Committee. This could be explained by the fact that many of the e-learning courses were not included due to the instability of internet services at remote locations. Almost one quartile of the students responded to the questionnaires carelessly and a half was in doubt about the secrecy of the responses, which seemed to be very decisive in the nature of the students’ responses.

Despite the high proportion of students disinterested in the Committee results, many of them did see the results by consulting notice boards, web-sites or by attending the meetings scheduled with directors or supervisors In addition, the majority of students affirmed that the results were likely to have reflected the reality of university routines (Table 2).

Table 1:
Support of Institutional Self-Assessment Committee in the completion of weakness and strengths towards university management

Table
2: Importance of Institutional Self-Assessment Committee among students’ opinion

3 Conclusion

One of the consequences arising from the evaluation of complex educational institutions, specifically relative to their dimensional characteristics, such as the institution targeted in this article is to establish organizational partnerships. The performance of the Institutional Self-Assessment Committee has been deeply supported by Pro-Rectory of Undergraduate Courses which has divided the responsibilities of the institutional assessment process with all administrative and pedagogical directors to assist self-assessment reviews. However, the independence and the transparency of the self-assessment were respectfully guaranteed. The tasks were numerous; however, there was a common behind the scenes feature: the mission of Uniube - to promote education and to generate knowledge for upcoming professional committed to a fairer society -which has guided the actions of all sectors of the university staff.

The Institutional Self-Assessment Committee took on the broad communication using a variety of channels as an important strategy for the dissemination of their activities, by using all available web media for this purpose (intranet, internet, short videos on YouTube™, radio web-cast/podcast). As effective communication depends on vehicles that are appropriate to the content of the messages to be sent, the Committee also used these communication channels and media apparatuses as they help to convey the results and their impacts. Thus, the Institutional Self-Assessment Committee framework had support from other sectors of Uniube, directly from the Department of Institutional Marketing and from the Division of E-Learning Production which were responsible for the fulfillment of the communication needs of the institution.

The annual institutional self-assessment reports were based on data obtained or provided by various sectors of the university resulting from the gathering segments of institution. Thus, the Institutional Self-Assessment Committee organized and reviewed its processes to cope with the very large amount of data that formed the basis of the reports. Evaluative information and analyzes contained in these reports are legitimized in regular meetings with the community during the Institutional Assessment Week, or during official academic calendar events where the Committee’s participation was expected, and at times on the web-site. In 2008-2011 it was successfully fulfilled Committee planning, production, application and analysis of the results of 328 questionnaires to discuss all dimensions of SINAES. Still, according to the Ministry of Education timetable, the Committee intensified self-assessment of the courses by applying them as an Assessment tool for undergraduate courses. It had helped to reduce distortions in the pedagogical projects, equate investment in laboratory infrastructure and update library resources, and also contributed to the reviews of day to day teaching practices in classrooms.

However, difficulties were observed to meet all legal requirements in view of the progressive expansion in the number of professors, students and sectors spread across the campuses, requiring careful attention to matters of planning, assessment and performance of the different institutional groups, in view of the need for continual updating of the information network organized by administrative staff and academics. Infrastructure, hardware, software and human resources seemed to be compatible with the current institutional complexity, but it will require a massive investment to fulfill all the goals of the Institutional Development Plan.

This article can highlight the articulation of the components from a pedagogical perspective and incentive policies for teaching, research, post-graduate and pedagogical extension. The results of the self-assessment revealed a gap between undergraduate and post graduate. The career development, research and extension are inseparable, since all graduation courses presented the own specifications that should integrate master/doctoral programs in existing different knowledge areas. The relevance of postgraduate courses for many graduate students already constitutes the continuation of their vocational training. These courses were rare in Uniube and the gap represents a lost opportunity to meet and to develop core competences for research and post graduation in all knowledge fields.

The advances in the assessment policies of higher education in Brazil seem to be in accordance with The Program of the Joint Committee of Standards for Educational Evaluation (YARBROUGH, 2011YARBROUGH, Donald ; SHULHA, Lyn; HOPSON, Rodney; CARUTHERS, Flora. Joint committee on standards for educational evaluation: the program evaluation standards. Newbury Park, California: Sage, 2011) and beyond that stated by Bunoti (2014BUNOTI, Sarah. The quality of higher education in developing countries needs professional support. 2009 Disponível em: <http://www.intconfhighered.org/FINAL%20Sarah%20Bunoti.pdf>. Acesso em: 30 jul. 2014. ) regarding the quality of higher education in developing countries (RISTOFF, 2011RISTOFF, Dilvo. Construindo outra educação: tendências e desafios da educação brasileira. Florianópolis: Insular, 2011.). However, some of the major challenges facing the higher education in Brazil such as the rapid rise of privatization, underfunding public, lack of access of the poor, the weak investment in research and development, the little scientific production, high academic dropout rate and low rate enrollment should be considered in the comparison among universities’ quality abroad. (SEGRERA, 2016SEGRERA, Francisco López. Educación Superior Comparada: Tendencias Mundiales y de América Latina y Caribe. Avaliação, Campinas; Sorocaba, v. 21, n. 1, p. 13-32, 2016.).

This scenario prompted several attempts of quality evaluation of higher education witnessing the guidelines of a Brazilian neoliberal model, under an educational policy that involved a break with traditional higher education policies, and much less action planned to translate the political will to change paradigms (SEGRERA, 2016SEGRERA, Francisco López. Educación Superior Comparada: Tendencias Mundiales y de América Latina y Caribe. Avaliação, Campinas; Sorocaba, v. 21, n. 1, p. 13-32, 2016.). As complementary, another assessment paradigm on the institution as a whole, recovering the concept of superior more complex education, whose main purpose is to train citizens capable of critical thinking and whose reference is the center of society (DIAS SOBRINHO, 2010DIAS SOBRINHO, José. Avaliação e transformação da educação superior brasileira (1995-2009) do Provão ao SINAES. Avaliação, Campinas; Sorocaba, v. 15, n. 1, p. 195-224, 2010.).

The results presented herein showed the institutional evaluation as opportune and necessary towards a model of university management. It is believed that the greatest obstacle to the development of institutional evaluation is the treatment of the results of the evaluation processes, which should support better decisions that promote the improvement of institutional activities (PINTO et al 2016PINTO, Rodrigo; MELLO, Simone; MELO, Pedro. Metaevaluation: a decade of Institutional Evaluation Process of SINAES. Avaliação, Campinas, Sorocaba, v. 21, n. 1, p.89-108, 2016.); there is also some concern in the dissemination of evaluation results as unsatisfactory. The latter may result as the possibility of unfair comparisons among different educational institutions damaging the image of internal and external relations of those having the least grades.

Recently, Pinto and colleagues (2016PINTO, Rodrigo; MELLO, Simone; MELO, Pedro. Metaevaluation: a decade of Institutional Evaluation Process of SINAES. Avaliação, Campinas, Sorocaba, v. 21, n. 1, p.89-108, 2016.) have also indicated that private universities haven’t had any grade above the satisfactory level of quality at all the SINAES dimensions. On the other hand, the public ones got grades under satisfying level of quality on mission and strategic institutional plan - and also - planning and evaluation. Despite reaching the dimensions of a satisfactory concept they are lower than expected, so it is believed that somehow improving these concepts must be agreed with the institution of higher education in training, and systemic vision for the undergraduate performance improvements (PINTO et al 2016PINTO, Rodrigo; MELLO, Simone; MELO, Pedro. Metaevaluation: a decade of Institutional Evaluation Process of SINAES. Avaliação, Campinas, Sorocaba, v. 21, n. 1, p.89-108, 2016.).

However, in this case study there are some disagreements. In the effort to pursue SINAES’ directives, the Institutional Self-Assessment Committee failed to appropriately identify the stakeholders, since the people involved in the evaluation and those who will be affected by the evaluation must be clearly identified so that their needs can be addressed. The students put the Committee’s credibility in check, because the people conducting the assessment must be trustworthy and competent to perform the evaluation in order to achieve maximum credibility and acceptance. The latter deserve attention since a lack of student involvement arose because for the key stakeholders (students) the questionnaires were taken just as an obligation that would not bring practical results.

Indeed, some of the weaknesses pointed out by the Institutional Self-Assessment Committee still persist due to lack of more efficient management to solve the problems. Another aspect is related to the report's timelines and dissemination of results, evaluation reports and any significant interim findings should be disseminated to intended users so that they may be used in a timely fashion.

However, the results from the institutional self-assessment were important to reduce distortions between undergraduate courses. In the period of 2008-2011, the indexes (POLIDORI, 2009POLIDORI, M. M. Higher education evaluation policies in Brazil: PROVÃO, SINAES, IDD, CPG, IGC and… other performance indicators. Avaliação, Campinas; Sorocaba, v. 14, n. 2, p. 253-266, 2009.; RISTOFF, 2011RISTOFF, Dilvo. Construindo outra educação: tendências e desafios da educação brasileira. Florianópolis: Insular, 2011.) reveal a greater conformity with SINAES’ standards as being “sufficient” or “satisfactory” from pedagogical, infrastructural and administrative points of view. Criticisms the evaluation of higher education in all models are advocated by confront them with the prevailing thought in its structure and to present the main features, guidelines and consequences of each model. In a recent article, Francisco and colleagues (2015FRANCISCO, Thiago Henrique Almino; NAKAYAMA, Marina Keiko; MELO, Pedro Antônio de; PITTA, Márcio Alexandre; OLIVEIRA, Fabiano Pires de. Análise epistemológica da avaliação institucional da educação superior brasileira: reflexões sobre a transposição de paradigmas. Avaliação, Campinas; Sorocaba, v. 20, n. 2, p. 531-562, 2015.) proposed a conceptual reflection on the current model, with the current paradigm of institutional assessment, consequences, impacts on consolidation of the epistemology of scientific field of management of higher education and the opportunities that can contribute toward moving from the current summative view to an emancipatory process.

More importantly, it was clear that the investment of Uniube must emphasize a broader academic experience, by using robust learning resources embedded in a culture of research and technological development to better stimulate students and professors.

Finally, the authors propose appropriate tools for assessing the quality of a Brazilian university management model, since there is no way to have quality without a critical and continuous evaluation of their performance; which is fundamental in a contemporary university. We did not just look at the assessment from the view of accreditation, but also from the point of view of internal improvement, struggling achieving and raising the level of quality. Recent achievements of our institution allow us to note that the culture of self-evaluation begins to consolidate in the institution's organizational framework, with the midpoint excellence in teaching and training being not only technically, but also socially responsible.

4 Acknowledgments

The authors wish to thank Cleber Henrique Moreira Dias, Alan Carlos da Silva, and Ricardo Aidar for the support and zealous motivation. The authors are grateful to the CPA 2008-2011 members whose dedication turned this article possible.

References

  • ALONSO, Katia Morosov. The expansion of higher education in Brazil and distance learning: Dynamics and places. Educação e Sociedade, Campinas, v. 31, n. 113, p. 1319-1335, 2010.
  • BRAZIL. Framework, Guidelines and Bases of Education. Presidency of the Republic Law No. 9394/1996. 1996 Disponível em: <Disponível em: http://www.planalto.gov.br/ccivil_03/leis/l9394.htm >. Acesso em: 30 jul. 2014.
    » http://www.planalto.gov.br/ccivil_03/leis/l9394.htm
  • BRAZIL. Higher Education Secretariat. Ministry of Education. Regulatory act No. 02/2009. 2009. Disponível em: <Disponível em: http://download.inep.gov.br/download/superior/condicoesdeensino/Extrato_reconhecimento_licenciatura.pdf >. Acesso em: 30 jul. 2014.
    » http://download.inep.gov.br/download/superior/condicoesdeensino/Extrato_reconhecimento_licenciatura.pdf
  • BRAZIL. Higher Education Secretariat. Ministry of Education. Regulatory act No. 130/1993. 1993. Disponível em: <Disponível em: http://pesquisa.in.gov.br/imprensa/jsp/visualiza/index.jsp?jornal=1&pagina=73&data=15/07/1993 >. Acesso em 30 jul. 2014.
    » http://pesquisa.in.gov.br/imprensa/jsp/visualiza/index.jsp?jornal=1&pagina=73&data=15/07/1993
  • BRAZIL. Ministry of Education. Basis for a new proposal for evaluation of Brazilian higher education. 2003Disponível em:< Disponível em:http://portal.mec.gov.br/arquivos/pdf/sinaes.pdf >. Acesso em: 17 jul. 2014.
    » http://portal.mec.gov.br/arquivos/pdf/sinaes.pdf
  • BRAZIL. National Evaluation System of Higher Education. Ministry of Education. Law No 10861/2004.2004. Disponível em: <Disponível em: http://www.planalto.gov.br/ccivil_03/_ato2004-006/2004/lei/l10.861.htm >. Acesso em: 17 jul. 2014.
    » http://www.planalto.gov.br/ccivil_03/_ato2004-006/2004/lei/l10.861.htm
  • BUARQUE, Cristovam. A refundação da universidade. Brasília: ABMES, 2005.
  • BUNOTI, Sarah. The quality of higher education in developing countries needs professional support. 2009 Disponível em: <http://www.intconfhighered.org/FINAL%20Sarah%20Bunoti.pdf>. Acesso em: 30 jul. 2014.
  • CAPES. Coordenação de Aperfeiçoamento de Pessoal de Nível Superior. Plano Nacional de Pós-graduação 2011-2020. Brasilia: CAPES, 2010.
  • DIAS SOBRINHO, José. Avaliação e transformação da educação superior brasileira (1995-2009) do Provão ao SINAES. Avaliação, Campinas; Sorocaba, v. 15, n. 1, p. 195-224, 2010.
  • FRANCISCO, Thiago Henrique Almino; NAKAYAMA, Marina Keiko; MELO, Pedro Antônio de; PITTA, Márcio Alexandre; OLIVEIRA, Fabiano Pires de. Análise epistemológica da avaliação institucional da educação superior brasileira: reflexões sobre a transposição de paradigmas. Avaliação, Campinas; Sorocaba, v. 20, n. 2, p. 531-562, 2015.
  • IBGE - Brazilian Institute of Geography and Statistics . Archives of national census. 2000. Disponível em: <Disponível em: http://www.sidra.ibge.gov.br/bda/territorio/infounit.asp?codunit=267&codunitibge=3105&nomeunit=Tri%E2ngulo+Mineiro%2FAlto+Parana%EDba+-+MG&n=8&nomenivel=Mesorregi%E3o+Geogr%E1fica&z=t&o=4 >. Acesso em: 12 jan. 2014.
    » http://www.sidra.ibge.gov.br/bda/territorio/infounit.asp?codunit=267&codunitibge=3105&nomeunit=Tri%E2ngulo+Mineiro%2FAlto+Parana%EDba+-+MG&n=8&nomenivel=Mesorregi%E3o+Geogr%E1fica&z=t&o=4
  • MINAYO, M. C. S. Pesquisa social: teoria, método e criatividade. Rio de Janeiro: Vozes, 2001.
  • OLSSEN, Mark; PETERS, Michael. Neoliberalism, higher education and the knowledge economy: From the free market to knowledge capitalism. Journal of Education Policy, London, v. 20, n. 3, p. 313-345, 2005.
  • PINTO, Rodrigo; MELLO, Simone; MELO, Pedro. Metaevaluation: a decade of Institutional Evaluation Process of SINAES. Avaliação, Campinas, Sorocaba, v. 21, n. 1, p.89-108, 2016.
  • POLIDORI, M. M. Higher education evaluation policies in Brazil: PROVÃO, SINAES, IDD, CPG, IGC and… other performance indicators. Avaliação, Campinas; Sorocaba, v. 14, n. 2, p. 253-266, 2009.
  • RISTOFF, Dilvo. Construindo outra educação: tendências e desafios da educação brasileira. Florianópolis: Insular, 2011.
  • SEGRERA, Francisco López. Educación Superior Comparada: Tendencias Mundiales y de América Latina y Caribe. Avaliação, Campinas; Sorocaba, v. 21, n. 1, p. 13-32, 2016.
  • STENSAKER, Björn. Accreditation of higher education in Europe-moving towards the US model? Journal of Education Policy, London, v. 26, n. 6, p. 757-769, 2011.
  • UNIUBE.Archives From Self Institutional Committee. 2009. Disponível em: <Disponível em: http://www.uniube.br/cpa/relatorios.php >. 2014.Acesso em: 10 jan. 2014.
    » http://www.uniube.br/cpa/relatorios.php
  • UNIUBE. Acontece na UNIUBE - Universidade de Uberaba. Disponível em: <Disponível em: http://www.uniube.br/conteudo2.php?p=4&m=&c=412 >2016. Acesso em: 18 maio 2016.
    » http://www.uniube.br/conteudo2.php?p=4&m=&c=412
  • YARBROUGH, Donald ; SHULHA, Lyn; HOPSON, Rodney; CARUTHERS, Flora. Joint committee on standards for educational evaluation: the program evaluation standards. Newbury Park, California: Sage, 2011

Publication Dates

  • Publication in this collection
    Jan-Apr 2017

History

  • Received
    06 Nov 2015
  • Accepted
    27 May 2016
Publicação da Rede de Avaliação Institucional da Educação Superior (RAIES), da Universidade Estadual de Campinas (UNICAMP) e da Universidade de Sorocaba (UNISO). Rodovia Raposo Tavares, km. 92,5, CEP 18023-000 Sorocaba - São Paulo, Fone: (55 15) 2101-7016 , Fax : (55 15) 2101-7112 - Sorocaba - SP - Brazil
E-mail: revistaavaliacao@uniso.br