Performance of Brazilian universities in view of the General Course Index (IGC)

Abstracts

A educação superior exerce papel fundamental no desenvolvimento econômico de determinado país, no que condiz ao atendimento das demandas da sociedade. A qualidade no contexto da educação superior tem sido tema recorrente nos últimos anos, sobretudo a partir da criação do Sistema Nacional de Avaliação da Educação Superior (SINAES), que instituiu um sistema de avaliação institucional global e integrador condizente a todas as Instituições de Ensino Superior (IES) brasileiras, sendo responsável por produzir índices para mensuração da qualidade como o Indicador de Diferença dentre os Desempenhos Observado e Esperado (IDD), o Conceito Preliminar de Curso (CPC) e o IGC, Índice Geral de Cursos, que mede o desempenho global da instituição. Diante disso, este estudo tem como objetivo analisar o IGC das universidades públicas e privadas das cinco regiões brasileiras, no intuito de caracterizar o desempenho das IES por região e verificar possíveis discrepâncias intra e inter-regionais identificando oportunidades de melhoria. Os resultados evidenciaram desempenho superior das universidades públicas em todas as regiões, tendo maior destaque as regiões norte e sudeste. Quanto à variabilidade, as regiões Centro-Oeste e Norte apresentaram os melhores desempenhos ambos condizentes ao setor privado. No entanto, para realizar uma avaliação consolidada do desempenho das IES por região faz-se necessário analisar, de forma integrada, os resultados do IGC alinhados aos demais subsistemas de avaliação que integram a avaliação multidimensional do SINAES.

Qualidade na educação superior; Desempenho do IGC nas regiões brasileiras; Avaliação de instituições de educação superior

Performance of Brazilian universities in view of the General Course Index (IGC)

Celina Hoffmann^I; Roselaine Ruviaro Zanini^I; Ângela Cristina Corrêa^II; Julio Cezar Mairesse Siluk^I; Vitor Francisco Schuch Júnior^I; Lucas Veiga Ávila^I

^IUniversidade Federal de Santa Maria, Santa Maria, RS, Brasil. Contacts: [e-mail], [e-mail], [e-mail], [e-mail] Federal de Santa Catarina, Florianópolis, SC, Brasil. Contact: [e-mail]

ABSTRACT

Higher education plays a fundamental role in the economic development of a country in terms of meeting society's demands. Quality in higher education has been a recurring theme in recent years, especially after the creation of National Higher Education Assessment System (SINAES), which established a global and integrative institutional assessment system in line with all Brazilian Higher Education Institutions (HEIs), responsible for producing quality measurement indicators such as Indicator of Difference between Expected and Observed Performance (IDD), Preliminary Course Program Score (CPC), and General Course Index IGC, which measures the overall performance of a higher education institution. Thus, this study aims to analyze the IGC of public and private universities of the five Brazilian regions in order to describe the performance of HEIs by region, identify possible intraregional and interregional discrepancies, and suggest opportunities for improvement. The results showed that public universities outperformed private ones in all regions, particularly the north and southeast regions. Regarding variability, private universities had the best performance in the center-west and north of Brazil. However, a thorough assessment of the performance of HEIs by region requires an integrative analysis of IGC results aligned with other assessment subsystems that integrate the multidimensional assessment of SINAES.

Keywords: Quality in higher education, Performance for IGC in Brazilian regions, Assessment of higher education institutions.

Introduction

An assessment is an instrument of control and improvement of performance as regards Higher Education Institutions (HEIs). According to Dias Sobrinho (2010, p.195):

[...] it is a tool capable of producing changes in curricula, in teaching methodologies, in concepts and training practices, in management, in power structures, in institutional models, in the settings of the education system.

Therefore, an assessment should be seen as an important tool for making decisions about public policies, as well as changing and improving the quality of each HEI within their working reality.

Quality in the education scenario has been a recurring theme in recent years, often linked to the issue of institutional assessment systems. Burlamaqui (2008), in a bibliometric study, pointed out that the concept of quality as seen by an institution must include the notions of multidimensionality and complexity, because they are inherent features to the environment of a HEI. Burlamaqui also advocates the use of both quantitative and qualitative data for measuring institutional performance, as this will enable an integrated view of the multifaceted reality of HEIs.

According to Brazil's higher education census 2010, there is total of 2,378 HEIs, most of which are private (88.3%), followed by state (4.5%), federal (4.2%) and municipal (3%) institutions. However, federal universities have the highest average enrollment rate (9481.4) because of the number of federal institutions, while private HEIs have 2256.6 enrollments per institution (INEP, 2012). These figures reflect the number of higher education institutions that play a key role in the socioeconomic development of Brazil by educating training professionals who will work in the labor market; they are classified as universities, university centers, and colleges.

The Ministry of Education (MEC), by Law No. 10.861, in 2004, established the National Higher Education Assessment System (SINAES). It is a complex and comprehensive institutional assessment system based on self-assessment, external assessment, teaching conditions, and information tools such as the higher education census and a registration completed by HEIs.

SINAES is guided by indicators such as the Preliminary Course Program Score (CPC) and the General Course Index (IGC), which support assessment processes in loco and provide course program scores (CC) and Institution scores (IC).Institutinaly, they are considered as measures of quality in Higher Education (INEP, 2011).

These indicators play an important role in guiding public policy initiatives for higher education. According to Burlamaqui (2008), the use of indicators is advantageous because it displays information that can be accessed by society at large, especially the users of the system. Having acknowledged the importance of assessing higher education and understanding the instruments, indicators and results that arise from such assessment, this study aimed to analyze IGC values of public and private universities located in the five Brazilian regions.

In this context, it should be noted that the quality measurement indices used by INEP (National Institute for Educational Studies and Research) are not widely accepted by the academic community that is dedicated to studies on higher education. On the contrary, these indicators, including IGC, have been subjects of fierce polemic since they were created, according to Schwartzman (2008, p.20). In criticism of CPC, which is an important indicator for the composition of IGC, he argues:

It is not legitimate because it was not created with the participation and involvement of relevant sectors of the Brazilian higher education community, which were surprised by their announcement.

Another view expressed by a wave of authors considers the use of quality indicators as a reductionist view of the higher education quality assessment system, as states Dias Sobrinho (2008, p.821): "[...] as if the numbers, grades, indices could be the assessment itself and account for the complexity of the educational phenomenon". While neglecting to consider aspects such as identity, context, and cultural factors inherent to each HEI assessed.

It is worth mentioning that this study discusses the view of the controversy aroused over such numbers in the theoretical background below.

Theoretical Background

In order to elucidate the objective proposed by this study, it is necessary to discuss the conceptual topics that will contextualize the issue presented and provide the parameters required for the analysis of results. Thus, the following constructs regard: Brazilian higher education assessment; institutional assessment and quality in higher education; higher education quality assessment System, and GCI as a quality indicator for HEIs.

Quality Assessment of Higher Education in Brazil

The history of higher education assessment in higher education institutions began in 1993 with the Institutional Assessment Program of Brazilian Universities (PAIUB), prepared by the National Assessment Committee (CNA) with the advice of the Secretary of Higher Education (SESu), and whose proposal was forwarded by the National Association of Presidents of Federal Higher Education Institutions (ANDIFES) to the Ministry of Education and Culture (ZANDAVALLI, 2009).

The design of the proposed assessment relied on a wide representativeness of the academic community, with several delegates from various universities actively participating in the process. CNA was comprised of various entities involved in the management of higher education, such as: Forum of Deans of Undergraduate Studies; Forum of Deans of Research and Graduate Studies; Forum of Deans of Planning and Management; Forum of Deans of Extension Studies; National Association of Deans of Higher Education in Federal Institutions (ANDIFES): National Association of Private Universities(ANUP); Brazilian Association of State and Municipal Universities (ABRUEM) and the National Association of Catholic Schools (ABESC) (BRASIL, 1993, apud Zandavalli, 2009).

The Program of Institutional Assessment (PAIUB) was created in order to find some latent demands at the time. The idea was that by means of institutional assessment there would be the possibility of continuous improvement of academic performance; serve as a tool for planning and management of HEIs, as well as provide a systematic process of social accountability, assuming that education is seen as a public good, supported by public funds and, therefore, by the whole society. The main objective of PAIUB was to analyze and improvement of the academic and sociopolitical project of HEIs, promoting the continuous improvement of quality and adequacy of institutional actions.

The initiatives proposed by PAIUB served as support for preparing the National Higher Education Assessment System. Dias Sobrinho (2002 apud Zandavalli 2009, p.421), states the importance of the program for the:

[...] fact that's a collective open work, which contemplates plurality, creates consistent theoretical and practical bases for achieving socially constructed goals and is unequivocally pedagogical and formative.

In addition, it has a structure formed by three articulated processes: internal assessment (self-assessment of subjects and hetero-assessment of structures, processes, and colleagues); external assessment (performed by groups of peers in the academic community), and reassessment (critical thinking on assessment processes).

One of the changes brought about by Law No. 9131 of November 1995 , the National Education Guidelines and Framework Law (LDB) among other changes, the creation of the National Education Exam, the so-called "Provão" ("Big Test"), in order to assess knowledge and skills acquired by students by the time they have finished their undergraduate studies. In addition, there was an intention of using the results to promote initiatives for improving the quality of education.

The creation of SINAES is closely associated with the main aspects of PAIUB, especially as regards the experience gained in the field of institutional assessment applied to the context of higher education, as stated by Ristoff and Giolo (2006, p.197):

In fact, the new National Higher Education Assessment System has adopted many of the principles and guidelines of PAIUB, e.g., formative commitment of assessments, comprehensiveness, organic integration of self-assessment with external assessment, continuity, active participation the academic community, respect for institutional identity and acknowledgment of the diversity of the system. Unlike PAIUB, however, Sinaes did not adopt the principle of voluntary membership. Abiding by the law established for Sinaes, and in line with the provisions of the Constitution, LDB and PNE

As explained by Zandavalli (2009), understanding such antecedents is crucial to identify the advances and retreats of SINAES, whose role was that of restructuring the assessment model of Brazilian higher education while having the implicit challenge of bringing together instruments and assessment spaces, thus overcoming fragmentation by organizing means of assessment.

Institutional Assessment and Quality in Higher Education

According to Ribeiro (2012), society has increasingly demanded government accountability of the set of services provided by the State. Within this demand, it is included the supply and assurance of quality in higher education, whose tasks of supervision, error correction, and publication of major results are critical. The right to education, ensured by the State, does not suffice to guarantee its performance in the context of a country's strategic development plan. Some norms consistent with current reality must also be ensured, for example the search for benchmarks that address the principles of quality (INEP, 2009).

The term quality is not restricted to one single concept capable of considering all the dimensions that it can reach, but it may be associated with the conformity between the expectation and the result achieved and, moreover, it may be linked to the perception of the subject whose task is to judge or assign levels of value to a given characteristic or phenomenon.

In this context, Júnior (2009, p. 259) states:

[...] what is considered to be quality by academics may conflict with the quality sought by governments, with quality as perceived by society or quality that meets the demands of the productive sector.

Thus, the quantitative or qualitative variables are supported by measurable indicators so that there is no predominance of subjectivity in the assessment process, because the latter has to be objectively based on compliance, standardization and fairness.

According to Silva (2008), the role of the term "quality in education" is that of distinguishing one or more characteristics considered as superior or of excellence, and setting them as a target to be achieved. However, the author emphasizes that the meanings of quality may vary in two directions: from object to object and according to the historical context, i.e., references about quality change over time. This way, what was seen as a criterion of excellence in the past – for example "discipline" and "rigor" – is not such a valued aspect today.

According to Dal Magro and Rausch (2012, p. 432): "development and quality in teaching also depend on continuous assessment in the teaching, administrative, and structural processes". In this sense, the guidelines of SINAES include such perspectives, because the regulatory mechanisms of the state are organized with the aim of improving academic quality and institutional management (INEP, 2009). Viewed from the standpoint of market logic, HEIs lie within in a competitive environment where demand from a greater number of students reflects the excellence of the services they offer (DAL MAGRO; RAUSCH, 2012).

Thus, competition for improving the quality of higher education occurs in the macroeconomic scenario, where competition takes place between developed countries that produce technological innovations with countries that consume and reproduce the technology created previously. It also occurs in microeconomic perspective, with direct competition between HEIs as regards their performance as regards number of students, courses offered and places available, faculty training, infrastructure conditions, scientific and technological production, etc.

Reflection upon quality in higher education inevitably rests on the notions of efficiency and performance, consistent with the business logic that associates quality with the concept of production whereby students are the inputs, the interaction between teachers and students where learning takes place is effectively the "process", and students who are about to finish their higher education studies are the output.

In this sense, Burlamaqui (2008, p.138), agrees with this perspective, except for the stigma that is created around education as a process. He claims: "it is observed that teaching corresponds to a process permeated by various aspects or variables that, in the end, will bring some result (product)". This results in the use of quality measuring indices which, when applied in each step of the process, become amply justified insofar as there are strengths and weaknesses in this context, and actions for error correction and improvement can, thus, be planned.

In the context of higher education management from the perspective of business logic, Sander (2007) lists four trajectories and their respective education management models under the concepts of efficiency, efficacy, effectiveness, and cultural relevance as a type of evolution of education management. These would be a way to run the progressive process of education. At this point, it is relevant to consider the concept of efficiency in education, which is similar to the technocratic vision found in enterprises. The author acknowledges efficiency as a criterion of economic performance that accentuates the extrinsic and instrumental characteristics of the organizational context and their respective activities, in which the individuals involved, are guided by economic logic, instrumental rationality and operational productivity. This concept is very similar to the current practice of quality indices being used by the higher education assessment system, which Sander repudiates because he states that the concept of efficiency in education is not consistent with the subjectivity and ethics that are inherent to educational practice.

Higher Education Quality Assessment System

SINAES started with the enactment of Law No. 10,861 on April 14, 2004, in order to consolidate the assessment of Brazilian higher education. It was established as a state policy, which corresponds to the adoption of initiatives and assessment processes included in the system, regardless of the election of new authorities or the administrative sphere. It includes both public and private institutions in order to improve the quality of higher education.

According to Dias Sobrinho (2010), in its initial conception, SINAES was based on the assumptions of assessment and global and inclusive education, geared towards designing a higher education quality assessment system. This initiative is structured under the common sense that the task of higher education quality assessment is complex enough; hence it should not be restricted to a single evaluative dimension. Thus, SINAES proposed the integration between various assessment tools in order to encompass the plurality of variants of quality indicators that HEIs are supposed to satisfy.

SINAES consists of three main pillars: institutional assessment, assessment of course programs and performance assessment of undergraduate students, the latter supported by the administration of the National Student Performance Exam (ENADE).Whereas the first two pillars are monitored by processes of local assessment (POLIDORI, 2009). In this sense, it is guided by the following assessment dimensions: mission and institutional development plan, policies on teaching, research, and undergraduate, graduate and university extension activities; social corporate responsibility, communication with society; personnel policies; administration and institutional organization; physical infrastructure, planning and assessment policies for student services and financial sustainability (BRASIL, 2004).

In this context, SINAES seeks to ensure, among other things, the articulation of the internal and external, individual and global dimensions, as well as take into consideration the assumptions of both qualitative and quantitative methodology. This principle of SINAES attempts to meet the demands of a higher education quality assessment system that addresses the complexity of the subject and of the assessment process, the participation of their respective agents of governance, institutional, learning, administrative and social dimensions (BRAZIL, 2004.)

However, the development and establishment of performance indices aroused great controversy over the theme and challenged the whole meaning of SINAES, which now has a distorted conception in the eyes of the academic community. In this context, Ribeiro (2012) reports that in the early months of 2008, the academic community learned about the changes in the philosophy of SINAES and the role of the state from that moment onwards through the media.

There was controversy around the creation of two indices: the Preliminary Course Program Score (CPC), regulated by Normative Rule No. 4 of August 5^th, 2008, preceded by Normative Rule No. 40 of 2007, which evaluates undergraduate courses, and the General Course Index (IGC), regulated by Normative Rule of 12 September 5^th, 2008, also preceded by Normative No. 40 of 2007, which evaluates the performance of the institution as a whole (INEP, 2011). In both indicators, the main base of calculation came from ENADE, which is an instrument aimed at measuring the academic performance of undergraduate students.

The adoption of such indices for measuring the quality of Brazilian higher education caused lively controversy about the subject, and several criticisms spread among researchers and scholars. According to Polidori (2009), the fact that the indicators are based on student performance through ENADE results shows a transgression of SINAES itself, which prioritizes only one pillar of its whole systemic view. In this sense, Barryero (2008, p.867) argues about the harm caused by the creation of the CPC and IGC indices because it "[...] seems to lead us back to the time of the rankings, market assessments and media simplifications, closer to advertising visibility rather than true quality assessment" [our translation].

In this sense, Dias Sobrinho (2010, p.216) suggests that the creation of such indices changed the original proposal of SINAES, which was originally created as a systemic process of higher education quality assessment. As the author says:

Although the original proposal of SINAES focused on the ideas of system, focused on institutions in particular and repudiated the practice of rankings, it has not fully occurred in practice.

This argument is reaffirmed by Ribeiro (2012, p. 307):

In fact, SINAES is showing signs that it is exhausted and has not complied with the provisions, contrary to the expectation that SINAES was going to have a central role in the regulation.

One of the major misrepresentations of SINAES listed by the authors is the disregard of institutional assessment in favor of the publication of IGC outcomes: 1) the non- -consideration on the institutional assessment for the divulgation of IGC; 2) the overlap of the results of ENADE that comprise the calculation of such indices, and as a consequence; 3) the induction for preparing and disseminating the rankings, which are unable to portray reality as it does not consider the identity and institutional specificities and the agents involved, as well as disregards the social and pedagogical training of university citizens.

Limana (2008, p.872) criticizes the indices even more severely regarding the indices:

[...] only good for giving a scientific veneer to an imbroglio that means absolutely nothing in terms of higher education quality assessment, and only makes Brazilian society confused with false rankings of excellence [...]

While it is the role of SINAES itself to disseminate such results to society, the controversy seems to lie not only in the method for calculating these indicators, but also in overvaluing the ranking and the publication of the results, whereby the discrepancies among institutions become evident. However, it is not an objective of this paper to discuss the philosophical and pedagogical design of such indices, and even less so to associate this theme with a historical-political approach. Based on the existence and availability of such indices, this study aims to analyze the values of the IGC continuum of universities in the Brazilian regions, focusing on quantitative methods of analysis.

The General Course Index as a Quality Indicator for HEIS

Despite such controversy surrounding indices drawn from the instruments of SINAES, it is recognized that it is important to concurrently use qualitative and quantitative data for evaluating performance, as they are both able to support the possible conclusions. The calculation of the preliminary score of a course is essential for producing IGC, so it is pertinent to understand what is relevant to compose CPC.

According to INEP's manual of quality indicators 2011, the unit of observation to be considered in this index corresponds to the undergraduate course, and the calculation is made with the following quality indicators: information on infrastructure, teaching-learning resources and faculty, performance achieved at ENADE by first-year and final-year students, and the results of the Indicator of Difference between Expected and Observed Performance (IDD).

IGC, in turn, is published annually and is the weighted average of the scores assigned to undergraduate and strictu sensu graduate courses. For calculation purposes, CPC values are used for scoring undergraduate courses, while for graduate courses, the scores assigned by Coordination for the Improvement of Higher Education Personnel (Capes) are converted. The weighting is based on the number of students in the undergraduate, masters and PhD programs. The result shown is a continuous variable in the range between 0 and 5, and for the purpose of classification of HEIs, the results are converted into discrete values from 1 to 5, according to Table 1. According to INEP (2011), this indicator will serve as a guiding reference for institutional assessment committees.

The calculation of IGC uses CPC values of the three year-period prior to the year of observation, with weighting given by the number of enrollments in the corresponding years. The three-year period is taken into account because of the results of ENADE, which is administered every three years for each area of knowledge and serves as the basis for calculating CPC values. In addition, for graduate courses, scores given by Capes are used for the three-year period regarding the year of calculation.

According to INEP (2011), Normative Rule No. 40, 2007, republished in 2010, establishes that the assessment results in SINAES evaluative cycle should be based on quality indicators referenced on 1-5 scale. The status of satisfactory results corresponds to results equal or above 3 points. Values below 3 points may be subject to notification. For institutional assessment, Burlamaqui (2008) says that for variables or statistics to be relevant, they should be able to influence the outcome; moreover, if two variables show causality or interdependence, they should identify flaws or defects that support the initiatives of the decision-making process and possible adaptations aimed at increasing and improving the quality of HEIs.

Method

According to Normative Rule No. 40 of 2007, established by the Ministry of Education, universities are defined as institutions for multi-curriculum training of higher education professionals, research, extension activities and domain and cultivation of human knowledge. They must have at least a third of faculty with Master's or Doctoral degrees, and one third of the faculty working on a full time basis. These characteristics are conditions for regulatory purposes; however they also represent quality indicators, as shown in the composition of CPC, referenced earlier.

According to INEP (2012), the number of universities has increased from 156 in 2001 to 190 in 2010, with 54.3% of total enrollments, considering the number of colleges and universities. In this context, the purpose of SINAES is that of making a diagnostic, formative and regulatory assessment of higher education institutions. For this reason, it is important to analyze the population of Brazilian universities.

The complexity inherent to the context of higher education requires the use of several sources of information that will make up an indicator. In this case, the composition of IGC is associated with other indicators such as CPC, which, in turn, is dependent on a number of factors associated with undergraduate programs. In addition, the composition IGC requires the scores assigned by Capes to graduate courses. In this sense, IGC can be considered a comprehensive indicator to assess institutional performance, as it encompasses several assessment items.

The present study is descriptive because it consists in the observation, analysis and characterization of a certain reality in order to aggregate information about facts or investigated phenomena and to establish possible relationships between variables (GIL, 2008). The method of collection and analysis of data used the assumptions of quantitative research, based on the positivist paradigm, rationality prevails and the methods used are highly quantitative, i.e., based on figures that simply attempt to represent a temporal reality observed (GOMES; ARAÚJO, 2005).

Data were collected from the excel spreadsheet available on INEP's website with data for the IGC continuum/2011 of a population comprised of 221 universities sorted by their respective geographic regions and administrative categories (public and private). The software statistica 9.1 was used for a descriptive analysis of IGC in HEIs corresponding to the five Brazilian regions: South, Southeast, Midwest, Northeast, and North. In addition, Student's t- test was used in order to check the difference between the means of groups of public and private universities in each region, considering a 5% level of significance.

The results are not intended only for comparisons, which are often seen as a discriminatory practice, because reality differs across the five Brazilian regions, as argued by several authors cited in the theoretical framework of this study. Their purpose is also that of mapping patterns of regional identities, taking IGC as the point of observation, as an index of representativeness of the performance of HEIs. Our goal is, through this, help to identify interregional and intra-regional discrepancies, with weighting of the public and private categories.

Results

The results allow the characterization of the distribution of IGC values in the five Brazilian regions, considering the private and public administration, as shown in Table 2.

In terms of the values of the average IGC in the South, it was observed that public universities outperform private ones, and they also have both the highest and the lowest IGC in the region. This is confirmed in the values of standard deviation and coefficient of variation, which are greater for public than for private HEIs. A significant difference was observed between the mean values of IGC (p = 0.01) when public and private institutions were compared. The general analysis showed that the average in the South is lower than that of the public sphere and higher than that of the private sphere, given that public universities scored twice as much as private ones.

The Southeast region has the largest number of both public and private universities, and the results show better performance of public universities. Likewise, they have higher average IGC values compared to private universities. The difference between the averages of the groups was significant (p-value < 0.001). The values corresponding to the standard deviation and coefficient of variation were also higher for public universities than private ones; this represents a greater heterogeneity among the IGCs of this group.

Just like in the above-mentioned regions, public universities outperformed private ones for the average IGC in the Midwest. Thus, the maximum IGC value of this region was achieved by a public university, and the lowest value was found in the group of private universities. However, there was no statistically significant difference in the means of IGC between groups of public and private universities (p-value = .16). As for variability, private universities were more homogeneous for IGC, showing lower values for standard deviation and coefficient of variation when compared to the group of public universities.

In the Northeast, the overall average is very similar to that of the South. Like in the other regions, public universities had the best results for average IGC, with the highest value, while the lowest value was found in the group of private universities. It was found that there is significant difference between the average IGC values for public and private universities in the region (p-value <.01). In terms of variability, private universities show higher relative dispersion, and demonstrated higher value for coefficient of variation.

In the North region, the overall average IGC is higher than in the South, Midwest and Northeast; however, standard deviation is higher. As for administrative categories, public universities performed better than private ones for average IGC, and the highest IGC value belonged to a public university, while the lowest value was found in the private category. A significant difference was also observed between the average IGC values (p <0.001). However, private universities had a lower coefficient of variation, which indicates greater homogeneity in this group.

The analysis of the results shows that public universities outperform private ones in all regions analyzed. The best performance for IGC was observed in the North region, and the Southeast Region had the second best performance for average IGC. The other regions showed lower results, but there was a certain balance between the average IGC values, presented in descending order: South, Northeast and Midwest.

Regarding variability, the highest homogeneity was clearly seen in the Midwest and North, both for the group of private universities, while the remaining regions showed higher heterogeneity for both private and the public HEIs. Public universities had the greatest variability observed in the Midwest.

The analysis of IGC helps identify opportunities for institutions to improve, and stimulate public policies in this sector. Thus, institutional assessment should be integrated into the management of HEIs, as stated by Junior (2009, p. 265):

For institutions to be prepared to face contemporary challenges, it is essential that their reality, their virtues, capabilities and limitations are known by their members.

The development of higher education must include the pursuit of regional equity, in terms of both the performance of quality indices and the subjective criteria of institutional assessment, while respecting cultural, economic and demographic differences. Considering the theoretical framework of IGC and other indicators addressed in this study, including the controversy about the interpretations of its values and dissemination of the outcomes in the media, the results of this research point to the reflection precisely on the basis of such a discussion. The arguments against the use of such indicators emphasize the importance of considering the differences of culture and identity among institutions to avoid discrimination and stigmas about HEIs and their respective regions.

The results of this investigation reflect a revealing reality, precisely, taking into account the highest IGC average was observed in the North, a fact that is particularly worth of notice: the North had a better performance than the other regions. Regions such as the North are historically stigmatized due to the performance of their socio-economic indicators are generally below the national average. Moreover, this region doesn't present inherent tradition in HEIs as those in the South and Southeast regions.

Thus, this result may be questionable: can IGC really reflect the quality of a given HEI? In other words, is the value calculated for IGC able to reflect reality? The values of Table 1 show a reality so far unknown as they reveal a better performance for HEIs in the North and Northeast over the South and Southeast, while common sense acknowledges exactly the opposite. This creates the need for formulating explanatory hypotheses for this result, i.e., the need to map the causes or evaluative components that led to such revealing results. On the other hand, it can be said that the results of this study can break paradigms and question the basis of the arguments against IGC, whose composition is assumed to be unbiased, and that the results reported did not cause discrimination between HEIs and regions, as it highlighted historically stigmatized regions.

Graph 1 shows some uniformity in the performance of the Brazilian regions for IGC.

Because Brazil is considered to be a country of huge dimensions, there are all types of discrepancies across regions. However, IGC establishes minimum performance criteria for HEIs and the quest for a uniform and constant performance should be followed closely and monitored. Higher education, seen through the prism of a system, assumes coexistence and interaction between different institutions whose point in common is academic organization (universities); therefore, it should not be restricted to its own reality.

The search for equity in development needs the feedback of institutional assessment that also occurs with the interaction among peers. Figure 1 reveals some hegemony among Brazilian regions, although the South, Midwest and Northeast are slightly below the national IGC average. However, the average IGC in the Southeast and North have was above the national average, with greater variability observed in the Southeast.

Final Remarks

Based on the distinction between public and private universities, especially better results of the public category for average performance of IGC, the composition of this quality index has to be taken into account. It is formed by weighted average of CPCs, which, in turn, contain assessment sub-items, e.g. number of professors with Master's and doctoral degrees, number of professors working on full-time or part-time bases, infrastructure, didactic-pedagogic organization, scores of first-year and final-year undergraduate students at ENADE, and the Indicator of Difference between Observed and Expected Performance (IDD) (INEP 2011).

It should be pointed out that the majority of the items that compose CPC favor public universities, because it is easier for them to comply with the requirements of faculty qualification and working arrangements. The superior performance for GCI in public universities can be explained by the weighting made by Bittencourt et al. (2010) in a study on the items that compose the CPC values collected in 2008. The authors found that the public sector outperformed the private one in most of the items that compose the CPC values, while the strengths of private HEIs relied only on infrastructure and didactic-pedagogical organization, with both items accounting for only 10% of CPC.

Such consideration was envisioned by Cunha (2004, p.795) who, in an article on the subject, argues that the development of higher education occurs unevenly between the public and private sectors:

The most dramatic effect of this process is teachers' improvisation in the private sector, which produces negative effects on the quality of education, at the undergraduate and graduate levels.

A possible solution to minimize this disparity would be to increase the supply of places in graduate courses (Master's/PhD), in order to balance the labor market for these professionals and, thus, offer conditions for private universities to meet these requirements.

In summary, all regions showed superior performance of the public sphere in comparison with the private category, with average values showing significant differences between the public and private classes in some regions.

The Northern region had the best performance for average IGC (2.91), which sheds light on the perspective of the performance of higher education across the Brazilian regions. Therefore, it is seen some degree of impartiality is found in IGC values in so far the components that compose IGC are calculated objectively. The next best results were achieved by the Southeast (2.84), the Northeast (2.76), the South (2.77) and the Midwest (2.72). The South had the lowest absolute variability (0.52), followed by the Northeast (0.58), the Midwest (0.59), the North (0.60) and the Southeast (0.65). As for individual performance, it is clear that the Southeast region has the university with the highest IGC (4.28), followed by the Midwest (4.21) and the North (4.07), all of which are public universities. The university with the worst performance is private and located in the Southeast (1.44), followed by public universities of the Southeast, again, (1.65) and the South (1.57).

Higher education plays a fundamental role in the economic development of a country, regarding society's demands and the education and training of qualified professionals to monitor the constant changes arising from the economy, technology and information. This is the reason why it is important to assess institutions and monitor their indicators. As explained by Ribeiro (2012, p.177):

HEIs need to fulfill the duty of continuous quality pursuit in academic performance, the constant improvement of planning and university management [...].

In this context, this paper aimed, through IGC, which gathers information about HEIs, to contribute to studies on higher education by highlighting the implications of IGC outcomes when interregional and intra-regional analyses are performed and discussing characteristics such as performance and variability, considering both public and private HEIs.

This work was motivated by the possibility of addressing performance of higher education with a focus on a quantitative approach that can support qualitative analyses. Thus, this study sought to collaborate with preliminary results on the comparison of quality in higher education across Brazilian regions, based on the analysis of IGC. However, there is a real need for further studies that can look into explanatory hypotheses for the results of this study. Therefore, an assessment on the consolidated performance of HEIs by region should occur seamlessly, when the analysis of the results of IGC are aligned with other assessment subsystems that integrate the multidimensional assessment by SINAES.

References

• BARREYRO, Gladys Beatriz. De exames, rankings e mídia. Avaliação, Campinas; Sorocaba, v. 13, n. 3, p. 863-868, nov. 2008.
• BITTENCOURT, Hélio Radke et al. Mudanças nos pesos do CPC e seu impacto nos resultados de avaliação em universidades federais e privadas. Avaliação, Campinas; Sorocaba, v. 15, n. 3, p. 147-166, nov. 2010.
• BRASIL. Lei n. 10.861, de 14 de abril de 2004: Institui o Sistema Nacional de Avaliação da Educação Superior (Sinaes) e dá outras providências.
• BURLAMAQUI, Marco Guilherme Bravo. Avaliação e qualidade na educação superior: tendências na literatura e algumas implicações para o sistema de avaliação brasileiro. Estudos em Avaliação Educacional, v. 19, n. 39, jan./abr. 2008.
• CUNHA, Luiz Antônio. Desenvolvimento desigual e combinado no ensino superior estado e mercado. Educação e Sociedade, Campinas, v. 25, n. 88, p. 795-817, out. 2004.
• DAL MAGRO, Cristian Bau; RAUSCH, Rita Buzzi. Plano de desenvolvimento institucional de universidades federais brasileiras. Administração: Ensino e Pesquisa, Rio de Janeiro. v.13, n.3. p. 427-454, jul./set. 2012.
• DIAS SOBRINHO, José. Avaliação e transformações da educação brasileira (1995 2009): do Provão ao SINAES. Avaliação, Campinas; Sorocaba, v. 15, n. 1, p. 195-224, mar. 2010.
• ______. Qualidade, avaliação: do SINAES a índices. Avaliação, Campinas; Sorocaba, v. 13, n. 3, p. 817-825, nov. 2008.
• GIL, Antônio Carlos. Como elaborar projetos de pesquisa 5. ed. São Paulo: Atlas, 2008.
• GOMES, Fabrício Pereira; ARAÚJO, Richard Medeiros de. Pesquisa quanti-qualitativa em administração: uma visão holística do objeto em estudo. In: SEMINÁRIOS EM ADMINISTRAÇÃO, 8, 2005, São Paulo. Anais.. São Paulo: FEA/USP, 2005.
• INSTITUTO NACIONAL DE ESTUDOS E PESQUISAS EDUCACIONAIS ANÍSIO TEIXEIRA (INEP). Sistema Nacional de Avaliação da Educação Superior: da concepção à regulamentação. 5. ed., Brasília: INEP, 2009, 328 p.
• ______. Manual dos indicadores de qualidade Brasília: INEP, 2011.
• ______. Censo da educação superior: 2010 – resumo técnico. Brasília:INEP, 2012, 85 p.
• JUNIOR. Nelson de Abreu. Sistema(s) de avaliação da educação superior brasileira. Caderno Cedes, Campinas, v. 29, n. 78, p. 257-269, maio/ago. 2009.
• LIMANA, Amir. Desfazendo mitos: o que estão fazendo com o SINAES. Avaliação, Campinas; Sorocaba, v. 13, n. 3, p. 869-873, nov. 2008.
• POLIDORI, Marlis Morosini. Políticas de avaliação da educação superior brasileira: provão, SINAES, IDD, CPC, IGC e outros índices. Avaliação, Campinas; Sorocaba, SP, v. 14, n. 2, p. 267-290, jul. 2009.
• RIBEIRO, Elisa Antonia. As atuais políticas públicas de avaliação para a educação superior e os impactos na configuração do trabalho docente. Avaliação, Campinas; Sorocaba, SP, v. 17, n. 2, p. 299-316, jul. 2012.
• ______. Convergências e divergências das práticas de autoavaliação institucional em IES pública e privada na percepção da comunidade acadêmica. Estudos em Avaliação Educacional São Paulo, v. 23, n. 51, p. 168-189, jan./abr. 2012.
• RISTOFF, Dilvo; GIOLO, Jaime. O SINAES como Sistema. RBPG - Revista Brasileirade Pós-Graduação, Brasília, v. 3, n. 6, p. 193-213, dez. 2006.
• SANDER, Benno. Administração da educação no Brasil: genealogia do conhecimento. Brasília: Liber Livro, 2007.
• SCHWARTZMAN, Simon. O "conceito preliminar" e as boas práticas de avaliação do ensino superior. Estudos: Revista da Associação Brasileira de Mantenedoras de Ensino Superior, v. 26, n. 38, p. 9-22, dez. 2008.
• SILVA, Vandré Gomes da. A narrativa instrumental da qualidade na educação. Estudos em Avaliação Educacional, v. 19, n. 40, maio/ago. 2008.
• ZANDAVALLI, Carla Buzato. Avaliação da educação superior no Brasil: os antecedentes históricos do SINAES. Avaliação, Campinas; Sorocaba, v. 14, n. 2, p. 267-290, jul. 2009.

¹

, all Brazilian HEIs, not only the federal ones, must participate in the assessment processes that compose the system [our translation].

Publication Dates

• Publication in this collection
  29 Apr 2014
• Date of issue
  Sept 2014

History

• Received
  28 June 2013
• Accepted
  30 Sept 2013