Contributions to the improvement of ENADE indicators: a study with coordinators of undergraduate courses

Abstract

This study purpose to identify the perception of the coordinators of the undergraduate course regarding strategies to improve indicators at Enade. To this end, a survey was applied to 110 coordinators of undergraduate courses in higher education institutions (HEIs). The collected data were analyzed using the techniques of descriptive statistics and calculation of information entropy. As for the results, in relation to the level of importance attributed to the indicators, there was a relevant importance attributed by the respondents to the proposed indicators. All indicators were considered by most respondents to be very important, so that the averages presented were higher than four points. Regarding the level of execution, levels below the level of importance were observed, however, the averages obtained show satisfactory levels of execution. It is concluded that the proposed model of indicators was validated by the coordinators, serving as a basis for the management of the courses.

Keywords:
higher education institutions; undergraduate course coordinators; ENADE

Resumo

Este estudo teve por objetivo identificar a percepção dos coordenadores de curso de graduação quanto às estratégias contributivas para melhorias dos indicadores no Enade. Para tal, foi aplicada uma survey a 110 coordenadores de cursos de graduação de instituições de ensino superior (IES). Os dados coletados foram analisados a partir das técnicas da estatística descritiva e cálculo de entropia da informação. Quanto aos resultados, em relação ao nível de importância atribuído aos indicadores, constatou-se relevante importância atribuída pelos respondentes aos indicadores propostos. Todos os indicadores foram considerados pela maioria dos respondentes como muito importantes, de forma que as médias apresentadas foram superiores a quatro pontos. Em relação ao nível de execução, observou-se níveis inferiores de importância, entretanto, as médias apuradas mostram níveis satisfatórios de execução. Conclui-se que, o modelo de indicadores proposto foi validado pelos coordenadores servindo de base para a gestão dos cursos.

Palavras-chave:
instituições de ensino superior; coordenadores de cursos de graduação; ENADE

Resumen

Este estudio tuvo como objetivo identificar la percepción de los coordinadores de cursos de pregrado sobre las estrategias de contribución para mejorar los indicadores en Enade. Para ello, se aplicó una encuesta a 110 coordinadores de cursos de pregrado en instituciones de educación superior (IES). Los datos recolectados fueron analizados mediante estadística descriptiva y cálculo de entropía de información. En cuanto a los resultados, en relación al nivel de importancia atribuido a los indicadores, se verificó que los encuestados atribuyeron importancia relevante a los indicadores propuestos. Todos los indicadores fueron considerados por la mayoría de los encuestados como muy importantes, por lo que los promedios presentados fueron superiores a cuatro puntos. En cuanto al nivel de ejecución, se observaron niveles inferiores al nivel de importancia, sin embargo, los promedios calculados demuestran niveles de ejecución satisfactorios. Se concluye que el modelo de indicadores propuesto fue validado por los coordinadores, sirviendo de base para la gestión de los cursoss.

Palavras clave:
instituciones de educación superior; coordinadores de cursos de pregrado; ENADE

1 Introduction

Assessment is part of everyday education and is present throughout the pedagogical process, from basic to higher education. It is not restricted only to self-assessments but also expands externally through third-party assessments, whether peers or the government itself (GONTIJO, 2014). According to Gontijo (2014), educational evaluation occurs at different levels; for example, the evaluation carried out in the classroom refers to learning; institutional evaluation evaluates the pedagogical process; large-scale assessment encompasses education systems and is intended to guide educational public policies, etc.

Thus, Educational evaluation fulfills the role of disclosing to civil society the levels of development and the quality of education offered (COSTA, 2009). Thus, evaluation cannot be considered as an end, but as part of a set of policies that contribute to a process of revaluation of education and the development of society (FALLEIROS; PIMENTA; VALADÃO JÚNIOR, 2016).

In Brazil, there is the National System for the Evaluation of Higher Education (Sinaes), formed by three main components: the evaluation of institutions, courses and the performance of higher education students. Sinaes evaluates aspects related to teaching, research, extension, social responsibility, student performance, institution management, faculty and facilities. With this, it allows higher education institutions (HEIs) evaluated to dialogue and participate in the evaluation mechanism (POLIDORI, 2009), given that Sinaes has a series of complementary instruments such as self-evaluation, external evaluation, the National Performance Examination of Students (Enade) etc. Enade, for example, has been presented as an essential instrument in the management of undergraduate courses, and its data enable HEIs to assess the need for adjustments or not in their educational processes (RISTOFF; LIMANA, [2007]).

Of the three main evaluation components proposed by Sinaes, two (course evaluation and student performance) have course coordinators as the main actors in the evaluation process. Bittencourt, Casartelli and Rodrigues (2009) point out that Enade is the most notable element of Sinaes, due to its data collection potential and, consequently, the generation of information through reports.

Given the above, HEIs need to use indicators capable of expressing their limitations and potential to improve their management process. Thus, this study sought to build a set of indicators that allow each undergraduate course to assess whether its strategic actions contribute to improving student performance in Enade. To this end, the perception of undergraduate course coordinators of HEIs linked to ACAFE (Associação Catarinense das Fundações Educacionais), which is a non-profit civil society and brings together 16 HEIs located in the State of Santa Catarina, regarding the strategies to improve the indicators in Enade.

The indicators and their strategies were proposed, considering the course evaluation instrument as the main object and those obtained in previous studies. For this, the model was divided into five sets: didactic-pedagogical indicators, indicators of adopted methodologies, evaluation indicators, faculty indicators and infrastructure indicators. Subsequently, this model was applied to ACAFE undergraduate course coordinators to identify its importance and execution to limitations and potentialities, thereby enabling better management of the course and resources.

The relevance of this study lies in highlighting the importance of the role of coordinators, in addition to expanding other study possibilities (CORDAZZO; WERNKE; ZANIN, 2021), highlighting how undergraduate course coordinators seek to implement actions and strategies capable of ensuring the permanence and supply of courses in the market, meeting the quality requirements proposed by the market, but, above all, by regulation. It is also relevant to the coordinators themselves, as it can help them better understand the Enade assessment and the assessment systems in general and their role as managers in the face of the various requirements. It is also justified by the need to meet the processes and demands imposed by law under penalty of discontinuing the course or even the institution.

2 Literature review

2.1 Evaluation of higher Education in Brazil

In Brazil, the rapid expansion of higher education at the end of the 20th century led to the need to implement evaluation systems to ensure quality levels of professional and academic training (PEIXOTO, 2009). With the growth in the number of HEIs, especially the private ones and, consequently, the reduction of the State's presence in educational funding, the exams gained notoriety and importance as quality control instruments (DIAS SOBRINHO, 2010). Thus, the evaluation of Brazilian higher education was more effective from the 1980s, going through different experiences and gradual evolution.

Polidori (2009) described the evolution of the development of higher education in Brazil in four cycles comprised in the periods: (i) from 1986 to 1992, in which several initiatives to organize an evaluation process and the existence of isolated evaluations in the country did not constitute a national assessment; (ii) from 1993 to 1995, called policy formulation and Installation of the Institutional Evaluation Program of Brazilian Universities; (iii) from 1996 to 2003, called the consolidation or implementation of the government proposal, and in which the development of the National Course Examination, the Provão, and the Evaluation of Offer Conditions, later called the Evaluation of Teaching Conditions; (iv) from 2003 to the present, called the construction of emancipatory assessment, with the implementation of Sinaes, in a proposal to develop a formative assessment that considers the specificities of HEIs in the country.

In force since 2004, by Law n. 10.861/2004, Sinaes is the current higher education evaluation system created to create an integration between evaluation and regulation (PEIXOTO, 2009). He proposed a global assessment integrating all aspects of HEIs and various application instruments to comply with their components (PEIXOTO, 2009; DIAS SOBRINHO, 2010).

Regarding evaluating students' performance, Enade was instituted with the purpose of a dynamic evaluation that proposes to consider the change and development of the student during the course (DIAS SOBRINHO, 2010). Enade is a mandatory component and is included in the students' diplomas. Data collection takes place through the following instruments: (i) test, intended to assess the student's performance concerning the syllabus provided in the course's curricular guidelines, their abilities to adjust to the demands arising from the evolution of knowledge and their skills to understand issues outside the scope of their profession; (ii) student questionnaire, designed to gather information to characterize the profile of students and the context of training processes; (iii) test perception questionnaire, intended to gather information that allows assessing the students' perception about the instrument; and (iv) the course coordinator's questionnaire, designed to gather information to characterize the profile of the course coordinator and the context of the training processes.

It is noticed that evaluation in an institution is one of the main tools for implementing and organizing educational policies and reforms. It causes changes in teaching practices, curricula, management model and structure, research priorities and the entity's social responsibility (DIAS SOBRINHO, 2010).

2.2 HEI Management and the Role of the undergraduate course coordinator

Over the last 40 years, HEI management has become based on rational models inspired by the reality of companies. However, such models did not generate compelling results, as they were institutions in which the management process involved rational, political and symbolic aspects (MEYER JÚNIOR, 2005). In this context, it was understood that the management of HEIs must be differentiated from other types of organizations for several reasons, including the mission. While the assignment of a business organization is easily identified, as they are aimed at profit, HEIs have the mission of providing teaching, research and extension (BARBOSA; MENDONÇA, 2016). Thus, the mission of an HEI is more complex and subjective, requiring more management work.

Rebelo and Erdmann (2007) point out that among HEIs, universities are more complex organizations with multiple relationships, which must be evaluated in all their specificities. Understanding corroborated by Silva and Cunha (2012), HEIs (public, private or community) are organizations composed of particularities and belong to an economic segment differentiated from business organizations. Furthermore, business techniques tend to value the quantification of results, which would prioritize the number of enrolled or graduated students (for example), but without assessing the effectiveness of the teaching-learning process (SILVA; CUNHA, 2012).

In this context, Meyer Júnior, Pascucci and Mangolin (2012) point out that HEIs are no longer inserted in a peaceful environment, where students arrive continuously and increasingly, functioning in a climate of stability. More recently, given the increase in HEIs, private institutions, for example, have inserted strategic planning to improve management as a way to guarantee short-term viability and long-term permanence.

The elaboration of a strategic plan, with a political and institutional character, serves as a reference and justification for the main actions of the institution, in addition to legitimizing the management's intentions before the academic community. However, in the conception of Meyer Júnior, Pascucci and Mangolin (2012), the practice of strategic management in HEIs has revealed a gap between formal plans and the effective practice of strategies due to the differences in contexts between business and educational organizations and, because programs and models created for companies are unlikely to be useful for HEIs. In compliance with this, Ferronato (2017) highlights that implementing Sinaes and its adaptations implies a management model for HEIs, since the system indicates actions and strategies that make up the indicators for the final evaluation. Thus, considering that an HEI is premised on the quality of education offered, managers, especially course coordinators, now significantly contribute to the institution's management. The coordinators act as catalysts for numerous processes and activities, as they immediately control the courses (BOTELHO et al., 2018).

The undergraduate course coordinator's role, duties and profile may vary according to the HEI. Bonzanini, Ferreira-da-Silva and Leite (2017) explain, for example, that in public HEIs, in addition to administrative attributions, there is a character more focused on research and academic activities, while, in private and community HEIs, functions are more characteristic of management, since these purpose at results that should contribute to their sustainability. However, regardless of the nature of the HEI, the coordinator's main attributions are to comply with the legislation, accompanied by institutional commitment, in line with the Course Pedagogical Project (CPP) provisions.

Thus, the coordinator is perceived as the articulator between professors and students in the teaching-learning process, seeking to make the course objectives viable through pedagogical mediation (DOMINGUES et al., 2011). He is the conductor of qualifying the education offered, as he must monitor the development of students, aiming to train qualified professionals. However, like the other HEI managers, the course coordinator is a professor who, according to institutional norms, can take over by appointment or election (COSTA, 2009). When assuming the coordination, he will have a new identity, that of a manager, but without ceasing to be a teacher (MARCON, 2011). Botelho et al. (2018) point out that many assume they need to gain experience in a similar position since they are professors in different areas. Those who take the role without knowledge present, at first, the behavior of professors concerned only with the classroom; those with previous experience in managerial positions, however, current behaviors characteristic of managers, in addition to those of teaching (MARCON, 2011).

Given the competitive scenario in which HEIs are inserted, improvisations or doubts regarding the activities to be developed are not allowed. Thus, more and more qualified and prepared professionals are required to occupy management positions at the university level (BARBOSA; MENDONÇA, 2016). Therefore, HEIs and course coordinators need to use indicators capable of expressing their limitations and potential to improve the management process.

Given this, this study proposed to build a set of indicators that allow each undergraduate course to assess whether its strategic actions contribute to improving students' performance in Enade. With this focus, the indicators were proposed observing as the main object of the Assessment of Undergraduate Courses instrument, which focuses on identifying the teaching conditions offered to students, in the dimensions of the faculty, the didactic-pedagogical organization and the infrastructure (BRASIL, 2004). In addition, the methodology and evaluation indicators were also observed in the postponed, according to the contributions of Zanin (2014) and Molozzi (2015). For this, the model was divided into five axes: didactic-pedagogical indicators, adopted methodologies, evaluation, faculty and infrastructure.

3 Methodological Procedures

3.1 Population and sample

The population of this research comprised 547 coordinators of the most different undergraduate courses of 14 HEIs, linked to the ACAFE System, maintained by foundations instituted by municipal law in the State of Santa Catarina. ACAFE, founded in 1974, aims to promote the integration of HEI consolidation efforts, carry out technical-operational support activities, and represent them before state and federal government bodies. It currently integrates 16 HEIs, 11 universities and five university centers (ACAFE, 2020).

For sample selection, two institutions were excluded from the population. The first is the State University of Santa Catarina, a foundation created by the State of SC, thus characterizing itself as a state public university. The second is the Municipal University Center of São José, maintained by the Municipal Educational Foundation of the city of São José, characterizing itself as a municipal public institution. The exclusions are justified by both having structural organization and management models differentiated from the others due to their public character, and by the fact that the budget comes from the Public Sector, while the others, even if instituted by municipal law, are not necessarily linked to the respective municipal governments, having their management, with independent councils.

Among the list of institutions participating in the research are: the Regional University of Blumenau; University of Contestado; University Center – Catholic of Santa Catarina; University of the Extreme South of Santa Catarina; Barriga Verde University Center; University Center for the Development of Alto Vale do Itajaí; University Center of Brusque; University of Planalto Catarinense; University of Southern Santa Catarina; the University of Vale do Itajaí; University of Joinville Region; Community University of the Chapecó Region; University of West Santa Catarina and the University of Alto Vale do Rio do Peixe.

The final research sample comprised 110 undergraduate coordinators from the 14 participating HEIs, equivalent to 20.11% of the population. The questionnaire was sent to the coordinators through an electronic link (Google docs.), via email from December 2017 to February 2018.

3.2 Research tool

The research instrument was elaborated in two parts. The first consisted of 14 questions related to the respondents' profile, and the second initially proposed 31 indicators, subdivided into five sets, related to the improvement of Enade’s indicators.

To validate the questions, the questionnaire was sent for analysis by five specialists in the area, four professors from different HEIs, and one professional linked to Ministry of Education with a management position related to Enade. Of these, only one evaluator still needs to return his suggestions. However, it should be noted that one of the evaluators who responded invited another professor who is also a reference in the area to contribute to the analysis, and with that, there were a total of five respondents.

After this first stage, a pre-test was carried out with three coordinators with experience managing undergraduate courses which were not participants in the research. This step aimed to assess whether the proposed model and how the questions were presented were adequate and easy to understand. The reviewers considered that it was following the proposed objectives and could be forwarded to the respondents.

After the steps above, the instrument was finalized with 51 questions, 14 of which were about the profile of the respondents and 37 related to the improvement of the Enade indicators. The profile questions dealt with age, gender, area and time of training, area of expertise, previous professional experience, time at the institution and acting in the course coordination, weekly workload and participation in management-related training. As for the questions about the improvement of the Enade indicators (Table 1), they were presented to verify the degree of importance assigned (Likert scale, 1 being unimportant and 5 being very important) and the degree of execution (Likert scale, 1 being not executed and 5 being executed very often).

Tabela 1
Research instrument

Indicators	Items
Didactic-pedagogical (DPI)	DPI1. Discuss the CPP of the Course with the freshmen DPI2. Discuss CPP with students throughout the course DPI3. Discuss the CPP of the Course with the graduates DPI4. Keep the professional profile and skills of the graduate in the CPP up to date DPI5. Keep the course objectives consistent with the egress profile, curriculum structure and educational context DPI6. Compose the Curriculum Structure in an interdisciplinary way DPI7. Propose curricular contents that enable the development of the graduate's professional profile (updated contents, adequate bibliographies, etc.) DPI8. Provide opportunities for curricular internships (mandatory and non-mandatory) DPI9. Carry out complementary activities to the course DPI10. Institute student support policies DPI11. Insert students in research and extension projects DPI12. Integrate students with the practical areas of action (education network, health network, companies, etc.) DPI13. Carry out practical teaching activities DPI14. Elaborate the PPC according to the National Curricular Guidelines of the course DPI15. Use Enade reports (area, course and HEI summary report produced by INEP) to define improvement strategies DPI16. Use didactic methodologies in the development of classes that provide student leadership in the development of knowledge DPI17. Carry out activities aimed at the interdisciplinary articulation of the faculty
Evaluation (EI)	EI1. Structure the evaluation questions in the format of the Enade questions EI2. Promote interdisciplinary evaluations during the course EI3. Carry out evaluations in the form of an Enade simulation EI4. Encourage teachers to give feedback to students based on the assessments carried out
Methodologies adopted (IMA)	IMA1. Present the higher education evaluation model to HEI entrants IMA2. Make students aware of their responsibility for the grade obtained in Enade IMA3. Make contact with students who will participate in Enade IMA4. Conduct specific classes preparatory for Enade IMA5. Use active methodologies as a teaching-learning method
From faculty (FI)	FI1. Keep the Structuring Teaching Nucleus active FI2. Provide the course coordinator's work schedule compatible with the number of students FI3. Improve the qualification of the faculty (higher percentage of masters and doctors) FI4. Observe the teaching staff's work regime (greater number of teachers with full-time work) FI5. Prioritize the professional experience of the faculty FI6. Stimulate the performance of the course collegiate (with faculty and student representation)
Infrastructure (II)	II1. Provide classrooms suitable for the number of students per class II2. Provide adequate amount of equipment per class II3. Provide basic course bibliography II4. Provide complementary bibliographies II5. Provide own equipped laboratories

• Source: prepared based on the Assessment of Undergraduate Courses instrument (INEP, 2017), Zanin (2014) and Molozzi (2015).

3.3 Análise dos dados

For data analysis, descriptive statistics were used, and information entropy calculation. Entropy is considered a simple but important measure, due to the amount of information it provides (ZELENY, 1982). Rocha (2010) describes the calculation of informational entropy as shown below.

Let $d_i=\left(d_i^1,d_i^2,\dots ,d_i^m\right)$ the values be normalized, where: $d_i^k=\frac{x_i^k}{x_i}$, characterizes the set D, in terms of the ith attribute. Lies $D_i={\displaystyle {\sum }_{k=1}^m{d}_i^k;i=1,2,\dots ,n}$.

The entropy measure of the intensity contrast is sought for the ith attribute calculated by $e\left(d_i\right)=-\alpha {\displaystyle {\sum }_{k=1}^m\frac{d_i^k}{D_i}}\text{Ln}\left(\frac{d_i^k}{D_i}\right)$, where $\alpha =\frac{1}{e_{\max }}>0\text{e}{\text{e}}_{\max }=\text{Ln}\left(\text{m}\right)$. It is also observed that $0\le d_i^k\le 1\text{e}{d}_i^k\ge 0$. If all $d_i^k$ are equal for a given i, then $\frac{d_i^k}{D_i}=\frac{1}{n}\text{e}\text{e}\left(d_i\right)$ assumes the maximum value, that is, ${\text{e}}_{\max }=\text{Ln}\left(\text{m}\right)$. When fixing $\alpha =\frac{1}{e_{max}}$, it is determined $0\le e\left(d_i\right)\le 1$ for all $e\left(d_i\right)\text{s}$ where this normalization is necessary for comparative purposes. In this way, the total entropy of D is defined by: $E={\displaystyle {\sum }_{i=1}^{n}e\left(d_i\right)}$. The larger e(d_i), the smaller the information conveyed by the ith attribute. If $\text{e}\left({\text{d}}_{\text{i}}\right)={\text{e}}_{\max }=\text{Ln}\left(\text{m}\right)$, then the ith attribute does not convey information and can be removed from the decision analysis. Because weight ${\lambda }^{~}{}_i$ is inversely related to e(d_i), use 1-e(d_i) instead of e(d_i) and normalize to ensure that $0\le {\lambda }^{~}{}_i\le 1\text{e}{\displaystyle {\sum }_{i=1}^n{\lambda }^{~}{}_i}=1$ Thus, the information entropy can be represented by:

${\lambda }^{~}{}_i=\frac{1}{n-E}\left[1-e\left(d_i\right)\right]=\frac{\left[1-e\left(d_i\right)\right]}{n-E}$

Next, the analyzes and discussion of the results are highlighted.

4 Analysis and Discussion of the Results

4.1 Results

Table 2 presents the profile of respondents participating in the survey.

Table 2
Respondents profile (n= 110)

Age	n	%	Gender	n	%
Up to 25 years	1	0.91	Female	61	55.45
26 and 35 years old	21	19.09	Male	49	44.55
36 and 45 years old	31	28.18
46 and 55 years old	35	31.82
Over 55 years old	22	20.00

Management experience	n	%	Time of performance in management	n	%
No	45	40.91	Less than 2 years	9	8.26
Yes - administrative management	45	40.91	2 and 5 years old	27	24.77
Yes - academic management	20	18.18	5 and 10 years old	32	29.36
10 and 15 years old	19	17.43
More than 15 years	23	20.18

HEI time	n	%	Time spent in coordination	n	%
Less than 5 years	23	20.91	Less than 2 years	26	23.64
5 and 10 years old	34	30.91	2 and 4 years old	34	30.9
10 and 15 years old	18	16.36	4 and 6 years old	18	16.36
15 and 20 years old	20	18.18	6 and 8 years old	16	14.55
More than 20 years	15	13.64	More than 8 years	16	14.55

Weekly workload dedicated to HEI	n	%	Weekly workload dedicated to course coordination	n	%
Up to 20 hours	13	11.82	up to 8 hours	7	6.36
From 20 and 30 hours	5	4.55	From 8 am to 4 pm	27	24.55
30 and 40 hours	62	56.36	16 and 24 hours	47	42.73
More than 40 hours	30	27.27	More than 24 hours	29	26.36

Highest degree	n	%	Completion time	n	%
Specialization/MBA	8	7.28	Less than 1 year	4	3.64
Master's degree	73	66.36	From 1 to 2.5 years	29	26.36
PhD degree	29	26.36	From 2.5 to 5 years	32	29.09
From 5 to 7.5 years	8	7.27
More than 7.5 years	37	33.64

Area - Undergraduate	n	%	Area - Graduate	n	%
Exact and Earth Sciences	4	3.64	Exact and Earth Sciences	3	2.73
biological Sciences	3	2.73	biological Sciences	2	1.82
Engineering	11	10.00	Engineering	12	10.91
Health Sciences	21	19.09	Health Sciences	18	16.36
Agricultural Sciences	9	8.18	Agricultural Sciences	6	5.45
Social and Applied Sciences	33	30.00	Social and Applied Sciences	29	26.36
Human Sciences	24	21.82	Human Sciences	17	15.45
Linguistics, Letters and Arts	2	1.82	Linguistics, Letters and Arts	4	3.64
Others	2	1.82	Others	4	3.64
Not identified	1	0.91	Not identified	15	13.64

Participates in management training	n	%	Participation in management training	n	%
Yes	105	95.45	At least 1 training per year	47	42.73
No	5	4.55	From 2 to 5 trainings per year	49	44.54
More than 5 trainings per year	9	8.18
does not participate	5	4.55

• Source: Own elaboration.

Regarding age, it was observed that more than 60% of coordinators are between 36 and 55 years old, and that women are predominant at the head of coordination, representing 55.45% of the sample. Concerning professional experience before the position, it was found that 45 of the respondents stated that they had not had previous experience in any area of management, while another 45 claimed to have expertise in the area of administrative management, and another 20 claimed to have experience in the area of academic leadership. In addition, the length of time working in the management area was verified, in which it was found that the majority (39.36%) stated that they had been working in the management area for between 5 and 10 years.

About the time of work in the institution, it was evidenced that the highest concentration is between 5 and 10 years (n=34), followed by those with less than 5 years (n=23). When asked about the time spent coordinating the course, the highest concentration (30.91%) has between 2 and 4 years, followed by those with less than 2 years (23.64%). Regarding the workload dedicated to the institution, it was found that most have a workload between 30 and 40 hours per week. Regarding the workload dedicated to coordination, the majority (69.09%) dedicate more than 16 hours per week to the function.

As for training, it was found that the respondents were instructed in 41 different courses, which, to facilitate the analysis, were organized according to the Areas of Knowledge established by CNPQ. The highest concentration of respondents was observed in the Social and Applied Areas (32.74%). This area includes courses in Administration (12), Architecture and Urbanism (5), Accounting (7), Social Communication (1), Industrial Design (2), Law (3), Economics (3), Journalism (2), Publicity and Propaganda (1) and Social Service (1). The second most representative area was Human Sciences, with 21.24%, with courses in Geography (2), History (2),

Pedagogy (11), Psychology (8) and Sociology (1). It is also noted that the majority followed in the same area to take their postgraduate courses (master's and/or PhD).

Finally, regarding participation in training courses related to management, the majority (95.45%) mentioned participating. Regarding the frequency with which they participate in these training courses, 47 respondents said that they experience at least once a year; 49 claimed to participate in 2 to 5 trainings per year, 9 respondents indicated participating in more than 5 trainings per year.

Table 3 presents the descriptive statistics and the information entropy of the didactic-pedagogical indicators.

Table 3
Didactic-pedagogical indicators (DPI)

	Importance			Execution
	Entropy e(di)	Weight	Média	Entropy e(di)	Weight	Média
DPI1	0.9925	0.1544	DPI1	0.9925	0.1544	DPI1
DPI2	0.9959	0.085	DPI2	0.9959	0.085	DPI2
DPI3	0.993	0.1443	DPI3	0.993	0.1443	DPI3
DPI4	0.9979	0.0424	DPI4	0.9979	0.0424	DPI4
DPI5	0.9986	0.0281	DPI5	0.9986	0.0281	DPI5
DPI6	0.9978	0.0462	DPI6	0.9978	0.0462	DPI6
DPI7	0.9989	0.0229	DPI7	0.9989	0.0229	DPI7
DPI8	0.9988	0.0255	DPI8	0.9988	0.0255	DPI8
DPI9	0.9984	0.0328	DPI9	0.9984	0.0328	DPI9
DPI10	0.9977	0.0479	DPI10	0.9977	0.0479	DPI10
DPI11	0.9984	0.0338	DPI11	0.9984	0.0338	DPI11
DPI12	0.9972	0.0578	DPI12	0.9972	0.0578	DPI12
DPI13	0.9982	0.0367	DPI13	0.9982	0.0367	DPI13
DPI14	0.9988	0.0243	DPI14	0.9988	0.0243	DPI14
DPI15	0.9974	0.053	DPI15	0.9974	0.053	DPI15
DPI16	0.9972	0.0572	DPI16	0.9972	0.0572	DPI16
DPI17	0.9948	0.1076	DPI17	0.9948	0.1076	DPI17
Total		1.0000	Total		1.0000	Total

• Source: Own elaboration.

Regarding the importance attributed to the DPI, it was obtained that the general average of the indicators was 4.58. The lowest average (4.05) was calculated for the DPI3 indicator and the highest (4.80) was for the DPI14. As for the information entropy coefficient, the lowest was presented by the DPI7 indicator, with e(d 7) of 0.9989 and ${\lambda }^{~}{}_i$ 0.0229 (2.29%); while the highest occurred in the DPI1 indicator, which obtained an e(d 1) of 0.9925 and a weight of 0.1544 (or 15.44%). These findings reveal that the DPI1 indicator showed the highest degree of dispersion between the scale levels, while the DPI7 had the lowest. This allows us to infer that most of the indicators did not present dispersion in the answers, that is, there is an alignment in the degree attributed by the respondents to the importance of the DPI.

Regarding the calculation of entropy related to execution, there was a general average of 4.29, with the lowest (3.49) being calculated in DPI3 and the highest (4.59) in DPI8. Regarding entropy, the lowest dispersion result obtained a weight of 2.84%, referring to DPI13, while the highest was calculated by DPI3 (15.89%). In addition to DPI3, DPI1 also showed a high dispersion coefficient (13.63%) when compared to the others, indicating that they had a higher degree of dispersion in the responses. Still, in relation to the indicators of this set, it is observed that the DPI2, DPI10 and DPI15 presented a median level of dispersion (6.99%, 6.79% and 7.63%) respectively, indicating little dispersion in the answers.

Table 4 highlights the entropy results of the evaluation indicators.

Table 4
Evaluation Indicators (EI)

	Importance			Execution
	Entropy e(di)	Weight	Média	Entropy e(di)	Weight	Média
EI1	0.9978	0.2038	EI1	0.9978	0.2038	EI1
EI2	0.9971	0.2632	EI2	0.9971	0.2632	EI2
EI3	0.9955	0.4163	EI3	0.9955	0.4163	EI3
EI4	0.9987	0.1168	EI4	0.9987	0.1168	EI4
Total		1,000	Total		1,000	Total

• Source: Own elaboration.

It is observed that the general average of the evaluation indicators was 4.57 for the level of importance and 4.11 for the level of execution. With regard to the level of importance, it is observed that the lowest mean (4.37) was found in EI3 and the highest (4.78) was in EI4. As for entropy, it is noted that EI3 was the only one to present a high coefficient of dispersion in the response scale with a weight of 41.63%, with the lowest average of the set (4.37). This was the indicator with the smallest number of respondents who attributed the maximum level of importance, implying that, while for some, it is essential, others evaluate it with little or no importance. EI4, on the other hand, obtained the highest average of the set and, consequently, presented the lowest degree of dispersion (weight 0.1168).

About the level of execution, it was found that the lowest average (3.94) corresponds to EI3, which showed a high level of dispersion in the response scale, as it showed a weight of 41.63%. In EI4, the degree of agreement between respondents was high, as in addition to obtaining the best average of the set, it presented low dispersion (11.68%), representing the lowest result of the group.

In Table 5, the findings regarding the entropy of the information of the indicators related to the questions of adopted methodologies are evidenced.

Table 5
Indicators of methodologies adopted(IMA)

	Importance			Execution
	Entropy e(di)	Weight	Média	Entropy e(di)	Weight	Média
IMA1	0.9966	0.2475	IMA1	0.9966	0.2475	IMA1
IMA2	0.998	0.1494	IMA2	0.998	0.1494	IMA2
IMA3	0.9988	0.0898	IMA3	0.9988	0.0898	IMA3
IMA4	0.9953	0.3414	IMA4	0.9953	0.3414	IMA4
IMA5	0.9977	0.1719	IMA5	0.9977	0.1719	IMA5
Total		1	4.59	Total	1	4.59

• Source: Own elaboration.

According to the data, the general average related to the importance of the indicators was 4.59. The lowest mean (4.34) corresponds to IMA4, while the highest (4.83) belongs to IMA3. IMA4 had the highest dispersion, compared to the other indicators in the set, with an entropy weight of 0.3414, and an average of 4.34, the lowest among the IMA.

Regarding the performance of the IMA, it is observed that the overall average was 4.25, with the lowest calculated by the IMA1 (3.93) and the highest calculated by the IMA3 (4.60). As for entropy, it is observed that the lowest result was for the IMA3 with a weight of 7.40%, indicating that there is no high degree of dispersion.

Next, Table 6 presents the results obtained for the entropy of the indicators related to the faculty.

Table 6
Faculty Indicators (FI)

	Importance			Execution
	Entropy e(di)	Weight	Média	Entropy e(di)	Weight	Média
FI1	0.9984	0.1419	FI1	0.9984	0.1419	FI1
FI2	0.9985	0.1397	FI2	0.9985	0.1397	FI2
FI3	0.9984	0.1422	FI3	0.9984	0.1422	FI3
FI4	0.9978	0.2027	FI4	0.9978	0.2027	FI4
FI5	0.9975	0.2266	FI5	0.9975	0.2266	FI5
FI6	0.9984	0.1468	FI6	0.9984	0.1468	FI6
Total		1	4.73	Total	1	4.73

• Source: Own elaboration..

The general average related to the importance of the indicators was 4.73, with the lowest (4.59) being for FI5 and the highest (4.80) for FI1. Regarding entropy, only FI4 and FI5 showed median levels of dispersion (weights 0.2027 and 0.2266) compared to the other indicators that make up a set, which showed similar consequences, indicating no entropy.

Concerning the performance of the indicators, the averages ranged from 4.12 (FI4) to 4.46 (FI6), resulting in an overall average of 4.31. Regarding the calculation of entropy, it is observed that three of the six indicators presented a degree of dispersion considered median in the responses. FI4 had the highest degree of dispersion in the responses, weighing 0.2476 (24.76%). To FI3, there is a median degree of dispersion (17.52), indicating that there is disagreement in the level of execution of this indicator in the courses represented in the research; however, to a lesser degree than FI2 and FI4, that is, the degree of agreement is greater than that of disagreement.

Next, the results of the infrastructure indicators are presented.

Table 7 Infrastructure Indicators (II)

	Importance			Execution
	Entropy e(di)	Weight	Média	Entropy e(di)	Weight	Média
II1	0.9987	0.1704	II1	0.9987	0.1704	II1
II2	0.9982	0.2403	II2	0.9982	0.2403	II2
II3	0.9988	0.1566	II3	0.9988	0.1566	II3
II4	0.9987	0.1688	II4	0.9987	0.1688	II4
II5	0.998	0.2638	II5	0.998	0.2638	II5
Total		1	4.78	Total	1	4.78

Source: Own elaboration.

Regarding importance, the overall average of the indicators was 4.78, ranging from (4.75) for II2 and II5 to (4.84) for II3. As with the indicators related to the faculty, none with a high degree of dispersion were identified, only two median ones, the II2 with a weight of 24.03% and the II5 with a weight of 26.38%.

Regarding the execution, it was identified that the II presented the smallest variation among themselves regarding the levels of dispersion and the averages of the indicators. An overall average of 4.45 was calculated, with the lowest (4.40) being calculated in II5 and the highest (4.58) in II1.

Concerning the calculation of entropy, it was identified that the II5 was the only one to present a degree of dispersion between the response scale (25.91%), representing some level of disagreement among the respondents. As for the others, there is no dispersion level, since their weights varied from 0.1724 (II4) to 0.2070 (II2).

After the identification of the levels of entropy of the indicators per set, the calculation of entropy between the sets of indicators was performed (Table 8).

Table 8
Entropy of sets of indicators

	Importance			Execution
	Entropy e(di)	Weight	Média	Entropy e(di)	Weight	Média
DPI	0.9991	0.1464	DPI	0.9991	0.1464	DPI
EI	0.9985	0.2469	EI	0.9985	0.2469	EI
IMA	0.9986	0.2233	IMA	0.9986	0.2233	IMA
FI	0.9988	0.1917	FI	0.9988	0.1917	FI
II	0.9988	0.1918	II	0.9988	0.1918	II
Total		1	Total		1	Total

• Source: Own elaboration.

With regard to importance, the general average of the sets of indicators was 4.65, with the lowest (4.57) being found in the set related to the EI questions and the highest (4.78) corresponding to the II set. As for entropy, the lowest coefficient was presented by the DPI set, with e(d₁) of 0.9991 and ${\lambda }^{~}{}_i$ 0.1464 (14.64%). On the other hand, the highest degree occurred in the EI questions, which obtained an e(d₂) of 0.9985 and a weight of 24.69%.

The IMA set also showed a higher level of dispersion to the other sets, with a weight of 22.33%. However, when analyzed together, it is observed that the EI and IMA sets did not present results with high degrees of dispersion, as they have similar weights to the others, indicating a low level of dispersion.

For the performance indicators, it was found that the overall average of the sets was 4.28, with the lowest (4.11) being calculated in the EI set and the highest (4.45) in the II set. As for entropy, the lowest result was presented by the DPI set, with e(d₁) of 0.9981 and ${\lambda }^{~}{}_i$ 0.1302 (ie, 13.02%). On the other hand, the highest degree occurred in the IMA questions, with a low level of dispersion in the answers and a weight of 22.08%.

With regard to the level of importance, it is observed that most of the indicators showed a good level of agreement between the responses, however in all sets indicators with some degree of dispersion were calculated. These findings suggest that, despite the importance averages having surpassed level 4 of the response scale, for some respondents these indicators are very important, while others evaluate them as unimportant or very unimportant.

The same occurred in relation to execution, which in general, most indicators showed a good level of agreement, however, all sets presented indicators with some degree of dispersion. These findings reveal that those who presented dispersion results have a higher degree of disagreement in the answers, that is, while some perform it very frequently, there is a relatively proportional number who perform it less frequently or do not perform it at all.

4.2 Discussions

Among the observed results, it was found that all the proposed indicators have an attribution of greater importance than execution in the responses of the coordinators; that is, although the respondents assess that it is important to carry out such actions and strategies, they do not do it in the same way. Several reasons can cause this situation, such as the institutional policy or even the coordinator's lack of knowledge due to the short time in the function (ZANIN, 2014; MOLOZZI, 2015). It is concluded that there is a convergence for the importance given by the coordinators to the curricular contents that allow the development of the professional profile of the egress (updated contents, adequate bibliographies, etc.). Still, on the other hand, there is disagreement about the importance of discussing the CPP of the course with first-year students. In addition, the coordinators converge on the execution of practical teaching activities. However, they have yet to focus so much on the discussion of the CPP of the course with the graduating students. These findings are out of line with what was highlighted by Soares (2013) when explaining that the role of the educational manager is premised on uniting the interests of students with the pedagogical project, noting the need of the market in relation to graduates of their institution.

Regarding the evaluation indicators, it is noted that the coordinators have a more significant divergence in the importance of carrying out evaluations in the Enade mock format. On the other hand, the coordinators have focused on encouraging teachers to give feedback to students based on the assessments carried out. The faculty indicator showed an excellent dispersion of coordinators regarding the importance of observing the faculty's work regime and prioritizing their professional experience.

Finally, it is concluded that the most critical evaluation indicators and which are most performed by the coordinators, involve the didactic-pedagogical, faculty, and infrastructure, which both in terms of importance and in terms of execution, present less dispersion of responses among the coordinators of the course of graduation. This is in line with what the legislation (Law 10.861/2004) currently recommends is to verify the teaching conditions offered to students in these three dimensions mentioned (BRASIL, 2004).

5 Final Considerations

This study purposed to identify the perception of undergraduate course coordinators regarding contributory strategies to improve indicators in Canada. In general, the findings reveal more experienced coordinators with a higher level of training in terms of course management. Although for some coordinators, this is their first experience in management, the respondents stated that they participated in training courses purposed at qualifying management and strengthening undergraduate courses. It was also found that most coordinators have adequate degrees for this function, which corroborates the understanding of Barbosa and Mendonça (2016) about the growing search for professional qualifications of the managers of these institutions.

It was found that the proposed indicators showed a higher level of importance attributed than the execution; that is, although the coordinators consider that it is crucial to execute such actions and strategies, they do not do it in the same proportion. Several reasons can cause this situation, such as institutional policy or, yet, need for more knowledge on the part of the coordinator due to the short time in the role. It is also observed in relation to the execution that, differently from the degree of importance, some indicators of the set of didactic-pedagogical, evaluation, and adopted methodologies presented averages between 3 and 4 points and the smallest general standards. These findings suggest that coordinators and courses are more aligned in terms of infrastructure and faculty. This requires attention and care on the part of the coordinators since the least performed actions are related to didactic activities and have the most significant potential to directly interfere with the student's training and their preparation for carrying out the evaluation proposed by Enade. However, it can be inferred that the proposed model with the five sets of indicators was validated by the coordinators, serving as a basis for the management of the courses.

To conduct this research, despite the theoretical and technical care that the researchers adopted, it is not free of limitations. Among them, the research sample characterized as non-probabilistic stands out, and the temporal aspect, because in other spaces and times, the answers may show different perceptions of course coordinators at HEIs accredited by ACAFE. Thus, as suggestions for future studies, improving the sample composition criteria is recommended, which can contribute to a broader group. It is also suggested the adoption of longitudinal perspectives in studies in the area of education of particular courses, such as, for example, only accounting, administration, etc.

Referências

• ACAFE. Conheça a ACAFE Disponível em: https://acafe.org.br/site/ Acesso em: 26 mar. 2020.
  » https://acafe.org.br/site/
• BARBOSA, M. A. C.; MENDONÇA, J. R. C. O professor-gestor e as políticas institucionais para formação de professores de ensino superior para a gestão universitária. Revista Economia & Gestão, Belo Horizonte, v. 16, n. 42, p. 61-88, 2016.
• BITTENCOURT, H. R.; CASARTELLI, A. O.; RODRIGUES, A. C. M. Sobre o índice geral de cursos (IGC). Avaliação: Revista da Avaliação da Educação Superior, Campinas; Sorocaba, v. 14, n. 3, p. 667-682, 2009.
• BONZANINI, O. A.; FERREIRA-DA-SILVA, A. C.; LEITE, T. G. Aspectos da Gestão nos cursos superiores em Ciências Contábeis do Brasil: um estudo sobre a atuação dos coordenadores. e3-Revista de Economia, Empresas e Empreendedores na CPLP, Funchal, v. 3, n. 1, p. 52-74, 2017.
• BOTELHO, L. L. R. et al Ser gerente em instituições de ensino superior: a percepção de ex-coordenadores de curso de graduação sobre suas aprendizagens gerenciais. Revista GUAL, Florianópolis, p. 190-209, 2018.
• BRASIL. Lei nº 10.861, de 14 de abril de 2004. Institui o Sistema Nacional de Avaliação da Educação Superior-SINAES e dá outras providências, 2004. Disponível em: http://www.planalto.gov.br/ccivil_03/_Ato2004-2006/2004/Lei/L10.861.htm Acesso em: 20 abr. 2020.
  » http://www.planalto.gov.br/ccivil_03/_Ato2004-2006/2004/Lei/L10.861.htm
• CORDAZZO, E. G.; WERNKE, R.; ZANIN, A. Estilos gerenciais dos coordenadores de cursos de graduação de uma instituição de ensino superior. Revista GUAL, Florianópolis, v. 14, n. 1, p. 84-107, 2021.
• COSTA, D. M. Avaliação educacional em larga escala: a opção pela democracia participativa. Jornal de políticas educacionais, Curitiba, v. 3, n. 6, 2009.
• DIAS SOBRINHO, J. Avaliação e transformações da educação superior brasileira (1995-2009): do provão ao SINAES. Avaliação: Revista da Avaliação da Educação Superior, Campinas; Sorocaba, v. 15, n. 1, p. 195-224, 2010.
• DOMINGUES, M. J. C. S. et al Identificação e análise do perfil dos gestores de cursos de ciências contábeis nos estados de São Paulo e Santa Catarina. Revista Base, São Leopoldo, v. 8, n. 2, p. 189-201, 2011.
• FALLEIROS, A. E. S.; PIMENTA, M. L.; VALADÃO JÚNIOR, V. M. O significado da autoavaliação institucional na perspectiva de técnicos-administrativos de uma universidade pública. Avaliação: Revista da Avaliação da Educação Superior, Campinas; Sorocaba, v. 21, n. 2, p. 593-618, 2016.
• FERRONATO, F. B. Interfaces entre avaliação institucional e gestão de instituições de ensino superior: um estudo a partir do SINAES. REGAE, Santa Maria, v. 6, n. 11, p. 21-43, 2017.
• GONTIJO, S. B. F. Implicações do Enade para a organização do trabalho pedagógico e as práticas avaliativas em um curso de Pedagogia 2014. Tese. (Doutorado em Educação) -Universidade de Brasília, Brasília. 2014.
• INEP. Instrumento de Avaliação de Cursos de Graduação Presencial e a Distância: autorização. Brasília, 2017. Disponível em: http://download.inep.gov.br/educacao_superior/avaliacao_cursos_graduacao/instrumentos/2017/curso_autorizacao.pdf Acesso em: 21 fev. 2020.
  » http://download.inep.gov.br/educacao_superior/avaliacao_cursos_graduacao/instrumentos/2017/curso_autorizacao.pdf
• MARCON, S. R. A. Atribuições dos cargos de coordenação e subcoordenação de cursos de graduação. In: CONGRESSO INTERNACIONAL DE GESTÃO UNIVERSITÁRIA NA AMÉRICA DO SUL, 11., 2011, Florianópolis. Anais [...] Florianópolis, Universidade Federal de Santa Catarina, 2011.
• MEYER JÚNIOR, V. Planejamento universitário: ato racional, político ou simbólico-um estudo de universidades brasileiras. Revista Alcance, Itajaí, v. 12, n. 3, p. 373-389, 2005.
• MEYER JÚNIOR, V.; PASCUCCI, L.; MANGOLIN, L. Gestão estratégica: um exame de práticas em universidades privadas. Revista de Administração Pública, Rio de Janeiro, v. 46, n. 1, p. 49-70, 2012.
• MOLOZZI, G. A. Indicadores de desempenho para Instituições de Ensino Superior: uma proposta adaptada aos eixos do SINAES. 2015. Dissertação (Mestrado Profissional em Administração) - Universidade do Oeste de Santa Catarina, Chapecó, 2015.
• PEIXOTO, M. C. L. A avaliação institucional nas universidades federais e as comissões próprias de avaliação. Avaliação: Revista da Avaliação da Educação Superior, Campinas; Sorocaba, v. 14, n. 1, p. 9-28, 2009.
• POLIDORI, M. M. Políticas de avaliação da educação superior brasileira: Provão, SINAES, IDD, CPC, IGC e.… outros índices. Avaliação: Revista da Avaliação da Educação Superior, Campinas; Sorocaba, v. 14, n. 2, p. 439-452, 2009.
• REBELO, L. M. B.; ERDMANN, R. H. Modelo de formação de estratégias de gestão em instituições de ensino superior sob o enfoque da teoria da complexidade. Revista de Administração e Inovação, São Paulo, v. 4, n. 1, p. 5-20, 2007.
• RISTOFF, D.; LIMANA, A. O ENADE como parte da avaliação da educação superior Brasília, DF, [2007]. Disponível em: http://mecsrv04.mec.gov.br/news/ArtigosDiaImp.asp?Id=1875 Acesso em: 20 de jan. 2020.
  » http://mecsrv04.mec.gov.br/news/ArtigosDiaImp.asp?Id=1875
• ROCHA, I. Grau de entropia da informação em indicadores econômico-financeiros das empresas que participam dos níveis de governança corporativa da BM&FBovespa. 2010. Dissertação (Mestrado em Ciências Contábeis) - Universidade Regional de Blumenau. Blumenau, 2010.
• SILVA, F. M. V.; CUNHA, C. J. C. A. A transição de contribuidor individual para líder: a experiência vivida pelo professor universitário. Revista GUAL, v. 5, n. 1, p. 145-171, jan./abr. 2012. Disponível em: https://www.researchgate.net/publication/314828174_A_transicao_de_contribuidor_individual_para_lider_a_experiencia_vivida_pelo_professor_universitario Acesso em: 20 jan. 2020.
  » https://www.researchgate.net/publication/314828174_A_transicao_de_contribuidor_individual_para_lider_a_experiencia_vivida_pelo_professor_universitario
• SOARES, L. H. Gestão de Instituições de Ensino: o ensino superior privado e os novos parâmetros de perenidade. Universitas: Gestão e TI, Brasília, DF, v. 3, n. 2, 2013.
• ZANIN, A. Definição de painel de indicadores de desempenho para instituições comunitárias de ensino superior. 2014. Tese (Doutorado em Engenharia de Produção) -Universidade Federal do Rio Grande de Sul, Porto Alegre, 2014.
• ZELENY, M. (ed.). Multiple criteria decision-making Kyoto 1975 Berlin: Springer Science & Business Media, 1982.

Publication Dates

• Publication in this collection
  07 July 2023
• Date of issue
  2023

History

• Received
  23 June 2021
• Accepted
  20 June 2023
• Reviewed
  21 June 2023