SciELO - Scientific Electronic Library Online

 
vol.27Grain intermodal terminals: evaluation of pure technical efficiency by Data Envelopment AnalysisSuccess factors in the implementation of statistical process control: action research in a chemical plant author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


Production

Print version ISSN 0103-6513On-line version ISSN 1980-5411

Prod. vol.27  São Paulo  2017  Epub June 05, 2017

http://dx.doi.org/10.1590/0103-6513.219116 

Research Article

Structural equation modeling applied to assess industrial engineering students’ satisfaction according to ENADE 2011

Agenor Sousa Santos Netoa  * 

Maria José Pereira Dantasb 

Ricardo Luiz Machadob 

aUniversidade Federal de Goiás, Aparecida de Goiânia, GO, Brasil

bPontifícia Universidade Católica de Goiás, Goiânia, GO, Brasil

Abstract

This research aims to propose and try out a model to assess the Industrial Engineering students’ satisfaction, with a focus on private higher education institutions. The research tool used in this study was the Student Questionnaire 2011 National Exam for the Assessment of Student Performance (ENADE), made available by the Brazilian Ministry of Education (MEC). The questions were designed to assess undergraduate students general satisfaction based on the following constructs: Teacher Involvement, Student Interest, Course Demands, Course Organization, and General Satisfaction. A new construct was added to this research: Infrastructure. The methods employed to analyze students’ responses concerning their satisfaction with the service provided by higher education institutions were: Confirmatory Factor Analysis and Structural Equation Modeling, with the support of the software package SPSS® (Statistical Package for the Social Sciences) and the AMOS® module. The hypotheses proposed in the model indicate that the construct Teacher Involvement is key to Student General Satisfaction.

Keywords Private higher education institutions; Service; Student interest; Teacher involvement

1. Introduction

In recent decades, government, companies, and society started to use several new services, inventions, and innovations for direct and indirect application in the lives of consumers. Therefore, it is considered that the companies began to look more carefully to the after sales and to continue enhancing the customer even after purchasing the product or service. In order to keep the customer satisfaction level is an increasingly arduous task and demands specific expertise to deal with these issues across the local, regional and global dimensions (Murali et al., 2016).

Customer satisfaction is considered critical to the success of organizations in a competitive market. How a company can provide continuing customer satisfaction for its customers? The satisfaction is related to the attendance of explicit and implicit requirements, the consumer through the set of characteristics, or attributes of the service. So, it becomes important to find out as the performance of the different attributes is related to customer satisfaction (Tontini & Sant’Ana, 2008).

Therefore, the evaluation of procedures or processes presents itself as an ideal tool for understanding the problems and identification of causes, while it may allow the indication of corrective or alternative measures to expand the effectiveness and efficiency of these transformations. It is observed that the evaluation in order to measure includes both the performance evaluation as the evaluation of satisfaction for a particular product or service (Murali et al., 2016). The research in question will include the service sector, specifically the field of education from private institutions of higher education.

The evaluation of institutional management gives support widely in the process of observing the social and administrative obstacles, as it provides obtaining information necessary to identify and understand the causes of successes and failures of projects, both in individual issues as those relating to teams or divisions (local, regional and global) (Murali et al., 2016).

The information obtained through the evaluation of an educational institution can assist in the preparation and review of strategic planning and redirect decision-making. Bernroider & Schmöllerl (2013) point out that concerning the management of an organization the availability of information about the market, the environment, the macro environment of the company itself, in quantity and quality is essential to minimizing the risk in decision-making.

Considering the decision-making process to assess the institutional management, this research combines statistical techniques such as structural equation modeling with the evaluation process of satisfaction by students about the course they are attending, developing a measurement scale, in which satisfaction items are located.

Structural Equation Modeling (SEM) combines factorial and regression analysis. According to Kaplan (2009), SEM has a hybrid historical origin and it is applied to practical problem solving for complex model tests with multiple simultaneous variables and latent traits, thus enables simultaneous testing of the relation series. Applications of SEM are able to estimate a series of interdependent multiple regressions simultaneously by specifying a structural model (Iacobucci, 2010). Furthermore, the system is analyzed in its entirety and in an integrated manner (Byrne, 2010).

However, the structural equation modeling has been very little used in research in Brazil, due to the complexity of the calculations and the application of a relatively large sample to evaluate the determinants involved (Tinoco & Ribeiro, 2007).

The evaluation of the measure applied to education in order to investigate the satisfaction of students concerning the course is essential to educational institutions as it provides a reflection on the teaching methods employed, and gives information to improve the teaching quality of courses offered by the institution (Lidice & Saglam, 2013).

2. Literature revision

2.1. Quality management in education

Due to many changes which are currently present in the daily lives of human beings, it can be seen that it is necessary to change paradigms within the educational system in the country. It should be emphasized that these changes must occur due to the adoption of perceptions and participative, democratic, and interactive practices. These ones are usually characterized by dynamic and global movements, through which are used to determine the characteristics of products and services, promote the interaction between directors, employees and customers and / or users, establishing alliances, networks, and partnerships in search for solutions to problems and expanding horizons (Abu-Duhou, 2002).

It is observed that during this change, not just educational institutions develop this awareness, as the company itself demands that this change should be put into practice (Azevedo, 1999).

The indiscriminate use weakened the term ‘quality’ when trying to impose it as a fad. Quality presents great advantages, values and positive results in organizations that have adopted it as part of its management system. Many scholars have suggested that everyone has an obligation to “do things right the first time” without taking into consideration that not all organizational structures respond in a similar way to its application, such is the case of the broad universe that involves the educational process (Larina, 2015).

The literature shows that studies, evaluations, and research conclude that the education quality is a highly complex phenomenon that involves multiple dimensions that cannot be defined only by recognition of the variety and quantity of the elements involved in this process (Sayed & Ahmed, 2015).

Education presents quality if it meets the specifications of scheme curricular programs and study plans specified in advance; introduces deserved importance and relevance to the content. If Education meets the expectations of parents and students; if the educated students contribute widely in search to meet fulfill the community needs when collaborating in problem-solving; and finally, if the student can from the acquired knowledge, improve their living standard and assist in the positive transformation of their neighbors and the environment in which it develops (Masino & Niño-Zarazúa, 2016).

2.2. Determinants of satisfaction of higher education student

Several studies handle factors related to the satisfaction of higher education students. Some of which show that the academic prestige or status of higher education institutions (HEIs), which would generate a certain degree of recognition both personal, between friends and family, as well as in relation to the labor market, would ensure them a job at the course’s end (Quevedo-Silva et al., 2012).

It is worth mentioning several factors that do not significantly affect the satisfaction, such as sports and social extracurricular activities on campus (Yin & Lei, 2007).

Other studies have characterized certain factors that may influence positively the student satisfaction. These are the factors in this category: the group of friends, personal academic identification with the study area, the external aspects of the student, as the objective conditions to get a job after graduating, good answers to their needs, expectations generally, as well as a good course structure (Kanan & Baker, 2006; Appleton-Knapp & Krentler, 2006; Petruzzellis et al., 2006; Bardagi et al., 2003).

In the literature, there are other studies that insert the factors that may affect negatively the satisfaction of academics. The ones that stand out in this category are lack of preparation and little faculty commitment to their respective classes, the disappointment with the poor organization and general failure to meet expectations, and finally the lack of availability and willingness by faculty to fulfill the daily needs of their students (Petruzzellis et al., 2006; Douglas et al., 2006).

A recent study in Brazil, based on the model of Paswan & Young (2002), was developed by Vieira et al. (2008). The authors used the structural equation modeling to analyze student satisfaction from the use of five constructs: Teacher Involvement; Student Interest; Teacher-student Interaction; Course Demands; Course Organization.

This study used as reference the research conducted by Paswan & Young (2002), enhanced by Vieira et al. (2008).

2.3. National Exam for the Assessment of Student Performance

Several steps were taken so the Brazilian Higher Education’s assessment was made using the current methodologies. In the decade of 1980, some universities began to evaluate themselves. In 1996, the National Course Examination (ENC) was created, and only in 2004, the Brazilian National Higher Education Evaluation System (SINAES) began to be used (Polidori et al., 2006).

SINAES include as evaluation components the first-year and last-year undergraduate students’ performance, as well as the assessment of institutional dimensions and the undergraduate program. The National Exam for the Assessment of Student Performance (ENADE) is the component used by SINAES to evaluate the students once the exam measures the students’ performance, according to various curricular contents of undergraduate courses (Brito, 2008).

In a comprehensible way, SINAES aims to analyze the performance of the students, the Higher Education courses, and institutions, considering teaching, research, and extension activities developed by these Higher Education institutions. To achieve this goal, the National Exam for the Assessment of Student Performance (ENADE) is used, as well as the Brazilian Higher Education Register, Brazilian Higher Education Census, the Evaluation of the Teaching Conditions (ACE) and the Brazilian Institutional Evaluation (AI). When these data are analyzed in an integrated manner, they allow the creation of a new trend for the Higher evaluation in the country (Marinho-Araujo, 2004).

According to Polidori et al. (2006), the National Exam for the Assessment of Student Performance (ENADE) is the most important instrument for measuring the higher education student’s performance in Brazil, using the curricular contents in the exam elaboration.

2.4. Structural equation modeling

Models based on Structural Equation can be defined as research models based on statistics, correlation and regression analysis, including several independent and dependent latent variables and measurement errors in variables, multiple parameters, reciprocal causes, simultaneity and interdependence (Koç et al., 2016).

It is observed that the structural equation models are useful for troubleshooting in the social sciences and human behavior sciences, being applied in marketing and traditional areas of Sociology, Psychology, Econometrics and Education (Jöreskog & Sörbom, 1982).

In education, there are studies that applied structural equation modeling. Gonçalves Filho et al. (2004) tested measurement models for the satisfaction construct related to Loyalty, Perceived Quality, Value, and Expectations in higher education institutions. They applied structural equations to answers from 604 respondents and verified the impact of satisfaction on loyalty in these institutions.

Neelaveni & Manimaran (2016) applied structural equation modeling to assess the quality impact of some important administrative practices on higher education institutions, such as vision and mission, leadership and governance, recruitment and qualification of teachers, infrastructure and the curriculum.

Finally, Pereira et al. (2016) propose and validate a satisfaction model for students of private educational institutions in Southern Brazil, considering the following constructs: Image, Perceived Quality, Perceived Value, Expectations, Loyalty, and Word-of-Mouth Outreach. For the study, they developed a predictive model using structural equation modeling. The model was based on the European Customer Satisfaction Index (ECSI). The results obtained showed that the image of educational institution influences positively on the student's perception of his satisfaction, loyalty to the university, and the attraction of new students.

For Karimi & Meyer (2014), the model is usually used when the phenomenon in question is specified in terms of variables of causes and effects. Thus, each equation in the model is a causal link rather than a simple empirical association between variables. This author considered the structural parameters representing process characteristics (procedure) that produce the observed variables.

Regarding the structural equation modeling (SEM), SEM can be defined, according to Klem (1995) analysis, as an extension of multiple regression, considering that in the application of regression the researcher is interested in predicting a single dependent variable, while in SEM there are more than one dependent variable. Thus, the concern on this technique is the variable order. In regression X influences Y; in SEM X influences Y and Y influences Z (Farias & Santos, 2000).

Klem (1995) and Maruyama (1998) emphasize that two main results can be expected from an analysis of the application of SEM. Initially, there is an estimated magnitude of the effects established between variables. These estimates are directly conditioned to the fact that the specified model (diagram) is correct. Second, it is possible to test whether the model is consistent with the verified data. If the model and the data are consistent, it can be said that this is plausible, although it cannot affirm that this is correct.

Maruyama (1998) describes the path analysis is directly related to models with the unidirectional causal flow, in which the measurements of each conceptual variable are perfectly reliable. Thus, based on this premise, it is expected that there is no measurement error (measurement) or specification (operation) of the variables. That is, each measure is seen as an exact manifestation of the theoretical variable. Obviously, in the social sciences, to assume that there is a perfect reliability is unreal.

Maruyama (1998) describes that SEM methods must start with a conceptual model that specifies the relationship between a set of variables. Thus, the theory provides the central point of this technique. SEM offers estimates of the strength of all the hypothesized relationships in a theoretical framework. Furthermore, the information available refers both to the impact of one variable in the other and the relationship of an indirect influence of a variable positioned between two other, called as intervening or mediating (Farias & Santos, 2000).

Hair Junior et al. (2009) argued that the theory provides the rationalization for almost every aspect of SEM - another method for confirmatory analysis, guided more by the theory than by empirical results.

Bollen (1989) presents a model of structural equations with latent variables that can be defined by the following equations:

η=α+Bη+Γξ+ζ (1)
y=μy+λyη+ε (2)
x=μy+λxξ+δ (3)

Equation 1 it is the structural part of the model, and the Equations 2 and 3 are parts relating to the measured variables.

The random vectors η’ = ( η1 , ..., ηm ) and ξ’ = ( ξ1 , ..., ξn ) both unobservable represent the dependent latent variables (endogenous) and independent (exogenous), respectively. The α vector indicates the intercept of the structural equation, but it will not appear in the model if the latent variables and observed variables are taken diverted from their respective averages. Β (m × m) and Γ (m × n) matrices represent coefficient matrices, and ζ '= ( ζ1 , ..., ζm ) designates the residual vector or random disorders (errors in the structural equation). The Β elements express direct causal effects of variables η in another η and Γ elements show the direct effects of the ξ variables in η variables. Φ (n × n) and Ψ (m × m) matrices, not represented in the equation, refer to the ξ and ζ covariance matrices, respectively. The model assumptions are configured like this: ζ is unrelated with ξ; ζi is homocedastic and it has no autocorrelations; I-Β is not unique.

The other two equations include the observable vector y’ = ( λ1 , ..., λp ) and x’ = ( x , ..., xp ). Vectors μy and μy represent the intercepts of the equation (averages of the variables). The vectors ε and δ indicate the measurement errors of y and x, respectively. Matrices λy (p×m) and λx (q×n)are related to regression matrices y in η and x in ξ, respectively. It is pertinent to title y and x of observable variables, η and ξ of latent variables. Matrices Θε (p×p) and Θδ (q×q), not represented in the equation, point to the covariance matrices of ε and δ, respectively. Hypothetically the measurement errors are unrelated with η, ξ and ζ, but they can be correlated.

3. Methodology

3.1. The theoretical model

The core and peripheral services had influenced the perceived quality of service. In the case of universities, teaching, research and extension can be identified as core services. Peripheral services include the secretariat, financial, the cafeteria, library, administrative support, laboratories, infrastructure, among others (Coutinho, 2007).

In this research, the construct Infrastructure was chosen as a peripheral service for a university to reach the general satisfaction of its students. Thus, the hypotheses related to this construct are:

Hypothesis 1: A higher degree of Infrastructure will lead to a higher level of Teacher involvement.

Hypothesis 2: A higher degree of Infrastructure will lead to a higher level of Student interest in the course.

Paswan & Young (2002) ensure that the way the teacher teaches content to students can influence negatively or positively in the evaluation on it. Therefore, the Course demands dimension relates to the way the teacher passes on the content to their students, how the activities are demanded, the support material and the readings are indicated, the development of the activities and work in the classroom (Paswan & Young, 2002; Vieira et al., 2008).

In addition, according to Clayson & Haley (1990) discussed, the severity of the course and how students evaluate their learning are characterized by a negative correlation. Therefore, the following hypotheses can be presented:

Hypothesis 3: A higher level of Course demands will lead to a lower level of Teacher involvement.

Hypothesis 4: A higher level of Course demands will lead to a lower level of Student interest.

The Course Organization dimension refers to the presentation of content in a systematic way by teachers; the definition of the curriculum appropriately and with well-defined sequence; student facility to make notes of the classes held by the teacher (Paswan & Young, 2002). Thus, the following hypotheses can be related to this construct:

Hypothesis 5: A higher level of Course demands will lead to a higher level of Teacher involvement.

Hypothesis 6: A higher level of Course demands will lead to a higher level of Student interest.

It can be said that student satisfaction is directly related to meeting their expectations, from the perspective of Student interest dimensions, Teacher involvement, Interaction student-teacher, Course organization and Course demand. Thus, according to Bronw & Mazzarol (2009), HEIs need to send a good image to the market and still be commercially competitive, so to achieve a satisfactory level of overall satisfaction by students is necessary to have a good level of quality of service.

Assuming that General satisfaction relates to Teacher involvement, Student interest, Infrastructure, Course demands and Course organization, the following assumptions can be elaborated:

Hypothesis 7: A higher level of Teacher involvement will lead to a higher level of Overall satisfaction by Industrial Engineering students.

Hypothesis 8: A higher level of Student interest will lead to a higher level of Overall satisfaction by Industrial Engineering students.

3.2. Data collection

To collect data, the tool used was the Student Questionnaire 2011 National Exam for the Assessment of Student Performance (ENADE), developed and applied by MEC (Brazilian Ministry of Education) of the Higher Education Institution that was answered voluntarily by undergraduate students of Industrial Engineering courses, totaling 2096 filled questionnaires from Private Non-Profit and For-Profit Private higher education institutions. However, they excluded all the questionnaires that had some cell without filling. The option adopted to remove them did not undermine the progress of analysis or substantially altered the expected sampling error. This refinement by the elimination of incomplete questionnaires led to exclude 306 respondents, resulting in 1790 valid interviews, and equivalent to 85.4% of the population.

According to data from 2013 Brazilian Higher Education Census, the private institutions gathered 5.3 million students (73.6%), while the public had 1.9 million students (26.3%), so we have a more representative sample of undergraduates from a private network. Moreover, according to National Confederation of Industry Brazil (2013), in Brazil, in the period of 2001 to 2011, the dropout rate in private universities was, on average, 60%, while in public universities, it represents, on average, 40%. There is an opportunity to associate the student dropout rate in correlation with satisfaction with the private university where he studies. Therefore, the focus of this research was Private Non-Profit and For-Profit Private Higher Education Institutions.

Questions from Student Questionnaire 2011 were grouped according to the model of Paswan & Young (2002), and Vieira et al. (2008). This model defines each Construct, which allowed to fit the variables according to their similarity to a given Construct. Paswan & Young (2002), and Vieira et al. (2008) in this research still exposed the inherent path diagram in Structural Equation Modeling. Therefore a similar representation in this research was made with adjustments for a better model fit. It is worth mentioning that the construct Infrastructure was added because it is a peripheral service and it is relevant to the customer satisfaction of educational institutions, as highlighted by Coutinho (2007). The data follow patterns of ordinal measurement scale, non-metric variables, characterized by Likert-type with values ​​from one to four, where the number represents the maximum correlation value.

The status of Socioeconomic Questionnaire section of the Student Questionnaire has received special attention since Structural Equation Modeling is built by means of these questions in this section. The questions used for each construct are presented in Appendix A.

The questions were grouped as follows: Instructor Involvement (Questions 34, 35, 41, 42), Student Interest (Questions 29, 46, 47, 48), Course Demands (Questions 36, 37, 38, 39, 40), Course Organization (Questions 43, 44, 45, 49, 50, 51, 52, 53), Infrastructure (Questions 22, 23, 24, 25, 26, 27, 28, 30, 31, 32), and Overall satisfaction (Question 54).

3.3. Data analysis

Hatch (2002) defines data analysis as the categorization process, sorting, handling and summarizing data. This research used Confirmatory factor analysis and the structural equation modeling as tools to evaluate the grades given by the students the variables that compose the satisfaction concerning the service provided by the educational institution. The software SPSS® (Statistical Package for Social Sciences) version 22 and its module, AMOS® version 22 supported the data analysis.

The procedures described by Hair Junior et al. (2009), and Anderson & Gerbing (1988) guided this research. We used a two-step approach where initially the measurement model is generated, and then the structural model (or estimation model) is built. The model is evaluated from the check values. These values exceed acceptable limits such as very high standard errors, negative error variances or standardized coefficients greater than 1. After confirming that these values are not present, an evaluation of overall adjustment of measurement and estimation models starts.

The following adjustment measures were used:

    •. CMIN represents the value of chi-square (χ2) (or minimal discrepancy) (Hooper et al., 2008);

    •. CMIN/DF: is the ratio χ2/degrees of freedom (df), the result should be ≤ 5 (Bentler & Bonett, 1980);

    •. GFI (Goodness-of-Fit Index) is the combined measurement provided by Σ for the relative amount of variance and covariance S. The range for this index ranges from 0 to 1, the closer to 1, better the adjustment (Hooper et al., 2008);

    •. NFI (Normed Fit Index): is generated by deriving the comparison between the independent model and the hypothetical model. This index has a range from 0 to 1, where values greater than 0.90 indicate a model with good adjustment (Bentler & Bonett, 1980);

    •. CFI (Comparative Fit Index): In order to take into account the size of the sample, since the adjustment for small samples is underestimated by NFI, CFI was created which is a fixed NFI. It has similar characteristics to the NFI (Hooper et al., 2008). The closer the CFI value is to 1 (ranges from 0 to 1), better is the adjustment of the model;

    •. Tucker-Lewis Index (TLI) or NNFI (Non-Normed Fit Index): provides interpretation similar to NFI. However, in the case of NNFI, it is considered an adjustment to the complexity of the data. Levels above 0.9 are considered satisfactory (Garver & Mentzer, 1999; Hair Junior et al., 2005);

    •. RMR (Root Mean Squares Residual): When there are no discrepancies between the observed covariances and those determined in the model, the RMR value is equal to 0 and therefore has a perfect fit. As discrepancies increase, this index also increases. According to Kline (1998), RMR of less than 0.10 is acceptable;

    •. RMSEA (Root Mean Square Error of Approximation): the index also known as the discrepancy per degree of freedom, or simply by RMS or RMSE, it has great informative character regarding modeling of covariance structures. There is a good fit when the index is less than 0.05, when there is a reasonable error in the approximation with the population there is the indication that the value of the index was greater than 0.08 (Maccallum et al., 1996).

In the estimation stage, for each construct a measurement model was produced, from which there was the specification of relations between the latent variable and the observable elements. It is possible to use just one indicator, but the researcher would have to provide reliability estimates.

The Maximum Likelihood (ML) was chosen to model estimation, once the variables have multivariate normality.

Adjustment indices report model peculiarities, therefore it is necessary the analysis of all indices know if the model can be accepted or not.

The hypothesis test was performed in order to calculate the level of significance of the estimated coefficients, being p values ​​greater than 0.05 considered as favorable.

Finally, for a hypothesis to get a tolerable acceptance, the Critical Ratio (CR or z) of the indicators must be greater than 1.96. However, the CR should ideally be greater than 2.58 in order to provide adequate significance.

4. Results and discussion

The construct Infrastructure, Course Organization, Course Demands, Teacher Involvement and Student Interest were built and validated by Confirmatory Factor Analysis (CFA).

First, each of the constructs and its variables were evaluated individually, so that the integrated model was built. Thus, the integrated model has been enhanced with the inclusion of amendments aimed at achieving the best-adjusted model.

The results of the final estimate of the proposed model are shown in Figure 1 and Tables 1 and 2.

Figure 1 Diagram of the Integrated Model - final model. Source: Elaborated by the authors with AMOS 22. 

Table 1 Standardized coefficients and significance of the variables of the Integrated Model - final model. 

Relationship between Constructs Standardized coefficients Z Sig.
Teacher_Involvement <--- Infrastructure 0.173 7.236 ***
Teacher_Involvement <--- Course_Organization 0.842 16.847 ***
Teacher_Involvement <--- Course_Demands 0.087 2.819 ***
Student_Interest <--- Course_Demands 0.609 12.412 ***
CO_RS_S54 <--- Student_Interest -0.080 -2.758 ***
CO_RS_S54 <--- Teacher_Involvement 0.761 20.922 ***

***Significant at 1%;

Source: Authors (2016).

Table 2 Adjustment indices of the Integrated Model - final model. 

Index Value
Chi-square 1766.158
p 0
Degrees of freedom 304
GFI 0.927
CFI 0.926
NFI 0.913
TLI 0.915
RMR 0.029
RMSEA 0.052

Source: Authors (2016).

When considering Table 2, it can be noticed that the chi-square is significant since its value is equal to 0 (should be greater than 0.05), so the matrix observed and the original matrix is not similar. When the absolute adjustment ratios, GFI, RMR, and RMSEA are observed, we noticed that the values are desirable. It occurs because GFI is greater than 0.9, RMR is less than 0.1 and RMSEA is less than 0.08. The comparative adjustment indices, CFI, NFI, and TLI also had desirable values above 0.9.

The standardized coefficients of relations between the constructs were all significant, five relationships were positive, in other words, if there is an increase or decrease of a construct the same behavior on the other will happen. One of the relations was negative. It indicates that if there is an increase or decrease in the construct Student Interest, the Overall Satisfaction will have an opposite behavior.

Given the results of the standardized coefficients and their respective degrees of significance observed in Table 1, it becomes feasible to answer the hypotheses raised in this research. Thus, the following claims are possible:

Hypothesis 1: A higher degree of Infrastructure will lead to a higher level of Teacher Involvement. Hypothesis 1 is accepted, since it is significant to 1%. In addition, as its standardized coefficient is positive, the behavior of both constructs is the same. If a University has a good level of Infrastructure, the Teacher Involvement also will be satisfactory and will grow to the extent that the Education Institution invests in improving its physical space, equipment and materials that provide quality lessons. The reverse is also true. If the University leaves its infrastructure to be desired, the Teacher Involvement will be lower than expected for the dissemination of diverse content. This content composes the Industrial Engineering class schedule, which will certainly affect the understanding of the disciplines by students.

Hypothesis 2: A higher degree of Infrastructure will lead to a higher level of Student Interest in the course. This hypothesis was rejected, because as seen in Table 1, the relationship between the constructs Student Interest and Infrastructure was not statistically significant.

Hypothesis 3: A higher level of Course Demands will lead to a lower level of Teacher Involvement. Hypothesis 3 is rejected because despite being significant at 1%, its standardized coefficient is positive; the behavior of both constructs will be the same. If the Course Demands are presented with a satisfactory level, Teacher Involvement also had a sufficient level to meet the students. So, if teachers can well develop the activities and work in the classroom, the student interest grows to the same extent. When the teacher cannot inform the content as expected, either for lack of resources at the University or even disinterest of the class, the Course Demands impact negatively on the Teacher’s Interest. The teacher ends up unmotivated and shows no interest in teaching and whether knowing if the student actually learned what was taught.

Hypothesis 4: A higher level of Course demands will lead to a lower level of Student Interest. Hypothesis 4 is rejected because despite being significant at 1%, its standardized coefficient is positive; the behavior of both constructs will be the same. If the Course Demands presents a satisfactory level, the Student Interest also responds similarly. So, if teachers can well develop the activities and work in the classroom, the student will have more interest in learning course content. However, if teachers do not present the contents in a nice way, students will evaluate negatively the discipline and even the teacher himself. They will not pay attention to what teachers are teaching them.

Hypothesis 5: A higher level of Course Organization will lead to a higher level of Teacher Involvement. Hypothesis 5 is accepted, as it showed the highest standardized coefficient (0.842), which implies a strong relationship between the constructs, as well as being significant to 1%. As the standardized coefficient of the relationship between Course Organization and Teacher Involvement is positive, the behavior of both constructs will be the same. Thus, if Course Organization shows a high level, it is expected that the Teacher Involvement level is also high. When the teacher can set an appropriate schedule for the discipline in order to cover each content at the right time and systematically feel more excited to teach and also seeks to know whether the student actually understands what was taught. However, if the content sequence is not well defined from the beginning, the teacher can be late to teach a given subject within the discipline, and virtually has no time to answer questions from students.

Hypothesis 6: A higher level of Course Organization will lead to a higher level of Student Interest. Hypothesis 6 is rejected, because as seen in Table 1, the relationship between the constructs Student Interest and Infrastructure was not statistically significant.

Hypothesis 7: A higher level of Teacher Involvement will lead to a higher level of Overall Satisfaction by Industrial Engineering Students. Hypothesis 7 is accepted as it is significant to 1%. In addition, its standardized coefficient is positive, indicating that the behavior of both constructs will be the same. Thus, if the construct Teacher Involvement increases, the Overall Satisfaction also grows. When the teacher shows enthusiasm and courage to teach the student is motivated to learn the content. The interest of the teacher in knowing the students were able to understand the content is essential to establish a link between the parties, so the student realizes the teacher as someone able to help him to understand the discipline and not a tyrant who intended to reprove him. However, if the Teacher Involvement is minimal, the Overall Satisfaction will also fall to minimum levels. Those apathetic teachers, who are not interested whether the students understand what is being explained, will certainly have unsatisfied students with their teaching methods.

Hypothesis 8: A higher level of Student Interest will lead to a higher level of Overall Satisfaction by Industrial Engineering Students. Hypothesis 8 is rejected, because despite being significant at 1%, its standardized coefficient is negative, indicating that the behavior of the constructs will be otherwise. Thus, a high Student Interest indicates a low Overall Satisfaction level. The fact that when the student needs to pay attention to a given content, he does not really feel satisfied, explains this low level. To achieve a satisfactory grade requires the understanding of a specified matter. If a discipline demands a little intellectual challenge to get a good grade, it is much easier; therefore, they will be more satisfied.

Thus, eight hypotheses were elaborated initially, three of them were accepted, they are Hypothesis 1, Hypothesis 5 and 7. Hypotheses 2, 3, 4, 6 and 8 were rejected.

5. Final considerations

To make students not only from the Industrial Engineering course, but any other ones, to reach motivation to remain until the end of academic training, it is necessary to consider their satisfaction as a key factor. The education of Higher Education institutions should provide a quality service in order to disseminate appropriate knowledge and put in the labor market qualified professionals to meet the society needs. Therefore, to understand which variables influence the satisfaction of academics is a strategic point for the survival of any university that intends to provide an educational service.

We estimated the construct Infrastructure, Course Demands, Course Organization, Teacher Involvement and Student Interest using the technique of the structural equation modeling. This technique had tested hypotheses concerning the Overall Satisfaction.

From the eight hypotheses defined initially, three of them were accepted. They were Hypothesis 1, Hypothesis 5 and Hypothesis 7.

The Hypothesis 2 relates the space of classrooms, materials and laboratory equipment and library resources with student interest in learning certain content. This hypothesis is rejected, so the Infrastructure does not affect the Student Interest. Hypothesis 6 is also rejected, which means that the way the teacher presents the curriculum of the discipline has no influence on the Student Interest in learning content.

In Hypotheses 3 and 4, there is a sign that the behavior of the constructs should be otherwise, they are therefore refuted. Thus, the way the teacher shows content to students and collecting activities leads to a higher level of Teacher Involvement and Student Interest. It is important to emphasize that Course Demands was the only construct able to influence the Student Interest in this research.

Neither Infrastructure nor Course Organization was significant to the Student Interest. Student motivation to the challenges required by the Course Demands plays an important role in the teaching-learning construction. Also, when the student is engaged in learning, he assigns to the teacher credit for the content taught, and the teacher feels interested in teaching. Thus, there is the teacher-student interaction that promotes greater effectiveness in education, as the student feels at ease to discuss, question and answers his questions during class.

Unlike appointed by Clayson & Haley (1990), in this research, a higher level of Course Demands implies a higher level of both Teacher Involvement and Student Involvement, so the scenario for Industrial Engineering Private Universities is encouraging. Students prefer a demanding course and that, in fact, provides them consistent learning to solve problems inherent in their future profession.

As shown in Paswan & Young (2002) research, many students can see the course as a means to get a degree or a job, instead of perceiving it as a unique learning experience. Feelings of discouragement or indifference to the course and the teacher may appear when the student realizes that the Course Demands are reducing their leisure time and activities that they really consider as pleasurable. Students often find that a lower requirement of the course may result in the rapid completion of tasks, which creates a sense of relief and perhaps conquest.

Hypothesis 8 had a negative standardized coefficient, indicating opposite behavior from the constructs. So either a higher level of Student Interest will lead to a lower level of Overall Satisfaction by Industrial Engineering Students or a lower level of Student Interest will lead to a higher level of Overall Satisfaction by Industrial Engineering Students. As Hypothesis 8 stands for the similar behavior of Student interest and Overall Satisfaction was also refuted.

More interested and participating students are usually also the most demanding since they assess critically with the service offered by the educational institution. This is what a recent study by Moreira Junior et al. (2015) shows, where 49.6% of students show that they have some sort of dissatisfaction, especially those related to Teacher Involvement, such as the teacher’s ability to motivate students and keep them aware of the content, the applicability of the themes addressed and curricular interaction with the discipline. So students very interested can become unsatisfied when realizing that their expectations are not satisfied.

The opposite can also be true, many students are satisfied with indulgence, so the fewer challenges the course offers more content they will be due to remaining more time to engage in activities they find most enjoyable. However, as many teachers would not incite in students the need to learn certain content, there is a pseudo-satisfaction. And this lack of satisfaction generates this indulgence. In order to strengthen Teacher Involvement as essential for students to leave the self-indulgence and become active elements of teaching and learning, Naimie et al. (2010) research shows the need for teachers to create situations that optimize the class learning outcomes. The author recommends that they seek to understand the claims and preferences of students to improve their academic achievements.

Hypothesis 1 and Hypothesis 7 show that the construct Teacher Involvement is a key element to achieve student Overall Satisfaction. So if the Universities invest in physical space, equipment, materials, and teachers can plan the curriculum of successful disciplines, Teacher Interest in educating the student and whether the student actually learned the content is high, consequently, Overall Satisfaction by Industrial Engineering Students will be satisfactory.

It should be noted that Hypothesis 5 is the one with the highest standardized coefficient (0.842), which implies a strong relationship between the constructs Course Organization and Teacher Involvement. Paswan & Young (2002) emphasize that more organized and structured courses lead the student to a more favorable assessment of the content taught by the teacher.

Students are smart consumers who know when the teachers are really investing energy and resources to make their course go well. To make a teaching-learning relationship to occur, it is not enough only the teacher willingness to present a given content. Another key point to remember is that students must seek knowledge. This knowledge should not be seen only as a way to be approved in the various course disciplines, otherwise, it will be a temporary knowledge and entirely limited to the time of that discipline. True knowledge acts as a lever to bring benefits to society during the full period.

Appendix A

Questions from Student Questionnaire 2011 grouped by constructs.

Question INFRASTRUCTURE
22 Are the general conditions of the physical facilities of classrooms, libraries and workplaces and study spaces for the course operation appropriate?
23 Are classrooms appropriate to the amount of students?
24 Are the lab facilities, equipment, materials and course specific support services adequate?
25 Are environments of specific practical classes appropriate to the amount of students?
26 Are equipment and / or materials available in the environments for practical classes enough for the number of students?
27 How does your institution provide access to the Internet for undergraduate students to meet course requirements?
28 How do you characterize the use of audiovisual and technological resources in your course?
30 Between times you had to use the library collection, were you able to access to the material?
31 How do you evaluate the library collection, regarding updates, given the curricular needs of your course?
32 Regarding the update, how do you evaluate the collection of scientific / academic journals available in the library?
Question COURSE ORGANIZATION
43 Does the course contextualize the knowledge of the area, such as theories, procedures, techniques, tools, etc., with the general themes and situations of everyday life of the Brazilian reality?
44 How do you evaluate your course curriculum in relation to the integration of the contents of different disciplines?
45 Does your course offer complementary activities?
49 Is your course financially supporting student participation in events (conferences, meetings, seminars, technical visits, etc.)?
50 How do you assess the course level of demand?
51 Do you think that your course contributes to the acquisition of general knowledge?
52 Do you think that your course contributes to the acquisition of theoretical training in the area?
53 Do you think that your course contributes to the preparation for professional practice?
Question COURSE DEMANDS
36 Do teachers ask in their disciplines to carry out research activities?
37 Do teachers indicate the use of textbooks as study material?
38 Do teachers indicate the use of specialized journal articles (papers) as study material?
39 Do teachers indicate the use of manuals or materials prepared by teachers in their disciplines?
40 Do course subjects require mastery of a foreign language?
Question INSTRUCTOR’S INVOLVIMENT
34 Most of the time, do school plans submitted by teachers contain the following: objectives, teaching methods and evaluation criteria, contents and bibliography of the discipline?
35 Do the contents worked with the majority of teachers are consistent with those presented in their teaching plans?
41 Are the teachers available to attend students outside of class time?
42 Do teachers demonstrate mastery of the discipline's content?
Question STUDENT’S INTEREST
29 How often do you typically use the library of your institution?
46 Have you participated in scientific initiation programs? Does this program contribute to your training?
47 Have you participated in mentoring programs? Does this program contribute to your training?
48 Have you participated in extension programs? Do these programs contribute to your training?
Question OVERALL SATISFATION
54 How do you assess the course’s contribution to your training?

How to cite this article: Santos Neto, A. S., Dantas, M. J. P., & Machado, R. L. (2017). Structural equation modeling applied to assess industrial engineering students’ satisfaction according to ENADE 2011. Production, 27(spe), e20162191. http://dx.doi.org/10.1590/0103-6513.219116

References

Abu-Duhou, I. (2002). Uma gestão mais autônoma das escolas. Brasília: UNESCO, IIEP. [ Links ]

Anderson, J. C., & Gerbing, D. W. (1988). Structural equation modeling in practice: a review and recommended two-step approach. Psychological Bulletin, 103(3), 411-423. http://dx.doi.org/10.1037/0033-2909.103.3.411. [ Links ]

Appleton-Knapp, S. L., & Krentler, K. A. (2006). Measuring student expectations and their effects on satisfaction: the importance of managing student expectations. Journal of Marketing Education, 28(3), 254-264. http://dx.doi.org/10.1177/0273475306293359. [ Links ]

Azevedo, J. C. (1999). A democratização da escola no contexto da democratização do estado: a experiência de Porto Alegre. In L. H. Silva (Ed.). Escola Cidadã: teoria e prática. Petrópolis: Vozes. [ Links ]

Bardagi, M. P., Lassance, M. C. P., & Paradiso, A. C. (2003). Trajetória acadêmica e satisfação com a escolha profissional de universitários em meio de curso. Revista Brasileira de Orientação Profissional, 4(1-2), 153-166. [ Links ]

Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588-606. http://dx.doi.org/10.1037/0033-2909.88.3.588. [ Links ]

Bernroider, E. W. N., & Schmöllerl, P. A. (2013). Technological, organisational, and environmental analysis of decision making methodologies and satisfaction in the context of it induced business transformations. European Journal of Operational Research, 224(1), 141-153. http://dx.doi.org/10.1016/j.ejor.2012.07.025. [ Links ]

Bollen, K. A. (1989). Structural equations with latent variables. New York: John Wiley & Sons. http://dx.doi.org/10.1002/9781118619179. [ Links ]

Brito, M. R. F. D. (2008). O SINAES e o ENADE: da concepção à implantação. Revista da Avaliação da Educação Superior, 13(3), 841-850. http://dx.doi.org/10.1590/S1414-40772008000300014. [ Links ]

Bronw, R. M., & Mazzarol, T. W. (2009). The importance of institutional image to student satisfaction and loyalty within higher education. Higher Education, 58(1), 81-95. http://dx.doi.org/10.1007/s10734-008-9183-8. [ Links ]

Byrne, B. M.(2010). Structural equation modeling with AMOS: basic concepts, applications, and programming. New York: Taylor & Francis. [ Links ]

Clayson, D. E., & Haley, D. A. (1990). Student evaluations in marketing: what is actually being measured? Journal of Marketing Education, 12(3), 9-17. http://dx.doi.org/10.1177/027347539001200302. [ Links ]

Coutinho, F. C. C. (2007). Avaliação da qualidade dos serviços de uma instituição de ensino superior (Dissertação de mestrado). Universidade Federal da Bahia, Salvador. [ Links ]

Douglas, J., Douglas, A., & Barnes, B. (2006). Measuring student satisfaction at a UK University. Quality Assurance in Education, 14(3), 251-267. http://dx.doi.org/10.1108/09684880610678568. [ Links ]

Farias, S. A., & Santos, R. C. (2000). Modelagem de equações estruturais e satisfação do consumidor: uma investigação teórica e prática. Revista de Administração Contemporânea, 4(3), 107-132. http://dx.doi.org/10.1590/S1415-65552000000300007. [ Links ]

Garver, N. S., & Mentzer, J. T. (1999). Logistics research methods: employing structural equation modeling to test for construct validity. Journal of Business Logistics, 20(1), 33-57. [ Links ]

Gonçalves Filho, C., Guerra, R. S., & Moura, A. I. (2004). Mensuração de satisfação, qualidade, lealdade, valor e expectativa em instituições de ensino superior: um estudo do Modelo ACSI através de equações estruturais. Revista Eletrônica de Gestão Organizacional, 2(1), 1-14. Retrieved in 4 February 2017, from http://www.revista.ufpe.br/gestaoorg/index.php/gestao/article/viewFile/90/80Links ]

Hair Junior, J. F., Babin, B., Money, A. H., & Samouel, P. (2005). Fundamentos de métodos de pesquisa em Administração. Porto Alegre: Bookman. [ Links ]

Hair Junior, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2009). Multivariate data analysiS. Englewood Cliffs: Prentice-Hall. [ Links ]

Hatch, J. A. (2002). Doing qualitative research in education settings. Albany: SUNY Press. [ Links ]

Hooper, D., Coughlan, J., & Mullen, M. R. (2008). Structural equation modelling: guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53-60. [ Links ]

Iacobucci, D. (2010). Structural equations modeling: fit indices, sample size, and advanced topics. Journal of Consumer Psychology, 20(1), 90-98. http://dx.doi.org/10.1016/j.jcps.2009.09.003. [ Links ]

Jöreskog, K. G., & Sörbom, D. (1982). Recent developments in structural equation modeling. Journal of Marketing Research, 19(2), 404-416. http://dx.doi.org/10.2307/3151714. [ Links ]

Kanan, H. M., & Baker, A. M. (2006). Student satisfaction with an educational administration preparation program: a comparative perspective. Journal of Educational Administration, 44(2), 159-169. http://dx.doi.org/10.1108/09578230610652042. [ Links ]

Kaplan, D. (2009). Structural equation modeling: foundations and extensions. Newbury Park: Sage. [ Links ]

Karimi, L., & Meyer, D. (2014). Structural equation modeling in psychology: the history, development and current challenges. International Journal of Psychological Studies, 6(4), 123-133. http://dx.doi.org/10.5539/ijps.v6n4p123. [ Links ]

Klem, L. (1995). Path analysis. In L. G. Grimm & P. R. Yarnold. Reading and understanding multivariate statistics. Washington: American Psychological Association. [ Links ]

Kline, B. (1998). Principles and practice of structural equation modeling. New York: The Guilford Press. [ Links ]

Koç, T., Turan, A. H., & Okursoy, A. (2016). Acceptance and usage of a mobile information system in higher education: an empirical study with structural equation modeling. The International Journal of Management Education, 14(3), 286-300. http://dx.doi.org/10.1016/j.ijme.2016.06.001. [ Links ]

Larina, L. N. (2015). Practical application of total quality management system to education of international students. Procedia: Social and Behavioral Sciences, 215(8), 9-13. http://dx.doi.org/10.1016/j.sbspro.2015.11.566. [ Links ]

Lidice, A., & Saglam, G. (2013). Using students’ evaluations to measure educational quality. Procedia: Social and Behavioral Sciences, 70(25), 1009-1015. http://dx.doi.org/10.1016/j.sbspro.2013.01.152. [ Links ]

Maccallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130-149. http://dx.doi.org/10.1037/1082-989X.1.2.130. [ Links ]

Marinho-Araujo, C. M. (2004). O desenvolvimento de competências no enade: a mediação da avaliação nos processos de desenvolvimento psicológico e profissional. Avaliação: Revista da Rede de Avaliação Institucional da Educação Superior, 9(4), 77-97. Retrieved in 4 February 2017, from http://periodicos.uniso.br/ojs/index.php/avaliacao/article/view/1289Links ]

Maruyama, G. M. (1998). Basics of structural equation modeling. London: Sage. [ Links ]

Masino, S., & Niño-Zarazúa, M. (2016). What works to improve the quality of student learning in developing countries? International Journal of Educational Development, 48, 53-65. http://dx.doi.org/10.1016/j.ijedudev.2015.11.012. [ Links ]

Moreira Junior, F. J., Zanella, A., Lopes, L. F. D., & Seidel, E. J. (2015). Avaliação da satisfação de alunos por meio do Modelo de Resposta Gradual da Teoria da Resposta ao Item. Ensaio: Avaliação e Políticas Públicas em Educação, 23(86), 129-158. http://dx.doi.org/10.1590/S0104-40362015000100005. [ Links ]

Murali, S., Pugazhendhi, S., & Muralidharan, C. (2016). Modelling and investigating the relationship of after sales service quality with customer satisfaction, retention and loyalty: a case study of home appliances business. Journal of Retailing and Consumer Services, 30, 67-83. http://dx.doi.org/10.1016/j.jretconser.2016.01.001. [ Links ]

Naimie, Z., Siraj, S., Abuzaid, R. A., & Shagholi, R. (2010). Did you cook your lesson based on right recipe? Accommodating the students preferences in class. Procedia: Social and Behavioral Sciences, 2(2), 383-387. http://dx.doi.org/10.1016/j.sbspro.2010.03.030. [ Links ]

National Confederation of Industry Brazil – CNI. (2013). Mais da metade dos estudantes abandona cursos de engenharia. Agência de Notícias CNI. Retrieved in 4 February 2017, from http://www.portaldaindustria.com.br/agenciacni/noticias/2013/07/mais-da-metade-dos-estudantes-abandona-cursos-de-engenharia/Links ]

Neelaveni, C., & Manimaran, S. (2016). Structural equation modeling for analysing the impact of quality of administrative practices in higher educational institutions. Quality & Quantity, 50(4), 1663-1674. http://dx.doi.org/10.1007/s11135-015-0227-8. [ Links ]

Paswan, A. K., & Young, J. A. (2002). Student evaluation of instructor: a nomological investigation using structural equation modeling. Journal of Marketing Education, 24(3), 193-202. http://dx.doi.org/10.1177/0273475302238042. [ Links ]

Pereira, B. A. D., Cavalheiro, E. A., Potrich, A. C. G., Campara, J. P., & Paraboni, A. L. (2016). Evaluation of satisfaction and students’ loyalty: tool validation in brazilian universities using the Model European Customer Satisfaction Index (ECSI). Revista Perspectivas Contemporâneas, 11(1), 100-110. Retrieved in 4 February 2017, from http://revista.grupointegrado.br/revista/index.php/perspectivascontemporaneas/article/view/1840Links ]

Petruzzellis, L., D’Uggento, A. M., & Romanazzi, S. (2006). Student satisfaction and quality of service in italian universities. Managing Service Quality, 16(4), 349-364. http://dx.doi.org/10.1108/09604520610675694. [ Links ]

Polidori, M. M., Marinho-Araújo, C. M., & Barreyro, G. B. (2006). SINAES: perspectives and challenges in evaluation of brazilian higher education system. Ensaio: Avaliação e Políticas Públicas em Educação, 14(53), 425-436. http://dx.doi.org/10.1590/S0104-40362006000400002. [ Links ]

Quevedo-Silva, F., Lima Filho, D. O., Sauer, L., & Reinert, J. N. (2012). Fatores discriminantes no grau de satisfação de estudantes de administração. Revista de Economia e Administração, 11(1), 28-45. [ Links ]

Sayed, Y., & Ahmed, R. (2015). Education quality, and teaching and learning in the post-2015 education agenda. international Journal of Educational Development, 40, 330-338. http://dx.doi.org/10.1016/j.ijedudev.2014.11.005. [ Links ]

Tinoco, M. A. C., & Ribeiro, J. L. D. (2007). Uma nova abordagem para a modelagem das relações entre os determinantes da satisfação dos clientes de serviços. Produção, 17(3), 454-470. http://dx.doi.org/10.1590/S0103-65132007000300005. [ Links ]

Tontini, G., & Sant’Ana, A. J. (2008). Interação de atributos atrativos e obrigatórios de um serviço na satisfação do cliente. Produção, 18(1), 112-125. http://dx.doi.org/10.1590/S0103-65132008000100009. [ Links ]

Vieira, K. M., Milach, F. T., & Huppes, D. (2008). Equações estruturais aplicadas à satisfação dos alunos: um estudo no Curso de Ciências Contábeis da Universidade Federal de Santa Maria. Revista de Contabilidade e Finanças, 19(48), 65-76. http://dx.doi.org/10.1590/S1519-70772008000300006. [ Links ]

Yin, D., & Lei, S. A. (2007). Impacts of campus involvement on hospitality student achievement and satisfaction. Education, 128(2), 282-293. [ Links ]

Received: August 27, 2016; Accepted: March 24, 2017

Creative Commons License This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.