## Services on Demand

## Journal

## Article

## Indicators

- Cited by SciELO
- Access statistics

## Related links

- Cited by Google
- Similars in SciELO
- Similars in Google

## Share

## Revista de Administração Pública

##
*Print version* ISSN 0034-7612*On-line version* ISSN 1982-3134

### Rev. Adm. Pública vol.54 no.3 Rio de Janeiro May/June 2020 Epub June 26, 2020

#### https://doi.org/10.1590/0034-761220190232x

ARTICLE

Efficiency in higher education. Empirical study in public universities of Colombia and Spain

^{1}Universidad del Cauca / Facultad de Ciencias Contables Económicas y Administrativas, Cauca - Colombia.

^{2}Universidad de Valencia / Facultad de Economía, Valencia - Spain.

In recent decades, Iberoamerican universities have introduced new quality assessment and accountability schemes, inspired by the New Public Management (NGP) model. In this context, efficiency in the distribution of public funds and obtaining the maximum possible return are a priority. Thus, measuring efficiency in the public sector, and specifically in higher education, has become a challenge for accounting science. The objective of this work is a proposal to calculate efficiency indices with Data Envelopment Analysis (DEA) models, introducing a previous step through the Analysis of Canonical Correlation (ACC). Using this technique, the aim is to improve discrimination capacity and overcome monodimensionality and lack of reliability in the representativeness of the chosen input and output variables. The study is applied in the public universities of Colombia and Spain during the years 2015 and 2016. The results obtained demonstrate the convenience of applying this preliminary step in the multivariate analysis. This reinforces the need to explore more rigorous methodologies in stages before and after the calculation of the efficiency indices. This practice increases confidence when using the indices to formulate policies and manage resources for the sector.

**Keywords: **higher education; canonical correlation analysis; data envelopment analysis DEA; efficiency; university rankings

En las últimas décadas, las universidades de Iberoamérica han introducido nuevos esquemas de evaluación de calidad y rendición de cuentas, inspirados en el modelo de la nueva gestión pública (NGP). En este contexto, la eficiencia en el reparto de los fondos públicos y la obtención del máximo rendimiento posible son una prioridad. Así, medir la eficiencia en el sector público, y específicamente en la educación superior, se ha convertido en un desafío para la ciencia contable. El objetivo de este trabajo es una propuesta para el cálculo de índices de eficiencia con modelos de análisis envolvente de datos (DEA), introduciendo un paso previo a través del análisis de correlación canónica (ACC). A través de esta técnica se pretende mejorar la capacidad de discriminación y superar la monodimensionalidad y falta de confiabilidad en la representatividad de las variables input y output elegidas. El estudio se aplicó en las universidades públicas de Colombia y España durante los años 2015 y 2016. Los resultados obtenidos demuestran la conveniencia de aplicar este paso preliminar en el análisis multivariante. Con ello, se refuerza la necesidad de explorar metodologías más rigurosas en etapas previas y posteriores al cálculo de los índices de eficiencia, que permitan generar confianza, a efectos de ser utilizados en la formulación de políticas y gestión de recursos para el sector.

**Palabras clave: **educación superior; análisis de correlación canónica; análisis envolvente de datos (DEA); eficiencia; productividad; calidad educativa; rankings universitarios

Nas últimas décadas, as universidades iberoamericanas introduziram novos esquemas de avaliação e prestação de contas, inspirados no modelo da Nova Gestão Pública (NGP). Nesse contexto, a eficiência na distribuição de recursos públicos e a obtenção do máximo retorno possível são uma prioridade. Assim, medir a eficiência no setor público, e especificamente no ensino superior, tornou-se um desafio para a ciência contábil. O objetivo deste trabalho é uma proposta para o cálculo de índices de eficiência com os modelos DEA (Data Envelopment Analysis), introduzindo uma etapa anterior da Análise de Correlação Canônica (ACC). O objetivo dessa técnica é melhorar a capacidade de discriminação e superar a monodimensionalidade e a falta de confiabilidade no quão representativas são as variáveis de entrada e saída escolhidas. O estudo é aplicado nas universidades públicas da Colômbia e Espanha durante os anos de 2015 e 2016. Os resultados obtidos demonstram a conveniência de aplicar esta etapa preliminar na análise multivariada. Isso reforça a necessidade de explorar metodologias mais rigorosas nas etapas antes e depois do cálculo dos índices de eficiência, os quais gerarão confiança, para serem utilizados na formulação de políticas e gestão de recursos para o setor.

**Palavras-chave: **ensino superior; análise de correlação canônica; análise de envelope de dados DEA; eficiência; produtividade; qualidade educacional; ranking universitário

1. INTRODUCTION

In recent decades, society has increasingly demanded an increase in transparency and accountability from public organizations. In response to this, and aiming to improve quality and ensure efficient use of public resources (C. R. M. ^{Silva & Crisóstomo, 2019}), most countries have introduced new management models in their institutions, inspired by the principles of new public management (NPM) (^{Andrews, Beynon, & McDermott, 2019}; ^{Broucker, De Wit, & Verhoeven, 2018}; ^{Lane, 2002}), introducing management techniques from the private sector.

Within this new paradigm, public higher education institutions have been pressured to improve their performance. Thus, many governments have implemented new regulations to professionalize universities in search of excellence. This market approach has fostered an interest in analyzing and comparing results between different universities, with particular emphasis on research (^{Mateos-González & Boliver, 2019}).

However, the increase in university quality should not be linked only to the university’s effectiveness, that is, achieving its objectives in terms of the number of publications, citations, or graduates (^{De-Juanas Oliva & Beltrán Llera, 2013}; ^{Giménez-Toledo & Tejada-Artigas, 2015}), regardless of the cost or effort required. It is also important to consider efficiency, that is, the relationship between resources used and output produced, something indisputable given extreme resource constraints (^{Gómez-Sancho & Mancebón-Torrubia, 2005}; ^{Mateos-Gonzalez & Boliver, 2019}).

In the public sector, the concepts of quality and efficiency should be inseparable. As stated by ^{Gómez-Sancho & Mancebón-Torrubia (2005}), it is hard to imagine that a high-quality university could be inefficient. Quality should also be associated with optimizing resource use, thus improving the services provided to the population and contributing to socioeconomic development (^{Debnath & Shankar, 2014}; ^{Tiana Ferrer, 2018}).

On this background, this study aims to calculate efficiency scores using data envelopment analysis (DEA) by applying a preliminary calculation with canonical correlation analysis (CCA). Our main interest lies in multivariate methodological questions that are conducive to overcoming the unidimensionality and unreliable representativeness of the selected input and output variables, to improve the discriminatory power of efficiency analysis. The study considers Colombian and Spanish public universities in 2015 and 2016. Its key contribution is not so much the numerical results obtained (efficiency scores) for each university evaluated but the discussion of various methodological aspects arising from the evaluation process: formulation, delimitation, significance, and representativeness of inputs and outputs specific to public universities; technique selection; model evaluation; and selection of the units of analysis.

The results obtained show the convenience of using CCA in a preliminary step in multivariate analysis to provide reliability and representativeness to the variables used for efficiency calculations in the public higher education sector. Colombian universities obtain high average efficiency scores (0.7107 and 0.7911) in 2015 and 2016 along with high inefficiency scores (0.2280 and 0.3792), with a high dispersion of the input and output data used in the calculation. Spanish universities show lower average efficiency (0.6537 and 0.5865) and dispersion levels. Eleven out of 32 Colombian universities and six out of 48 Spanish universities are fully efficient, showing that data refinement and the selection of appropriate methods increase the reliability of the final results and therefore their usefulness.

This study is subdivided into four sections: the first reviews the literature on the subject of public management and efficiency in higher education; the second describes the CCA and DEA models applied in the study and the variables and units of analysis involved; the third describes and analyzes the empirical results obtained; finally, the fourth proceeds to the discussion and conclusions.

2. NEW PUBLIC MANAGEMENT AND EFFICIENCY IN HIGHER EDUCATION

NPM emerged at the end of the 20th century from the need to use public resources with maximum efficiency, meet citizens’ demands, take advantage of the opportunities of a globalized and competitive world, and make societies more aligned with the collective will (^{Frey & Jegen, 2001}; ^{Agasisti & Haelermans, 2016}). Thus, NPM aims to create a more efficient and effective administration in areas where a better supplier is not found, eliminating bureaucracy, adopting more rational processes, and having greater administrative autonomy (^{García Sánchez, 2007}).

In this context, measuring public sector efficiency, specifically in higher education, becomes a challenge for accounting (A. F. ^{Silva, J. D. G. Silva, M. C. Silva & 2017}). However, measuring university efficiency is not trivial; in fact, obtaining an easy and objective measurement is one of the main problems (^{Moreno-Enguix, Lorente-Bayona, & Gras-Gil, 2019}).

Efficiency has been a widely addressed topic in the context of private and for-profit organizations and generally implies doing things well, i.e., ensuring adequate distribution of the means utilized relative to the outcomes achieved (^{Álvarez, 2001}). In the public sector, efficiency consists in optimizing resource use to obtain the maximum of goods and services in both quantitative and qualitative terms (^{Hauner & Kyobe, 2010}; ^{Mukokoma & Dijk, 2013}; ^{Peña 2008}; A. F. ^{Silva et al., 2017}; ^{Soto Mejía & Arenas Valencia, 2010}).

To assess organizational efficiency, it is necessary to specify a production function reflecting the process by which the entities under evaluation transform inputs into outputs (^{Johnes, 2006}; ^{Kuah & Wong, 2013}; A. F. ^{Silva et al., 2017}). To construct a production function for universities, their normal activities must be considered (^{Moncayo-Martínez, Ramírez-Nafarrate, & Hernández-Balderrama, 2020}). The productive efforts of universities involve simultaneously performing several activities of different kinds (activities related to the creation of knowledge—research activities—and its dissemination through teaching, transfer, and extension activities, along with other activities that universities perform as social agents) while sharing most resources (faculty, administrative and service staff, facilities, equipment, supplies, etc.).

Like any other public organization, universities find it difficult to assign monetary values to the inputs and outputs of their production process given that they both generate multiple outputs (e.g., graduates and publications) and use multiple inputs (e.g., speakers and facilities) (^{Kuah & Wong, 2013}).

Many studies have sought to facilitate efficiency calculation in higher education from various perspectives (^{Abbott & Doucouliagos, 2003}; ^{Avkiran, 2001}; ^{Bougnol & Dulá, 2006}; ^{Cloete & Moja, 2005}; ^{Fandel, 2007}; ^{Johnes, 2006}; ^{Johnes & Li, 2008}; ^{Moncayo-Martínez et al., 2020}; ^{Shi & Wang, 2004}). In the Ibero-American context, Colombia and Spain — which have undergone important transformations in the public management of their universities (^{Brunner & Miranda, 2016}) — have specifically seen the emergence of studies addressing efficiency measurement in higher education (^{García & González, 2011}; ^{González, Ramoni, & Orlandoni, 2017}; ^{Maza-Ávila, Quesada-Ibargüen, & Vergara-Schmalbach, 2013}; ^{Maza Ávila, Vergara-Schmalbach, & Román Romero, 2017}; ^{Melo-Becerra, Ramos-Forero, & Hernández-Santamarí, 2014}) that have aimed to assess and classify institutions, either to inform citizens or the government or to disclose their management capacity, impact, coverage, or social mission.

It should be noted many of these papers focus more on the relationship between institutional inputs and outputs than on their overall performance since they compare universities by the units with best practices, based on their ability to maximize outputs given some available inputs (^{Johnes, 2006}). The present study aims to provide a comprehensive view of both inputs and outputs — which must be relevant and significant — of the processes they engage in, by using appropriate methods, and of the results that ultimately allow the assessment and classification of the institutions. We finally aim to propose options for improvement and put forward individual and sector management policies.

4. METHODS AND DATA

4.1 CCA

A previous step before using DEA in efficiency analysis is to select the most representative variables. This step is very important since the variables used directly affect the final score. The selection of these variables seeks to obtain good discrimination between efficient and inefficient units and set a boundary that best fits the observed data. Despite their significance, few studies propose preliminary methods to construct the variables that best represent the set of inputs and outputs used for efficiency analysis (^{Azor Hernández, Sánchez García, & DelaCerda Gastélum, 2018}; ^{Friedman & Sinuany-Stern, 1997}; ^{Moreno Sáez & Trillo del Pozo, 2001}; Sabando ^{Vélez, & Cruz Arteaga, 2019}). To optimize this process, as a previous step before using DEA, the present study applies CCA to analyze the significance and representativeness of the selected variables (inputs, outputs) to calculate efficiency scores.

CCA is a linear, multivariate statistical analysis method (^{Hotelling, 1935}) used to identify, measure, and analyze associations between two sets of variables. While multiple regression predicts a single dependent variable from a set of independent variables, CCA facilitates the study of the interrelationships between multiple criterion variables (dependent) and multiple predictor variables (independent) (^{Badii & Castillo, 2007}; ^{Soto Mejía, Vásquez Artunduaga, & Villegas Flórez, 2009}; Soto Mejía & Arenas Valencia, 2010). The mathematical expression of CCA is:

CCA is a valuable tool in human factor research, as it involves a clear distinction between independent and dependent variables, multiple dependent variables, and the potential for multidimensional relationships between these two sets of variables.

4.2 Data and Variables

According to ^{Gómez-Sancho and Mancebón-Torrubia (2005}), it has not been possible to specify a generally accepted production function of higher education. Inputs are usually proxies of capital and labor factors. While for the labor factor there seems to be agreement on using the number of full-time equivalent faculty (^{Chang, Chung, & Hsu, 2012}; ^{Johnes, 2006}; ^{Laureti, Secondi, & Biggeri, 2014}; ^{Rhodes & Southwick, 1993}; ^{Sarafoglou & Haynes, 1996}; ^{Sav, 2012}), in the case of capital, the approaches are sufficiently different (infrastructure, technology, operating expenses, among others) that it continues to be an open topic for discussion. Outputs in all cases are related to the results of the two main activities in universities: teaching and research (^{Pérez-Esparrels & Gómez-Sancho, 2011}), measured, for example, by the number of graduates (^{Athanassopoulos & Shale, 1997}; ^{Avrikan, 2001}; ^{Laureti et al., 2014}; ^{Rhodes & Southwick, 1993}) and the number of publications (^{Chang et al., 2012}; ^{García-Aracil, 2013}; ^{Kao & Hung, 2008}; ^{Munoz, 2016}; ^{Kuah & Wong, 2013}), respectively. The conclusion is there is no definitive standard to guide the selection of inputs and outputs in assessing university efficiency (^{Kuah & Wong, 2013}). According to ^{Buitrago-Suescún et al. (2017}), the literature reports approximately 254 inputs and 230 outputs to measure education efficiency worldwide.

In the present study, starting from the bibliometric and systemic analysis in ^{Ramírez-Gutiérrez, Barrachina-Palanca, and Ripoll-Feliu (2019}) of the existing literature within the area of efficiency in higher education, we have selected those variables (see Table 1) that have had the most impact in previous studies and were available in the databases (National System of Higher Education Institutions of Colombia - SNIES, for its initials in Spanish, and Integrated University Information System of Spain - SIIU, for its initials in Spanish) of Colombia and Spain. The study periods are 2015 and 2016.

To formulate canonical functions, the smallest number of variables is considered, i.e., five for Colombia and three for Spain (see Table 1), since the number of possible canonical random variables (canonical dimensions) is equal to the number of variables in the smallest set (^{Badii & Castillo, 2007}).

COLOMBIA | SPAIN | ||||||
---|---|---|---|---|---|---|---|

INPUT VARIABLES (Independent) (Prdedictive) (Explicative) | OUTPUT VARIABLES (Dependent) (Criterion) (Explained) | INPUT VARIABLES (Independent) (Predictive) (Explicative) | OUTPUT VARIABLES (Dependent) (Criterion) (Explained) | ||||

PROF_TCE: | Full-Time Equivalent (FTE) Faculty | GRAD_PREG | Degree graduates | PROF_TCE | Full-Time Equivalent research and teaching staff | GRAD_PREG | Degree graduates |

ICAL_PDI | Quality index of teaching and research staff | GRAD_POST | Postgraduate graduates | ICAL_PDI | Percentage of pdi doctor | GRAD_POST | Masters graduates |

NUM_ADM | Full-Time Equivalent Administrative Staff | GRUP_INV | Research Groups | NUM_ADM | Administrative and service staff PAS | PUB_WOS | Publications in Scopus and Web of Science |

M2_USO_MISIONAL | Missionary space in m2 | REV_INDEX | Indexed Journals | TRANSF_EST | Total public transfers | ||

TRANSFER_NACION | Resources transferred by the State in COP | TOTAL_PUB_SCOPUS | Cumulative total number of Publications in Scopus | ||||

TOTAL: 10 Variables (5 inputs - 5 outputs) | TOTAL: 7 Variables (4 inputs - 3 outputs) | ||||||

INPUT VARIABLES | OUTPUT VARIABLES | ||||||

Variable name | Description | Variable name | Description | ||||

Full-Time Equivalent Faculty PROF_TCE | The conversion of time dedicated by professors to the universities is carried out. Per hour of teaching equivalent to 0.25 FTE, part-time equivalent to 0.5 FTE, and full-time to 1 FTE. | Degree graduates GRAD_PREG | Total students graduated from university undergraduate programs during each academic year. | ||||

Quality index of the teaching and research staff ICAL_PDI | This indicator is made taking into account the level of professor training. A rate is calculated = (graduates*5 + specialists*6 + masters*8 + doctors*10)/TotalFTE. This percentage or rate is already established in the Spanish university information system, but only for professors with doctoral training. | Postgraduate graduates GRAD_POST | Total students graduated from postgraduate programs (specializations, Masters, and Doctorates) during each academic year. | ||||

FTE administrative staff NUM_ADM | The information is contained in the information systems, and is reported by the Planning Offices. It is the number of people dedicated to the administrative functions of the universities. | Research Groups GRUP_INV | Total Research Groups categorized by Colciencias in Colombia, by university. A direct relationship is found between productivity in research terms and the number of categorized research groups. This variable is not used for the Spanish universities, since it does not operate in the same sense as previously described. | ||||

Missionary space in m2 M2_USO_ MISIONAL | It is considered an important resource, and a proxy of capital (installed capacity) for the universities. It must be reported from the Planning offices to the Ministry of National Education. Many studies in Colombia use it as an input (Garcia & Gonzalez, 2011; Maza Avila, Vergara Schmalbach, & Roman Romero, 2017; Melo, Ramos, & Hernandez, 2014; Ramos, Morero, Almanza, Picon, & Rodriguez, 2015; Rodriguez Murillo, 2014). This variable can only be obtained in the SNIES database of the Colombian Ministry of Education. For the Spanish universities, this variable is not available. | Indexed Journals REV_INDEX | Total Indexed Journals by university in Colombia. A direct relationships is found between productivity in research terms and the number of indexed journals, since in Colombia the journals are attached to university institutions. For the Spanish universities this variable is not used, since most of the journals are attached to entities of a different nature. | ||||

Resources transferred by the State in COP TRANSFER_ NACION | It is the monetary value of the public transfers to each of the public universities. Also known as current transfers or transfers of the autonomous communities. | Total number of publications accumulated in Scopus NUM_PUB_ SCOPUS | Total number of publications in the Scopus database. It only works for Colombian universities, since it is the database that consolidates production for most of them in terms of research articles. |

Source: Elaborated by the authors.

4.3 DEA

This model, developed by ^{Charnes, Cooper, and Rhodes (1978}), is a nonparametric and deterministic procedure to assess the relative efficiency of a set of homogeneous production units. Using the number of inputs consumed and outputs produced by each unit, and by linear programming techniques, DEA constructs, from the current best practice, the efficient production frontier against which the efficiency of each unit is evaluated (^{Salinas-Jiménez & Smith, 1996}).

The conceptual foundations of DEA were laid by ^{Farrel (1957}), who defined technical (relative) efficiency as the ability to achieve certain objectives through the desirable combination of certain inputs and products (^{Ramos Ruiz, Moreno Cuello, Almanza Ramírez, Picón Viana, & Rodríguez Albor, 2015}). Following Farrel (1957), DEA calculates efficiency from the following equation:

Where:

r = 1…s Subscript identifying an output

j = 1…n Subscript identifying the different decision-making units (DMUs)

i = 1…m Subscript identifying the input

j_{o} Subscript identifying the decision-making unit (DMU) for which the efficiency is being calculated.

h_{j0}Efficiency of the decision-making unit (DMU) that is being calculated

u_{r}Relative weight of the output yr for the DMU j0 that is being calculated.

v_{i}Relative weight of input xi in the DMU j0 that is being calculated.

The weights obtained (U_{r} and V_{i}) represent the values attributed to each input and output that provide the highest possible efficiency index to each decision-making unit (DMU). This weight combination, when applied to the rest of the DMUs must yield an efficiency indicator between 0 and 1. Thus, the objective is to find the DMUs producing the highest output levels from the lowest input levels. To do this, it maximizes the ratio of weighted outputs and weighted inputs for each DMU under consideration (^{Ray, 1991}).

Since 1978, according to ^{Soto Mejía and Arenas Valencia (2010}), many multivariate models have been developed. The two basic models, named after the initials of their innovators, are CCR (^{Charnes, Cooper, & Rhodes, 1981}) and BCC (^{Banker, Charnes, & Cooper, 1984}), which may differ in their orientation (inputs*,* outputs, or none), diversification, returns to scale (CRS: constant returns to scale; NIRS: nonincreasing returns to scale; NDRS: nondecreasing returns to scale; and VRS: variable returns to scale), measurement type (radial, nonradial, additive, multiplicative, hyperbolic, etc.), and other features. Equations (3), (4), (5), and (6) in Box 1 show the linear programming models for CCR (constant returns) and BCC (variable returns to scale), with input orientation and output orientation.

Source: Elaborated by the authors. Note: The BCC-O model aims to determine how much output could be obtained from the same level of inputs if all the DMUs were efficient, once the scale effects are removed.

The BCC model is designed to measure efficiency under variable returns. In this procedure, the inefficient DMUs are compared only with efficient units operating on a similar scale (^{Soto Mejía & Arenas Valencia, 2010}). The output-oriented BCC model is the best suited to evaluate the efficiency of public universities (^{Ramos Ruiz et al., 2015}; ^{Visbal-Cadavid, Mendoza Mendoza, & Causado Rodríguez, 2016}), as these may have different sizes in terms of the number of students, faculty, and/or financial resources allocated, may not control their inputs, and may rely on models of state funding and budget allocation. Thus, the BCC-O model aims to determine how much output could be obtained from the same level of inputs if all DMUs were efficient, once scale effects were eliminated.

4.4 Units of Analysis or DMUs

A DMU is the unit subject to efficiency measurement compared to others of its kind or typology. The DMU controls the process of transforming resources (inputs) into products. To identify the DMU, it must comply with an essential homogeneity characteristic, which is evident when it is verified that all DMUs use the same type of resource (inputs) to obtain the same type of output, albeit in different amounts (^{Soto Mejía & Arenas Valencia, 2010}).

Thus, Colombian and Spanish public universities can be seen as production units transforming resources into products. Each institution—treated as a DMU—can be considered a multiproduct organization (^{Ray, 1991}). This study focuses on 32 DMUs (Colombian public universities) and 48 DMUs (Spanish public universities) belonging to the public university system (SUE, for its initials in Spanish) of each country (see Table 2).

COLOMBIA - SUE | SPAIN - SUE | ||||
---|---|---|---|---|---|

No. | University Name (DMU) | Acronym | No. | University Name (DMU) | Acronym |

1 | University of Antioquia | udea | 1 | A Coruña | UDC |

2 | University of Caldas | unicaldas | 2 | Alcalá | UAH |

3 | University of Cartagena | unicart | 3 | Alicante | UA |

4 | University of Córdoba | unicord | 4 | Almería | UAL |

5 | University of Cundinamarca | udecun | 5 | Autonomous U. of Barcelona | UAB |

6 | University of the Amazon | uniamaz | 6 | Autonomous U. of Madrid | UAM |

7 | University of the Guajira | uniguajira | 7 | Barcelona | UB |

8 | University of the Llanos | unillanos | 8 | Burgos | UBU |

9 | University of Nariño | unariño | 9 | Cádiz | UCA |

10 | University of Pamplona | unipamp | 10 | Cantabria | UNICAN |

11 | University of Sucre | unisucre | 11 | Carlos III de Madrid | UC3M |

12 | University of the Atlantic | uniatlantico | 12 | Castilla-La Mancha | UCLM |

13 | University of Cauca | unicauca | 13 | Complutense U. of Madrid | UCM |

14 | University of Magdalena | unimag | 14 | Córdoba | UCO |

15 | University of the Pacific | unipac | 15 | Extremadura | UNEX |

16 | University of Quindio | uniquindio | 16 | Girona | UDG |

17 | University of Tolima | udetol | 17 | Granada | UGR |

18 | University of Valle | univalle | 18 | Huelva | UHU |

19 | Francisco José de Caldas University | udist | 19 | Illes Balears (Les) | UIB |

20 | Francisco de Paula Santander University - Cúcuta | ufpsc | 20 | Jaén | UJAEN |

21 | Francisco de Paula Santander University - Ocaña | ufpso | 21 | Jaume I de Castellón | UJI |

22 | Industrial University of Santander | uis | 22 | La Laguna | ULL |

23 | Military University - Nueva Granada | militar | 23 | La Rioja | UNIRIOJA |

24 | National Open and Distance University UNAD | unad | 24 | Las Palmas de Gran Canaria | ULPGC |

25 | National university of Colombia | unal | 25 | León | UNILEON |

26 | National Pedagogical University of Colombia | upnal | 26 | Lleida | UDL |

27 | Pedagogical and Technological University of Colombia - UPTC | uptc | 27 | Málaga | UMA |

28 | Popular University of Cesar | upoc | 28 | Miguel Hernández de Elche | UMH |

29 | Surcolombiana University | unisur | 29 | Murcia | UM |

30 | Technological University of Pereira | utp | 30 | National University of Distance Education | UNED |

31 | Technological University of Chocó | utch | 31 | Oviedo | UNIOVI |

32 | University - College of Cundinamarca | ucolm | 32 | Pablo de Olavide | UPO |

33 | Basque Country/Euskal Herriko Unibertsitatea | EHU | |||

34 | Polytechnic U. of Cartagena | UPCT | |||

35 | Polytechnic U. of Catalonia | UPC | |||

36 | Polytechnic U. of Madrid | UPM | |||

37 | Polytechnic U. of Valencia | UPV | |||

38 | Pompeu Fabra | UPF | |||

39 | Public U. of Navarra | UPNA | |||

40 | Rey Juan Carlos University | URJC | |||

41 | Rovira i Virgili | URV | |||

42 | Salamanca | USAL | |||

43 | Santiago de Compostela | USC | |||

44 | Sevilla | US | |||

45 | València (Estudi General) | UV | |||

46 | Valladolid | UVA | |||

47 | Vigo | UVIGO | |||

48 | Zaragoza | UNIZAR |

Source: Elaborated by the authors.

5. RESULTS

5.1 Significance Analysis by CCA

Table 3 shows the canonical correlation scores and the multivariate dimensional analysis for five independent and five dependent variables selected for the Colombian SUE and four independent and three dependent variables processed for the Spanish SUE.

The most important canonical correlation index for each country is that of Function 1 (0.9078 for Colombia and 0.8779 for Spain). A significant relationship between the two sets of variables is established at the 1% level, representing the greatest possible correlation between any linear combination of independent variables (faculty, teaching quality, administrative staff, and public transfers) and any linear combination of dependent variables (graduates, postgraduates, and publications).

Colombia | Spain | ||||||||
---|---|---|---|---|---|---|---|---|---|

Canonical Function | Canonical Correl. | Canonical R2 | F-Statistic | Prob. | Canonical Function | Canonical Correl. | Canonical R2 | F-Statistic | Prob. |

1 | 0,908 | 0,824 | 37,032 | 0.000* | 1 | 0,878 | 0,771 | 55,299 | 0.000* |

2 | 0,522 | 0,273 | 10,910 | 0 | 2 | 0,236 | 0,056 | 3,905 | 0,001 |

3 | 0,398 | 0,159 | 7,659 | 0 | 3 | 0,124 | 0,015 | 2,472 | 0,086 |

4 | 0,217 | 0,047 | 3,747 | 0,05 | |||||

5 | 0,038 | 0,001 | 0,425 | 0,515 | |||||

Multivalent contrast of significance | Multivalent contrast of significance | ||||||||

Statistical | Value | Approx F-Statistic | Probability | Statistical | Value | Approx F-Statistic | Prob. | ||

Wilks’ Lambda | 0,102 | 37,032 | 0 | Wilks’ Lambda | 0,213 | 55,299 | 0 | ||

Pillai’s Trace | 1,304 | 21,023 | 0 | Pillai’s Trace | 0,842 | 30,999 | 0 | ||

Hotelling’s Trace | 5,299 | 61,972 | 0 | Hotelling’s Trace | 3,436 | 90,105 | 0 | ||

Roy’s Largest Root | 4,685 | 279,204 | 0 | Roy’s Largest Root | 3,362 | 267,267 | 0 |

Source: Authors’ calculation (Stata Software) based on data from the Colombian Ministry of National Education (2015-2016) and the University Information System of Spain SIIU (2015-2016). Note: The statistical test used to evaluate the significance of the respective correlation indices is Wilks’ Lambda, contrasted with the F-test.

Note that the highest coefficient of determination (canonical R^{2}) corresponds to the first pair of canonical variables (U_{1}, V_{1}) (R^{2} can = 0.8241 for Colombia and 0.7707 for Spain), which are high values indicating high practical significance (^{Badii & Castillo, 2007}). These results mean 82.41% of the variability of U_{1} (linear combination of dependent variables) is explained by V_{1} (linear combination of independent variables). This preliminary approach demonstrates for each public university system the importance and representativeness of the selected variables and the explanatory power of the set of independent variables with respect to the set of dependent variables. Both sets of variables (inputs, outputs), specifically selected for each university system, have interdependencies with each other and a high explanatory value, which corroborates their selection as proxies to perform efficiency calculations using DEA models.

5.2 CCA Redundancy Analysis

*Amount of shared variance*. Table 4 shows the correlation between the first dependent canonical variable and each original dependent variable. Each correlation is interpreted as a factor loading, which identifies the value of each item’s relative contribution to its canonical item.

Colombia - SUE | Spain - SUE | ||
---|---|---|---|

Variable | Canonical Function 1 | Variable | Canonical Function 1 |

GRAD_PREG | 0,84 | GRAD_PREG | 0,738 |

GRAD_POST | 0,712 | GRAD_POST | 0,833 |

GRUP_INV | 0,79 | PUB_WOS | 0,977 |

REV_INDEX | 0,637 | ||

TOTAL_PUB_SCOPUS | 0,915 |

Source: Authors’ calculation (Stata Software) based on data from the Colombian Ministry of National Education (2015-2016) and the University Information System of Spain SIIU (2015-2016). Note: For both countries the variable with the greatest relative contribution is Scopus publications (Colombia) and WOS publications (Spain), followed by the graduates variables, indicating the two output variables most representative of two university activities (research and teaching).

*Redundancy rates.* In Table 5, the canonical Function 1 for Colombia shows a percentage of 50.78%, which is high, indicating the explanatory power of the input variables (independent) in the variances of the original (dependent) output variables. For Spain, this percentage is 57.06%, showing the variables associated with Spanish universities have a high explanatory power for the variability of their original outputs (^{Soto Mejía et al., 2009}; ^{Badii & Castillo, 2007})

Colombia | Spain | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|

Standardized variance of dependent variables explained by: | Standardized variance of dependent variables explained by: | ||||||||||

Own Theoretical Value (Shared Variance) | Opposite Theoretical Value (Redundancy) | Own Theoretical Value (Shared Variance) | Opposite Theoretical Value (Redundancy) | ||||||||

Canonical Function | % | % Accum. | Canonical R2 | % | % Accum. | Canonical Function | % | % Accum. | Canonical R2 | % | % Accum. |

1 | 61,62 | 61,62 | 82,41 | 50,78 | 50,78 | 1 | 74,04 | 74,04 | 77,07 | 57,06 | 57,06 |

2 | 11,40 | 73,02 | 27,26 | 3,11 | 53,89 | 2 | 9,78 | 83,82 | 5,56 | 0,54 | 57,61 |

3 | 8,06 | 81,08 | 15,85 | 1,28 | 55,16 | 3 | 7,26 | 91,08 | 1,53 | 0,11 | 57,72 |

4 | 13,09 | 94,17 | 4,73 | 0,62 | 55,78 | ||||||

5 | 5,83 | 100,00 | 0,14 | 0,01 | 55,79 | ||||||

Standardized variance of independent variables explained by: | Standardized variance of independent variables explained by: | ||||||||||

Own Theoretical Value (Shared Variance) | Opposite Theoretical Value (Redundancy) | Own Theoretical Value (Shared Variance) | Opposite Theoretical Value (Redundancy) | ||||||||

Canonical Function | % | % Accum. | Canonical R2 | % | % Accum. | Canonical Function | % | % Accum. | Canonical R2 | % | % Accum. |

1 | 73,20 | 73,20 | 82,41 | 60,32 | 60,32 | 1 | 74,12 | 74,04 | 77,07 | 57,12 | 57,12 |

2 | 6,87 | 80,07 | 27,26 | 1,88 | 62,19 | 2 | 9,49 | 83,53 | 5,56 | 0,53 | 57,65 |

3 | 7,25 | 87,32 | 15,85 | 1,15 | 63,34 | 3 | 16,47 | 100,00 | 1,53 | 0,25 | 57,90 |

4 | 8,18 | 95,50 | 4,73 | 0,38 | 63,72 | ||||||

5 | 3,95 | 99,45 | 0,14 | 0,00 | 63,73 |

Source: Authors’ calculation (Stata Software) based on data from the Colombian Ministry of National Education (2015-2016) and the University Information System of Spain SIIU (2015-2016).

Thus, for the first canonical correlation (Function 1), the independent canonical variables explain 82.41% (Colombia) and 77.07% (Spain) of the variance of the dependent canonical variables, while the first variables predict 50.78% and 57.06% of the variance in the original dependent variables, respectively. Dependent canonical variables predict 61.62% and 74.04% of the variance in the original dependent variables, and independent canonical variables predict 73.20% and 74.12% of the variance in the original independent variables. The above shows, in percentages, all the relationships and interdependencies between the two sets of variables and their respective linear combinations, providing reliability (Azor Hernández et al., 2018; ^{Friedman & Sinuany-Stern, 1997}; Moreno Sáez & Trillo del Pozo, 2001; Sabando ^{Vélez & Cruz Arteaga, 2019}) and encouraging researchers to continue using the proposed sets of variables to perform further efficiency calculations using DEA models.

Regarding independent theoretical values (canonical loadings), as seen in Table 6, the three items that significantly contribute to teaching and research activities in Colombian universities are the number of full-time faculty (prof_tce), the space available for missionary use (m2_uso_misional), and the resources transferred from the state (transfer_nacion). These variables imply high representativeness and significance as inputs and as explanatory variables for the outputs in each country.

The most significant dependent variables are degree graduates (grad_preg) and total Scopus publications (total_pub_scopus) for Colombia and postgraduates (grad_post) and total Web of Science publications (pub_wos) for Spain, consistent with previous studies (^{Kao & Hung, 2008}; ^{Kuah & Wong, 2013}; ^{Chang et al., 2012}; ^{García-Aracil, 2013}; ^{Munoz, 2016}), as they are representative variables for teaching and research activities in universities.

Colombia | Spain | ||||||
---|---|---|---|---|---|---|---|

Variables | Standar. Canon. Coeff. Function1 | Canonical Loadings Function 1 | Canonical cross Loadings Function 1 | Variables | Standar. Canon. Coeff. Function 1 | Canonical Loadings Function 1 | Canonical cross Loadings Function 1 |

Dependent | Dependent | ||||||

GRAD_PREG | 0,391 | 0,840 | 0,763 | GRAD_PREG | 0,220 | 0,738 | 0,647 |

GRAD_POST | 0,099 | 0,712 | 0,646 | GRAD_POST | 0,167 | 0,833 | 0,731 |

GRUP_INV | 0,105 | 0,791 | 0,718 | PUB_WOS | 0,715 | 0,977 | 0,858 |

REV_INDEX | 0,031 | 0,637 | 0,578 | ||||

TOTAL_PUB_SCOPUS | 0,545 | 0,915 | 0,830 | ||||

Independent | Independent | ||||||

PROF_TCE | 0,507 | 0,926 | 0,841 | PROF_TCE | 0,149 | 0,975 | 0,856 |

ICAL_PDI | 0,269 | 0,758 | 0,688 | ICAL_PDI | 0,093 | 0,363 | 0,319 |

NUM_ADM | -0,056 | 0,790 | 0,717 | NUM_ADM | 0,210 | 0,949 | 0,833 |

M2_USO_MISIONAL | 0,307 | 0,897 | 0,814 | TRANSF_EST | 0,629 | 0,990 | 0,869 |

TRANSFER_NACION | 0,113 | 0,851 | 0,773 |

Source: Authors’ calculation (Stata Software) based on data from the Colombian Ministry of National Education (2015-2016) and the University Information System of Spain SIIU (2015-2016).

5.3 DEA: BCC-O results

Following the preliminary results obtained with CCA, two models (sets of variables) are proposed to calculate efficiency in higher education using DEA.

*Model 1.* This method aims to perform efficiency calculations using DEA based on the representative and/or significant input and output variables explained by the results obtained from the canonical loadings shown in Table 6. Table 7 shows the descriptive details. The inferences are made for each set of universities (Colombian and Spanish) independently and to characterize their specific components, variables, data, and efficiency results, considering the aim is not to compare them but to demonstrate that the proposed method applies to the university sector of any country.

Colombia | ||||||
---|---|---|---|---|---|---|

Input Variables | Average | Min | Max | |||

2015 | 2016 | 2015 | 2016 | 2015 | 2016 | |

PROF_TCE | 591 | 648 | 59 | 91 | 2.426 | 2.501 |

M2_USO_MISIONAL | 117.732 | 117.732 | 20.278 | 20.278 | 491.956 | 491.956 |

TRANSFER_NACION* | 78.800 | 84.400 | 16.000 | 17.000 | 610.000 | 650.000 |

Output Variables | ||||||

GRAD_PREG | 2.199 | 1.593 | 218 | 124 | 6.793 | 5.705 |

TOTAL_PUB_SCOPUS | 1.115 | 1.445 | - | - | 13.704 | 17.419 |

Spain | ||||||

Input Variables | Average | Min | Max | |||

2015 | 2016 | 2015 | 2016 | 2015 | 2016 | |

PROF_TCE | 1.462 | 1.460 | 329 | 332 | 4.124 | 4.099 |

TRANSF_EST** | 141.000 | 147.417 | 36.100 | 36.000 | 363.000 | 400.000 |

Ouput Variables | ||||||

GRAD_POST | 1.054 | 1.260 | 122 | 131 | 3.306 | 4.622 |

PUB_WOS | 1.108 | 1.308 | 142 | 218 | 4.722 | 5.273 |

Source: Authors’ calculation (Stata Software) based on data from the Colombian Ministry of National Education (2015-2016) and the University Information System of Spain SIIU (2015-2016). Descriptive note: in Colombia, the number of full-time equivalent professors increased from 2015 to 2016, while infrastructure remained the same and graduates decreased. For the Spanish system, little variation is noted in the professors variable, while the graduates and publications increased. For both countries, transfers increased from one period to another: in Colombia an increase of 2.1% and in Spain an increase of 4.6%. ** Measured in thousands EUR. *Measured in millions COP.

*Model 2.* The prediction of the variables U_{1} and V_{1}
*,* corresponding to the canonical Function 1, is made with a coefficient of determination of 82.4% for the Colombian SUE and 77.07% for the Spanish SUE. This study proposes transforming the input and output variables into fictitious variables, a product of the canonical Function 1, as explained in the methods and data section using CCA. The variable U_{1} will be named U_{input}, and V_{1} will be V_{output.} The prediction model for each variable is shown in Table 8.

From raw canonical coefficients and canonical correlations, and based on the most representative variables (Model 1) and transformed variables (Model 2), efficiency scores are calculated using DEA models to facilitate the analysis.

Colombia | Spain | ||
---|---|---|---|

Input Variable (U_{input}) |
Output Variable (V_{output}) |
Input Variable (U_{input}) |
Output Variable (V_{output}) |

U_input = 0.6941prof_tce + 0.4802ical_pdi- 0.0735num_adm + 0.3520m2_uso_misional + 0.1169transfer_nacion | V_output = 0.5050grad_preg + 0.0435grad_post + 0.0824grup_inv + 0.0206rev_index + 0.2840total_pub_scopus | U_input = 0.2541prof_tce + 0.0097ical_pdi + 0.3524num_adm + 1.06transfer_nacion | V_output = 0.2539grad_preg + 0.2286grad_post + 0.9576pub_wos |

Source: authors’ calculation (Stata Software) based on data from the Colombian Ministry of National Education (2015-2016) and the University Information System of Spain (SIIU) 2015-2016.

The coefficients shown for each variable, both input and output, are obtained as unrotated or raw canonical coefficients, for each set of variables (dependent and independent), and they will be the*a priori* weights for each input and output, leaving a single variable as the input and another as the output.

Data are processed in Stata and DEA-solver software. Figure 1 shows the aggregate results from efficient and inefficient universities. Tables 10 and 11 show the disaggregated efficiency scores for each model, method, period, and university.

Figure 1 shows that, in Colombia, in 2015 and 2016, 34% of universities are fully efficient (11/32). In Spain, 14.5% (7/48) and 12.5% (6/48) of the public universities are considered fully efficient for 2015 and 2016, respectively.

Model 1 shows more efficient universities than Model 2. Its higher number of inputs and outputs limits the discriminatory power of the model, where some variables considered critical might be zero-weighted so that they do not affect relative calculations (^{Pedraja Chaparro, Salinas Jimenez, & Smith, 1994}).

The importance of the results shown by Model 2 is that, using a single input variable and a single output variable (transformed variables), it groups items with their respective weight coefficients, using the canonical function described in the method section above.

Table 15 shows that the average efficiency of Colombian universities is 0.7107 in 2015 and 0.7911 in 2016 (Model 1 BCC-O, 2015-2016; see Table 15). The average efficiency of Spanish universities, using the same model and periods, is 0.6537 and 0.5865.

Table 9 lists Colombian and Spanish universities with recurring relative efficiency, these being universities with scale effects. For Colombia, they are the Francisco de Paula Santander University-Ocaña and University of the Llanos; the latter is not considered efficient by any of the recent efficiency studies conducted on Colombian public universities (^{García & González, 2011}; ^{Ramos Ruiz et al., 2015}; ^{Rodríguez-Varela & Gómez-Sancho, 2015}; ^{Visbal-Cadavid et al., 2016}). For Spain, the consistent results of the University of La Rioja stand out, a university also deemed efficient by ^{Parellada and Duch (2006}) for the years 2003 and 2004.

COLOMBIA - SUE UNIVERSITIES | SPAIN - SUE UNIVERSITIES | |||||||
---|---|---|---|---|---|---|---|---|

Year 2015 | Year 2016 | Year 2015 | Year 2016 | |||||

Model 1 | Model 2 | Model 1 | Model 2 | Model 1 | Model 2 | Model 1 | Model 2 | |

Militar | Udetol | Upoc | Udist | UB | UAB | UB | UAL | |

Udecun | Uniquindío | Udecun | Unillanos | UCM | UIB | UCM | UMH | |

Ufpso | Ufpso | Ufpso | Unirioja | Unirioja | Unirioja | |||

Utch | Unillanos | UV | UPF | |||||

Unillanos | ||||||||

Source: Elaborated by the authors.

DMU | MODEL 1 | MODEL 2 | ||||||
---|---|---|---|---|---|---|---|---|

Efficiency Score CCRO | Efficiency Score BCCO | Efficiency Score CCRO | Efficiency Score BCCO | |||||

2015 | 2016 | 2015 | 2016 | 2015 | 2016 | 2015 | 2016 | |

udea | 1,000 | 1,000 | 1,000 | 1,000 | 0,979 | 0,964 | 0,979 | 0,968 |

unicaldas | 0,562 | 0,773 | 0,653 | 0,793 | 0,926 | 0,901 | 0,928 | 0,917 |

unicart | 0,536 | 0,628 | 0,564 | 0,636 | 0,921 | 0,896 | 0,923 | 0,912 |

unicord | 0,396 | 0,380 | 0,409 | 0,410 | 0,866 | 0,798 | 0,868 | 0,815 |

udecun | 0,546 | 0,862 | 1,000 | 1,000 | 0,756 | 0,775 | 0,791 | 0,806 |

uniamaz | 0,198 | 0,561 | 0,273 | 0,632 | 0,774 | 0,813 | 0,841 | 0,895 |

uniguajira | 0,293 | 0,356 | 0,437 | 0,379 | 0,784 | 0,754 | 0,822 | 0,771 |

unillanos | 0,381 | 0,776 | 1,000 | 1,000 | 0,829 | 0,850 | 0,980 | 1,000 |

unariño | 0,326 | 0,561 | 0,348 | 0,573 | 0,848 | 0,853 | 0,850 | 0,869 |

unipamp | 0,502 | 0,423 | 0,539 | 0,457 | 0,881 | 0,826 | 0,883 | 0,839 |

unisucre | 0,305 | 0,774 | 0,679 | 0,952 | 0,760 | 0,838 | 0,881 | 0,961 |

uniatlantico | 0,345 | 0,476 | 0,384 | 0,770 | 0,842 | 0,832 | 0,844 | 0,844 |

unicauca | 0,386 | 0,439 | 0,400 | 0,458 | 0,880 | 0,871 | 0,881 | 0,884 |

unimag | 1,000 | 0,710 | 1,000 | 0,710 | 0,968 | 0,876 | 0,979 | 0,895 |

unipac | 0,125 | 0,157 | 1,000 | 1,000 | 0,631 | 0,585 | 0,933 | 0,909 |

uniquindio | 0,797 | 0,383 | 0,871 | 0,450 | 0,939 | 0,781 | 1,000 | 0,797 |

udetol | 1,000 | 0,731 | 1,000 | 0,777 | 0,998 | 0,872 | 1,000 | 0,891 |

univalle | 0,877 | 0,948 | 0,885 | 0,949 | 0,967 | 0,954 | 0,968 | 0,962 |

udist | 1,000 | 1,000 | 1,000 | 1,000 | 0,966 | 0,978 | 0,969 | 1,000 |

ufpsc | 0,702 | 1,000 | 0,735 | 1,000 | 0,900 | 0,883 | 0,944 | 0,948 |

ufpso | 0,552 | 0,765 | 1,000 | 1,000 | 0,618 | 0,612 | 1,000 | 1,000 |

uis | 1,000 | 1,000 | 1,000 | 1,000 | 0,974 | 0,945 | 0,977 | 0,957 |

militar | 0,955 | 0,867 | 1,000 | 0,908 | 0,897 | 0,897 | 0,899 | 0,913 |

unad | 0,700 | 1,000 | 0,762 | 1,000 | 0,810 | 0,818 | 0,812 | 0,832 |

unal | 1,000 | 1,000 | 1,000 | 1,000 | 1,000 | 1,000 | 1,000 | 1,000 |

upnal | 0,217 | 0,584 | 0,228 | 0,624 | 0,827 | 0,825 | 0,833 | 0,841 |

uptc | 0,245 | 0,401 | 0,323 | 0,496 | 0,894 | 0,890 | 0,895 | 0,903 |

upoc | 0,411 | 0,903 | 0,519 | 1,000 | 0,858 | 0,856 | 0,907 | 0,913 |

unisur | 0,351 | 0,588 | 0,380 | 0,593 | 0,850 | 0,837 | 0,874 | 0,860 |

utp | 0,561 | 0,796 | 0,623 | 0,853 | 0,930 | 0,911 | 0,932 | 0,927 |

utch | 0,542 | 1,000 | 1,000 | 1,000 | 0,848 | 0,838 | 0,955 | 0,942 |

ucolm | 0,340 | 0,708 | 0,729 | 0,897 | 0,821 | 0,799 | 0,909 | 0,881 |

Source: Authors’ calculation (Stata Software) based on data from the Colombian National System of Information for Higher Education SNIES (2016-2016).

DMU | MODEL 1 | MODEL 2 | ||||||
---|---|---|---|---|---|---|---|---|

Efficiency Score CCRO | Efficiency Score BCCO | Efficiency Score CCRO | Efficiency Score BCCO | |||||

2015 | 2016 | 2015 | 2016 | 2015 | 2016 | 2015 | 2016 | |

EHU | 0,386 | 0,335 | 0,553 | 0,528 | 0,927 | 0,921 | 0,937 | 0,995 |

UA | 0,477 | 0,397 | 0,552 | 0,417 | 0,871 | 0,865 | 0,907 | 0,999 |

UAB | 1,000 | 1,000 | 1,000 | 1,000 | 0,992 | 0,904 | 1,000 | 0,911 |

UAH | 0,634 | 0,502 | 0,638 | 0,506 | 0,873 | 0,850 | 0,932 | 0,919 |

UAL | 0,517 | 0,483 | 0,593 | 0,560 | 0,814 | 0,869 | 0,904 | 1,000 |

UAM | 0,842 | 0,791 | 0,853 | 0,799 | 0,966 | 0,908 | 0,983 | 0,914 |

UB | 0,976 | 0,882 | 1,000 | 1,000 | 1,000 | 0,910 | 1,000 | 0,986 |

UBU | 0,367 | 0,341 | 0,664 | 0,720 | 0,764 | 0,903 | 0,964 | 0,968 |

UC3M | 0,540 | 0,525 | 0,615 | 0,574 | 0,868 | 0,896 | 0,922 | 0,922 |

UCA | 0,347 | 0,304 | 0,378 | 0,306 | 0,826 | 0,932 | 0,878 | 0,947 |

UCLM | 0,382 | 0,329 | 0,392 | 0,334 | 0,867 | 0,894 | 0,898 | 0,917 |

UCM | 0,593 | 0,542 | 1,000 | 1,000 | 0,950 | 0,917 | 0,977 | 0,930 |

UCO | 0,551 | 0,420 | 0,595 | 0,426 | 0,878 | 0,919 | 0,937 | 0,930 |

UDC | 0,460 | 0,369 | 0,490 | 0,384 | 0,852 | 0,861 | 0,910 | 0,934 |

UDG | 0,579 | 0,561 | 0,811 | 0,751 | 0,874 | 0,860 | 0,992 | 0,978 |

UDL | 0,510 | 0,463 | 0,782 | 0,704 | 0,834 | 0,921 | 0,996 | 0,927 |

UGR | 0,568 | 0,445 | 0,852 | 0,733 | 0,941 | 0,961 | 0,953 | 0,979 |

UHU | 0,385 | 0,363 | 0,476 | 0,450 | 0,790 | 0,959 | 0,901 | 0,980 |

UIB | 0,670 | 0,600 | 0,883 | 0,786 | 0,880 | 0,783 | 1,000 | 1,000 |

UJAEN | 0,473 | 0,452 | 0,533 | 0,471 | 0,838 | 0,784 | 0,920 | 0,974 |

UJI | 0,520 | 0,375 | 0,561 | 0,436 | 0,849 | 0,927 | 0,931 | 0,929 |

ULL | 0,399 | 0,357 | 0,442 | 0,382 | 0,863 | 0,942 | 0,905 | 0,949 |

ULPGC | 0,297 | 0,247 | 0,308 | 0,250 | 0,813 | 0,922 | 0,864 | 0,925 |

UM | 0,587 | 0,464 | 0,666 | 0,506 | 0,898 | 0,843 | 0,928 | 0,950 |

UMA | 0,358 | 0,372 | 0,436 | 0,509 | 0,872 | 0,951 | 0,885 | 0,990 |

UMH | 0,884 | 0,668 | 0,915 | 0,675 | 0,878 | 0,960 | 0,985 | 1,000 |

UNED | 1,000 | 1,000 | 1,000 | 1,000 | 0,864 | 0,841 | 0,918 | 0,942 |

UNEX | 0,454 | 0,396 | 0,463 | 0,409 | 0,849 | 0,855 | 0,902 | 0,973 |

UNICAN | 0,510 | 0,453 | 0,640 | 0,567 | 0,860 | 0,928 | 0,943 | 0,942 |

UNILEON | 0,426 | 0,371 | 0,546 | 0,488 | 0,830 | 0,931 | 0,933 | 0,931 |

UNIOVI | 0,459 | 0,408 | 0,471 | 0,411 | 0,885 | 0,905 | 0,908 | 0,941 |

UNIRIOJA | 0,348 | 0,349 | 1,000 | 1,000 | 0,759 | 0,917 | 1,000 | 0,918 |

UNIZAR | 0,416 | 0,415 | 0,421 | 0,435 | 0,907 | 0,940 | 0,913 | 0,943 |

UPC | 0,409 | 0,411 | 0,414 | 0,469 | 0,894 | 0,928 | 0,898 | 0,944 |

UPCT | 0,296 | 0,288 | 0,583 | 0,505 | 0,755 | 0,924 | 0,948 | 0,991 |

UPF | 1,000 | 1,000 | 1,000 | 1,000 | 0,917 | 0,925 | 1,000 | 0,931 |

UPM | 0,332 | 0,333 | 0,351 | 0,395 | 0,884 | 0,922 | 0,886 | 0,936 |

UPNA | 0,443 | 0,372 | 0,641 | 0,526 | 0,813 | 0,948 | 0,947 | 0,967 |

UPO | 0,753 | 0,632 | 0,968 | 0,794 | 0,825 | 0,910 | 0,970 | 0,946 |

UPV | 0,401 | 0,361 | 0,504 | 0,469 | 0,897 | 0,930 | 0,901 | 0,936 |

URJC | 0,877 | 0,784 | 0,898 | 0,842 | 0,873 | 0,946 | 0,949 | 0,968 |

URV | 0,597 | 0,547 | 0,734 | 0,650 | 0,888 | 1,000 | 0,995 | 1,000 |

US | 0,409 | 0,350 | 0,592 | 0,524 | 0,914 | 0,879 | 0,926 | 0,893 |

USAL | 0,536 | 0,427 | 0,552 | 0,428 | 0,895 | 0,983 | 0,927 | 0,998 |

USC | 0,491 | 0,417 | 0,566 | 0,438 | 0,897 | 0,889 | 0,907 | 0,904 |

UV | 0,698 | 0,529 | 1,000 | 0,807 | 0,964 | 0,911 | 0,968 | 0,932 |

UVA | 0,350 | 0,276 | 0,350 | 0,280 | 0,857 | 0,911 | 0,887 | 0,933 |

UVIGO | 0,594 | 0,479 | 0,701 | 0,515 | 0,878 | 0,940 | 0,922 | 0,943 |

Source: Authors’ calculation (Stata Software) based on data from the University Information System of Spain SIIU (2016-2016).

6. ANALYSIS OF RESULTS

To complement the analysis of the efficiency scores shown in Tables 11 and 12, these are distributed in quartiles (see Table 13) for each higher education system analyzed.

In Model 1, efficiency scores are lower than in Model 2, due to the sensitivity shown in the number of inputs and outputs, and variable transformation by previous CCA weighting.

As for the Colombian university system, Table 13 shows that 37.5% of the Colombian universities went from having scores above 0.88854 in 2015 to above 0.9517 in 2016 (Model 1), which shows improved efficiency, analyzed among the last 12 universities. Universities with low efficiency scores, located in the first quartile (8 institutions) showed scores below 0.4093 in 2015 and less than 0.5925 in 2016.

Of the Spanish universities, as shown in Table 13, 25% went from having scores above 0.852 in 2015 to higher than 0.733 in 2016 (Model 1), which implies a worsening from one period to the next for universities in the last quartile. Institutions with low efficiency scores, corresponding to the first 25%, showed scores below 0.4755 in 2015 and less than 0.4275 in 2016, confirming the downward trend in efficiency scores in 2016.

COLOMBIA - SUE | SPAIN - SUE | |||||||||
---|---|---|---|---|---|---|---|---|---|---|

QUARTILE | EFFICIENCY SCORES | QUARTILE | EFFICIENCY SCORES | |||||||

% ACCUM | BCC-O 2015 | BCC-O 2016 | % ACCUM | BCC-O 2015 | BCC-O 2016 | |||||

DMU | MODEL 1 | MODEL 2 | MODEL 1 | MODEL 2 | DMU | MODEL 1 | MODEL 2 | MODEL 1 | MODEL 2 | |

25% | 0,409 | 0,868 | 0,593 | 0,844 | 25% | 0,476 | 0,905 | 0,428 | 0,930 | |

50% | 0,729 | 0,909 | 0,853 | 0,903 | 50% | 0,593 | 0,928 | 0,509 | 0,943 | |

62,50% | 0,885 | 0,944 | 0,952 | 0,925 | 75% | 0,852 | 0,968 | 0,733 | 0,978 |

Source: Authors’ calculation (Stata Software) based on data from the Colombian Ministry of National Education (2015-2016) and the University Information System of Spain SIIU (2015-2016).

The efficiency scores are categorized in Table 14 to classify each public university, both in Colombia and Spain, for each model and period (2015-2016), into the following groups: fully efficient (index = 1), highly efficient (1 > index > average), and low-efficiency or inefficient (index < average).

COLOMBIA - SUE | ||||
---|---|---|---|---|

BCC-O 2015 | BCC-O 2016 | |||

MODEL 1 | MODEL 2 | MODEL 1 | MODEL 2 | |

AVERAGE EFFICIENCY INDEX | 0,711 | 0,914 | 0,791 | 0,902 |

FULLY EFFICIENT (EF= 1) | udetol | udetol | udea | unal |

udecun | ufpso | udecun | unillanos | |

unal | unal | udist | udist | |

unillanos | uniquindio | ufpso | ||

udist | uis | |||

ufpso | unal | |||

unimag | unillanos | |||

udea | utch | |||

uis | unad | |||

utch | ufpsc | |||

militar | upoc | |||

HIGH EFFICIENCY (EF > AVERAGE) | unipac | unillanos | unipac | ufpso |

univalle | udea | unisucre | udea | |

uniquindio | unimag | univalle | univalle | |

unad | uis | militar | unisucre | |

ufpsc | udist | ucolm | uis | |

ucolm | univalle | utp | ufpsc | |

utch | unicaldas | utch | ||

ufpsc | utp | |||

unipac | unicaldas | |||

utp | militar | |||

unicaldas | upoc | |||

unicart | unicart | |||

unipac | ||||

uptc | ||||

NOT EFFICIENT (EF < AVERAGE) | unisucre | ucolm | udetol | udetol |

unicaldas | upoc | uniquindio | ucolm | |

utp | militar | unimag | uniquindio | |

unicart | uptc | unicart | unimag | |

unipamp | unipamp | uptc | unipamp | |

upoc | unisucre | unipamp | unicauca | |

uniguajira | unicauca | unicauca | unisur | |

unicord | unisur | unisur | unicord | |

unicauca | unicord | unicord | unariño | |

uniatlantico | unariño | unariño | uniatlantico | |

unisur | uniatlantico | uniatlantico | uniamaz | |

unariño | uniamaz | uniamaz | upnal | |

uptc | upnal | upnal | uniguajira | |

uniamaz | uniguajira | uniguajira | unad | |

upnal | unad | udecun | ||

udecun | ||||

SPAIN - SUE | ||||

BCC-O 2015 | BCC-O 2016 | |||

MODEL 1 | MODEL 2 | MODEL 1 | MODEL 2 | |

AVERAGE EFFICIENCY INDEX | 0,654 | 0,937 | 0,587 | 0,951 |

FULLY EFFICIENT (EF= 1) | UAB | UAB | UAB | UMH |

UB | UB | UB | URV | |

UNED | UNIRIOJA | UNIRIOJA | UAL | |

UNIRIOJA | UPF | UPF | ||

UPF | UIB | UNED | ||

UCM | UCM | |||

UV | ||||

HIGH EFFICIENCY (EF > AVERAGE) | UPO | UDL | URJC | UIB |

UMH | URV | UV | UA | |

URJC | UDG | UAM | USAL | |

UIB | UMH | UPO | EHU | |

UAM | UAM | UIB | UPCT | |

UGR | UCM | UDG | UMA | |

UDG | UPO | UGR | UB | |

UDL | UV | UBU | UHU | |

URV | UBU | UDL | UGR | |

UVIGO | UGR | UMH | UDG | |

UM | URJC | URV | UJAEN | |

UBU | UPCT | UNEX | ||

UPNA | URJC | |||

UNICAN | UBU | |||

UCO | UPNA | |||

EHU | ||||

NOT EFFICIENT (EF < AVERAGE) | UPNA | UNILEON | UC3M | UM |

UNICAN | UAH | UNICAN | ULL | |

UAH | UJI | UAL | UCA | |

UC3M | UM | EHU | UPO | |

UCO | USAL | UPNA | UPC | |

UAL | US | US | UVIGO | |

US | UC3M | UVIGO | UNIZAR | |

UPCT | UVIGO | UMA | UNED | |

USC | UJAEN | UAH | UNICAN | |

UJI | UNED | UM | UNIOVI | |

EHU | UNIZAR | UPCT | UPV | |

UA | UDC | UNILEON | UPM | |

USAL | UNIOVI | UJAEN | UDC | |

UNILEON | USC | UPV | UVA | |

UJAEN | UA | UPC | UV | |

UPV | ULL | UHU | UNILEON | |

UDC | UAL | USC | UPF | |

UHU | UNEX | UJI | UCO | |

UNIOVI | UHU | UNIZAR | UCM | |

UNEX | UPV | USAL | UJI | |

ULL | UCLM | UCO | UDL | |

UMA | UPC | UA | ULPGC | |

UNIZAR | UVA | UNIOVI | UC3M | |

UPC | UPM | UNEX | UAH | |

UCLM | UMA | UPM | UNIRIOJA | |

UCA | UCA | UDC | UCLM | |

UPM | ULPGC | ULL | UAM | |

UVA | UCLM | UAB | ||

ULPGC | UCA | USC | ||

UVA | US | |||

ULPGC |

Source: Elaborated by the authors. Note: For the Colombian university system there are 11 fully efficient universities, 6 with high efficiency and 15 with low efficiency, for both periods (2015-2016), under model 1, and with all the representative input-output variables. The Colombian universities with the lowest score in 2015 and 2016 are upnal and uniguajira (0.2280 and 0.3792). In the Spanish system, 3 efficient universities in the years 2015 and 2016 stand out, both in constant and variable returns: the UB, the UNED and the UPF. The ULPGC is the institution with the lowest score in both periods (0.3083 and 0.2503).

For the Colombian SUE, ^{Ramos Ruiz et al. (2015}) calculate efficiency scores under DEA BCC-O models, classifying 13 and 15 institutions in the efficient category for the years 2007 and 2013, with average efficiency scores of 0.836 and 0.827, respectively. Some of these universities are still in that category in 2015 and 2016 in the present study (unal, udea, udetol, ufpso, and ufpsc). ^{Visbal-Cadavid et al. (2016}) classified 20 Colombian universities as fully efficient in 2011, also with the BCC-O model, and five universities (unal, udea, uis, ufpso, and udist) are still in that category. ^{García and González (2011}) classified 17 Colombian universities as efficient in the period 2003-2009, with an average efficiency index of 89%, and three of those classified in the present study as fully efficient (uis, udetol, and udist) are still in the top 10. ^{Rodríguez-Varela and Gómez-Sancho (2015)}, by applying variable returns, calculated efficiency scores for 2015 and found only three Colombian universities (unicord, uniatlantic, and udetol) to be fully efficient, of which only udetol appears in the classification for 2015 made in the present study, while the other two universities are considered to have low efficiency (below average).

Regarding the Spanish university system, Table 12 details the efficiency scores calculated for each university. For the year 2016, under the BCC-O method, there are only three efficient universities (UAL, URV, and UMH), which are also considered by ^{Gómez-Sancho and Mancebón-Torrubia (2012}) as efficient in research, and although they differ from those present in Model 1, this is surely due to the transformation of variables by CCA.

Although few studies have been published in the last 5 years at the level of Spanish public higher education institutions (^{Parellada & Duch, 2006}; ^{Vásquez Rojas, 2010}; ^{Gómez-Sancho & Mancebón-Torrubia, 2012}; ^{Martí-Selva, Puertas-Medina, & Calafat-Marzal, 2014}), since most have been conducted at the departmental level within universities, the most recent study (^{Martí-Selva et al., 2014}) stands out because it classified 18 Spanish universities as efficient for the year 2006, of which URV and UMH are still efficient in the present study in both 2015 and 2016.

^{Vásquez Rojas (2010}) reports average efficiency scores for Spanish universities for 2005 and 2007 that are very close to each other, 0.9608 and 0.9378, respectively, while the two values in the present study differ substantially, with average efficiency scores in 2015 and 2016 of 0.6537 and 0.5865, respectively.

Table 14 shows the aggregate of the calculated efficiency scores, for each public university system (Spain and Colombia), to evaluate the methods and models used in both periods (2015, 2016).

Colombian public universities show highly variable results, with high average efficiency and inefficiency scores, indicating high inequality in the public higher education system. The average efficiency score with Model 1 BCC-O was 0.7107 and 0.7911 in 2015 and 2016, respectively. The minimum efficiency was 0.1250 and 0.1568 for the same years.

Spanish public universities show low variability in the results and low average efficiency and inefficiency scores, indicating homogeneity in grouping data. The average efficiency index with Model 1 BCC-O was 0.6537 and 0.5865 in 2015 and 2016, respectively, and the minimum efficiency was 0.3083 and 0.2503.

CCR-O METHOD (Charnes, Cooper, and Rhodes. Constant returns - output-oriented) | ||||||||
---|---|---|---|---|---|---|---|---|

MODEL 1 | MODEL 2 | |||||||

SPAIN | COLOMBIA | SPAIN | COLOMBIA | |||||

2015 | 2016 | 2015 | 2016 | 2015 | 2016 | 2015 | 2016 | |

Average Efficiency | 0,544 | 0,483 | 0,567 | 0,705 | 0,873 | 0,909 | 0,867 | 0,848 |

Standard Deviation | 0,196 | 0,189 | 0,284 | 0,240 | 0,054 | 0,044 | 0,094 | 0,089 |

Minimum | 0,296 | 0,247 | 0,125 | 0,157 | 0,755 | 0,783 | 0,618 | 0,585 |

Maximum | 1,000 | 1,000 | 1,000 | 1,000 | 1,000 | 1,000 | 1,000 | 1,000 |

BCC-O METHOD (Banker, Charnes, and Cooper. Variable returns - output-oriented) | ||||||||

MODEL 1 | MODEL 2 | |||||||

SPAIN | COLOMBIA | SPAIN | COLOMBIA | |||||

2015 | 2016 | 2015 | 2016 | 2015 | 2016 | 2015 | 2016 | |

Average Efficiency | 0,654 | 0,587 | 0,711 | 0,791 | 0,937 | 0,951 | 0,914 | 0,902 |

Standard Deviation | 0,214 | 0,214 | 0,275 | 0,221 | 0,039 | 0,030 | 0,062 | 0,063 |

Minimum | 0,308 | 0,250 | 0,228 | 0,379 | 0,864 | 0,893 | 0,791 | 0,771 |

Maximum | 1,000 | 1,000 | 1,000 | 1,000 | 1,000 | 1,000 | 1,000 | 1,000 |

Source: Elaborated by the authors, based on the results processed in DEA-solver.

7. DISCUSSION AND CONCLUSIONS

Higher education has been facing challenges, not only in terms of its contribution to the generation and dissemination of knowledge to society but also in its use of limited resources to generate and disseminate knowledge. Challenges for this and the coming decades, such as internationalization and mobility, the empowerment of Ibero-American identity, and university social responsibility, are only possible if combined with a strengthening of university autonomy, governance, and funding through accountability, along with efficiency and effectiveness in resource use.

Thus, to be efficient and to appear as such — in terms of quality and excellence — to citizens and governments, is a challenge that has been raised by applying NPM methods to all kinds and categories of universities.

This study has sought to apply efficiency measurement to the Colombian and Spanish public higher education institutions, entering a field of research with all its complexities related to the very nature of this type of organization, which performs multiple activities, with multiple shared resources, to deliver multiple results.

Thus, the relevant contribution of the present study has been to put forth a specific method to specify a production function that can be applied in higher education, delving into the typology and quantification of relevant inputs and outputs and the mathematical description of their relationship.

For this purpose, CCA was used as a multivariate analysis method. CCA has been little used in this field but is useful to give significance and representativeness to the variables considered in order to calculate relative efficiency using DEA models. This study confirms the usefulness of this method in higher education systems in one or more countries (^{Wolszczak-Derlacz & Parteka, 2011}; ^{Rodríguez-Varela & Gómez-Sancho, 2018}; ^{Agasisti & Wolszczak-Derlacz, 2015}).

The results obtained allow us to contrast some hypotheses previously put forth about the number of universities classified as efficient with each method and model. With the BCC method, efficient DMUs always exceed those obtained with CCR (^{Ramos Ruiz et al., 2015}; ^{Visbal-Cadavid et al., 2016}).

As for higher education systems for each country, there is a close relationship with context and public policy. In Colombia, inequality stands out in terms of resources vs. products, as it has a higher number of efficient universities than Spain but a greater difference between efficient and inefficient universities. In Spain, there is greater data homogeneity, with lower and more clustered efficiency scores and a smaller difference between efficient and inefficient universities.

Other results on the efficiency of Colombian and Spanish higher education institutions differ significantly from the results obtained in this study, mainly due to methodological and sample aspects (delimitation of inputs and outputs of university activity, selection of the technique and evaluation model, and selection of the sample and periods). This corroborates the difficulty in making such comparisons (^{Gómez-Sancho & Mancebón-Torrubia, 2012}) and reinforces the great criticisms made of the homogenizing classifications that have prevailed since the beginning of the century.

Another contribution of this study lies in demonstrating that, even with independent and not necessarily related systems of public higher education in Ibero-America, a preliminary step of analysis can be performed with CCA to provide representativeness to the input and output variables necessary to calculate efficiency using DEA models. At this preliminary step, CCA can transform variables to reduce their number and generate *a priori* weightings, thereby improving their ability to discriminate and providing more accurate, reliable, and meaningful scores.

The most important conclusion is that, when addressing efficiency measurement in higher education, special care should be taken when selecting variables, methods, periods, and units to be evaluated. The stated objective should always be kept in mind, allowing comparability of resource management, improvement plans, and monitoring strategies. Therefore, this study highlights alternative research paths for efficiency measurement in higher education, as a public management priority.

REFERENCES

Abbott, M., & Doucouliagos, C. (2003). The efficiency of Australian universities: A data envelopment analysis. *Economics of Education Review*, *22*(1), 89-97. Recuperado de https://doi.org/10.1016/S0272-7757(01)00068-1
[ Links ]

Agasisti, T., & Haelermans, C. (2016). Comparing Efficiency of Public Universities among E uropean Countries: Different Incentives Lead to Different Performances.*Higher Education Quarterly*,*70*(1), 81-104.
[ Links ]

Agasisti, T., & Wolszczak-Derlacz, J. (2015). Exploring efficiency differentials between Italian and Polish universities, 2001-11. *Science and Public Policy*, *43*(1), 128-142.
[ Links ]

Álvarez, A (2001) *La Medición de la Eficiencia y la Productividad*. Madrid, ES: Editorial Pirámide.
[ Links ]

Andrews, R., Beynon, M. J., & McDermott, A. (2019). Configurations of New Public Management reforms and the efficiency, effectiveness and equity of public healthcare systems: a fuzzy-set Qualitative Comparative Analysis. *Public management review*, *21*(8), 1236-1260.
[ Links ]

Athanassopoulos, A. D., & Shale, E. (1997). Assessing the comparative efficiency of higher education institutions in the UK by the means of data envelopment analysis.*Education economics*, 5(2), 117-134.
[ Links ]

Avkiran, N. K. (2001). Investigating technical and scale efficiencies of Australian universities through data envelopment analysis. *Socio-economic planning sciences*, *35*(1), 57-80.
[ Links ]

Azor Hernandez, J. L., Sánchez García, J. E., & DelaCerda Gastélum, J. (2018). Generalization of the canonical correlation method applied to an economy problem Generalización del método de correlaciones canónicas aplicado a un problema de economía. *Revista Internacional de Gestión Del Conocimiento y La Tecnología*, 6(1), 1-14.
[ Links ]

Badii, M. H., & Castillo, J. (2007). Análisis de correlación canónica (ACC) e investigación científica. *Innovaciones De Negocios*, 4(2), 405-422.
[ Links ]

Banker, R. D., Charnes, A., & Cooper, W. W. (1984). Some models for estimating technical and scale inefficiencies in data envelopment analysis. *Management science*, *30*(9), 1078-1092.
[ Links ]

Bougnol, M. L., & Dulá, J. H. (2006). Validating DEA as a ranking tool: An application of DEA to assess performance in higher education. *Annals of Operations Research*, *145*(1), 339-365.
[ Links ]

Broucker, B., De Wit, K., & Verhoeven, J. C. (2018). Higher education for public value: taking the debate beyond New Public Management. *Higher Education Research* & *Development*, *37*(2), 227-240.
[ Links ]

Brunner, J. J., & Miranda, D. (2016). *Educación Superior en Iberoamérica*(Reporte 2016). Santiago, Chile: Cinda.
[ Links ]

Buitrago-Suescún, O. Y., Espitia-Cubillos, A. A., & Molano-García, L. (2017). Análisis envolvente de datos para la medición de la eficiencia en instituciones de educación superior: una revisión del estado del arte. *Revista Científica General José María Córdova*, *15*(19), 147-173. Recuperado dehttp://dx.doi.org/10.21830/19006586.84
[ Links ]

Chang, T. Y., Chung, P. H., & Hsu, S. S. (2012). Two-stage performance model for evaluating the managerial efficiency of higher education: Application by the Taiwanese tourism and leisure department.*Journal of Hospitality, Leisure, Sport* & *Tourism Education*,*11*(2), 168-177.
[ Links ]

Charnes, A., Cooper, W. W., & Rhodes, E. (1978). Measuring the efficiency of decision making units. *European Journal of Operational Research*, 2(6), 429-444.
[ Links ]

Charnes, A., Cooper, W. W., & Rhodes, E. (1981). Evaluating program and managerial efficiency: an application of data envelopment analysis to program follow through.*Management science*,*27*(6), 668-697.
[ Links ]

Cloete, N., & Moja, T. (2005). Transformation tensions in higher education: Equity, efficiency, and development. *Social Research*, *72*(3), 693-722.
[ Links ]

Debnath, R. M., & Shankar, R. (2014). Does good governance enhance happiness: a cross nation study. *Social indicators research*, *116*(1), 235-253.
[ Links ]

De-Juanas Oliva, Á, & Beltrán Llera, J. A. (2013). Valoraciones de los estudiantes de ciencias de la educación sobre la calidad de la docencia universitaria. *Educación XX1*, *17*(1), 59-82.
[ Links ]

Fandel, G. (2007). On the performance of universities in North Rhine-Westphalia, Germany: Government’s redistribution of funds judged using DEA efficiency measures. *European Journal of Operational Research*, *176*(1), 521-533.
[ Links ]

Farrel, M. J. (1957). The measurement of Productive efficiency. *Journal of the Royal Statistical Society. Series A (General)*, *120*(3), 253-290.
[ Links ]

Frey, B. S., & Jegen, R. (2001). Motivation crowding theory. *Journal of economic surveys*, *15*(5), 589-611.
[ Links ]

Friedman, L., & Sinuany-Stern, Z. (1997). Scaling units via the canonical correlation analysis in the DEA context. *European Journal of Operational Research*, *100*(3), 629-637.
[ Links ]

García Sánchez, I. M. (2007). La nueva gestión pública: Evolución y tendencias. *Presupuesto Y Gasto Público*, *47*, 37-64.
[ Links ]

García, A., González, M., (2011). *La evaluación de la eficiencia de las universidades públicas de Colombia utilizando el Análisis Envolvente De Datos (AED)*. Bucaramanga, Colombia: Universidad Industrial de Santander.
[ Links ]

García-Aracil, A. (2013). Understanding productivity changes in public universities: Evidence from Spain.*Research evaluation*,*22*(5), 351-368.
[ Links ]

Giménez-Toledo, E., & Tejada-Artigas, C. M. (2015). Proceso de publicación, calidad y prestigio de las editoriales científicas en educación. *EducaciónXX1*, *18*(1), 17-44.
[ Links ]

Gómez-Sancho, J. M., & Mancebón-Torrubia, M. J. (2005). Algunas reflexiones metodológicas sobre la evaluación de la eficiencia productiva de las instituciones de educación superior. *Ekonomiaz: Revista Vasca De Economía*, *60*(1), 140-167.
[ Links ]

Gómez-Sancho, J. M., & Mancebón-Torrubia, M. J. (2012). La evaluación de la eficiencia de las universidades públicas españolas: En busca de una evaluación neutral entre áreas de conocimiento. *Presupuesto y gasto público*, *67*(2), 43-70.
[ Links ]

González, A., Ramoni, J., & Orlandoni, G. (2017). Evaluación de la eficiencia de las Universidades Estatales Colombianas. Comunicaciones en Estadística,*10*(1), 83-100.
[ Links ]

Hauner, D., & Kyobe, A. (2010). Determinants of government efficiency. World Development, *38*(11), 1527-1542.
[ Links ]

Hotelling, H. (1935). Demand functions with limited budgets. *Econometrica: Journal of the Econometric Society*, 3(1), 66-78.
[ Links ]

Johnes, J. (2006). Data envelopment analysis and its application to the measurement of efficiency in higher education. *Economics of education review*, *25*(3), 273-288.
[ Links ]

Johnes, J., & Li, Y. U. (2008). Measuring the research performance of Chinese higher education institutions using data envelopment analysis. *China economic review*, *19*(4), 679-696.
[ Links ]

Kao, C., & Hung, H. T. (2008). Efficiency analysis of university departments: An empirical study.*Omega*,*36*(4), 653-664.
[ Links ]

Kuah, C. T., & Wong, K. Y. (2013). Data Envelopment Analysis modeling for measuring knowledge management performance in Malaysian higher educational institutions.*Information Development*,*29*(3), 200-216.
[ Links ]

Lane, J. E. (2002). *New public management: an introduction*. London, UK: Routledge.
[ Links ]

Laureti, T., Secondi, L., & Biggeri, L. (2014). Measuring the efficiency of teaching activities in Italian universities: An information theoretic approach.*Economics of Education Review*,*42*, 147-164.
[ Links ]

Martí-Selva, M. L., Puertas-Medina, R., & Calafat-Marzal, C. (2014). Calidad y eficiencia de las Universidades Públicas Españolas.*Revista de Estudios Regionales*,*99*, 135.
[ Links ]

Mateos-González, J. L., & Boliver, V. (2019). Performance-based university funding and the drive towards ‘institutional meritocracy’in Italy. *British Journal of Sociology of Education*, *40*(2), 145-158.
[ Links ]

Maza-Ávila, F. J., Quesada-Ibargüen, V. M., & Vergara-Schmalbach, J. C. (2013). Efficiency and productivity of the quality of education in municipalities in the State of Bolivar, Colombia.*Entramado*, 9(2), 28-39.
[ Links ]

Maza Ávila, F. J., Vergara Schmalbach, J. C., & Román Romero, R. (2017). Eficiencia y productividad en la cobertura de las Universidades públicas colombianas. Efficiency and productivity in access to Colombian public universities. *Investigación y Desarrollo*, *25*(2), 6-33.
[ Links ]

Melo-Becerra, L. A., Ramos-Forero, J. E., & Hernández-Santamarí, P. O. (2017). La educación superior en Colombia: situación actual y análisis de eficiencia. *Revista Desarrollo y Sociedad*, *78*, 59-111.
[ Links ]

Moncayo-Martínez, L. A., Ramírez-Nafarrate, A., & Hernández-Balderrama, M. G. (2020). Evaluation of public HEI on teaching, research, and knowledge dissemination by Data Envelopment Analysis. *Socio-Economic Planning Sciences*, *69*(100718), 1-15.
[ Links ]

Moreno-Enguix, M. D. R., Lorente-Bayona, L. V., & Gras-Gil, E. (2019). Social and Political Factors Affect the Index of Public Management Efficiency: A Cross-Country Panel Data Study. *Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement*, *144*(1), 299-313.
[ Links ]

Mukokoma, M. M. N., & van Dijk, M. P. (2013). New public management reforms and efficiency in urban water service delivery in developing countries: blessing or fad? *Public Works Management* & *Policy*, *18*(1), 23-40.
[ Links ]

Munoz, D. A. (2016), Assessing the research efficiency of higher education institutions in Chile: A data envelopment analysis approach.*International Journal of Educational Management*, *30*(6), 809-825.
[ Links ]

Parellada, M., & Duch, N. (2006). Descentralización autonómica y sistema universitario. *Mediterráneo Económico: Un balance del estado de las Autonomías*, *10*, 405-426.
[ Links ]

Pedraja Chaparro, F., Salinas Jimenez, J., & Smith, P. (1994). La restricción de las ponderaciones en el análisis envolvente de datos: Una fórmula para mejorar la evaluación de la eficiencia. *Investigaciones Económicas*, *18*(2), 365-380.
[ Links ]

Peña, C. R. (2008). Um modelo de avaliação da eficiência da administração pública através do método análise envoltória de dados (DEA). *Revista de Administração Contemporânea*, *12*(1), 83-106.
[ Links ]

Pérez-Esparrels, C., & Gómez-Sancho, J. M. (2011). Los rankings internacionales de las instituciones de educación superior y las clasificaciones universitarias en España: visión panorámica y prospectiva de futuro. In *Anales do*
*18º*
*Encuentro de Economía Pública*, Málaga, España.
[ Links ]

Ramírez-Gutiérrez, Z., Barrachina-Palanca, M., & Ripoll-Feliu, V. (2019). University Rankings disclosure and efficiency in higher education: A bibliometric analysis and systematic review. *Revista de educación*, *384*, 247-286.
[ Links ]

Ramos Ruiz, J. L., Moreno Cuello, J., Almanza Ramírez, C., Picón Viana, C. J., & Rodríguez Albor, G.(2015). *Universidades públicas en Colombia: Una perspectiva de la eficiencia productiva y la capacidad científica y tecnológica*. Barranquilla, Colombia: Universidad del Norte.
[ Links ]

Ray, S. C. (1991). Resource-use efficiency in public schools: A study of Connecticut data. *Management science*,*37*(12), 1620-1628.
[ Links ]

Rhodes, E. L. y Southwick, L. (1993). “Variations in public and private university efficiency”. *Public Policy Applications of Management Science*, 7, 145-170.
[ Links ]

Rodríguez-Varela, D., & Gómez-Sancho, J. M. (2018). La evaluación de la eficiencia en universidades públicas de Colombia y Chile. In *Anales do*
*28º*
*Jornadas de la asociación de economía de la educación*, Barcelona, España.
[ Links ]

Sabando Vélez, Yonaida Ismenia, & Cruz Arteaga, Kerly Cecilia (2019). La Metodología no Paramétrica Data Envelopment Analysis en la medición de la eficiencia de los programas de vinculación universitaria. *Revista Electrónica Cooperación Universidad Sociedad*, 4(2), 15-23.
[ Links ]

Salinas-Jiménez, J., & Smith, P. (1996). Data envelopment analysis applied to quality in primary health care.*Annals of Operations Research*,*67*(1), 141-161.
[ Links ]

Sarafoglou, N., & Haynes, K. E. (1996). University productivity in Sweden: a demonstration and explanatory analysis for economics and business programs.*The Annals of Regional Science*,*30*(3), 285-304.
[ Links ]

Sav, G. T. (2012). Productivity, efficiency, and managerial performance regress and gains in United States universities: a Data Envelopment Analysis.*Advances in Management and Applied Economics*, 2(3), 13.
[ Links ]

Shi, Q., & Wang, D. (2004). A new perspective for solving the contradiction between equity and efficiency in higher education. *Chinese Education* & *Society*, *37*(1), 72-88.
[ Links ]

Silva, A. F., Neto, Silva, J. D. G., & Silva, M. C. (2017). Análise da eficiência da gestão pública das capitais brasileiras. *Revista de Administração, Contabilidade e Sustentabilidade*, 7(2), 85-100.
[ Links ]

Silva, C. R. M., & Crisóstomo, V. L. (2019). Gestão fiscal, eficiência da gestão pública e desenvolvimento socioeconômico dos municípios cearenses. *Revista de Administração Pública*, *53*(4), 791-801.
[ Links ]

Soto Mejía, J. A., & Arenas Valencia, W. (2010). *Análisis envolvente de datos de la teoria a la práctica: Fundamentos teóricos y prácticos*. Pereira, Risaralda: Universidad Tecnológica de Pereira, Facultad de Ingeniería Industrial.
[ Links ]

Soto Mejía, J. A., Vásquez Artunduaga, S., & Villegas Flórez, J. A. (2009). Medición de la eficiencia en las instituciones educativas oficiales del municipio de dosquebradas (risaralda) 2007. *Scientia Et Technica*, 3(43), 95-99.
[ Links ]

Tiana Ferrer, A. (2018). Treinta años de evaluación de centros educativos en españa. *Educación XX1*, *21*(2), 17-36.
[ Links ]

Vásquez Rojas, A. M. (2010). Estudio sobre la eficiencia técnica de las universidades públicas presenciales españolas. In M. J. Mancebón-Torrubia, D. P. Ximénez-de-Embún, J. M. Gómez-Sancho, & G. Gim (Eds.), *Investigaciones de Economía de la Educación 5*, (1 ed., vol. 5, cap. 35, pp. 689-702). Zaragoza, España: Asociación de Economía de la Educación.
[ Links ]

Visbal-Cadavid, D., Mendoza Mendoza, A., & Causado Rodríguez, E. (2016). Eficiencia en las instituciones de educación superior públicas colombianas: Una aplicación del análisis envolvente de datos. *Civilizar.Ciencias Sociales Y Humanas*, *16*(30), 105-118.
[ Links ]

Wolszczak-Derlacz, J., & Parteka, A. (2011). Efficiency of European public higher education institutions: a two-stage multicountry approach. *Scientometrics*, *89*(3), 887.
[ Links ]

Received: June 29, 2019; Accepted: April 29, 2020