A model for multidimensional efficiency analysis of public hospital management

Introduction: Multidimensional efficiency analysis can provide important insights into the performance of hospitals. In this paper, we propose a multidimensional model based on Data Envelopment Analysis (DEA) to investigate and compare the efficiency of public hospitals in Brazil. Methods: Data from 21 public hospitals were collected from public databases (OECD Organization for Economic Co-operation and Development; SIH-SUS SUS Hospital Information System, Datasus, Brazil). Four inputs (Number of medical and non-medical staff, Annual revenue, Number of beds, Average length of patient hospitalization), four Variables of Influence (Type of hospital, Accredited hospital, Number of medical specialties, Resources from government) and four Outputs (Number of outpatient care services, Number of hospitalizations, Number of surgeries, Number of exams) were used to feed the DEA model. Results: Seven hospital units reach 100% efficiency and, according to DEA, can be considered efficient units. Two units were considered “almost efficient” and the remaining twelve units perform poorly, considering the data supplied to the DEA model. As a whole, the average efficiency of the hospitals investigated was 79% (0.79). Conclusion: A very heterogeneous performance has been found among the Brazilian public hospitals investigated. Besides, the reasonably low average efficiency seems to indicate that the system has a large potential for improvement in almost all areas associated with the input and output variables investigated in this paper.


Introduction
In recent years, there have been a number of changes in primary health care in Brazil. Among them, the most important are related to the creation of the Unified Health System (UHS) in 1990 and adjustments in the financing policy, expanding governmental investment towards improving access to health care for the general population (Machado, 2007), as well as better management of the resources in the area. UHS transformed and expanded medical care in the country, offering a universal national public health system aimed at providing equitable care and promoting the health of the entire population.
According to a health report published by the Foundation Institute of Administration of the University of São Paulo (FIA/USP -Brazil), in 2007 the Brazilian government invested R$50 billion in the health sector. Although the gross number was higher than the amount invested in the early 90's, the expenditure per capita was basically the same -about US$280 per person per year (Fundação…, 2011). That number is above the average recorded in Latin America, but less than half the world average of US$806 per capita. Furthermore, when considering the percentage of resources invested in relation to what the countries raised from tax revenues and so on, the Brazilian spending falls to just over half of that registered in the Latin American neighbor countries.
UHS is a unique social project that materializes itself through actions of promotion, prevention and health care for Brazilians, with a number of associated hospitals. Among the units of the public network that operate with UHS, there are 2,411 public hospitals (88 federal, 625 state and 1,698 municipal hospitals) spread across all Brazilian states (Brasil, 2017).
The annual report of the Brazilian Federal Court of Accounts for 2011, shows that, between 2007 and 2011, the total expenditure of the Union with health care increased up to 58%, reaching around R$74.47 billion in 2011 (Tribunal…, 2011). However, the percentage expenditure as a function of Gross Domestic Product (GDP) ranged from 2.14% to 2.09% in the same period. In 2015, health investment reached R$113.4 billion -about 1.92% of GDP (Tribunal…, 2015). Nevertheless, the system has not been capable of executing (spending) a significant amount of money invested by the Union, as shown in Table 1. Those numbers seem to point out that the public administration is still struggling to overcome a number of challenges for proper execution, application, and use of the resources allocated by the government for health care.
Hospitals should be as efficient as possible when it comes to the administration of financial resources, always looking for new means of improving the overall results, while providing the best possible service to the population. Hence, it is important to study all possible reasons that lead to results such as those depicted in Table 1. A superficial analysis shows that a number of factors may play important roles in the efficiency of the health establishment -from the heavy bureaucracy associated with the publish service, to historical elements. In other words, multidimensional models are required to study such systems.
Data Envelopment Analysis (DEA) is a non-parametric mathematical model capable of evaluating organizational performance in terms of relative efficiency between similar units or decision-making operational units with a multidimensional perspective (with multiple inputs and outputs, such as the case of public hospitals). In this sense, this technique could be used as a tool to model and analyze the operational efficiency of public and private hospitals by means of input-output type indicators that can be further developed and extended towards a final model. In addition, a multidimensional analysis of the efficiency of such organizations may also provide historical understandings for current results. Khushalani and Ozcan (2017) used DEA to evaluate hospitals in the United States, computing efficiency scores for hospital sub-divisions and quality. They found that the efficiency of quality production improved significantly between 2009 and 2013 with no trade-off between efficiency of producing quality outputs and efficiency of producing medical care. Another interesting finding is was that urban and teaching hospitals were less likely to improve efficiency of quality production. Biørn et al. (2003) studied the effect of a new regulation in the Norwegian health sector on the organizational outcome of the hospitals. The authors found that the introduction of a contract management model based on targets for the definition of funding, associated with the new regulation, has increased the efficiency of the system, as observed in the historical series from 1992 to 2000. The effect of the budget on efficiency was not determined in the study. A similar research conducted in Greece, showed that the expected benefits from the government reforms have not, in general, been achieved (Aletras et al., 2007).
Efficiency has also been used as possible index to define if a hospital should remain open or not. Lynch and Ozcan (1994) investigated the hypothesis that inefficient and underutilized hospitals in competitive markets were the ones with greater risk of closure. They found that efficiency was not directly correlated with nor was capable of predicting hospital closure in the United States. Pilyavsky et al. (2006) studied the Ukrainian hospital management model and sought to understand if cultural differences influence in the organizational and economic behaviors. Data were obtained from 3 geopolitical regions, one in the west and two in the east of the country. According to the authors, eastern regions were heavily influenced by Russo-Soviet rule, while western regions have more of a European outlook. The results showed that the European model, based on market-oriented management practices, is more efficient (besides being faster to adopt new techniques for improving health care) than the Russian model, which is normative and based on controlled and centralized state management.
As shown, there may be a number of variables that could influence on the performance and efficiency of hospitals, especially those directly associated with public services. In this sense, besides investigating the efficiency levels of public hospitals in Brazil, this work aims at proposing a new multidimensional model of efficiency evaluation and comparison among institutions. In so doing, we hope to contribute towards more coherent application of the resources and improvement of the organizational management. This multidimensional study focuses on the effect of multiple factors, based on data collected at the initial point of the exposure (data discovery) and at a later moment. Specifically, we seek to identify the possible variables that influence hospital efficiency levels and what are the differences in efficiency among public hospitals in Brazil.  (Tribunal…, 2009;2011;2013;.

Evolution of federal budget for health services: Authorized x Executed
Year

Methods
In this research, we used analytical methods to better understand the many variables involved and to lay the foundations for our model. According to Bogdan and Biklen (1994), analytical methods can be used for the examination of the components of a whole when seeking for the causes and nature of the problem; i.e., the evaluation of the information available to explain the context of a certain phenomenon.
In this work, we apply Data Envelopment Analysis (DEA) as a tool to model and analyze the operational efficiency of hospitals. However, in order allow for a better understanding our proposal, it is necessary to define the main terms that will be used throughout this paper (Almeida, 2010): • Decision-making units (DMUs): DMUs are autonomous units in the analysis of efficiency -here, hospitals are considered DMUs; • Inputs: the resources available for the DMU -in hospitals, inputs can be understood, for instance, as the number of staff; • Outputs: the resources available as a result of the processes in the DMU -in hospitals, outputs can be understood, for example, as the number of surgeries per surgeon; • Efficiency score: DEA scores efficiency on a scale of 0 to 1, where 1 indicates an efficient unit. Hence, the efficiency frontier is at 1 (or 100%) and represents the locus of highly efficient DMUs. An efficient DMU serves as a reference for other units, enabling the development of new strategies so that less efficient units can achieve better performance; • Reference set: The reference set is defined for a given DMU i and consists of the list of efficient DMUs of which the performance was used to calculate the efficiency of that given DMU i . To develop the proposed model, we collected data from various hospitals, as published on the following databases: OECD -Organization for Economic Co-operation and Development; SIH-SUS -SUS Hospital Information System, Datasus, Brazil.
Although there are more than 10,500 health centers in Brazil, this work will be focused on public institutions, as described before. Hence, the following information, hereafter called 'Variables', were collected from Brazilian public hospitals (DMUs) in order to feed our DEA model: I -Inputs • Number of medical and non-medical staff; • Annual revenue; • Number of beds; • Average length of patient hospitalization.
II -Variables of influence -also called moderating or intervening variables -can alter the level of efficiency of a certain DMU.

III -Outputs
• Number of outpatient care services; • Number of hospitalizations; • Number of surgeries; • Number of exams. The choice for these variables was based on what can be considered primary outputs of a health unity -number of in-and outpatients treated and the number of exams and surgeries required to do so (Kao et al., 2011). The inputs were then defined as those most correlated to the outputs. The variables of influence, as the name implies, were selected as those that have a strong indirect impact on the results. For instance, accredited units are expected to have implemented a number of procedures to ensure, among other aspects, patient safety, integrated management and organizational culture towards continuous improvement in all areas (Organização…, 2014).
DEA was performed with the Frontier Analyst  software Version 4 (Banxia Software Ltd, UK). This software can used to assist in comparative analysis of the relative efficiency of organizational units such as bank branches, hospitals, stores and other cases where units perform similar tasks. The method considers that there may be differences between the units caused by different elements, such as the technology they have at their disposal, their geographic location or population covered. As such, there may be a number of factors that determine the operational effectiveness of a unit. The software uses DEA to compile all information available and provide a comparative performance assessment that allows for an in-depth analysis of the relative performance of the DMUs performing similar functions. An efficiency frontier can than be constructed. Units located closer to the border (at the 100% frontier) are doing better than others father away.
The efficiency of a certain unit j can be calculated as the ratio between the weighted sum of the outputs and the weighted sum of the inputs: (1) where: j = unit under analysis; n = number of outputs; m = number of inputs; u n = weight attributed to output n; y nj = amount of output n for unity j; v m = weight attributed to input m; x mj = amount of input m for unity j.
To define the efficiency of a DMU, DEA solves Equation (1) by means of a Linear Programming method (see Sierksma (2001) and Cooper et al. (2000) for detailed explanation) to achieve the best linear optimization for the weights to obtain the best outcome (maximum output or minimum input).
As explained by Gonçalves et al. (2007), there is a maximum and minimum for each set of values for each variable that define its limits and importance in DEA, without direct interference from a decision-maker. DEA compares all inputs and all the results (outputs) provided by a certain DMU with those from all other units, and identifies units that are operating inefficiently. In so doing, the best practice (efficient) units can be selected, to which an efficiency rating of 1 (or 100%) is attributed. All inefficient units will be scored below 1. The displacement of a unit to the efficient frontier (1 or 100%) implies the optimum input value that would make inefficient units efficient. Furthermore, it is possible to compare how different units are doing by checking their distances to the efficient frontier.
Although a direct comparison of efficiency among the various DMUs brings valuable insights, it does not necessarily carries all the information to clearly understand different performances. To help uncovering more detailed information, we will also investigate the following indicators provided by DEA: • Reference frequency: It shows how many times an efficient DMU has been part of the reference set of inefficient DMUs. The greater the frequency with which an efficient unit appears in reference sets, the more it is considered an example of good performance. An efficient DMU that appears in most reference sets could be called "Global Leader" and, as such, should provide an example of good operating practice to be applied in inefficient units; • Total potential for improvement: DEA can provide an indicator for each variable that defines the potential for improvement of that variable for any given DMU. This is an extremely valuable information for managers, who can easily spot the inputs and outputs with potential, or not, for improvements; • Efficiency with regard to the frontier: One of the most striking outputs of DEA is a graphic diagram were the frontier of efficiency is show along with the efficiency of all DMUs. This information can be very useful to find out the locus of DMUs of interest and to evaluate how the health system is performing as a whole.

Results
Data from 21 Brazilian public hospitals were extracted from the OECD and SIH-SUS databases. Although there are more than 21 public hospitals in the country, those were the units with the complete set of variables (Inputs, Outputs and Variables of influence) available. Hence, the sample comprised 18 federal hospitals and 3 state hospitals. To preserve confidentiality of the data, hospitals are named H1-H18 (federal units) and H19-H21 (state units).
From each one of those 21 DMUs, 4 inputs, 4 variables of interference and 4 outputs were collected, as described in methods. These data were fed into the Frontier Analyst system, as shown in Table 2 (the maximum/limits for each input are shown in the bottom row).
These data were processed using the DEA model and the results are shown next. Figure 1 shows the reference frequency for the DMUs under investigation. As seen, only seven DMUs appear in the reference sets. With the highest frequency, the DMU H12 is here considered the "Global Leader" among the 21 DMUs under study and can be considered the best example of good performance. Table 3 shows the efficiency scores calculated for all 21 DMUs (public hospitals). It is possible to observe that only 07 reach 100% efficiency and, according to DEA, can be considered efficient units. Two can be considered "almost efficient" with scores below 100%, but above 80%. All other twelve units perform poorly, considering the data supplied to the DEA model. As a whole, the average efficiency of the DMUs evaluated was 79% (0.79). Res. Biomed. Eng. 2017 December;33(4): 352-361    Res. Biomed. Eng. 2017 December;33(4): 352-361 Total improvement potential Table 4 shows the estimated percentage of improvement when all inefficient DMUs are compared, variable-by-variable, with the set of efficient DMUs.

Efficiency score
In other words, it shows the percentage of improvement that an inefficient DMU would have to reach in order to reach the same level of efficient DMUs for that variable. As seen, among the described variables, those with greatest potential for improvement among all inefficient DMUs are the number of surgeries and the number of exams. On the other hand, for instance, many of those units do not have room for improvement in elements such as the number of outpatient care services, reflecting a possible reality of the current Brazilian heath sector, with the ambulatory sectors of hospitals throughout the country already operating at maximum capacity.

Efficiency with regard to the frontier
One of the aims of this study was to determine the efficiency levels of public hospitals in Brazil, using public data, in order to identify potential differences and compare performances. The efficiency frontier, defined by means of the DEA model and illustrated in Figure 2 as a radar chart, can be used to respond to that aim while allowing a bird's eye view of the whole system. A very heterogeneous performance can be seen, with some units, such as H3, far below the average performance of the units under evaluation.

Discussion
In this paper we proposed a DEA model to evaluate the performance of public hospital in Brazil based on 4 main inputs (Number of medical and non-medical staff, Annual revenue, Number of beds and Average length of patient hospitalization) and four main sets of outputs (Number of outpatient care services, Number of hospitalizations, Number of surgeries and Number of exams) for the model. As a result, a very heterogeneous performance has been found, with an average efficiency of 79%, showing a great potential for improvement in, at least, 12 of the 21 units under study.
In the study developed by Kao et al. (2011), the results showed that DEA was able to distinguish differences in performance among DMUs based on 5 inputs (Total number of registered beds within the hospital -including acute, chronic, and special beds; Total number of physicians who are full-time employees -including dentists and Chinese medicine doctors; Total number of nurses employed in hospitals -including midwives; Total number of health service providers employed in hospitals -including pharmacists, dietitians, physiotherapists, occupational therapy technologists, and radiological technologists; and Total number of full-time equivalent personnel -including social workers, researchers, and nonprofessionals) and 3 outputs (Total number of patients to outpatient departments within a year; Total number of patients to emergency room within a year; Total number of inpatient and outpatient surgeries within a year). Similar to our findings, Kao et al. also did not find homogeneity in hospital efficiency.
A recent study conducted by Johannessen et al. (2017), investigated the impact of several political reforms of the Norwegian hospital sector occurred since 2002. All public hospitals were transferred from a system of county ownership to central government ownership. In so doing, the government expected to increase hospital efficiency by providing greater autonomy with respect to planning, budgeting and workforce policies of the health units (Hagen and Kaarboe, 2006). Johannessen et al. (2017) used a non-parametric DEA model and panel analyses to study the overall physician productivity. Their results showed that, in spite of comprehensive governmental efforts, there was a significant variation in productivity among the country's hospitals, along with a noticeable decline in physician productivity. Their DEA model showed that, although cost efficiency did not change in the study period, allocative efficiency (refers to how different resource inputs are combined to produce a mix of different outputs -Medeiros and Schwierz (2015)) decreased significantly. Similar results and significantly different efficiency levels were also find by Khushalani and Ozcan (2017), who studied the efficiency of general hospitals in the United States between 2009 and 2013, also using the DEA.
The studies depicted before seem to reinforce that, in general, the overall efficiency of different hospitals is not homogeneous, regardless the regions of a country or even among different countries. This is also corroborated by the study conducted by Asandului et al. (2014), in which the health systems efficiency of 30 European countries was compared by means of DEA, using three production variables (life expectancy at birth, adjusted life expectancy adjusted to health and child mortality rate) and three input variables (number of doctors, number of hospital beds and public health expenditure as a percentage of GDP). Their findings also revealed that there are a number of developed and developing countries at the frontier of efficiency.
The efficiency obtained in the Brazilian public hospitals analyzed was on average 0.79, a number similar to that found by Pérez-Romero et al. (2017) Mitropoulos et al. (2015), combining stochastic DEA with Bayesian analysis to extract statistical properties of efficiency scores of 117 Greek hospitals, concluded that while similar production technologies were used by tertiary and secondary hospitals, a large technology gap existed between primary care hospitals and those at the frontier of efficiency.
As show, regardless of which country, or culture, the various indicators provided by DEA can contribute significantly to better identify hospital efficiency, compare different units and also to provide valuable information from efficient units that could be helpful for improving non-efficient ones. In this sense, we believe that our study, besides providing information about a large group of Brazilian Public hospitals, could also be used for those involved in the public health system, researchers, students and community in general, in order to better understand the complexity of the Brazilian health system. This tool (DEA -Frontier) was found to be a relevant instrument for measuring the efficiency of activities and operations in the public hospital network, drawing attention to the clear identification of hospital efficiency levels that are not based in one or two isolated indicators, but in a complex set of indicators and inputs and outputs that, after modeled and processed within the concept of DEA, resulted in a multidimensional measure of efficiency.
Although multidimensional analyses are usually hard to understand or even hard to grasp all possible outcomes, DEA provides tools capable of allowing for a clear and unambiguous overall evaluation of the mains results. Figure 2 illustrates one possible form of summarizing the various indicators in a single element that, when showed in a graph, can easily be used to locate units according to their levels of efficiency and perspectives for evolution. For instance, considering that 08 of the public hospitals under evaluation reached the efficiency frontier (if we also include H20 with 99.9% efficiency), we noticed that all other 13 hospitals investigated in this study have a solid set of "role models" from which good practices of management and operation could be learned and assimilated.
The multidimensional DEA model proposed in this paper is based on four Inputs, four Variables of Influence and four Outputs, and is intended to serve as a supporting tool for the descriptive representation of efficiency of the Brazilian public health system.