Dynamic network data envelopment analysis for university hospitals evaluation

ABSTRACT OBJECTIVE To develop an assessment tool to evaluate the efficiency of federal university general hospitals. METHODS Data envelopment analysis, a linear programming technique, creates a best practice frontier by comparing observed production given the amount of resources used. The model is output-oriented and considers variable returns to scale. Network data envelopment analysis considers link variables belonging to more than one dimension (in the model, medical residents, adjusted admissions, and research projects). Dynamic network data envelopment analysis uses carry-over variables (in the model, financing budget) to analyze frontier shift in subsequent years. Data were gathered from the information system of the Brazilian Ministry of Education (MEC), 2010-2013. RESULTS The mean scores for health care, teaching and research over the period were 58.0%, 86.0%, and 61.0%, respectively. In 2012, the best performance year, for all units to reach the frontier it would be necessary to have a mean increase of 65.0% in outpatient visits; 34.0% in admissions; 12.0% in undergraduate students; 13.0% in multi-professional residents; 48.0% in graduate students; 7.0% in research projects; besides a decrease of 9.0% in medical residents. In the same year, an increase of 0.9% in financing budget would be necessary to improve the care output frontier. In the dynamic evaluation, there was progress in teaching efficiency, oscillation in medical care and no variation in research. CONCLUSIONS The proposed model generates public health planning and programming parameters by estimating efficiency scores and making projections to reach the best practice frontier.


INTRODUCTION
Hospitals are complex, multi-professional and interdisciplinary institutions, with specific technological density, according to the Brazilian National Policy for Hospital Care, implemented in the Health Care Networks. They are responsible for providing health care to users on an inpatient basis and via services covering health promotion, disease prevention, diagnosis, therapy, and rehabilitation a .
Barata et al. 1 consider health to be a social right. They stress the complementary contribution of teaching hospitals in training health professionals to adequately tackle priority problems of the Brazilian population and developing research to help cope with these problems. Teaching hospitals include all hospital units, general or specialized, public or private, independent or attached to a higher education institution, which provide curricular activities in the field of health b . University Hospitals (UH) belong to a federal or state higher education institution, with coordinated activities in care, teaching, and research.
Since the 1990s, the funding and management crisis of Brazilian hospitals, in particular university hospitals, has been placed on the agenda, with important repercussions in care, teaching, and research activities. Initiatives by the Brazilian Federal Government to face the crisis were based on financial, material and human resources support, and the introduction of improved management practices, among them the National Policy for the Certification of Teaching Hospitals, the National Program for Restructuring Federal University Hospitals, and the creation of the Brazilian Hospital Services Corporation b,c,d . Such initiatives aimed to attain excellence in teaching hospitals in care, teaching, and research by establishing management goals that would ensure the best use of resources in carrying out their tasks.
The development of performance studies using efficiency frontiers is appropriate to analyze the use of resources in achieving objectives. Among efficiency studies applied to the field of health, data envelopment analysis (DEA) is the most frequently used technique (48.0% of publications) 5 .
This study aimed to develop an assessment tool to evaluate efficiency in federal general university hospitals. This tool considers coordination between teaching, care, and research and the monitoring of changes over time.

METHODS
The model is based on dynamic network DEA to help decision-making processes among managers. A longitudinal analysis of interrelations is carried out between the dimensions of care, teaching, and research regarding their efficiency.
DEA is a linear programming technique first described by Charnes et al. 2 (1978). It measures efficiency by comparing similar units (DMU or Decision Making Units) that use the same resources (inputs) and generate the same products (outputs). A DMU is considered efficient when, compared to the others, it produces more outputs with a fixed amount of inputs (output-oriented model), or uses fewer resources to generate a fixed amount of outputs (input-oriented model). Efficient DMUs receive the maximum score of 100% and shape the best practice frontier, which involves the set of compared units. A DMU is only considered efficient when it reaches the Pareto-Koopmans optimality frontier, in which it is not possible to increase an output (or reduce an input) without reducing another output (or increasing another input) at the same time.
In classic models, the frontier may consider constant returns to scale (CRS, when an increase in outputs proportional to the increase in inputs is expected) or variable returns to scale (VRS, usually preferred when comparing units very different in size and outcome scale), a format established a priori by the researcher. Being a deterministic technique, any distance from the frontier (score lower than 100%) results from inefficiency, i.e., from the difference between current (X%) and projected values (100-X%). The projection of inefficient DMUs considers the shortest distance from or the nearest point to the frontier, where similar efficient units are located as benchmarks 14 .
In the development of classic models, when the production process generates intermediate outputs that serve as inputs for new processes, network DEA can be used. Network DEA is composed of a family of DEA models that establishes linear constraints for each one of the analyzed dimensions. It is thus possible to consider link variables belonging to more than one dimension and generate efficiency scores for all of them 3 .
The models applied to case studies in the literature do not usually pay attention to projections in Pareto-inefficient regions of the frontier, and the projections with slacks in relation to the Pareto-efficient target result in efficiency estimation mistakes 8 . Prominent among the non-radial models that determine efficiencies based on Pareto-efficient targets are the SBM (Slacks-based Model) and Russel models. In this work, we chose to use the DEA model in SBM networks to evaluate efficiency, as proposed by Tone and Tsutsui 15,16 .
For longitudinal monitoring, dynamic DEA enables the generation of efficiency scores for distinct and consecutive times by considering efficiency at each moment and the frontier shift over time. In each studied period, the DMUs use up the same inputs and generate the same outputs. However, some outputs of a period (capital and human resources, for example) can serve as inputs for the following period. These link variables between periods are called carry-over, which can be treated as fixed or mutable according to frontier projection.
The network DEA and dynamic DEA models are integrated (DNSBM -Dynamic Network Slack-based Model) in this study. Vertically, it deals with multiple dimensions linked in a networks structure within each period, and, horizontally, each structure is combined in networks by carry-over variables that link two distinct periods. The model enables the evaluation of: a) overall and single-dimension efficiency, at each evaluated moment; b) changes in overall and single-dimension efficiency, over the studied period. A modified Malmquist Index was developed, which evaluates changes in individuals scores (catch-up component) and the technological frontier shift, or of all hospitals over time ( frontier shift) 16 . Values higher than 1.0 mean performance progression in the interpretation of the index and its components; lower values mean frontier regression.
The DNSBM model used is output-oriented (aiming at increasing output) and considers variable returns to scale (given the different sizes of the analyzed hospitals). The network structure considers the relation between the dimensions of care, teaching and research, and the dynamic assessment analyzes the 2010-2013 period. The DMUs are the 31 federal general university hospitals, run by the Brazilian Ministry of Education (MEC), identified here by their respective universities. The variables were taken from the Ministry of Education Information System (SISREHUF). Figure 1 illustrates the studied model schematically and Table 1 describes in detail the variables in the several dimensions: inputs, outputs, link and carry-over. No variables -other than link variables -were repeated in different dimensions.
In treating variables missing in SISREHUF, the study used the highest value (input variable); the lowest value (output variable); and the mean value (link and carry-over variables).

RESULTS
We observed heterogeneity in size and care-related characteristics of these hospitals. Mean care production (8.2% for adjusted outpatient visits and 8.0% for adjusted admissions) and technological structure (16.1%) increased over the period, despite the decrease in the number of active beds (3.7%) and non-medical staff (1.8%). There was an increase in the number of multi-professional residents (193.0%), of medical residents (21.8%) and of undergraduate students in health areas (13.9%) in the teaching dimension. Reported research projects fell by 64.7%, with no proportional fall in graduate students (increase of 1.5%) in the research dimension. We observed no significant numerical change in the number of professors with or without doctoral degrees or of doctors in most institutions (the high mean professor percentage is due to two hospitals with extreme reported values) ( Table 2). Hospitals of different sizes coexisted as benchmarks of best practice (100% score). This indicated the existence of at least two frontier regions of different scales, one with large hospitals of over 200 beds, which were efficient due to high output and complexity, and the second with smaller hospitals, with maximum efficiency measured by low resource consumption. These differences are important for the definition of peers and the choice of benchmarks of inefficient units. Five hospitals were present in the best practice frontiers ( (Figure 2).
The frontier projection suggested possible priorities and goals to be agreed on between managers and UHs. In 2010, at the beginning of the National Program for Restructuring Federal University Hospitals, there was a pressing need to increase output, especially in adjusted outpatient visits (131.0%) and admissions (88.0%), besides graduate students (83.0%). In 2013, the greater need was for an increase in research activities, which additionally require a significant increase in graduate students (80.0%) and care activities for outpatients (69.0%) (Figure 3).    The annual output budget (in millions of reais) is the input defrayal for one year and the proxy output measure for the following year. Since negotiation of budget values is based on the historical output series of previous years, it is useful to program a more efficient funding system.      Table 3 presents, besides the overall score and the specific score for care, the financial budget, in millions of reais, of each UH, and the projection for the following year to reach the efficiency frontier. The approach of making projections for the best practice frontier generated financial planning guidelines based on the care output efficiency observed in each hospital. In 2013, projections for the following period suggested that seven DMUs would need a budget increase (a total of R$97.2 million), 11 would need a reduction (a total of R$175.6 million) and the others could keep it at the same level. All other parameters being the same, there would be scope for a 4.6% reduction in the total amount of funds apportioned to these hospitals in 2014, without any impact to the system' s efficiency. These guidelines would allow negotiations to increase production or redistribute resources among the units. The best performance year (2012) was the only one in which the projected funds exceeded the actual budget, i.e., the output justified the budget in a year of greater efficiency.

DISCUSSION
The proposed model enables managers to monitor efficiency, plan goals, and establish budget guidelines over time, thus being a tool to support public policy in the field of health.
As shown in Figure 3, the projections for each year generated parameters and goals to be reached in the following period. For example, comparing the 2012 goals with the actual 2013 data, there was an increase in 20.0% in medical residents (against a reduction guideline of 9.0%). For the remaining link variables, adjusted admissions were 19.0% lower than the goal and research projects were 67.0% lower. For the remaining outputs, adjusted outpatient visits were 35.0% lower than expected; undergraduate students, 5.0% lower; graduate students, 15.0% lower. Multi-professional residents, on the other hand, were 7.0% higher than the goal. Such findings partly explain the drop in mean efficiency scores in the three dimensions in 2013.
The comparison between general hospitals, with and without teaching and research activities, not considering these dimensions, minimizes the latter' s efficiency scores. The reason is that various resources of the production process are, in practice, also geared towards non-care activities 4 . The network DEA model, on the other hand, enables comparison of the portion of resources used in each specific dimension, reducing this skew.
Since 2000, university hospitals have shown an effort in academic production, initially with classic DEA models to evaluate the efficiency of MEC units 12 . New analysis approaches have been developed for this set of hospitals, with the dimensions either treated separately or grouped in a hierarchical model or by networks 7,10,13 . Longitudinal evaluation has been used to analyze the impact of the funding reform created by the certification policy for university hospitals 9 .
The networks model was enhanced in this study by addressing the research dimension, considering only projections in the Pareto-efficient regions of the frontier and monitoring the frontier shift over the years. Such enhancement makes the model a more accurate tool for managers to analyze efficiency.
The tool proposed to monitor teaching hospitals enables managers to: a) generate efficiency scores, overall and for each dimension; b) monitor annual performance changes, overall and for each dimension; c) define and manage goals to improve performance based on the best practice frontier projection; d) program and evaluate dynamically the funding of UHs, based on efficiency. The findings have produced examples for each one of these functions. The data and projections can be analyzed separately, for each unit, or jointly, for a broader assessment of public policy.
The current set of federal university hospitals is heterogeneous in size, output capacity and operation time. Dealing with such differences required adjusting for complexity.
There was an increase in adjusted care output of approximately 8.0% in the last few years.
The efficiency analysis shows that use of some resources increased simultaneously, and in a greater proportion, such as technological structure (16.1%) and funding (37.8%). On the other hand, there was a reduction in the number of beds, following a world trend in the past few years, with an increase of those in areas of greater complexity, such as intensive care units. The performance of procedures geared towards outpatients also reduced the need for beds, but there is scope to expand this activity (goal of 69.0% in 2013).