Avaliação de desempenho e integração docente-assistencial nos hospitais universitários Assessment of teaching-health care integration and performance in university hospitals

MÉTODOS: Um modelo de data envelopment analysis em redes (network DEA) foi elaborado para aferir o desempenho de hospitais universitários federais, o qual permite considerar a relação entre as dimensões de ensino e de assistência, simultaneamente. Foram utilizados os dados do Sistema de Informação dos Hospitais Universitários do Ministério da Educação, referentes ao segundo semestre de 2003, e os resultados do modelo network foram comparados àqueles dos modelos DEA tradicionais para avaliação das vantagens da nova proposta metodológica.


INTRODUCTION
According to the World Health Organization 18 (WHO), university hospitals (UH) play an important role in high-complexity health care, being heavily involved with teaching and research activities, gathering a high amount of health resources (physical, human and fi nancial ones), and playing a relevant political role in the communities of which they are a part.However, they still need greater integration with the local health care network.This integration would be important to prevent wastage of resources, try out new forms of health management and adapt teaching to the human resource qualifi cation requirements so that the social and epidemiological community demands are met. 18An important development of the characterization of UHs is the recognition of the existence of multiple dimensions in each hospital -health care, teaching and research -whose performance and quality infl uence each other.
In Brazil, the regulation of activities developed in each dimension has been the responsibility of the following institutions: Ministry of Health, Ministry of Education and Ministry of Science and Technology, which regulate health care, teaching and research developed in teaching hospitals, respectively.Operational management models of units depend on rules and legislation adopted by the institutions that maintain university hospitals (Ministry of Education, state and municipal health departments, and private and philanthropic organizations).In the case of federal UHs, local managers have human resources that are partly guaranteed by the Ministry of Education budget, and the funds for additional costs primarily come from the Sistema Único de Saúde (SUS -National Health System) budget, by contract and/or from health procedures.Local managers have a low administrative capacity and little information about the teaching and research processes developed in hospitals.This is because teaching and research regulations are made directly by faculty and teaching department managers and research groups and laboratories, respectively, without the involvement of the hospital management.The fi nancial resources coming from teaching and research are not included in the budget of teaching hospitals, nor is there a systematic source of fi nancial support to invest in equipment and infrastructure.In addition, in the federal sphere, the respective databases do not have an inter-operational characteristic that enables communication among these dimensions.
In 2003, the Ministry of Health created departments related to the organization of human resources in health (Departamento de Gestão do Ensino e Trabalho em Saúde/ SGETS -Department of Health Education and Work Management) and the development of health research projects (Departamento de Ciência e Tecnologia/ Decit -Department of Science and Technology), seeking to integrate health care, teaching and research activities with SUS priority policies.A more organic integration among these dimensions began to be established in 2004, with the Política de Reestruturação dos Hospitais de Ensino (Policy of Restructuring of Teaching Hospitals).Thus, the process of certifi cation of university and teaching hospitals started, by means of biannual visits from Ministries of Education and Health representatives, when the fulfi llment of hospital prerequisites related to the teaching-health care integration, integration with the SUS and quality of management is assessed.Once certifi ed, these hospitals begin to establish mutual agreements (health care, teaching and research ones) with the respective local health managers and budgets for medium-complexity procedures are made.
Currently, although there is a positive qualitative assessment of the certifi cation process, it is still debated what the main indicators of follow-up of the assessed dimensions are and what the policy impact measures are, so as to improve the performance of these hospitals and the effi ciency of management of resources allocated by means of a goal contract.
In the international literature, teaching hospitals are defi ned by the presence of residents and/or by the affi liation with governmental associations or councils of medical education. 9It is known that the number of residents, product of the teaching dimension resulting from training and specialization in the service, is also a resource for the health care dimension, infl uencing the cost and effi ciency of procedures. 7This example shows the importance of bringing the relationship between dimensions and/or university hospital missions closer together.
The objective of the present study was to assess the performance of general (not specialized) federal university hospitals, associated with the Ministry of Education, considering the integration between health care and teaching activities developed in them.

Performance was assessed with Data Envelopment
Analysis -DEA.The relationship between teaching and health care was approached with the network DEA model.This approach allows for further assessment of effi ciency of each of the 30 hospitals and of each dimension inside them, enabling comparisons among teaching hospitals and between these and hospital institutions that do not develop academic activities.
The measure of productivity and effi ciency in the DEA, generated by linear programming, is used by comparing similar units, or Decision Making Units (DMU), which show multiple inputs and several outputs, only differing in terms of the amounts consumed and produced.One DMU will be effi cient if it shows, comparatively to others, higher production for fi xed amounts of resources (output-oriented) and/or if it uses fewer resources to generate a fi xed amount of products (input-oriented).By defi ning the DMUs with the best practices, the DEA creates an empirical production frontier, and the level of effi ciency varies between 0.00 and 1.00 (or between 0 and 100%), depending on the distance between the unit and the frontier.In the following formula, this distance is represented in the DEA envelopment model by the lambda intensity variable.The same model considers Variable Returns to Scale (VRS) and is output-oriented.According to the radial projection of ineffi cient units on the frontier, their benchmarks -or reference DMU -can be observed, as well as the ideal input and output values for the unit to become effi cient.
In addition, for the units to be considered effi cient, the "Pareto-Koopmans" logic is applied, where a unit situated on the frontier will only be effi cient if it is not possible to reduce any input, or increase any output, without having to also increase another input or reduce another output simultaneously.In other words, a hospital with maximum effi ciency, but situated in an ineffi cient-Pareto area, has "technical", weak or Farell effi ciency.Only the projection in a Pareto-effi cient area achieves maximum unit effi ciency. 2 Knowing that the application of the DEA in any sphere of the health sector must consider a systemic context, full of connections between dimensions and variables, these must be well understood before the modeling itself. 3In the specifi c literature on DEA for teaching hospitals, inputs, such as beds, cost-related resources (in the case under study, the budget originated from the SUS), equipment, laboratories and workers, generate outputs, such as health care production, students (several levels) and technologies resulting from research, while the relationships occurring among these same variables in each DMU are not usually considered. 5For this reason, the traditional DEA model has been known as the aggregate or "Black Box" model, as illustrated in Figure 1.In this Figure, teaching and health care are considered as a single block; residents were not included in the example, because they are in the "Black Box" (as teaching output and health care input); although they could be included, whether as input or output, through the "Black Box" model.Medical doctors and professors were considered in their entirety (full-time equivalent) together with beds and budget as inputs, whereas the outputs were the undergraduate medical students and the hospitalizations adjusted by complexity.
A proxy measure of case-mix was used to adjust complexity, created by the Ministry of Education team of technicians, based on the number of high-complexity procedures performed by the unit and registered with the Ministry of Health.A specifi c weight (scores from 1 to 5) was attributed to each procedure that requires registering (such as neurosurgeries, cardiac surgeries and transplants), according to the volume of resources necessary to perform them.The weighted sum of highcomplexity procedures generated an index for each hospital, detailed in a previous study. 14According to La Forgia & Couttolenc, 11 adjustment by case-mix must consider the heterogeneity of patients cared for, in terms of the amount of resources used and treatment costs; there are several forms to proceed, although it is essential to guarantee validity of the model.Another approach to deal with these differences consists in analyzing more homogeneous sub-samples individually, according to the size of the unit, level of complexity of services or technological level of the equipment available.
"The Black Box" model has predominated among scientifi c publications that assess performance of health units, such as hospitals, medical services, health centers and planning areas, with several variable confi gurations.On the national level, Marinho & Façanha, 17 used Ministry of Education university hospitals as DMU, although with a group where a set of units with different profi les was considered in the same sample, such as general, specialty and maternity hospitals.Gonçalves et al 6 compared the public hospital system in Brazilian capitals, proposing a methodology to avoid null weights in the model.
Certain works approach the multi-dimensionality found in teaching hospitals with assessments performed separately, creating a frontier for each dimension, and being subsequently gathered 13 or compared according to the relative effi ciency of each dimension (Figure 2). 21The set of variables proposed by Ozcan to deal with the health care dimension in a hospital environment has been a consensus in the literature.Among inputs, this set considers human resources, costs and beds (along with service-mix, it is proxy for capital); while, among outputs, the production adjusted for severity.The author made an analysis of sensitivity of several combinations of inputs and outputs to achieve this model, obtaining stability in effi ciency scores. 19,20 ever, there are variables present in more than one dimension (such as doctors and professors, who share teaching and health care actions) and fl exible variables, which function as inputs for a dimension and as outputs for another (residents).These peculiarities are not approached by the separate models, which can thus harm their validity and reliability.In Figure 2, two independent frontiers are built: the teaching border (with ⅓ of doctors and ⅔ of professors as inputs, and number of undergraduate students and residents as output) and the health care border (with residents, ⅔ of doctors, ⅓ of professors, beds and the budget as inputs, and hospitalizations adjusted for complexity as outputs).It should be noted that the choice of ⅔ and ⅓ was arbitrary, considering a greater volume of teaching activities among professors and a greater volume of health care tasks for doctors, provided that both perform in the two dimensions.
Systems with two or more processes connected to each other, whether in series or parallel, form networks.The network model consists of a family of DEA models, with the linear restrictions for each subprocess analyzed.The DEA network designs fl ows of relationship between variables and generates a score of total effi ciency, in addition to a score for each dimension or process that develops in each of the DMU (divisional score).
Data from the second semester of 2003 were used to develop the network DEA model, referring to the universe of 30 Ministry of Education general university hospitals, considering (Figure 3): two dimensions (teaching and health care), four external inputs (monthly budget and number of beds for health care; ⅔ of doctors for health care and ⅓ for teaching; ⅓ of professors for health care and ⅔ for teaching), one intermediate input/output (residents), two fi nal outputs (hospitalizations, adjusted for complexity; undergraduate medical students).The dimension of research was not considered in this model due to the low validity of data (systematic errors in collection and storage) in that period, as demonstrated in a previous publication. 13he selected network model considers the VRS, given the variation in the size of hospitals, and it is outputoriented, due to the need to improve management of resources, in addition to the respective managers' low capacity to administer the human resources of their units (inputs).The health care and teaching dimensions had similar weights in the model.Moreover, in this model, the connection between dimensions (residents) was dealt with as a non-discretionary variable ("free" link value). 23The software used was DEA Solver Pro (Professional Version 6.0).

RESULTS
Table 1 shows mean, maximum and minimum values and standard deviation of input and output variables of each dimension considered in the model.The Ministry of Education UHs showed a great variation in size and scale, which should be considered when comparing similar units identifying benchmarks.
Table 2 shows the hospital effi ciency scores according to different models: a) "Black Box"; b) Separate: pure teaching and pure health care; c) Network DEA.In the "Black Box" model, the DMU were considered effi cient and the mean of scores was equal to 0.78 (or 78%).In the separate models, means were 0.75 and 0.74, with nine and ten effi cient units on the teaching and health care frontiers, respectively.
According to the Network DEA model, one DMU will only be effi cient if it has a score of 100% in all dimensions assessed.As this is a model with variable returns to scale, it is also expected that each dimension should have at least one effi cient DMU (something that does not occur with constant returns to scale).In the Network model, the mean of effi ciency was 0.54 and the minimum value was 0.19, precisely for a hospital unit that favors the presence of residents and transfers students (teaching output) to other partner hospitals.This same hospital was effi cient for the "Black Box" and separate models of the teaching dimension, as it attributed a null weight to its number of students.Only two units were simultaneously effi cient in the dimensions analyzed, both with at least 150 beds and working with a low volume of resources (thus, lower consumption of inputs).
In view of the network DEA assessing the relative efficiency of each dimension and analyzing the correspondence among them, the fact that all units could maintain the score of technical effi ciency in the health care dimension draws attention, even if at the expense of the teaching dimension.In other words, the relative effi ciency for the health care dimension was 100% for all units, whereas the relative effi ciencies of the teaching dimension varied from 0.11 to 1.00 (11% to 100%), with a mean of 0.39 (39%).One alternate situation, attributing a weight of 70% to the teaching dimension, was tested, although signifi cant differences in results were not found.
In addition, Table 2 shows the benchmarks for hospitals, according to each dimension.In the teaching dimension, units that worked with low inputs, in this case, a lower volume of human resources, stood out.This aspect needs to be much better analyzed for larger hospitals, which also include research activities in their agenda, both from professors and medical doctors.If the eight hospitals with 300 beds are considered, the UFMG university hospital obtains the highest score (equal to  0.49).The benchmarks of the healthcare dimension define hospitals that resembled one another more closely in size and institutional mission.
Table 3 shows the difference between the value projected on the frontier and the value observed in each variable of the model, for each hospital.The values of effi cient units, closer to the projected points, defi ned the benchmarks shown in Table 2.In the teaching dimension, there is an excess of 261 (9%) professors, although they have other research activities, as previously pointed out.There is not an excess of doctors and the number of undergraduate students could be increased by approximately 200% (from the fourth to the last semester, there were 14,206 medical students).In this universe, only the hospital with the lowest teaching score would require more than 3,788 students.The need for an increase in the number of students was maintained, even when a model with professors as the only input variable was tested.
In the health care dimension, all hospitals could have reductions in inputs to achieve improvement in efficiency.Going back to the Pareto-Koopmans logic, even for units with a 100% score, if it is possible to reduce inputs without the need to increase any other input, this unit is not considered Pareto-effi cient, i.e. its projection occurs in a geometric area of the frontier parallel to the axis of one of the variables.In practice, divisional effi ciencies would be maintained, even with a reduction of 27% in doctors, 13% in professors, 20% in beds and 8% in the budget.With this perspective, the concept of effi ciency in the DEA should consider, in addition to scores, the amount of resources that needs to be changed to defi ne effi cient hospitals (Table 3).
As regards the required changes in the variable connecting the dimensions, that of the residents, only three hospitals should not change the number of residents (UFJF, UFBA, UFPA), nine should reduce it (FMTM, FUAM, HCPA, UFCE, UFPEL, UFPR, UFU, UNB and UNIFESP); and the remaining ones should increase it.The sum of the needs for increase in this number, considering all hospitals, was 488 or 14% of the total number of residents present in these units.

DISCUSSION
The DEA enables multiple inputs and outputs to be put together, when assessing the effi ciency of autonomous units, improving the traditional cooperative analysis of ratios that rely on only one numerator (of production) and only one denominator (of resources).In contrast, the network DEA model increases its discriminatory power, once it brings about a score for the total efficiency of the unit and another for each dimension, in addition to its measuring the infl uence of variables connecting the dimensions analyzed.
The pioneering work that investigated the infl uence of processes existing in the "Black Box" is the one by Färe & Grosskopf. 4These network models were improved by Lewis & Sexton 12 and had important applications to assess sectors of the economy of countries of the Organization for Economic Cooperation and Development (OECD). 22As an application to health, the work of Löthgren & Tambour 15 used this model to assess Swedish drugstores, considering production and user satisfaction as complementary processes in each unit.More recently, Tone & Tsutsui 23 proposed the use of slack-based measures, when the equiproportional change in inputs and/or outputs would not be expected as an excellent solution.Kao 10 formalized the DEA multiplier model and inferred some of the properties of the systems when considering the relationships among different system activities, in series and in parallel.
The use of network DEA models is particularly useful in studies on performance applied to health, where each unit of analysis can be seen as an open system, comprised of many parts that are interconnected, in addition to being dynamically infl uenced by external and environmental joining variables.In terms of UHs, health care, teaching and research are dimensions that are present and interactive in the mission of each hospital, although each group of activities is regulated by distinct governmental organizations.The unit manager is usually the one responsible for the task of connecting the dimensions, which is sometimes confl icting.In this context, the professor-healthcare interaction is at times pointed out as a key point for the institution's organizational sustainability. 16Following the logic of funding from the Fundo Nacional de Saúde (National Health Fund), if the health care management of university hospitals is usually aimed at meeting the demand of the hierarchical health system, where the hospital provides higher-complexity care, the teaching management still emphasizes a predominantly intrahospital teaching, not necessarily focused on diseases prevalent in the immediate surroundings. 1e quantitative Network DEA model enabled this process of connection to be approached, bringing about some interesting conclusions and recommendations, with a positive impact on hospital effi ciency.In the international literature, residents are sometimes included as inputs, and, at other times, as outputs, without distinction, precisely because they represent a link between the teaching and health care dimensions.If they are included as inputs, the frontierborder of teaching hospitals tends to become more distant from that which considers hospitals without academic activities. 8cording to the dimensional scores found, UHs had expectations about the guarantee of effi ciency of health care, once this is the dimension that originates and guarantees the resources to fund hospitals, by means of a monthly budget from the Ministry of Health.The emphasis on health care was maintained, even when the weight of this dimension decreased to 30% (versus 70% of the teaching dimension).This aspect could not have been observed by the model of separate dimensions.
As regards the teaching dimension, the possibility of doubling the current number of students seems to be the major obstacle identifi ed to justify the low effi ciencies found.This measure should be analyzed with caution, according to the demand resulting from the inclusion of new doctors in the labor market, although indicating that there is still room to receive these students in the UHs before new medical courses are created.
Moreover, with regard to the teaching dimension, the model tended to favor the units considered "simpler", i.e. those working with low resources, by giving them higher scores.This could be considered a limitation to the model that needs to be dealt with.Larger, more complex hospitals develop other research and technological assessment activities that need to be analyzed.
Even with a lack of information about research, the model could include certain weight restrictions that reduce this potential bias.Likewise, as described in the Methods, the choice of volume of work spent in the dimensions by professors and doctors was arbitrary.
In addition, a certain analysis of sensitivity could be introduced to suggest an ideal distribution of working hours.In the present study, authors observed that the distribution proposed did not show changes, when compared to the model where 100% of professors worked exclusively with teaching and 100% of doctors worked exclusively with health care (in this case, the number of variables in the model was reduced, without a signifi cant change in its discriminatory power).
The model also showed what changes are necessary to optimize the number of residents, in terms of its being a strategic variable of the teaching-health care relationship in each unit.If, for the teaching dimension, the higher the number of students, the greater the teaching hospital production; for the health care dimension, an increase in this input, from a certain value on, can cause a decrease in the effi ciency of the unit, a phenomenon known as congestion. 7The model enabled the question of the number of students to be calculated, thus becoming another instrument for the unit manager.This information is also particularly useful for the organization that regulates medical residency programs, in this case, the Ministry of Education, which estimates the need for vacancies and grants medical residency scholarships to university hospitals.The same model can also consider different medical specialties to help decision-making.
The dimension of quality of service in the UHs can be created, where the output is the score of user satisfaction, in addition to improving the analysis of teaching quality, using the weighting of units, according to the assessment grade of the teaching institution or the newly graduates' knowledge level scores.
As a future development, authors in this study expect to design a methodology, considering the measures of non-radial projection, to guarantee that all effi cient units are projected in Pareto-effi cient areas of the frontier; to include restriction to weights based on the opinion of specialists; and to apply the network DEA methodology to the dynamic assessment of effi ciencies throughout time.

Table 1 .
Descriptive statistics of the network DEA model variables in university hospitals.Brazil, 2003.

Table 2 .
Effi ciency scores for the "Black Box", separate and network DEA models of university hospitals, with the respective reference units.Brazil, 2003.

Table 3 .
Difference between values projected on the frontier and values observed in variables of the model of university hospitals.Brazil, 2003.