Assessment of teaching-health care integration and performance in university hospitals

Abstracts

OBJETIVO: Avaliar o desempenho e a integração entre as dimensões de assistência e de ensino dos hospitais universitários brasileiros. MÉTODOS: Um modelo de data envelopment analysis em redes (network DEA) foi elaborado para aferir o desempenho de hospitais universitários federais, o qual permite considerar a relação entre as dimensões de ensino e de assistência, simultaneamente. Foram utilizados os dados do Sistema de Informação dos Hospitais Universitários do Ministério da Educação, referentes ao segundo semestre de 2003, e os resultados do modelo network foram comparados àqueles dos modelos DEA tradicionais para avaliação das vantagens da nova proposta metodológica. RESULTADOS: A eficiência dos hospitais avaliados variou entre 0,19 e 1,00 (média = 0,54). O escore dimensional mostrou que os hospitais priorizam o ganho de eficiência assistencial. Observou-se que há necessidade de dobrar o número de alunos de medicina e de aumentar os residentes em 14% para que se tornem eficientes na dimensão de ensino. CONCLUSÕES: O modelo mostrou utilidade de aplicação tanto para os gestores das unidades, visando à integração docente-assistencial, como para os órgãos reguladores, na definição de políticas e incentivos.

Serviços de Integração Docente-Assistencial; Internato e Residência; Hospitais Universitários; Eficiência Organizacional; Administração Hospitalar; Análise Envoltória de Dados

ORIGINAL ARTICLES

Assessment of teaching-health care integration and performance in university hospitals

Evaluación del desempeño e integración docente-asistencial en los hospitales universitarios

Maria Stella de Castro Lobo^I; Marcos Pereira Estellita Lins^II; Angela Cristina Moreira da Silva^II; Roberto Fiszman^I

^IServiço de Epidemiologia e Avaliação. Hospital Universitário Clementino Fraga Filho, Universidade Federal do Rio de Janeiro. Rio de Janeiro, RJ, Brasil

^IIDepartamento de Engenharia de Produção. Instituto Alberto Luiz Coimbra de Pós-graduação e Pesquisa em Engenharia - Coppe. Universidade Federal do Rio de Janeiro. Rio de Janeiro, RJ, Brasil

^{Correspondence}

ABSTRACT

OBJECTIVE: To assess the performance and integration between the health care and teaching dimensions in Brazilian university hospitals.

METHODS: A network data envelopment analysis (DEA) model was designed to measure the performance of federal university hospitals, which enables the relationship between the teaching and health care dimensions to be considered simultaneously. Data from the Ministry of Education Information System of University Hospitals, in the second semester of 2003, were used. Results of the network model were compared to those of classical DEA models to assess the advantages of the new methodological proposal.

RESULTS: The efficiency of the hospitals assessed varied between 0.19 and 1.00 (mean = 0.54). The dimensional score showed that hospitals prioritize the gain in health care efficiency. It was observed that there was a need to double the number of medical students and increase the number of residents by 14% to obtain efficiency in the teaching dimension.

CONCLUSIONS: The model was useful for both unit managers, aiming to integrate teaching and health care, and regulatory organizations, when defining policies and incentives.

Descriptors: Teaching Care Integration Services. Internship and Residency, organization & administration. Hospitals, University. Efficiency, Organizational. Hospital Administration. Data Envelopment Analysis.

RESUMEN

OBJETIVO: Evaluar el desempeño y la integración entre las dimensiones de asistencia y de enseñanza de los hospitales universitarios brasileros.

MÉTODOS: Un modelo de data envelopment analysis en redes (network DEA) fue elaborado para contrastar el desempeño de hospitales universitarios federales, permitiendo considerar la relación entre las dimensiones de enseñanza y de asistencia, simultáneamente. Fueron utilizados los datos del Sistema de Información de los Hospitales Universitarios del Ministerio de Educación de Brasil, correspondientes al segundo semestre de 2003, y los resultados del modelo network fueron comparados con aquellos modelos DEA tradicionales para evaluación de las ventajas de la nueva propuesta metodológica.

RESULTADOS: La eficiencia de los hospitales evaluados varió entre 0,19 y 1,00 (promedio=0,54). El escore dimensional mostró que los hospitales priorizan la ganancia de eficiencia asistencial. Se observó que, para obtener eficiencia en la dimensión de enseñanza, hay necesidad de duplicar el número de alumnos de medicina y de aumentar los residentes en 14% para que se tornen eficientes en la dimensión de enseñanza.

CONCLUSIONES: El modelo mostró utilidad de aplicación tanto para los gestores de las unidades, buscando la integración docente-asistencial, como para los órganos reguladores, en la definición de políticas e incentivos.

Descriptores: Servicios de Integración Docente Asistencial. Internado y Residencia, organización & administración. Hospitales Universitarios. Eficiencia Organizacional. Administración Hospitalaria. Análisis Envolvente de Datos.

INTRODUCTION

According to the World Health Organization¹⁸ (WHO), university hospitals (UH) play an important role in high-complexity health care, being heavily involved with teaching and research activities, gathering a high amount of health resources (physical, human and financial ones), and playing a relevant political role in the communities of which they are a part. However, they still need greater integration with the local health care network. This integration would be important to prevent wastage of resources, try out new forms of health management and adapt teaching to the human resource qualification requirements so that the social and epidemiological community demands are met.¹⁸ An important development of the characterization of UHs is the recognition of the existence of multiple dimensions in each hospital - health care, teaching and research - whose performance and quality influence each other.

In Brazil, the regulation of activities developed in each dimension has been the responsibility of the following institutions: Ministry of Health, Ministry of Education and Ministry of Science and Technology, which regulate health care, teaching and research developed in teaching hospitals, respectively. Operational management models of units depend on rules and legislation adopted by the institutions that maintain university hospitals (Ministry of Education, state and municipal health departments, and private and philanthropic organizations). In the case of federal UHs, local managers have human resources that are partly guaranteed by the Ministry of Education budget, and the funds for additional costs primarily come from the Sistema Único de Saúde (SUS - National Health System) budget, by contract and/or from health procedures. Local managers have a low administrative capacity and little information about the teaching and research processes developed in hospitals. This is because teaching and research regulations are made directly by faculty and teaching department managers and research groups and laboratories, respectively, without the involvement of the hospital management. The financial resources coming from teaching and research are not included in the budget of teaching hospitals, nor is there a systematic source of financial support to invest in equipment and infrastructure. In addition, in the federal sphere, the respective databases do not have an inter-operational characteristic that enables communication among these dimensions.

In 2003, the Ministry of Health created departments related to the organization of human resources in health (Departamento de Gestão do Ensino e Trabalho em Saúde/ SGETS - Department of Health Education and Work Management) and the development of health research projects (Departamento de Ciência e Tecnologia/ Decit - Department of Science and Technology), seeking to integrate health care, teaching and research activities with SUS priority policies. A more organic integration among these dimensions began to be established in 2004, with the Política de Reestruturação dos Hospitais de Ensino (Policy of Restructuring of Teaching Hospitals). Thus, the process of certification of university and teaching hospitals started, by means of biannual visits from Ministries of Education and Health representatives, when the fulfillment of hospital prerequisites related to the teaching-health care integration, integration with the SUS and quality of management is assessed. Once certified, these hospitals begin to establish mutual agreements (health care, teaching and research ones) with the respective local health managers and budgets for medium-complexity procedures are made.

Currently, although there is a positive qualitative assessment of the certification process, it is still debated what the main indicators of follow-up of the assessed dimensions are and what the policy impact measures are, so as to improve the performance of these hospitals and the efficiency of management of resources allocated by means of a goal contract.

In the international literature, teaching hospitals are defined by the presence of residents and/or by the affiliation with governmental associations or councils of medical education.⁹ It is known that the number of residents, product of the teaching dimension resulting from training and specialization in the service, is also a resource for the health care dimension, influencing the cost and efficiency of procedures.⁷ This example shows the importance of bringing the relationship between dimensions and/or university hospital missions closer together.

The objective of the present study was to assess the performance of general (not specialized) federal university hospitals, associated with the Ministry of Education, considering the integration between health care and teaching activities developed in them.

METHODS

Performance was assessed with Data Envelopment Analysis - DEA. The relationship between teaching and health care was approached with the network DEA model. This approach allows for further assessment of efficiency of each of the 30 hospitals and of each dimension inside them, enabling comparisons among teaching hospitals and between these and hospital institutions that do not develop academic activities.

The measure of productivity and efficiency in the DEA, generated by linear programming, is used by comparing similar units, or Decision Making Units (DMU), which show multiple inputs and several outputs, only differing in terms of the amounts consumed and produced. One DMU will be efficient if it shows, comparatively to others, higher production for fixed amounts of resources (output-oriented) and/or if it uses fewer resources to generate a fixed amount of products (input-oriented). By defining the DMUs with the best practices, the DEA creates an empirical production frontier, and the level of efficiency varies between 0.00 and 1.00 (or between 0 and 100%), depending on the distance between the unit and the frontier. In the following formula, this distance is represented in the DEA envelopment model by the lambda intensity variable. The same model considers Variable Returns to Scale (VRS) and is output-oriented. According to the radial projection of inefficient units on the frontier, their benchmarks - or reference DMU - can be observed, as well as the ideal input and output values for the unit to become efficient.

In addition, for the units to be considered efficient, the "Pareto-Koopmans" logic is applied, where a unit situated on the frontier will only be efficient if it is not possible to reduce any input, or increase any output, without having to also increase another input or reduce another output simultaneously. In other words, a hospital with maximum efficiency, but situated in an inefficient-Pareto area, has "technical", weak or Farell efficiency. Only the projection in a Pareto-efficient area achieves maximum unit efficiency.²

Knowing that the application of the DEA in any sphere of the health sector must consider a systemic context, full of connections between dimensions and variables, these must be well understood before the modeling itself.³ In the specific literature on DEA for teaching hospitals, inputs, such as beds, cost-related resources (in the case under study, the budget originated from the SUS), equipment, laboratories and workers, generate outputs, such as health care production, students (several levels) and technologies resulting from research, while the relationships occurring among these same variables in each DMU are not usually considered.⁵ For this reason, the traditional DEA model has been known as the aggregate or "Black Box" model, as illustrated in Figure 1. In this Figure, teaching and health care are considered as a single block; residents were not included in the example, because they are in the "Black Box" (as teaching output and health care input); although they could be included, whether as input or output, through the "Black Box" model. Medical doctors and professors were considered in their entirety (full-time equivalent) together with beds and budget as inputs, whereas the outputs were the undergraduate medical students and the hospitalizations adjusted by complexity.

A proxy measure of case-mix was used to adjust complexity, created by the Ministry of Education team of technicians, based on the number of high-complexity procedures performed by the unit and registered with the Ministry of Health. A specific weight (scores from 1 to 5) was attributed to each procedure that requires registering (such as neurosurgeries, cardiac surgeries and transplants), according to the volume of resources necessary to perform them. The weighted sum of high-complexity procedures generated an index for each hospital, detailed in a previous study.¹⁴ According to La Forgia & Couttolenc,¹¹ adjustment by case-mix must consider the heterogeneity of patients cared for, in terms of the amount of resources used and treatment costs; there are several forms to proceed, although it is essential to guarantee validity of the model. Another approach to deal with these differences consists in analyzing more homogeneous sub-samples individually, according to the size of the unit, level of complexity of services or technological level of the equipment available.

"The Black Box" model has predominated among scientific publications that assess performance of health units, such as hospitals, medical services, health centers and planning areas, with several variable configurations. On the national level, Marinho & Façanha,¹⁷ used Ministry of Education university hospitals as DMU, although with a group where a set of units with different profiles was considered in the same sample, such as general, specialty and maternity hospitals. Gonçalves et al⁶ compared the public hospital system in Brazilian capitals, proposing a methodology to avoid null weights in the model.

Certain works approach the multi-dimensionality found in teaching hospitals with assessments performed separately, creating a frontier for each dimension, and being subsequently gathered¹³ or compared according to the relative efficiency of each dimension (Figure 2).²¹ The set of variables proposed by Ozcan to deal with the health care dimension in a hospital environment has been a consensus in the literature. Among inputs, this set considers human resources, costs and beds (along with service-mix, it is proxy for capital); while, among outputs, the production adjusted for severity. The author made an analysis of sensitivity of several combinations of inputs and outputs to achieve this model, obtaining stability in efficiency scores.^{19, 20}

However, there are variables present in more than one dimension (such as doctors and professors, who share teaching and health care actions) and flexible variables, which function as inputs for a dimension and as outputs for another (residents). These peculiarities are not approached by the separate models, which can thus harm their validity and reliability. In Figure 2, two independent frontiers are built: the teaching border (with S! of doctors and T! of professors as inputs, and number of undergraduate students and residents as output) and the health care border (with residents, T! of doctors, S! of professors, beds and the budget as inputs, and hospitalizations adjusted for complexity as outputs). It should be noted that the choice of T! and S! was arbitrary, considering a greater volume of teaching activities among professors and a greater volume of health care tasks for doctors, provided that both perform in the two dimensions.

Systems with two or more processes connected to each other, whether in series or parallel, form networks. The network model consists of a family of DEA models, with the linear restrictions for each sub-process analyzed. The DEA network designs flows of relationship between variables and generates a score of total efficiency, in addition to a score for each dimension or process that develops in each of the DMU (divisional score).

Data from the second semester of 2003 were used to develop the network DEA model, referring to the universe of 30 Ministry of Education general university hospitals, considering (Figure 3): two dimensions (teaching and health care), four external inputs (monthly budget and number of beds for health care; T! of doctors for health care and S! for teaching; S! of professors for health care and T! for teaching), one intermediate input/output (residents), two final outputs (hospitalizations, adjusted for complexity; undergraduate medical students). The dimension of research was not considered in this model due to the low validity of data (systematic errors in collection and storage) in that period, as demonstrated in a previous publication.¹³ The selected network model considers the VRS, given the variation in the size of hospitals, and it is output-oriented, due to the need to improve management of resources, in addition to the respective managers' low capacity to administer the human resources of their units (inputs). The health care and teaching dimensions had similar weights in the model. Moreover, in this model, the connection between dimensions (residents) was dealt with as a non-discretionary variable ("free" link value).²³ The software used was DEA Solver Pro (Professional Version 6.0).

RESULTS

Table 1 shows mean, maximum and minimum values and standard deviation of input and output variables of each dimension considered in the model. The Ministry of Education UHs showed a great variation in size and scale, which should be considered when comparing similar units identifying benchmarks.

Table 2 shows the hospital efficiency scores according to different models: a) "Black Box"; b) Separate: pure teaching and pure health care; c) Network DEA. In the "Black Box" model, the DMU were considered efficient and the mean of scores was equal to 0.78 (or 78%). In the separate models, means were 0.75 and 0.74, with nine and ten efficient units on the teaching and health care frontiers, respectively.

According to the Network DEA model, one DMU will only be efficient if it has a score of 100% in all dimensions assessed. As this is a model with variable returns to scale, it is also expected that each dimension should have at least one efficient DMU (something that does not occur with constant returns to scale). In the Network model, the mean of efficiency was 0.54 and the minimum value was 0.19, precisely for a hospital unit that favors the presence of residents and transfers students (teaching output) to other partner hospitals. This same hospital was efficient for the "Black Box" and separate models of the teaching dimension, as it attributed a null weight to its number of students. Only two units were simultaneously efficient in the dimensions analyzed, both with at least 150 beds and working with a low volume of resources (thus, lower consumption of inputs).

In view of the network DEA assessing the relative efficiency of each dimension and analyzing the correspondence among them, the fact that all units could maintain the score of technical efficiency in the health care dimension draws attention, even if at the expense of the teaching dimension. In other words, the relative efficiency for the health care dimension was 100% for all units, whereas the relative efficiencies of the teaching dimension varied from 0.11 to 1.00 (11% to 100%), with a mean of 0.39 (39%). One alternate situation, attributing a weight of 70% to the teaching dimension, was tested, although significant differences in results were not found.

In addition, Table 2 shows the benchmarks for hospitals, according to each dimension. In the teaching dimension, units that worked with low inputs, in this case, a lower volume of human resources, stood out. This aspect needs to be much better analyzed for larger hospitals, which also include research activities in their agenda, both from professors and medical doctors. If the eight hospitals with 300 beds are considered, the UFMG university hospital obtains the highest score (equal to 0.49). The benchmarks of the healthcare dimension define hospitals that resembled one another more closely in size and institutional mission.

Table 3 shows the difference between the value projected on the frontier and the value observed in each variable of the model, for each hospital. The values of efficient units, closer to the projected points, defined the benchmarks shown in Table 2. In the teaching dimension, there is an excess of 261 (9%) professors, although they have other research activities, as previously pointed out. There is not an excess of doctors and the number of undergraduate students could be increased by approximately 200% (from the fourth to the last semester, there were 14,206 medical students). In this universe, only the hospital with the lowest teaching score would require more than 3,788 students. The need for an increase in the number of students was maintained, even when a model with professors as the only input variable was tested.

In the health care dimension, all hospitals could have reductions in inputs to achieve improvement in efficiency. Going back to the Pareto-Koopmans logic, even for units with a 100% score, if it is possible to reduce inputs without the need to increase any other input, this unit is not considered Pareto-efficient, i.e. its projection occurs in a geometric area of the frontier parallel to the axis of one of the variables. In practice, divisional efficiencies would be maintained, even with a reduction of 27% in doctors, 13% in professors, 20% in beds and 8% in the budget. With this perspective, the concept of efficiency in the DEA should consider, in addition to scores, the amount of resources that needs to be changed to define efficient hospitals (Table 3).

As regards the required changes in the variable connecting the dimensions, that of the residents, only three hospitals should not change the number of residents (UFJF, UFBA, UFPA), nine should reduce it (FMTM, FUAM, HCPA, UFCE, UFPEL, UFPR, UFU, UNB and UNIFESP); and the remaining ones should increase it. The sum of the needs for increase in this number, considering all hospitals, was 488 or 14% of the total number of residents present in these units.

DISCUSSION

The DEA enables multiple inputs and outputs to be put together, when assessing the efficiency of autonomous units, improving the traditional cooperative analysis of ratios that rely on only one numerator (of production) and only one denominator (of resources). In contrast, the network DEA model increases its discriminatory power, once it brings about a score for the total efficiency of the unit and another for each dimension, in addition to its measuring the influence of variables connecting the dimensions analyzed.

The pioneering work that investigated the influence of processes existing in the "Black Box" is the one by Färe & Grosskopf.⁴ These network models were improved by Lewis & Sexton¹² and had important applications to assess sectors of the economy of countries of the Organization for Economic Cooperation and Development (OECD).²² As an application to health, the work of Löthgren & Tambour¹⁵ used this model to assess Swedish drugstores, considering production and user satisfaction as complementary processes in each unit. More recently, Tone & Tsutsui²³ proposed the use of slack-based measures, when the equiproportional change in inputs and/or outputs would not be expected as an excellent solution. Kao¹⁰ formalized the DEA multiplier model and inferred some of the properties of the systems when considering the relationships among different system activities, in series and in parallel.

The use of network DEA models is particularly useful in studies on performance applied to health, where each unit of analysis can be seen as an open system, comprised of many parts that are interconnected, in addition to being dynamically influenced by external and environmental joining variables. In terms of UHs, health care, teaching and research are dimensions that are present and interactive in the mission of each hospital, although each group of activities is regulated by distinct governmental organizations. The unit manager is usually the one responsible for the task of connecting the dimensions, which is sometimes conflicting. In this context, the professor-healthcare interaction is at times pointed out as a key point for the institution's organizational sustainability.¹⁶ Following the logic of funding from the Fundo Nacional de Saúde (National Health Fund), if the health care management of university hospitals is usually aimed at meeting the demand of the hierarchical health system, where the hospital provides higher-complexity care, the teaching management still emphasizes a predominantly intra-hospital teaching, not necessarily focused on diseases prevalent in the immediate surroundings.¹

The quantitative Network DEA model enabled this process of connection to be approached, bringing about some interesting conclusions and recommendations, with a positive impact on hospital efficiency. In the international literature, residents are sometimes included as inputs, and, at other times, as outputs, without distinction, precisely because they represent a link between the teaching and health care dimensions. If they are included as inputs, the frontierborder of teaching hospitals tends to become more distant from that which considers hospitals without academic activities.⁸

According to the dimensional scores found, UHs had expectations about the guarantee of efficiency of health care, once this is the dimension that originates and guarantees the resources to fund hospitals, by means of a monthly budget from the Ministry of Health. The emphasis on health care was maintained, even when the weight of this dimension decreased to 30% (versus 70% of the teaching dimension). This aspect could not have been observed by the model of separate dimensions.

As regards the teaching dimension, the possibility of doubling the current number of students seems to be the major obstacle identified to justify the low efficiencies found. This measure should be analyzed with caution, according to the demand resulting from the inclusion of new doctors in the labor market, although indicating that there is still room to receive these students in the UHs before new medical courses are created.

Moreover, with regard to the teaching dimension, the model tended to favor the units considered "simpler", i.e. those working with low resources, by giving them higher scores. This could be considered a limitation to the model that needs to be dealt with. Larger, more complex hospitals develop other research and technological assessment activities that need to be analyzed. Even with a lack of information about research, the model could include certain weight restrictions that reduce this potential bias. Likewise, as described in the Methods, the choice of volume of work spent in the dimensions by professors and doctors was arbitrary. In addition, a certain analysis of sensitivity could be introduced to suggest an ideal distribution of working hours. In the present study, authors observed that the distribution proposed did not show changes, when compared to the model where 100% of professors worked exclusively with teaching and 100% of doctors worked exclusively with health care (in this case, the number of variables in the model was reduced, without a significant change in its discriminatory power).

The model also showed what changes are necessary to optimize the number of residents, in terms of its being a strategic variable of the teaching-health care relationship in each unit. If, for the teaching dimension, the higher the number of students, the greater the teaching hospital production; for the health care dimension, an increase in this input, from a certain value on, can cause a decrease in the efficiency of the unit, a phenomenon known as congestion.⁷ The model enabled the question of the number of students to be calculated, thus becoming another instrument for the unit manager. This information is also particularly useful for the organization that regulates medical residency programs, in this case, the Ministry of Education, which estimates the need for vacancies and grants medical residency scholarships to university hospitals. The same model can also consider different medical specialties to help decision-making.

The dimension of quality of service in the UHs can be created, where the output is the score of user satisfaction, in addition to improving the analysis of teaching quality, using the weighting of units, according to the assessment grade of the teaching institution or the newly graduates' knowledge level scores.

As a future development, authors in this study expect to design a methodology, considering the measures of non-radial projection, to guarantee that all efficient units are projected in Pareto-efficient areas of the frontier; to include restriction to weights based on the opinion of specialists; and to apply the network DEA methodology to the dynamic assessment of efficiencies throughout time.

REFERENCES

Received: 11/5/2009

Approved: 12/20/2009

The authors declare there are no conflicts of interest.

• 1. Campos GWS. Educação médica, hospitais universitários e Sistema Único de Saúde. Cad Saude Publica. 2005;15(1):187-93. DOI:10.1590/S0102-311X1999000100019
• 2. Cooper WW, Seiford LM, Tone K. Data envelopment analysis: a comprehensive text with models, applications, references and DEA-Solver Software. 2. ed. New York: Springer; 2007.
• 3. Chilingerian JA, Sherman HD. Health care applications from hospitals to physicians: from productive efficiency to quality frontiers. In: Cooper W, Seiford, LM, Zhu J, editors. Handbook on data envelopment analysis. Boston: Kluwer Academic Publishers; 2004.
• 4. Färe R, Grosskopf S. Intertemporal production frontiers: with dynamic DEA. Boston: Kluwer Academic Press Publishers; 1996.
• 5. Färe R, Grosskopf S. Network DEA. Socioecon Plann Sci. 2000;34(1):35-49. DOI:10.1016/S0038-0121(99)00012-9
• 6. Gonçalves AC, Noronha CP, Lins MPE, Almeida RMVR. Análise Envoltória de Dados na avaliação de hospitais públicos nas capitais brasileiras. Rev Saude Publica. 2007;41(3):427-35. DOI:10.1590/S0034-89102006005000023.
• 7. Grosskopf S, Margaritis D, Valdmanis V. The effects of teaching on hospital productivity. Socioecon Plann Sci. 2001;35(3):189-204. DOI:10.1016/S0038-0121(01)00006-4
• 8. Grosskopf S, Margaritis D, Valdmanis V. Comparing teaching and non-teaching hospitals: a frontier approach (teaching vs. non-teaching hospitals). Health Care Manag Sci. 2001;4(2):83-90. DOI:10.1023/A:1011449425940
• 9. Grosskopf S, Margaritis D, Valdmanis V. Competitive effects on teaching hospitals. Eur J Oper Res. 2004;154(11):515-25. DOI:10.1016/S0377-2217(03)00185-1
• 10. Kao C. Efficiency Decomposition in network data envelopment analysis: a relational model. Eur J Oper Res. 2009;192(3):949-62. DOI:10.1016/j.ejor.2007.10.008
• 11. La Forgia GM, Coutollenc BF. Hospital performance in Brazil: the search for excellence. Washington: The World Bank Publications; 2008.
• 12. Lewis HF, Sexton TR. Network DEA. Efficiency analysis of organizations with complex internal structure. Comput Oper Res. 2004;31(9):1365-10. DOI:10.1016/S0305-0548(03)00095-9
• 13. Lins MPE, Lobo MSC, Fiszman R, Silva ACM, Ribeiro VJP. O uso da análise envoltória de dados - DEA - para avaliação de hospitais universitários brasileiros. Cienc Saude Coletiva. 2007;12(4):985-98. DOI: 10.1590/S1413-8123200700040002010
• 14. Lobo MSC, Bloch KV, Fiszman R, Oliveira MR, Ribeiro VJP. Sistema de Informações dos Hospitais Universitários (SIHUF/MEC): um banco de dados administrativo. Cad Saude Coletiva. 2006;14(1):149-62.
• 15. Löthgern M, Tambour M. Productivity and customer satisfaction in Swedish pharmacies: a DEA Network model. Eur J Oper Res. 1999;115(3):449-58. DOI:10.1016/S0377-2217(98)00177-5
• 16. Machado SP, Kuchenbecker R. Desafios e perspectivas futuras dos hospitais universitários no Brasil. Cienc Saude Coletiva. 2007;12(4):871-7. DOI:10.1590/S1413-81232007000400009
• 17. Marinho A, Façanha LO. Hospitais universitários: avaliação comparativa da eficiência técnica. Econ Apl. 2000;4(2):315-49.
• 18. Médici AC. Hospitais Universitários: passado, presente e futuro. Rev Assoc Med Bras. 2001;47(2):149-56. DOI:10.1590/S0104-42302001000200034
• 19. Ozcan Y. Sensitivity analysis of hospital efficiency under alternative output/input combinations and peer groupings. Knowl Policy 1993;5(4):1-31.
• 20. Ozcan Y. Health care benchmarking and performance evaluation: an assessment using Data Envelopment Analysis (DEA). Berlin: Springer; 2008. DOI:10.1007/978-0-387-75448-2
• 21. Ozcan Y, Lins MPE, Lobo MSC, Silva ACM, Fiszman R, Pereira BB. Evaluating the performance of Brazilian university hospitals. Ann Oper Res In press 2010. DOI:10.1007/s10479-009-0528-1
• 22. Prieto AM, Zofio JL. Network DEA efficiency in input-output models: with an application to OECD countries. Eur J Oper Res. 2007;178(1):292-304. DOI:10.1016/j.ejor.2006.01.015
• 23. Tone K, Tsutsui M. Network DEA: a slacks-based measure approach. Eur J Oper Res. 2008;197(1):243-52. DOI:10.1016/j.ejor.2008.05.027

Publication Dates

• Publication in this collection
  17 Sept 2010
• Date of issue
  Aug 2010

History

• Received
  05 Nov 2009
• Accepted
  20 Dec 2009