Use of comorbidity measures to predict the risk of death in Brazilian in-patients

OBJETIVO: Avaliar o uso de medidas de comorbidade para predizer o risco de óbito em pacientes brasileiros. MÉTODOS: Foram utilizados dados de internações obtidos do Sistema de Informações Hospitalares do Sistema Único de Saúde, que permite o registro de somente um diagnóstico secundário. Foram selecionadas 1.607.697 internações ocorridas no Brasil em 2003 e 2004, cujos diagnósticos principais foram doença isquêmica do coração, insuficiência cardíaca congestiva, doenças cérebro-vasculares e pneumonia. O Índice de Charlson e as comorbidades de Elixhauser foram as medidas de comorbidade utilizadas; o simples registro de algum diagnóstico secundário foi também empregado. A regressão logística foi aplicada para avaliar o impacto das medidas de comorbidade na estimava da chance de óbito. O modelo de base incluiu as seguintes variáveis: idade, sexo e diagnóstico principal. Os modelos de predição de óbitos foram avaliados com base na estatística C e no teste de Hosmer-Lemeshow. RESULTADOS: A taxa de mortalidade hospitalar foi 10,4% e o tempo médio de permanência foi 5,7 dias. A maioria (52%) das internações ocorreu em homens e a idade média foi 62,6 anos. Do total de internações, 5,4% apresentava um diagnóstico secundário registrado, mas o odds ratio entre óbito e presença de comorbidade foi de 1,93. O modelo de base apresentou uma capacidade de discriminação (estatística C) de 0,685. A melhoria nos modelos atribuída à introdução dos índices de comorbidade foi fraca – equivaleu a zero quando se considerou a estatística C com somente dois dígitos. CONCLUSÕES: Embora a introdução das três medidas de comorbidade nos distintos modelos de predição de óbito tenha melhorado a capacidade preditiva do modelo de base, os valores obtidos ainda são considerados insuficientes. A precisão desse tipo de medida é influenciada pela completitude da fonte de informação. Nesse sentido, o alto sub-registro de diagnóstico secundário, aliado à conhecida insuficiência de espaço para anotação desse tipo de informação no Sistema de Informações Hospitalares, são os principais elementos explicativos dos resultados encontrados.

Concerns about quality of care have triggered comparative analyses of health service performance indicators, especially hospital care.In several countries, governmental agencies, hospital associations, health insurance companies and consumer associations perform and publish comparative assessments of hospital performance, using mortality rates and other indicators. 4The availability of large computerized administrative databases has promoted this type of approach. 17It is necessary to consider the difference in prevalence of risk factors that change the prognosis and therapeutic response in inpatient care to assess quality of performance. 6e risk of a patient is associated with the severity of the case, and greater severity means higher risk or probability of occurrence of an undesirable outcome.Risk is a multidimensional concept that includes several attributes of a patient, such as age, sex, clinical instability, primary diagnosis, extension and severity of comorbidities and patient attitudes and preferences. 6veral methods to measure the severity of cases have been developed to enable the comparison of indicators from the case mix adjustment.The intensity (number and severity) of coexisting pathologies is one of the predictive factors of unfavorable outcomes and complications in in-patients. 6Methodologies that use comorbidities to weigh their effect on patient prognosis can be applied to administrative databases, once they usually include diagnostic information exclusively. 5wever, the quality and value of this type of method depend on the completeness and accuracy of diagnostic codes.With such characteristics, the Charlson Comorbidity Index (CCI) 2 and the methodology developed by Elixhauser et al 3 are used as the approach to risk adjustment in several studies. 5These two methods differ from each other mostly in terms of the number of comorbidities included and attribution of weights to weigh their prognostic effect.This weighing is present in 19 clinical conditions comprising the CCI. 2 Elixhauser's methodology does not attribute any weight to the 30 comorbidities defined, focusing exclusively on the number of pathologies present. 3Indications of the validity of such comorbidity indices to measure the severity of cases have been reported in the literature. 5,11e use of risk measures to adjust performance indicators is uncommon in Brazil, as are studies on the validity of such measures. 7,12,13Results from studies that used the Sistema de Informação Hospitalar do Sistema Único de Saúde (SIH/SUS -National Health System Hospital Information System) were limited to either the analysis of hospitalizations in specifi c cities (Rio de Janeiro) 7 or to the surgical procedure selected (myocardial INTRODUCTION revascularization). 12,13The SIH/SUS enables the record of only one secondary diagnosis, which is not mandatory for payment of hospital services.
The objective of the present study was to assess the use of comorbidity measures to predict the risk of death in in-patients, using the SIH/SUS databases and methodologies proposed by Charlson 2 and Elixhauser. 3

METHODS
The SIH/SUS includes anonymous information about the following variables: demographic profi le of patients (sex and age); primary and secondary diagnoses; surgical, therapeutic and diagnostic procedures; medical specialty of the case treated (general or specialized surgery and obstetrics, among others); days of stay; discharge status and hospital unit.First, 4,086,329 hospitalizations resulting from respiratory and circulatory problems, based on the International Classifi cation of Diseases, 10 th revision (ICD-10), and funded by the Sistema Único de Saúde (SUS -National Health System) in Brazil, between 2003 and 2004, were included to defi ne the universe of study.These two health problems were selected according to the following criteria: volume of hospitalizations in the period higher than 500,000, hospital mortality rate higher than 4.9%, and volume of deaths in the period higher than 99,000.The mean value of reimbursement for hospitalization and length of stay were used as secondary criteria, once they show the importance of hospitalizations in terms of use of hospital resources.For this selection, tabulations based on the SUS hospital morbidity were constructed, according to information available on the website of the Sistema de Informática do SUS (DATASUS -SUS Information Technology System). a The primary diagnosis of a patient is an essential dimension to risk adjustment, once severity may differ considerably among diagnostic categories.However, also at this stage, specifi c reasons for admission were selected to comprise the universe of study, considering the volume of hospitalizations and deaths per pathology as selection criteria.The hospitalizations selected were those whose ICD-10 codes were registered as primary diagnosis: ischemic heart disease (ICD-10: I21 and I25); congestive cardiac failure (ICD-10: I50); stroke (ICD-10: I60-I62, I64, I67, I69) and pneumonia (J15, J18).Hospitalizations of patients aged less than 18 years and those who provided incorrect information about sex, i.e. who used inexistent SIH/SUS codes, and had a length of stay above 30 days were excluded.The result of this process totaled 1,607,697 hospitalizations.Frequency measures and bivariate and multivariate analyses were used to assess the use of comorbidity measures.Logistic regression was employed to assess the impact of comorbidity measures on the estimate of risk of death.The deaths occurred during hospitalization were the dependent dichotomous variable.The impact of introduction of each of the comorbidity measures was tested to predict death in the baseline model.Considering the information available in the SIH/SUS databases, the baseline model included the following variables: age, sex and primary diagnosis.Age was treated as a continuous variable; the sex variable as a dichotomous variable and the male sex was the reference category.The primary diagnosis variable was considered a categorical variable with 11 groups and the reference category was chronic ischemic heart disease (ICD-10: I25), as it showed the lowest mortality rate.
The algorithm developed by Quan et al, 14 which defi nes the ICD-10 codes for each comorbidity included in Charlson's 2 and Elixhauser's 3 methodologies, was used to calculate the severity score.This choice is justifi ed because it is a proposal that aims to adopt a standardized coding for international use.These authors reviewed the different adaptations for ICD-10 available in the literature at that moment and made them compatible.
A total of three comorbidity measures were analyzed: (1) the CCI, 2 codifi ed according to Quan's 14 algorithm for ICD-10; (2) Elixhauser comorbidities, 3 also codifi ed according to Quan's 14 algorithm; and (3) the presence of comorbidity (secondary diagnoses -yes/no).Comorbidity measures were introduced in the models tested as an independent categorical variable and regrouped according to the distribution of frequency, based on CCI weighing.Weights were grouped into the following categories: category (1) weight equal to 0; category (2) weight equal to 1; category (3) weight equal to or higher than 2. Weight equal to zero (category 1) was used as reference category, because a score equal to zero means absence of severity.The other two comorbidity measures were considered dichotomous variables (0 = absence and 1 = presence).
The adequacy of the death prediction model was assessed based on the capacity to discriminate and on the adjustment of models.The statistics used were the percentage of improvement of the model in relation to the initial likelihood ( 2 ), C-statistic and Hosmer-Lemeshow goodness of fi t test.Statistical analyses were processed in the Stata software, version 10.0.

RESULTS
In the period of study, 1,607,697 hospitalizations occurred due to ischemic heart disease, congestive cardiac failure, stroke and pneumonia.Mean age of patients was 62.6 years and the percentage of hospitalizations in men was 51.9% (Table 1).The majority of hospitalizations occurred in private hospitals (63.9%).Patients remained hospitalized for 5.7 days and surgical interventions totaled 5.5% of cases (Table 1).Hospitalizations with a recorded secondary diagnosis (comorbidity) corresponded to 5.4% (Table 1).Stateowned hospitals were those that showed the highest percentage of recorded secondary diagnosis (18.7%).For the diagnoses selected, a hospital mortality rate of 10.4% was observed, of which 14.8% were associated with acute myocardial infarction (I21) and 7.2% with congestive cardiac failure (I50).As regards stroke, mortality varied between 6.4% and 32.0%, according to the diagnostic category.In cases of pneumonia, this variation was lower, between 6% and 8% (Table 1).
Tables 2 and 3 show the odds ratio (OR) between death and comorbidity measures.Of all clinical conditions that comprise the Charlson Index, 14 showed a statistically signifi cant OR.However, the OR was below 1.20 for four clinical conditions.Cases with ulcer showed an OR equal to 3.15 (Table 2).As regards the 30 comorbidities defi ned by Elixhauser, at least one third had an OR signifi cantly associated with the occurrence of deaths (Table 3).Of all these 30 comorbidities, 13 showed an OR higher than 1.50; of these, coagulopathies, weight loss, hydro-electrolytic imbalance and alcohol abuse are not included in the CCI (Table 3).
The percentage of cases with a score different from zero, i.e. with a certain level of severity, was low for both the CCI and Elixhauser comorbidities (Table 4).Mortality rates increased and were statistically signifi cant, indicating an association between these two comorbidity measures and the risk of death (Table 4).The mortality rate is higher due to the recording of comorbidity -patients without a comorbidity showed a mortality rate of 10%, whereas this rate was 17.6% among patients with one comorbidity (Table 4).
Of all models for hospital death prediction tested, model 4 showed the best discriminatory capacity (C-statistic = 0.691), incorporating the simple presence of comorbidity (Table 5).The effect on the discriminatory capacity of the baseline model (model 1), attributed to the incorporation of comorbidity measures, was insignifi cant in all models tested.Finally, all models tested showed calibration problems (Table 5).

DISCUSSION
The present study used information about secondary diagnosis recorded in the SIH/SUS to assess the severity of cases, based on comorbidity measures.In this assessment, the use of Elixhauser comorbidities, 3 which includes other pathologies previously excluded from the CCI, did not increase the predictive capacity, which was even lower than that observed for the CCI.The improvement in the predictive capacity of the baseline model (C-statistic = 0.685), attributed to the comorbidity measures, was poor -equal to zero, when considering C-statistic with only two digits, i.e. all models showed C-statistic equal to 0.69.The record of any secondary diagnosis (C-statistic of 0.691) was more important than other comorbidity measures assessed.
In addition, an OR equal to 1.93 was found between presence of comorbidity and death, a value higher than that obtained for the other two comorbidity measures.
The validity of use of severity score measures, such as the CCI, 2 or those that use the presence or not of a pathology for admission, such as Elixhauser comorbidities, 3 depends on the completeness and accuracy of diagnostic codes recorded in the databases.
Underreporting also interferes with the discriminatory capacity of these measures.In the data analyzed, the percentage of recording of secondary diagnosis in national hospitalizations was low (5%).The results found seem to indicate disregard for or unawareness of the importance of this type of information.A previous study 9 applied the CCI to hospitalizations occurred in 1993-1994, in the city of Rio de Janeiro, Southeastern Brazil, and obtained a score equal to zero in 94.3% of cases.This percentage is similar to that observed in this study for the CCI and Elixhauser comorbidities, whose values were higher than 95%.
As regards the completeness of diagnoses, the SIH/SUS databases enable only one secondary diagnosis to be recorded.In addition, this information is irrelevant for hospitalizations payments, resulting in underreporting.
The main impact of this situation is refl ected in the results of comparison of death prediction models -all of them showed a discriminatory capacity lower than 0.70, which is considered insuffi cient. 1 A Brazilian study that assessed the validity of use of the CCI with hospital data from the city of Ribeirão Preto, Southeastern Brazil, which recorded one primary diagnosis and two secondary diagnoses at that time, showed and compared models to predict death with a higher predictive capacity (C-statistic of 0.72). 8However, although the result was comparatively better when data from Ribeirão Preto were used, rather than SIH/SUS data, the C-statistics obtained were still lower than those reported in international studies.These studies tested the effect of the CCI, using databases with records of up to 15 diagnoses, and obtained better discriminatory capacity of death prediction models (C-statistics higher than 0.80). 11,16ong the study limitations, although including SUS-funded Brazilian hospitalizations, the population studied is restricted to specifi c diseases of the respiratory and circulatory systems.Even considering that the discriminatory capacity (C-statistic) is more important when the model is constructed to predict individual results, 6 all models showed adjustment problems, assessed by the Hosmer-Lemeshow test.
The high number of hospitalizations analyzed may be the main explanation for this problem of adjustment of models, once previous Brazilian studies did not report this type of fi nding. 8,9,12In view of the previously known limitations of the SIH/SUS, which encouraged researchers to perform the present study, and the magnitude of underreporting of secondary diagnoses found in the population analyzed, other studies are necessary.Future studies should be aimed at fi nding out the magnitude of underreporting, using medical records as source of information.Analyses of this type could help to assess the potential measurement bias resulting from the use of an administrative database.Considering the lack of studies on validity of comorbidity measures in the Brazilian population, 8 initiatives in this area are also important, especially in terms of Elixhauser et al's methodology, 3 which has not yet been validated in Brazil.
The results obtained do not promote the use of comorbidity measures, such as the proposals by Charlson 2 and Elixhauser, 3 based on the SIH/SUS.However, despite the limited effect on the capacity to predict death, probably associated with the quality of diagnostic information available in the national databases, the use of measures based on the presence of comorbidities is still recommended to adjust the risk of death or other performance indicators, as done in other countries.Thus, the CCI 2 and Elixhauser comorbidities 3 are still considered useful, especially with the adaptations to the ICD-10 performed by Quan et al. 14 The diagnostic information available in the Brazilian databases, particularly that from the SIH/SUS, need to be enriched.In other countries, discharge summaries enable secondary diagnoses to be recorded, including between 15 and 25 spaces 6,16 for this purpose.In addition, educational strategies for clinical professionals should be developed, aiming to promote recording of such information.Together with educational actions, the creation of mechanisms to monitor the quality of information recorded in different sources is important, whether in the patient's medical chart or in specifi c information system forms, such as the administrative database of hospital production.
In conclusion, fi ndings from the present study indicate the importance of allowing a higher number of spaces to record comorbidities of Brazilian in-patients, as reported by previous studies. 8,10,15Although the SIH/ SUS was conceived in the 1980s, opportunities for improvement have not been identifi ed or implemented, differently from the hospital information systems of other countries.This situation partly results from the diffi cult debate between type of payment for hospital services and the information system.Until now, there has not been an information system in Brazil that enables a complete description of hospital morbidity, when compared to those present in other countries, thus limiting the use of such information to assess the performance of services, among other things.This will require human and fi nancial investments, but it will result in the construction of more valid performance indicators, which enable monitoring and improvement of the health care provided by the Brazilian health system.

Table 5 .
Discriminatory capacity and adjustment of death prediction models, according to comorbidity measures.Brazil, 2003-2004.Likelihood 2 only for the intercept = 694745.46CCI: Charlson Comorbidity Index a