Predictive Validity of a Questionnaire to Identify Older Adults at Risk for Hospitalization

OBJECTIVE: To test the predictive validity of the Probability of Repeated Hospital Admissions questionnaire among older adults. METHODS: A population-based cohort study with a 6-month follow-up was implemented with 515 non-institutionalized older people (≥60 years) cared for by the Family Health Strategy in the city of Progresso, Southern Brazil, in 2005. Participants answered eight objective questions that were entered in a logistic regression model to estimate the risk of future hospital admission, by risk strata. Survival analysis and the receiver operating characteristics curve were utilized to assess instrument validity. RESULTS: Among participants, 56.1% were women and 10.1% were hospitalized. The high-risk group had a 6.5 times greater frequency of hospitalization in comparison to the low-risk category. CONCLUSIONS: The instrument is effective in assessing the risk of hospitalization among older adults attended by the Family Health Program of the National Unifi ed Health Care System.


INTRODUCTION
Spending by the Sistema Único de Saúde (SUS -Brazilian National Unifi ed Health System) will increase considerably over the next decades with an increased older population and greater frequency of chronic degenerative diseases.In 2004, the hospitalization rate in the SUS was 41.8 admittances per 1,000 residents among the 0 to 14 year age group, 61.2 for the 15 to 59 year age group and 143.3 for the 60 or older group. 3,5,8 2004, 25% of SUS expenses were attributed to hospitalization of older adults, which were 8.4% of national population in this period.Expenditures on hospitalizations were R$ 20.6 (US$ 7.36 a ) per person/year for the younger group, R$ 33.2 (US$ 11.86) for the 15 to 59 year age group and R$ 107.3 (US$ 38.32) for the group equal to or older than 60 years.b,c a US$ 1.00 was worth R$ 2. 80  Hospitalization of older adults generates a series of events that frequently culminate in diminished functional capacity and quality of life, complications unrelated to the reason for admission. 4,7,18erefore, for many older adults hospitalization does not result in improved health and on the contrary, is often correlated with increased mortality and morbidity rates, worse prognosis and predisposition for frailty. 7,18 SUS, health care for older adults is defi ned from an understanding of population heterogeneity in terms of health service utilization.According to this concept, care for older adults should be hierarchized and health actions for healthy older people should be prioritized, together with programs suited for those already sick.b To enable these actions, the development of an objective method to identify older adults at risk of hospitalization is fundamental.This procedure should have measurement validity, be easy to administer and produce easy to understand results so that Community Health Agents and Family Health Teams can utilize results to establish care priorities and preventive interventions among the population. 6,10 a cohort study, Boult et al 1 (1993) identifi ed eight risk factors for increased use of hospital services by older adults (age greater than 75 years, male sex, willingness of caretaker, self-perceived poor health, cardiovascular disease, diabetes mellitus, hospitalization in previous 12 months and more than six medical visits in previous 12 months), which are also valid for the identifi cation of fragile adults.Based on these results, Boult et al 1 developed a Probability of Repeated Hospital Admissions (Predição de Risco de Admissão Hospitalar -PRA) questionnaire. 1,3,12,13In accordance with the presence or not of risk factors, the logistic regression model applied to PRA generates a value from 0 to 1. Higher values correspond to increased risk of hospital admittance.
Two studies evaluated the validity of PRA in Brazil.The fi rst study, performed by Veras et al 19 (2000), administered PRA at the reception of a geriatric ambulatory clinic, and the second, by Negri et al 12 (2004), utilized PRA in the reception of a Family Health Basic Unit.In both studies, the PRA showed validity as an instrument to identify and classify older adults into risk strata for hospitalization.Nonetheless, these studies were crosssectional and therefore did not allow correlation of PRA risk strata for hospitalization to the outcome hospitalization.In addition, samples were restricted to older adults that sought care in specifi c health services.
d Instituto Brasileiro de Geografi a e Estatística.Resultados da amostra do Censo Demográfi co 2000 -Malha municipal digital do Brasil: situação em 2001.Rio de Janeiro: IBGE, 2004 [cited 2010 May 12].Available from: http://www.ibge.gov.br/cidadesat/topwindow.htm PRA validity for prediction of hospitalization among older adults in the United States can not be generalized to the Brazilian older population, due to characteristics of SUS and other country-specifi c factors, such as: availability of hospital sites, access to medical assistance and to basic health care and determinants of health service utilization. 20,21is study's objective was to evaluate PRA predictive validity for hospital admittance of older adults.

METHODS
A population-based prospective cohort study was performed in the municipality of Progresso, Southern Brazil, in 2005.Progresso has 6,256 residents, including 694 (11.1%) age 60 years or older.The human development index was 0.738.d The municipality relies on a low complexity hospital accredited by SUS.
The Municipal Health Secretary of Progresso, in partnership with the Medical and Health Sciences Post-Graduate Program of Pontifícia Universidade Católica do Rio Grande do Sul, developed a publicly available database on health status of older adults registered in the Basic Care Information System of the Family Health Strategy.The questionnaire, used to create the municipal database, has 20 questions, eight pertaining to the PRA and the others evaluating functioning and health.b Information on the PRA questionnaire, logistic equation and regression coeffi cients are described by Pacala et al 13 (1995).
Collection of primary data was performed in February and March of 2005, with non-institutionalized older people of both sexes, in rural and urban areas.Community Health Agents were trained to administer the research tool and conducted household visits.
In February of 2005, 551 people age 60 years or older were registered in the Basic Care Information System in Progresso municipality.Among these older people, 36 (6.5%) were excluded from analysis, 18 because the questionnaire was answered by or with assistance from the caretaker and 14 had a terminal or advancedstage illness.The sample contained 515 people, which corresponded to 74.2% of the elderly population in Progresso.
Descriptive statistics and a 95% confi dence interval were calculated to estimate average frequency of hospitalizations according to risk strata.
The association between the frequency distribution of nominal variables from the questionnaire and the outcome were analyzed by contingency tables.Chi-square test was used to analyze the association between qualitative variables.
Survival analysis was performed to determine the probability of older adults presenting the outcome hospitalization within six months, according to PRA risk strata.Outcome probability distribution was estimated by the Kaplan-Meier method with censoring.Survival functions were estimated separately for each group, and a log rank test was used to compare survival curves of different risk groups.
Receiver Operating Characteristics analysis was used to ascertain PRA validity for prediction of hospitalization within six months, by calculating sensitivity and specifi city of different cutoff points.
Data analysis was done with SPSS 11.5.
During the study period, the Municipal Health Secretary of Progresso and the Family Health teams did not have access to study data.This research project was approved by the Research Ethics Committee of São Lucas Hospital, Pontifi cal Catholic University of Rio Grande do Sul (Approval number: 05/02501 of 07/08/2005).

RESULTS
Of the total older adults studied (515), 289 were female, with average age 70.0 years (SD=7.16), and among the 226 males, average age was 69.9 years (SD=6.77).
In the high risk group, the frequency of hospital admittance was 6.5 times greater than low risk groups (Table 2).The outcome prevalence changes for PRA scores between 0.3 and 0.5.Using a value of 0.5 it was possible to identify 7% of the study population that had 33.3% hospital admittance prevalence (Table 3).
The ROC curve showed a predictive accuracy of 0.674 (95%CI: 0.588;0.760)(Figure 1).With a cutoff point of 0.4, the sensitivity and specifi city for prediction of hospitalization were 42.3% and 84.7% respectively, with 18.1% of older adults classifi ed in this stratum.
Survival curves (proportion of patients free of hospitalization) for the risk strata showed no statistical difference between the medium risk and medium-high risk strata (log rank test: p=0.618).The same phenomenon occurs when comparing medium-high risk to high risk (log rank test: p=0.076).However, statistically signifi cant differences were found between the other strata during study period (log rank test: p<0.01) (Figure 2).

DISCUSSION
PRA allowed identifi cation of 7% of older adults as high risk for hospitalization within six months, a similar result as found in United States (7%) and Europe (4% to 7%). 1,11,21Nonetheless, some study limitations should be mentioned.
The logistic regression model for prediction of hospitalization is based on data from the United States elderly population that participated in the original study. 3herefore, the present study has the same limitations, including the defi nition of elderly (age 70 years or older) and omission from the logistic equation of factors that could infl uence hospitalization, such as lifestyle, dementia and psychological conditions. 1,21e follow-up period was six months, while Boult et al 1 (1993) extended the follow-up for four years.Nonetheless, other studies evaluated PRA for shorter follow-up periods, such as Wagner et al 21 (one year), Pacala et al 13 (1995) and Vojta et al 20 (2001) (six months) and obtained a predictive performance similar to the original study. 1 Adjustment for confounding variables, such as exclusion of older adults that required caretaker assistance to answer the questionnaire, had cognitive defi cits or were gravely ill, can underestimate selection bias, prevalence of hospitalization and incapacity in the study population. 6 An estimated 25.8% (179) of older residents in Progresso did not participate in the study.It was not possible to determine any potential differences between the participant and non-participant groups.The possibility that the non-participant group had worse health conditions should be considered.
The group identifi ed as high risk experienced 6.5 times more hospitalizations than the low risk group.The four cutoff points for stratifi cation of low, medium, mediumhigh and high risk were not statistically signifi cant for identifying risk strata for hospitalization in six months.Nonetheless, the following strata were statistically signifi cant: low risk (PRA<0.30),medium risk (PRA=0.30-0.49)and high risk (PRA≥0.500). 1,11,19e observed predictive accuracy of the area under the ROC curve was 0.674, similar to results by Boult et al In our study, the PRA showed improved ability to identify elderly at risk of hospitalization within six months, when a cutoff value of 0.4 was used to defi ne the high risk population.
A group of 17 older adults (3.3%) was responsible for 60.2% of hospitalizations.These data reinforce the fact that the group of older adults is not homogeneous in regards to use of health services.The Medicare program in the United States found that 62% of hospital expenses with older adults are due to excessive demand generated by a relatively small subgroup (5%).In addition, these older people receive care only in the most advances stages of disease, and despite the excessive utilization of hospital resources, spending does not correspond to improved health status for this minority. 1,9,14,21e use of PRA would allow Family Health Teams to intervene with preventive actions and avoid hospitalization of older adults.b PRA would permit the Family Health team to defi ne appropriate actions and establish an intervention strategy with older adults to reduce hospitalizations, considering the amount of high risk individuals.Care can be hierarchized in accordance with risk of hospitalization, and prioritization in care for older adults can be established according to cutoff value selected.This decision will be based on the desired sensitivity and specifi city of PRA.The choice of a high cutoff value (PRA>0.4)would tend to increase the number of vulnerable individuals detected, with a lower proportion of misidentifi ed cases. 2,9,12,19is study identifi ed older adults at risk of hospitalization through household visits by Community Health Workers and concluded that PRA validity was similar to initial studies.Therefore, inclusion of PRA in the routine of Family Health Teams and in SUS information systems should be considered.
Note: Area under the ROC Curve = 0.

Figure 2 .
Figure 2. Survival curve of the population age 60 years and older for the outcome hospitalization within six months, by risk strata for hospitalization.Progresso, Southern Brazil, 2005.

Table 1 .
Distribution of the population age 60 years or older, by variables of the Probability of Repeated Hospital Admissions questionnaire.Progresso, Southern Brazil, 2005.
a Hospitalization during 12 months preceding household visit by Community Health Worker b Number of doctor visits during 12 months preceding household visit by Community Health Worker c Angina, myocardial infarction and vascular disease d Assistance from friend, family or neighbor, if necessary

Table 2 .
Hospitalization rates among the population age 60 years or older during six months by sex, according to risk strata.Progresso, Southern Brazil, 2005.

Table 3 .
Proportion of the population age 60 years and older, by score on the Probability of Repeated Hospital Admissions (PRA) Questionnaire.Progresso, Southern Brazil, 2005.