Prognostic studies for health care decision making

PRACTICAL SCENARIO

A multicenter retrospective cohort study was conducted to develop and validate a prognostic model to predict 1-year mortality among adult patients receiving at least 14 uninterrupted days of mechanical ventilation. Likely prognostic variables were chosen, a priori, based on published literature and clinical judgment (10 variables). During the development phase of the study, the prognostic variables were included in a logistic regression model to evaluate how well each variable predicted 1-year mortality by calculating discrimination (ability to correctly classify patients into those who did and did not die) using ROC curves and area under the curve (AUC). The authors found that 5 of the 10 variables maximized the prognostic capability of the model for 1-year mortality (age, platelet count, vasopressor use, hemodialysis, no trauma diagnosis) showing very good discrimination (AUC = 0.80; 95% CI: 0.76-0.83). For the validation phase, the authors used the β-coefficient values estimated for each variable in the development cohort logistic regression model to predict 1-year mortality in a new cohort of patients and showed that discrimination was also very good (AUC = 0.78; 95% CI: 0.72-0.83), thus showing that the 5-variable model was valid. Then the authors created a clinical prediction rule, a point system used to easily calculate the probability of 1-year mortality for each patient, based on the strength of association of each variable (β-coefficient) with mortality in the development model. All β-coefficients were assigned 1 point except for the category “age ≥ 65 years,” which was assigned 2 points. Lastly, the authors validated this point system by showing that, as the number of points increased, the probability of 1-year mortality increased.¹

WHY PROGNOSTIC STUDIES ARE USEFUL

The overall goal of prognostic research for clinical settings is to help clinicians, patients, and families make informed health-care decisions based on information available on each patient in the present to predict outcomes in the future. In our example, identifying patients at high risk of dying within 1 year justifies clinicians’ recommendation for closer outpatient-monitoring after discharge. Additionally, it helps patients and family think about appropriate end-of-life decisions for those at very high risk of dying, as well as to identify individualized interventions to prevent future hospitalizations due to respiratory failure.

HOW TO DEVELOP A CLINICAL PREDICTION RULE

The process involves designing a retrospective or prospective cohort study that measures prognostic variables among participants at study baseline (entry), that follows them during a pre-specified time and that assesses whether they develop the outcome or not. Using data from a subset of the participants, called the development cohort, a logistic regression model with the outcome (in our example, 1-year mortality) as the dependent variable and plausible predictive variables as independent variables is built and the AUC is calculated (Figure 1). In a second step, the mathematical equation (β-coefficients) from the development model is tested in another subgroup of similar patients, called the validation cohort. The clinical prediction rule is built by assigning points to each predictive variable based on their strength of association with the outcome.²

Figure 1
The ROC curve is used to quantify model discrimination by plotting the true positive rate (sensitivity) against the false positive rate (1 − specificity) for different possible cut-off values of a prognostic model. The greater the area under the curve (AUC), the better the model discriminates the subjects with the outcome from those without it. This figure was created with fictitious data.

REFERENCES

• ¹
  Hough CL, Caldwell ES, Cox CE, Douglas IS, Kahn JM, White DB, et al. Development and validation of a mortality prediction model for patients receiving 14 days of mechanical ventilation. Crit Care Med. 2015;43(11):2339-45. https://doi.org/10.1097/CCM.0000000000001205
  » https://doi.org/10.1097/CCM.0000000000001205
• ²
  Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, et al. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013;10(2):e1001381. https://doi.org/10.1371/journal.pmed.1001381
  » https://doi.org/10.1371/journal.pmed.1001381

Publication Dates

• Publication in this collection
  Jul-Aug 2017