ABSTRACT
Objective: Implementing the Checklist for Early Recognition and Treatment of Acute Illness and Injury (CERTAIN) decision support tool for structured intensive care unit admission and rounding was associated with an increased adherence to best care practices. We determined whether this association was patient sex-dependent.
Design: Post hoc analysis of CERTAIN.
Setting: prospective multinational quality improvement study.
Patients: Adult patients admitted to one of the participating intensive care units.
Interventions: Implementation of the CERTAIN decision support tool.
Measurements: We compared incidence rates of omission of delivery of ten best care practices, including deep vein thrombosis and peptic ulcer prophylaxis, head of bed elevation, daily oral care, spontaneous breathing trials, family conferences, assessment of need for central lines and urinary catheters, and prescription of antimicrobials and sedation, between sexes, before and after implementation of the decision support tool. In addition, we determined whether sex differences existed amongst high-and middle-income countries.
Main results: CERTAIN comprised a total of 4,256 patients, with 588 females and 859 males before the implementation of the decision support tool and 1,169 females and 1,640 males after its implementation. Overall, there was no notable difference in care between sexes, neither before nor after implementation, and both sexes in high-income and middle-income countries experienced equal benefits from checklist implementation.
Conclusion: The impact of a clinical decision support tool for structured intensive care unit admission and rounding on adherence to best care practices showed minimal variation between sexes.
Keywords:
Critical care; Sex; Inequity; Quality improvement global health; Middle-income countries; Intensive care units
INTRODUCTION
Numerous studies have highlighted the presence of sex inequity in critical care provision, with significant differences observed in the utilization of various interventions. Specifically, females tend to receive interventions such as deep venous thrombosis prophylaxis,(1) vasoactive medication,(2-4) fluid administration, transfusion and renal replacement therapy,(1-3,5-7) respiratory support,(1-5,8-10) neuromonitoring,(2) catheter use(1,2,5,8) and antibiotic therapy(7,11) less frequently than males. There is also a higher incidence of care limitations among females.(1,12,13)
A conveniently-sized multicontinental study, named Checklist for Early Recognition and Treatment of Acute Illness and Injury (CERTAIN), showed that implementation of a clinical decision support tool for structured intensive care unit (ICU) admission and rounding was associated with an increased adherence to important best care processes.(14) In a subgroup analysis, these improvements were notably more substantial in middle-income countries (MIC) than in high-income countries (HIC).
It is uncertain whether the implementation of CERTAIN had comparable effects on females and males. Therefore, we reanalyzed the CERTAIN database to test the hypothesis that the effects of implementation would be different between sexes. In a subgroup analysis, we also assessed the impact of implementation on females and males in HIC and MIC.
METHODS
Institutional Review Board statement
The Institutional Review Board of the Mayo Clinic, Rochester, Minnesota, United States (12-007998) approved the study protocol, and thereafter in all participating centers. Procedures were followed according to the ethical standards of the responsible committee on human experimentation (institutional or regional) and with the Helsinki Declaration of 1964 and its subsequent amendments or equivalent ethical standards.
Design, setting, and participants
Post hoc analysis of CERTAIN, a pragmatic, prospective, exploratory, multicontinental, international, multicenter, before-after quality-improvement study in 34 ICUs in 5 HIC and 10 MIC, conducted between November 2013 and December 2017.(14) The study was registered at clinicaltrials.gov (identifier NCT01973829). Informed consent was obtained from all individual participants included in the study.
Patients were eligible if admitted to one of the participating ICUs and ≥ 18 years of age. Patients admitted only for monitoring, patients with a planned admission for routine postoperative surveillance for less than 24 hours after uncomplicated surgery, readmitted patients, and patients transferred from an ICU outside a study hospital were excluded. Patients without a recorded admission date or an incorrectly recorded date did not participate. No additional exclusion criteria were used for this post hoc analysis.
Checklist
CERTAIN used a web-based or paper decision support tool displaying relevant clinical information, including the evidence-based checklist.(14) The core of the intervention in CERTAIN was a structured approach to admission and daily rounding by using the checklist to prompt clinicians to follow best care practices.
Data collected
Data was collected for 3 months before and 6 months after implementing the decision support tool and included ICU characteristics, patient demographics, baseline characteristics, comorbidities, and limitations on life support interventions. The Sequential Organ Failure Assessment (SOFA) score was collected for each patient. Daily care processes were recorded on the day of ICU admission (calendar day 0) and on calendar days 1, 2, 3, 7, 14, and 21 for as long as the patient remained in the ICU and included ten best care practices: Did the patient receive (1) deep vein thrombosis and (2) peptic ulcer prophylaxis?; Was there (3) head of bed elevation?; Is the (4) daily oral care performed?; Is there a documented assessment of (5) a spontaneous breathing trial?; Is there a documented (6) family conference or discussion?; Was there an assessment of the need for (7) central lines and (8) urinary catheters?; Was there a documented assessment to continue or discontinue the current prescription of (9) antimicrobials and current (10) sedation medication? Adherence to these daily care processes was recorded for each observation day as either "Yes" or "No". Patients were followed to hospital discharge to assess death in the ICU or hospital and duration of stay in ICU and hospital.
Definitions
Non-adherence to daily care processes was quantified as an incidence rate, defined as the ratio of the number of observations of not receiving basic care procedures among eligible patients (events) to the number of total observations in which the specific intervention was indicated (exposure) expressed per 1,000 days of specific intervention. Since we recorded biological sex rather than gender identity, we use the term "sex" to align with the available documentation and data structure.
Endpoints
We used the same endpoints as in the primary analysis of CERTAIN, i.e., the omission of delivery of the abovementioned best care practices; ICU, hospital, and mortality, and ICU and hospital length of stay served as secondary endpoints.
Sample-size calculation
No sample-size calculation was performed for this current analysis. The sample size was based on the number of patients available in the database.
Statistical analysis plan
Descriptive statistics were used to report differences between groups. Continuous distributed variables are expressed as medians and their interquartile ranges, and categorical variables are expressed as frequencies and proportions. Fisher's exact tests were used for categorical variables, and the Wilcoxon rank-sum test was used for continuous variables.
The cohort was divided into two groups: females and males. For all analyses, males were used as the reference group.
Nonadherence (the "No" outcome) was chosen as the endpoint. For nonadherence to the ten best care practices, we ignored day 0, which was the day of ICU admission before and after implementation of the decision support tool. Sexes were compared concerning the primary endpoint using Poisson regression. Then, to adjust for a center effect, the incidence rate ratio was calculated with the center modeled as a random effect in a generalized linear mixed model.
Logistic regression was used to compare mortality rates, and linear regression using geometric means was used to evaluate hospital and ICU length of stay, both before and after the checklist's implementation. A generalized linear mixed-effects model with the center modeled as a random effect and mechanical ventilation, life support limitation, and comorbidity as fixed effects was used.
The analyses were repeated in the subgroups from HIC and MIC. Country income status was defined by the World Bank 4. High-income countries included Croatia, Ireland, Poland, Saudi Arabia, and the United States. Middle-income countries included Bosnia and Herzegovina, China, India, Lebanon, Mexico, Pakistan, Philippines, Serbia, Tanzania, and Turkey.
All analyses were conducted in R version 4.0.3 (R Foundation for Statistical Computing (https://www.R-project.org/), Vienna, Austria); statistical significance was set at p value < 0.05. Details are provided in the Supplementary Material.
RESULTS
Patients
The analysis included 4,256 patients, 588 and 1,169 females and 859 and 1,640 males before and after implementing the decision support tool, respectively (Figure 1S - Supplementary Material). Females were older and had a lower median body weight but were more often obese (Table 1). Females more often had a history of hypothyroidism, rheumatoid arthritis, and collagen vascular disease; males more often had a history of alcohol abuse and liver disease. Compared to patients in HIC, patients in MIC were younger and shorter and had a lower median SOFA score (Tables 1SA and 1SB - Supplementary Material). Patients in HIC more often had a history of alcohol abuse.
Nonadherence to best care practices
Prior to implementation, nonadherence to best care practices did not vary significantly between sexes, except for peptic ulcer prophylaxis and daily oral care, which were more often omitted in females (Figure 1 and Table 2SA [Supplementary Material]). After implementation, head of bed elevation was more often omitted in female patients (Figure 1 and Table 2SB [Supplementary Material]).
The incidence rates of omissions in daily care processes between the sexes - middle-income countries + high-income countries. Checklist for Early Recognition and Treatment of Acute Illness and Injury (CERTAIN). An event refers to a non-adherence to an evidence-based Daily Care Process. Adjusted for center effect incident rate ratios. In this figure, the blue whisker compares the incident rates of events between female versus male patients before the checklist, and the red line compares the incident rates of events after the checklist.
The effects of implementing the checklist were not different between sexes (Figure 2). After implementation, there were no differences observed in ICU, hospital, mortality, and hospital length of stay between sexes (Tables 2 and 3). However, the ICU length of stay was shorter in female patients (Table 3).
The incidence rates of omissions in daily care processes before versus after the implementation. Checklist for Early Recognition and Treatment of Acute Illness and Injury (CERTAIN). Adjusted for center effect incident rate ratios. In this figure the blue whisker compares the incident rates of events between before versus after the checklist in male patients and the red line compares the incident rates of events in female patients.
High-income versus middle-income countries
The implementation of the checklist had consistent effects across the geo-economic settings (Figures 2S to 5S and Tables 3S to 11S - Supplementary Material).
DISCUSSION
The findings of this post hoc analysis of CERTAIN can best be summarized as follows: before implementation of a clinical decision support tool for structured ICU admission and rounding, nonadherence to ten best care practices showed minimal variation between sexes; and females and males experienced equal benefits from checklist implementation. In addition, the checklist implementation yielded consistent outcomes across various geo-economic settings.
Our analysis has strengths. CERTAIN was a worldwide study with patients originating from ten MIC and five HICs across five continents, university-affiliated hospitals, and teaching and non-teaching centers, increasing the generalizability of our findings. The original study was large and robust and used broadly accepted evidence-based checklists, algorithms, and educational modules. The dataset was rich, with a negligible amount of missing data. The large number of patients allowed us to perform several analyses of MIC and HIC, including a subgroup analysis. An analysis plan, in place before performing the analysis, was strictly followed.
Our analysis did not reveal significant differences in care between the sexes except for peptic ulcer prophylaxis and daily oral care, which were more frequently omitted in females before implementing clinical decision support tools for structured ICU admission and rounding. This finding is in contrast to the findings of previous investigations.(1-13) These prior studies showed important differences in care between the sexes. There is limited evidence suggesting a lower risk of peptic ulcers in women, particularly at a younger age,(15) which may explain the more frequent omission of peptic ulcer prophylaxis in female patients. This difference, though, disappeared after the implementation of the checklist. Studies looking into sex differences in the omission of daily best care practices that were part of the tool used in this study are lacking. One study showed no significant association between the omission of early (< 24 hours) thromboprophylaxis with the patient's sex,(16) and another study showed no sex difference in time for the head of the bed elevation.(17) It is unclear why we did not find differences in best care practices between sexes. One possibility is that we are dealing with observer bias: even before the implementation of the checklist, i.e., in the first period of the study, there may have already been awareness to apply care as effectively as possible. However, we cannot rule out the possibility that overall care has improved, for example, through implementation processes prior to our study.
After implementing the clinical decision support tool for structured ICU admission and rounding, differences in peptic ulcer prophylaxis and daily oral care between the sexes had disappeared, with the head of the bed elevation emerging as a differentiating factor. However, it should be noted that the confidence intervals for these three practices were relatively broad, indicating a degree of imprecision in the data. Additionally, the observed difference in head of bed elevation was minimal, likely not significantly affecting outcomes.
After implementation of the checklist, female patients had a statistically significant shorter ICU length of stay compared to male patients. Several factors may explain this observation. One possibility is that the same quality improvement would benefit female patients more. Another possibility is that sex-specific differences in care delivery influenced ICU length of stay. The fact that this difference emerged only after checklist implementation suggests that the standardized decision-support tool may have improved basic ICU care, potentially benefiting female patients more. This could be due to bias mitigation, as the tool may have reduced unconscious biases in clinical decision-making, leading to more consistent and appropriate care for female patients. However, it remains possible that the statistically significant shorter ICU length of stay occurred by chance, and further research is needed to confirm whether this difference is reproducible.
In the HIC and MIC subgroup analysis, we found no sex differences in best-care practices and outcome measures. However, only in MIC did the implementation of the checklist lead to a decrease in outcome measures for both sexes, including mortality and ICU- and hospital length of stay. This suggests that while implementing the tool may have less potential to improve outcomes in HIC, it holds strong potential to benefit patients in countries with room for improvement in care. Possible reasons why the checklist did not result in decreased outcome measures in HIC could be attributed to the fact that these countries have more resources for quality improvement interventions and an established organizational safety culture. It is also plausible that baseline adherence to processes of care was higher in HIC or that fewer centers and patients were enrolled from these countries. Nonetheless, the results indicate that a systematic checklist is feasible in resource-limited settings.
Future research should explore sex differences in treatment. We need to expand research in precision medicine for both sexes in the ICU since addressing these differences is challenging given the heterogeneity of patients, including hormonal status and physiology, socially constructed preferences, care limitations, and diagnoses.
Our analysis has limitations. CERTAIN was a before-after observational study, which somewhat limits the ability to establish causality. Awareness of the study may have affected the impact of the intervention. The selection of ICUs relied on personal contacts, potentially introducing selection bias. Only hospitals willing to participate and motivated to enhance the quality of care and those with the necessary time and resources completed the study and were included in the analysis.(14) Our analysis is susceptible to documentation bias, as adherence was determined from recorded compliance, which may not fully capture actual clinical practice. Best practices may have been followed but not consistently documented, potentially leading to underestimating adherence rates. Our data were collected between 2013 and 2017, which some may consider outdated. Clinical practices and protocols may have evolved. However, the fundamental principles of ICU care remain unchanged, and the key aspects of the studied interventions remain relevant today. Moreover, our primary research question focused on whether the checklist had comparable effects in female and male patients. Consistent with findings from other studies in various ICU cohorts, fewer female patients were admitted to the participating ICUs. Given our sample size, we do not anticipate this affecting the overall statistical power or the generalizability. While we adjusted for ventilation, life support limitations, and comorbidities, residual confounding may still be present. Not all relevant daily practices were collected and thus studied, including important bundles for catheter insertions, nutrition, or decubitus prevention. Furthermore, it is important to note that we had no data on pregnancy status. Staffing levels and logistical constraints were also not assessed daily, which may have influenced clinical practice. These factors should be considered when interpreting the findings. In our analysis, we have accounted for a center effect, which should have mitigated the effects of variations in resource availability and institutional practices. The findings may not be applicable to patients transferred from ICUs outside the participating hospital, as they were excluded from this study.(14)
Unfortunately, this study did not adequately capture cultural, traditional, and contextual factors. These elements may significantly influence healthcare practices, vary across geographic regions, and contribute to differences in care between sexes. More research is needed to understand their impact and ensure more comprehensive and equitable healthcare strategies.
CONCLUSION
The impact of a clinical decision support tool for structured intensive care unit admission and rounding on adherence to ten best care practices showed only minor variations between females and males, and this observation remained consistent across intensive care units in various geo-economic areas.
List of collaborators
REFERENCES
- 1 Pietropaoli AP, Glance LG, Oakes D, Fisher SG. Gender differences in mortality in patients with severe sepsis or septic shock. Gend Med. 2010;7(5):422-37.
- 2 Valentin A, Jordan B, Lang T, Hiesmayr M, Metnitz PG. Gender-related differences in intensive care: a multiple-center cohort study of therapeutic interventions and outcome in critically ill patients. Crit Care Med. 2003;31(7):1901-7.
- 3 Zettersten E, Jäderling G, Bell M, Larsson E. A cohort study investigating the occurrence of differences in care provided to men and women in an intensive care unit. Sci Rep. 2021;11(1):23396.
- 4 Modra LJ, Higgins AM, Pilcher DV, Bailey M, Bellomo R. Sex differences in vital organ support provided to ICU patients. Crit Care Med. 2024;52(1):1-10.
- 5 Blecha S, Zeman F, Specht S, Lydia Pfefferle A, Placek S, Karagiannidis C, et al. Invasiveness of treatment is gender dependent in intensive care: results from a retrospective analysis of 26,711 cases. Anesth Analg. 2021;132(6):1677-83.
- 6 Mikkelsen ME, Gaieski DF, Goyal M, Miltiades AN, Munson JC, Pines JM, et al. Factors associated with nonadherence to early goal-directed therapy in the ED. Chest. 2010;138(3):551-8.
- 7 Sunden-Cullberg J, Nilsson A, Inghammar M. Sex-based differences in ED management of critically ill patients with sepsis: a nationwide cohort study. Intensive Care Med. 2020;46(4):727-36.
- 8 Fowler RA, Sabur N, Li P, Juurlink DN, Pinto R, Hladunewich MA, et al. Sex-and age-based differences in the delivery and outcomes of critical care. CMAJ. 2007;177(12):1513-9.
- 9 Fernandez-Bustamante A, Wood CL, Tran ZV, Moine P. Intraoperative ventilation: incidence and risk factors for receiving large tidal volumes during general anesthesia. BMC Anesthesiol. 2011;11(1):22.
- 10 Chapa J, Nonas S. Differences in lung protective ventilation use in female vs male organ donors. Chest. 2018;154(4):1101A.
- 11 Madsen TE, Napoli AM. The DISPARITY-II study: delays to antibiotic administration in women with severe sepsis or septic shock. Acad Emerg Med. 2014;21(12):1499-502.
- 12 Purcell LN, Tignanelli CJ, Maine R, Charles A. Predictors of change in code status from time of admission to death in critically ill surgical patients. Am Surg. 2020;86(3):237-44.
- 13 McPherson K, Carlos WG 3rd, Emmett TW, Slaven JE, Torke AM. Limitation of life-sustaining care in the critically ill: a systematic review of the literature. J Hosp Med. 2019;14(5):303-10.
- 14 Vukoja M, Dong Y, Adhikari NK, Schultz MJ, Arabi YM, Martin-Loeches I, et al.; Checklist for Early Recognition and Treatment of Acute Illness and Injury (CERTAIN) Investigators of the SCCM Discovery Network. Checklist for Early recognition and treatment of acute illness and injury: an exploratory multicenter international quality-improvement study in the ICUs with variable resources. Crit Care Med. 2021;49(6):e598-612.
- 15 Wysocki A, Budzyński P, Kulawik J, Drożdż W. Changes in the localization of perforated peptic ulcer and its relation to gender and age of the patients throughout the last 45 years. World J Surg. 2011;35(4):811-6.
- 16 Sahle BW, Pilcher D, Peter K, McFadyen JD, Bucknall T. Trends and risk factors for omission of early thromboprophylaxis in Australian and New Zealand ICUs between 2009 and 2020. Intensive Care Med. 2022;48(5):590-8.
- 17 Martí-Hereu L, Arreciado Marañón A. Time of elevation of head of bed for patients receiving mechanical ventilation and its related factors. Enferm Intensiva. 2017;28(4):169-77.
Edited by
-
Responsible editor:
Pedro Povoa https://orcid.org/0000-0002-7069-7304
Publication Dates
-
Publication in this collection
24 Oct 2025 -
Date of issue
2025
History
-
Received
13 Jan 2025 -
Accepted
10 Apr 2025




