Open-access The accuracy of the fullPIERS model in predicting adverse maternal and perinatal outcomes: evidence from a tertiary care maternity unit

Abstract

Objective:  Low- and middle-income countries face significant challenges in managing women diagnosed with pre-eclampsia, from making the clinical decision about whether to deliver to transferring these women to healthy facilities where they can receive appropriate care. The aim of this study was to evaluate the performance and accuracy of the fullPIERS model in a referral Brazilian maternity hospital - to assess maternal and fetal morbidity and impatient mortality at birth admission.

Methods:  A cross-sectional study analyzed pregnant women with preeclampsia diagnosis, between 2014 and 2023. The full PIERS model was applied to a database retrospectively collected and its accuracy to predict maternal and perinatal outcomes during the hospital stay was determined through a receiver operating curve.

Results:  Analyzing 207 pregnant women with fullPIERS had an Area Under the Curve (AUC) for adverse maternal outcome discrimination of 0.672 (0.576-0.767 95% CI, p<0.001) and AUC 0.582, (0.504-0.6661 95% CI, p = 0.041) for maternal and perinatal outcomes. Nevertheless, the model had no discrimination utility to assess perinatal outcomes (AUC 0.561, 0.480-0.642 95% CI, p = 0.642).

Conclusion:  The fullPIERS model had limited performance in identifying women at increased risk of adverse outcomes birth admission and absent utility to assess perinatal outcomes. Future studies, combining different tools and validated in low- and middle-income countries should be carried out to improve maternal health.

Keywords
Pre-eclampsia; Maternal mortality; Infant, newborn; Pregnancy outcome; Developing countries

Introduction

Preeclampsia affects approximately 5-10% of pregnant women worldwide.(1) It is defined broadly as maternal hypertension after 20 weeks of gestation with signs or symptoms that indicate end-organ damage. Its prevalence may be influenced by many risk factors, such as previous history of preeclampsia, maternal age, obesity and the increase in the use of assisted reproduction techniques.(2)

Once preeclampsia is established, stroke, eclampsia, renal dysfunction, stillbirth, preterm delivery, and cerebral palsy are some of their maternal and fetal adverse consequences.(3) These harmful consequences are correlated to the disease onset (before or after 34 weeks of gestation) and severity, but they are also correlated to health care facilities. For instance, in a previous report, generated from 2012 to 2014, the United Kingdom had only two maternal deaths due to hypertensive disorders. During the same period, Brazil recorded 971 maternal deaths.(4) More than 90% of the maternal deaths related to hypertensive disorders occur in low- and middle-income countries (LMIC).(5) Clinical care, health service organization and research priorities are some of the interventions that can explain the difference between HIC (high-income countries) and LMIC regarding maternal morbidity and mortality.(4)

In this context, it would be helpful if maternal and fetal adverse outcomes could be predicted in time to improve the care of a pregnant woman diagnosed with preeclampsia. This includes avoiding extreme premature interruption of gestation or transferring the woman to a health service with better conditions for mother and baby. The fullPIERS (Preeclampsia Integrated Estimate of Risk) model was developed with the aim of identifying the risk of fatal or life-threatening complications in women with preeclampsia within 48 h of hospital admission for the disorder.(6) It is a free-of-charge risk calculator that includes gestational age data, maternal blood oxygen saturation, dyspnea or chest pain, hepatic transaminases, platelets and creatinine, which are routine laboratory parameters obtained from pregnant women with suspected preeclampsia. Although it is free of charge, its applicability worldwide, including low- and middle-income countries where it would be revolutionary, is modest. Therefore, our study aims to evaluate the performance and accuracy of the fullPIERS model in a hospital located in a scenario with shortage of resources and possibly to strengthen its utility in helping the reduction of maternal and fetal morbidity and mortality.

Methods

This is a cross-sectional study involving pregnant women with the diagnosis of preeclampsia admitted at the maternity ward between 2014 and 2023 at the Hospital of Clinics at the Federal University of Minas Gerais, Brazil. This referral university hospital is a tertiary reference for high-risk pregnancies in the public health network, with ICU units for women and/or their newborn, performing approximately 1,500 deliveries annually. Pregnant women were not directly assessed; data was collected from their electronic medical records. The inclusion criteria were based on a list of the ICD-10 codes included as a diagnostic hypothesis in the discharge records of pregnant women admitted to the hospital between 2014 and 2023. The list is available as (Supplementary Material – Chart 1S) Of the 867 pregnant women initially selected, 372 were not available for analysis (due to lack of important records in the medical record or complete lack of access to the medical record). Of the remaining 495 medical records of pregnant women with diagnosis of preeclampsia analyzed, 272 accounted for women who gave birth during this hospitalization and were therefore included in the study. The distribution of the analyzed cases can be observed in the flowchart in figure 1.

Figure 1
Flowchart of the analyzed pregnant women

In our study, we considered preeclampsia as the onset of hypertension after 20 weeks of gestation, followed by proteinuria. In the absence of it, the target organ dysfunction was considered to make the diagnosis according to The 2021 International Society for the Study of Hypertension in Pregnancy classification, diagnosis & management recommendations for international practice (Supplementary Material Chart 2S).(1)

Data was retrieved from electronic medical records. The variables included sociodemographic data, obstetric history, laboratory parameters, maternal and perinatal outcomes. The online fullPIERS calculator on the University of British Columbia website (https://pre-empt.obgyn.ubc.ca/home-page/past-projects/fullpiers/) was used to calculate the fullPIERS score for each case of pregnant women diagnosed with preeclampsia admitted to the study center. The parameters for fullPIERS were based on admission data. We tested the fullPIERS score to predict three composite adverse outcomes: maternal, perinatal, and conjoint maternal and perinatal outcomes. The choice to perform this approach was based on a previous publication from Guida et al.,(7) which made a study with a similar pregnant population.

The adverse maternal outcomes considered in our analyses are as follows: eclampsia, HELLP syndrome, placental abruption, maternal hemorrhage, pulmonary edema, stroke or other central nervous system bleedings, renal or hepatic failure, and maternal death. The perinatal adverse outcomes considered were: fifth-minute Apgar score lower than 5 and stillbirth (intrauterine death). This study takes into account the same timeframe for the assessment of maternal outcomes as the original fullPIERS study, i.e. within 48 hours and seven days of admission.(3,6)

Descriptive analysis of the data obtained was performed, expressing frequencies, medians, and interquartile intervals among the participants. ROC (receiver operating characteristics) curves were built to analyze the performance of fullPIERS to predict adverse outcomes. Values of p<0.05 were considered statistically significant and an area under the curve (AUC) >0.7 indicated good discrimination, 0.5< AUC ≤0.7 indicated poor discrimination, and ≤ 0.5 was considered non-informative.(8) The software used for analysis was SPSS Statistic version 26.

The Research Ethics Committee of the Federal University of Minas Gerais, Brazil (CEP-UFMG) approved the research project under the register number 6074180 (Certificado de Apresentação de Apreciação Ética: 67440323.8.0000.5149), according to the Resolution of the National Health Council 466/2012. Informed consent was waived due to the retrospective nature of the study.

Results

We analyzed 207 pregnant women with preeclampsia, with a median age of thirty years old (IQR 12 years). Table 1 shows sociodemographic data, obstetrical history and maternal and perinatal outcomes of the population studied. The median gestational age at the diagnosis of preeclampsia was 35 weeks (IQR 5.43 weeks), with most cases diagnosed before 34 weeks (40.1%). Maternal and perinatal outcomes were evaluated within 48 hours and seven days from admission. Cesarean section was the mode of delivery in 156 pregnant women, with a frequency of 75.4%. Among the pregnant women with preeclampsia, 68.8% presented signs of severity and 57.5% received magnesium sulfate (119 pregnant women). HELLP syndrome complicated 20,3% of preeclampsia cases and placental abruption 5.3%. Eclampsia happened in six cases, all of them in hospitalized women. Table 1 shows that 59 of 207 women had an adverse maternal outcome within 48 hours and 7 days from admission. Stillbirth was observed in 14 cases (7.7%).

Table 1
Clinical and obstetrical characteristics of women with preeclampsia and superimposed preeclampsia included in the study

Only 9 (4.33%) pregnant women had recorded information about chest pain or dyspnea. Table 2 shows the fullPIERS parameters assessed to determine the score.

Table 2
Assessment of clinical and laboratorial parameters of the fullPIERS score

The accuracy of the fullPIER to predict maternal outcomes showed poor discrimination (AUC = 0.672 (0.576-0.767 95% CI, p<0.001) (Figure 2). For a sensitivity of 65,4% and a specificity of 65,2%, the cut- off value of fullPIERS was 1.05. When analyzed to predict perinatal outcomes, the model had no discrimination utility to assess perinatal outcomes (AUC = 0.561, 0.480-0.642 95% CI, p = 0.642) (Figure 3). Taking maternal and perinatal outcomes together, a total of 97 adverse outcomes took place, performing 46.9% of the total of pregnant women. The accuracy of fullPIERS to predict those outcomes together demonstrated poor discrimination (AUC 0.582, 0.504-0.6661 95% CI, p = 0.041) (Figure 4).

Figure 2
FullPIER to predict maternal outcomes
Figure 3
Full PIER to predict perinatal outcomes
Figure 4
Full PIER to predict maternal and perinatal outcomes combined

Discussion

This study showed that the performance of the fullPIERS model to distinguish between pregnant women with potential severity from those with favorable outcomes in our population had low accuracy. Among women with preeclampsia, 68.1% had severe features and 20.3% had the diagnosis of HELLP syndrome. Nearly 16% of the women who were supposed to receive magnesium sulfate were not receiving it. Taken together, 46.9% of the pregnant women with preeclampsia analyzed presented a maternal or a perinatal adverse outcome.

Originally, the fullPIERS model was developed and internally validated in a cohort study involving high-income countries: Canada, New Zealand, Australia, and the United Kingdom. The original PIERS study by von Dadelszen et al.(9) demonstrated that the algorithm was able to predict adverse maternal outcomes within 48 h of admission, with an AUC of 0·88 (95%CI 0·84–0·92). Since then, the model has been validated in different settings, seen as a tool to maximize the temporal expectant management of pregnancy and a way to optimize pregnancy care. The performance varies according to those settings. Great predictive performance was shown in a study in the Netherlands, with an AUC of 0·97 (95% CI 0·94–0·99).(10) This cohort is similar to the cohort where the internal validation of the model fullPIERS occurred. On the other hand, studies conducted in some low- and middle-income countries showed modest performance of the fullPIERS model to predict perinatal outcomes. Ukah et al.(3) showed an AUC of 0.77 in a population of 5 LMIC (the population of the mini-PIERS validation): Brazil, Fiji, Uganda, South Africa and Pakistan.

Two Brazilian studies showed a better predictive ability of the fullPIERS score. The study by Guida et al.(7) showed an AUC of 0.845, despite being conducted in a hospital of similar complexity. Compared with the study by Almeida et al.,(11) the current study evaluated fullPIERS in a similar context but with different approaches: our study of 207 pregnant women showed limited utility of the model (AUC 0.672 for maternal outcomes), whereas Almeida's study of 325 women found an AUC of 0.72 for adverse maternal outcomes.(11)

We hypothesize that a possible reasoning for the fullPIERS model low prediction performance in our study (and perhaps in other ones in LMIC) is the presence of more comorbidities in our population, less availability of resources and differences in disease management, with less expectant management of preeclampsia. We supposed that: LMIC might be more likely to have shorter intervals between admission and delivery and more co intervention in pregnant women with preeclampsia. Thus, the natural history of preeclampsia was truncated by expedited delivery.

Our sample had a high rate of C-section of 75.4%. Considering the high rate of early-onset preeclampsia and HELLP syndrome in our population, probably, they would have contributed to this high rate of C-section delivery, in accordance with the institutional protocol. However, Brazil has demonstrated over the years an increment in C-section rates, with rates of 42.9% in national public healthcare and 87.9% within private healthcare, as demonstrated by Nakamura-Pereira et al.(12) Unfortunately, these numbers seem to follow a global trend.(13,14)

Preeclampsia with signs of severity was diagnosed in 141 participants (68.8%). The HELLP syndrome complicated 20.3% of pregnancies analyzed. Surprisingly, almost 16% of the women who were supposed to receive magnesium sulfate did not receive it. The magnesium sulfate is well established as the medication to halve the risk of eclampsia and maternal death, since the beginning of this century.(15) Hence, one could find it quite surprising that in a tertiary hospital one in five women who are supposed to receive magnesium sulfate did not receive it and is exposed to unnecessary risks. Fear of toxicity and misinformation are probably among the reasons for this discouraging reality. Our study showed 3% of eclampsia, which is the main cause of death related to hypertension in Brazil.(16) Six pregnant women had seizure crises in hospital, after admission and under the care of the health team. Given the underuse of magnesium sulfate, these six cases could have been avoided by the timely use of the medication designed to prevent eclampsia.

Considering maternal and perinatal adverse outcomes, 46.9% of the participants presented an unfavorable outcome. This number reinforces the severity of preeclampsia and the necessity to abandon concepts like "mild" preeclampsia, already proposed since 2014, given its great potential for complications.(17)

Dyspnea and chest pain were rarely mentioned in the medical records. Unfortunately, this data cannot be checked and a possible reason for the lack of information is the fact that this is not routinely inquired by the healthy team to the patients. These are important symptoms that should be incorporated at least to the first interview of pregnant women with preeclampsia diagnosis.

Recently, the interest in improving the fullPIERS model performance with maternal seric biomarkers, especially sFlt-1 (soluble FMS-like tyrosine kinase-1) and PlGF (placental growth factor) ratio regarding their property of adverse maternal prediction has increased.(18) In a study from Serbia involving preeclampsia cases with early-onset diagnosis, the accuracy (AUC) of sFlt-1/PlGF ratio was 0.853 and the accuracy of fullPIERS was 0.628.(18) However, the cost-effectiveness of this association in LMIC settings still needs to be studied. Even though the fullPIERS model applies information routinely available at presentation with preeclampsia in general, its applicability especially in primary care settings of LMIC still faces barriers. The inclusion of laboratory tests like sFlt-1 and PlGF (sometimes not performed in those settings) and external validation limitations make this tool underutilized.

More recently, the same group that developed the fullPIERS model proposed a machine learning model with the same purpose as the first, but developed and validated in high-, low- and middle-income countries. This time, instead of logistic regression, they developed and internally validated an 18-variable model for maternal risk stratification using a machine learning approach.(18) The population included 11 low-, middle- and high-income countries, including sub-Saharan Africa, South America, South Asia and Oceania, with a total of 8843 participants. The model identified almost 40% of women with preeclampsia for whom changes in care should be made: place of care, transfer of care, antenatal and postnatal monitoring, co-interventions and timed birth. The performance appears to be better than that demonstrated in the external validation of the fullPIERS model in lowand middle-income countries, with an AUC of 0.8, compared with 0.68 in the original study.(3) Of course, this needs to be tested in future studies.

Our study seems to have several limitations, and outcomes deserve caution in generalization. The main limitation of this study is the limited power due to the small sample size and a high proportion of missing cases of preeclampsia, in view of database changing in the local hospital, and the difficulty to access medical recorders. For this reason, a selection bias should be considered. Due to the small sample size, it wasn't possible to do a subgroup analysis of pregnant women with a gestational age of less than 34 weeks. This subgroup is actually the one of greatest interest, because after this gestational age, cases of preeclampsia with signs of severity benefit from bringing the birth forward.(19,20)

On the other hand, this study also has strengths. The conduction of this study in a low resourced setting is particularly useful, considering the high prevalence of morbidity and mortality attributable to preeclampsia in LMIC.

Conclusion

In our study, the performance of the fullPIERS model in predicting maternal and perinatal adverse outcomes was poor compared with other cohorts. We speculate that the heterogeneity of the population, the high proportion of morbidity and the practice of expedited delivery may have influenced our results. Future studies with larger sample sizes are needed to confirm our findings. We also encourage future studies using machine learning models with a higher number of variables and perhaps a powerful approach validated in LMIC, which may help in the management of preeclampsia.

Acknowledgements

We would like to thank the students, IT professionals, and the medical records archiving sector for all their help.

References

  • 1 Magee AL, Brown AM, Hall RD, Gupte S, Hennessy A, Karumanchi AS, et al. The 2021 International Society for the Study of Hypertension in Pregnancy classification, diagnosis & management recommendations for international practice. Pregnancy Hypertens. 2022;27:148-69. doi: 10.1016/j.preghy.2021.09.008
    » https://doi.org/10.1016/j.preghy.2021.09.008
  • 2 Poon CL, Shennan A, Hyett AJ, Kapur A, Hadar E, Divakar H, et al. The International Federation of Gynecology and Obstetrics (FIGO) initiative on pre‐eclampsia: a pragmatic guide for first-trimester screening and prevention. Int J Gynaecol Obstet. 2019;145 Suppl 1:1-33. doi: 10.1002/ijgo.12802
    » https://doi.org/10.1002/ijgo.12802
  • 3 Ukah UV, Payne B, Lee T, Magee AL, von Dadelszen P. External validation of the fullPIERS model for predicting adverse maternal outcomes in pregnancy hypertension in low- and middle-income countries. Hypertension. 2017;69(4):705-11. doi: 10.1161/HYPERTENSIONAHA.116.08706
    » https://doi.org/10.1161/HYPERTENSIONAHA.116.08706
  • 4 Costa ML, Cavalli RC, Korkes HA, Cunha Filho EV, Peraçoli JC. Diagnosis and management of preeclampsia: suggested guidance on the use of biomarkers. Rev Bras Ginecol Obstet. 2022;44(9):878-83. doi: 10.1055/s-0042-1744286
    » https://doi.org/10.1055/s-0042-1744286
  • 5 Wulf SK, Johns N, Lozano R. Non-fatal burden of maternal conditions: country-level results from the GBD 2010 Study. Lancet. 2013;381:S149.
  • 6 von Dadelszen P, Menzies JM, Payne B, Magee LA. Predicting adverse outcomes in women with severe pre-eclampsia. Semin Perinatol. 2009;33(3):152-7. doi: 10.1053/j.semperi.2009.02.009
    » https://doi.org/10.1053/j.semperi.2009.02.009
  • 7 Guida JP, Cralcev C, Santos JC, Marangoni-Junior M, Sanchez MP, Costa ML. Validation of the fullPIERS model for prediction of adverse outcomes in preeclampsia at a referral center. Pregnancy Hypertens. 2021;23:112-5. doi: 10.1016/j.preghy.2020.11.013
    » https://doi.org/10.1016/j.preghy.2020.11.013
  • 8 Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29-36. doi: 10.1148/radiology.143.1.7063747
    » https://doi.org/10.1148/radiology.143.1.7063747
  • 9 von Dadelszen P, Payne B, Li J, Ansermino JM, Pipkin FB, Côté AM, et al. Prediction of adverse maternal outcomes in pre-eclampsia: development and validation of the fullPIERS model. Lancet. 2011;377(9761):219-27. doi: 10.1016/S0140-6736(10)61351-7
    » https://doi.org/10.1016/S0140-6736(10)61351-7
  • 10 Akkermans J, Payne B, von Dadelszen P, Groen H, Vries J, Magee LA, et al. Predicting complications in pre-eclampsia: external validation of the fullPIERS model using the PETRA trial dataset. Eur J Obstet Gynecol Reprod Biol. 2014;179:58-62. doi: 10.1016/j.ejogrb.2014.05.021
    » https://doi.org/10.1016/j.ejogrb.2014.05.021
  • 11 Almeida ST, Katz L, Coutinho I, Amorim MM. Validation of fullPIERS model for prediction of adverse outcomes among women with severe pre-eclampsia. Int J Gynaecol Obstet. 2017;138(2):142-7. doi: 10.1002/ijgo.12197
    » https://doi.org/10.1002/ijgo.12197
  • 12 Nakamura-Pereira M, Leal MC, Esteves-Pereira AP, Domingues RM, Torres JA, Dias MA, et al. Use of Robson classification to assess cesarean section rate in Brazil: the role of source of payment for childbirth. Reprod Health. 2016;13 Suppl 3:128. doi: 10.1186/s12978-016-0228-7
    » https://doi.org/10.1186/s12978-016-0228-7
  • 13 Boerma T, Requejo J, Victora GC, Amouzou A, George A, Agyepong I, et al. Countdown to 2030: tracking progress towards universal coverage for reproductive, maternal, newborn, and child health. Lancet. 2018;391(10129):1538-48. doi: 10.1016/S0140-6736(18)30104-1
    » https://doi.org/10.1016/S0140-6736(18)30104-1
  • 14 Betrán PA, Ye J, Moller AB, Zhang J, Gülmezoglu AM, Torloni MR, The increasing trend in caesarean section rates: global, regional and national estimates: 1990-2014. PLoS One. 2016;11(2):e0148343.
  • 15 Altman D, Carroli G, Duley L, Farrell B, Moodley J, Neilson J, et al. Do women with pre-eclampsia, and their babies, benefit from magnesium sulphate? The Magpie Trial: a randomised placebo-controlled trial. Lancet. 2002;359(9321):1877-90. doi: 10.1016/s0140-6736(02)08778-0
    » https://doi.org/10.1016/s0140-6736(02)08778-0
  • 16 Giordano JC, Parpinelli MA, Cecatti JG, Haddad SM, Costa ML, Surita FG, et al. The burden of eclampsia: results from a multicenter study on surveillance of severe maternal morbidity in Brazil. PLoS One. 2014;9(5):e97401. doi: 10.1371/journal.pone.0097401
    » https://doi.org/10.1371/journal.pone.0097401
  • 17 Tranquilli AL, Dekker G, Magee L, Roberts J, Sibai BM, Steyn W, et al. The classification, diagnosis and management of the hypertensive disorders of pregnancy: a revised statement from the ISSHP. Pregnancy Hypertens. 2014;4(2):97-104. doi: 10.1016/j.preghy.2014.02.001
    » https://doi.org/10.1016/j.preghy.2014.02.001
  • 18 Mirkovic L, Tulic I, Stankovic S, Soldatovic I. Prediction of adverse maternal outcomes of early severe preeclampsia. Pregnancy Hypertens. 2020;22:144-50. doi: 10.1016/j.preghy.2020.09.009
    » https://doi.org/10.1016/j.preghy.2020.09.009
  • 19 Montgomery-Csobán T, Kavanagh K, Murray P, Robertson C, Barry SJ, Ukah UV, et al. Machine learning-enabled maternal risk assessment for women with pre-eclampsia (the PIERS-ML model): a modelling study. Lancet Digital Health. 2024;6(4):e238-e50. doi: 10.1016/S2589-7500(23)00267-4
    » https://doi.org/10.1016/S2589-7500(23)00267-4
  • 20 van der Tuuk K, Tajik P, Koopmans CM, van den Berg PP, Mol BW, van Pampus MG, et al. Blood pressure patterns in women with gestational hypertension or mild preeclampsia at term. Eur J Obstet Gynecol Reprod Biol. 2017;210:360-5. doi: 10.1016/j.ejogrb.2017.01.021
    » https://doi.org/10.1016/j.ejogrb.2017.01.021

Supplementary Material 

Chart 1S
List of CIDs included in the survey
Chart 2S
Diagnostic criteria for preeclampsia

Edited by

Publication Dates

  • Publication in this collection
    04 Aug 2025
  • Date of issue
    2025

History

  • Received
    18 Dec 2024
  • Accepted
    06 Mar 2025
location_on
Federação Brasileira das Sociedades de Ginecologia e Obstetrícia Av. Brigadeiro Luís Antônio, 3421, sala 903 - Jardim Paulista, 01401-001 São Paulo SP - Brasil, Tel. (55 11) 5573-4919 - Rio de Janeiro - RJ - Brazil
E-mail: editorial.office@febrasgo.org.br
rss_feed Acompanhe os números deste periódico no seu leitor de RSS
Reportar erro