Acessibilidade / Reportar erro

C-reactive protein and neutrophil–lymphocyte ratio as predictors of mortality in coronavirus disease 2019

SUMMARY

OBJECTIVE:

This study investigates whether C-reactive protein, platelet–lymphocyte ratio, and neutrophil–lymphocyte ratio could be useful to predict mortality in COVID-19.

METHODS:

Data of 635 patients with COVID-19 followed up in Sinop Ataturk State Hospital from February to May 2020 were evaluated retrospectively. Diagnosis of COVID-19 was made according to the interim guidance of the World Health Organization. Patients were grouped into two groups based on mortality as survived and non-survived patients. Age, gender, neutrophil–lymphocyte ratio, platelet–lymphocyte ratio, and C-reactive protein of the groups were investigated and compared.

RESULTS:

The mean age of the participants was 55.8±22.3 years. Among the patients, 584 survived and 51 patients died. Age was significantly different between the groups, 54.2±22.3 in the survived group and 75.6±11.1 in the dead group (p=0.000). In addition, neutrophil, C-reactive protein, and neutrophil–lymphocyte ratio values were significantly higher in the dead group (p=0.000). platelet–lymphocyte ratio was slightly higher in the dead group, but this difference was not significant (p=0.42). The area under the curve values for age, lymphocyte, platelet, C-reactive protein, and neutrophil–lymphocyte ratio are 0.797, 0.424, 0.485, 0.778, and 0.729, respectively.

CONCLUSIONS:

Our results showed that neutrophil–lymphocyte ratio and C-reactive protein are significantly higher in patients leading to death and could be effective biomarkers in predicting COVID-19 fatality. Furthermore, C-reactive protein could be used as an independent biomarker to predict death in patients with COVID-19, regardless of gender and age (p=0.000).

KEYWORDS:
COVID-19; C-Reactive Protein; Neutrophil; Lymphocyte; Platelet; Mortality

INTRODUCTION

The first case of the spread of abnormal pneumonia was observed on December 29, 2010, in Wuhan, China, and the first case of which was discovered on December 12 in the same year11 Abrishami A, Khalili N, Dalili N, Khaleghnejad Tabari R, Farjad R, Samavat S, et al. Clinical and radiologic characteristics of COVID-19 in patients with CKD. Iran J Kidney Dis. 2020;14(4):267-77. PMID: 32655021. Later, an abnormal outbreak was reported to the World Health Organization (WHO) on December 31. After various speculations about the origin of the disease, China CDC has introduced a new coronavirus called 2019-novel coronavirus disease (nCoV-2019) or COVID-1922 Cao X. COVID-19: immunopathology and its implications for therapy. Nat Rev Immunol. 2020;20(5):269-70. https://doi.org/10.1038/s41577-020-0308-3
https://doi.org/10.1038/s41577-020-0308-...
. The first nCoV-2019 genomic sequence went online one day after Zhang et al. approved it at Fudan University in Shanghai33 Asghar MS, Kazmi SJH, Khan NA, Akram M, Hassan M, Rasheed U, et al. Poor prognostic biochemical markers predicting fatalities caused by COVID-19: a retrospective observational study from a developing country. Cureus. 2020;12(8):e9575. https://doi.org/10.7759/cureus.9575
https://doi.org/10.7759/cureus.9575...
. Isolation and successful genomic sequencing of COVID-19 have helped understand the virus's origin and its infectious properties44 Gemelli Against COVID-19 Post-Acute Care Study Group. Post-COVID-19 global health strategies: the need for an interdisciplinary approach. Aging Clin Exp Res. 2020;32(8):1613-20. https://doi.org/10.1007/s40520-020-01616-x
https://doi.org/10.1007/s40520-020-01616...
. The new coronavirus outbreak has been declared a public health emergency worldwide, posing a threat to China and all countries55 Fauci AS, Lane HC, Redfield RR. Covid-19 – navigating the uncharted. N Engl J Med. 2020;382(13):1268-9. https://doi.org/10.1056/NEJMe2002387
https://doi.org/10.1056/NEJMe2002387...
. However, many ambiguities remain, and scientists are conducting extensive research on this new virus.

If the disease progresses, it will cause the immune system to overreact44 Gemelli Against COVID-19 Post-Acute Care Study Group. Post-COVID-19 global health strategies: the need for an interdisciplinary approach. Aging Clin Exp Res. 2020;32(8):1613-20. https://doi.org/10.1007/s40520-020-01616-x
https://doi.org/10.1007/s40520-020-01616...
,66 Chen L, Yu J, He W, Chen L, Yuan G, Dong F, et al. Risk factors for death in 1859 subjects with COVID-19. Leukemia. 2020;34(8):2173-83. https://doi.org/10.1038/s41375-020-0911-0
https://doi.org/10.1038/s41375-020-0911-...
. The chemical signals of cytokines cause inflammation that must be regulated22 Cao X. COVID-19: immunopathology and its implications for therapy. Nat Rev Immunol. 2020;20(5):269-70. https://doi.org/10.1038/s41577-020-0308-3
https://doi.org/10.1038/s41577-020-0308-...
. Inflammation of the lungs causes pneumonia, leading to multiple organ failure and subsequent mortality77 Asghar MS, Kazmi SJH, Khan NA, Akram M, Khan SA, Rasheed U, et al. Clinical profiles, characteristics, and outcomes of the first 100 admitted COVID-19 patients in Pakistan: a single-center retrospective study in a tertiary care hospital of Karachi. Cureus. 2020;12(6):e8712. https://doi.org/10.7759/cureus.8712
https://doi.org/10.7759/cureus.8712...
. If the immune system fails to resist the virus, it spreads to every organ of the body, causing further damage33 Asghar MS, Kazmi SJH, Khan NA, Akram M, Hassan M, Rasheed U, et al. Poor prognostic biochemical markers predicting fatalities caused by COVID-19: a retrospective observational study from a developing country. Cureus. 2020;12(8):e9575. https://doi.org/10.7759/cureus.9575
https://doi.org/10.7759/cureus.9575...
. Inflammatory processes usually cause changes in the body's biomarkers that can be measured to determine the state of inflammation and subsequent prognosis88 Eid M, Al-Kaisy M, Regeia W, Jiwa Khan H. The Prognostic accuracy of neutrophil-lymphocyte ratio in COVID-19 patients. Front Emerg Med. 2020; [cited on Oct. 22, 2021];5(1):e8. Available from: https://fem.tums.ac.ir/index.php/fem/article/view/472
https://fem.tums.ac.ir/index.php/fem/art...
. Some of these biomarkers are the neutrophil–lymphocyte ratio (NLR), platelet–lymphocyte ratio (PLR), and C-reactive protein (CRP), which are used as biomarkers predicting pneumonia. This study aimed to investigate the relationship of NLR, PLR, and CRP with COVID-19 mortality.

METHODS

This is a retrospective study of data from 635 patients with COVID-19 referred to Sinop State Hospital from March to November 2020. The diagnosis of COVID-19 was made according to the WHO interim guidance. Only patients whose COVID-19 were confirmed by the laboratory participated in this study. Fifty-one participants died, and 584 patients survived. Patients were grouped into two groups based on fatality. Due to anonymous, retrospective, and observational nature of this study, patients’ informed consent was waived.

Exposure history records, clinical signs, epidemiological characteristics, and laboratory data of patients were obtained from their electronic records and telephonic confirmation. Admission white blood cell (WBC), lymphocyte (LYM), neutrophil (NEU), NLR, platelet, PLR, and CRP were important variables selected from patients’ records. Other laboratory data obtained from the records included complete blood count and blood chemistry. Based on the patient's death or survival, this study group was divided into two groups. The exitus group included 51 patients, and the survivor group included 584 patients.

χ² test and Fisher's exact test were used to compare categorical variables, while Wilcoxon rank-sum test was used to compare continuous variables. Received operational curve (ROC) analysis was used to obtain the optimal cutoff values of PLR, CRP, and NLR variables to determine the mortality. 95% confidence interval (CI) and hazard risk (HR) were used to assess the relevant risks. Binary logistic regression analysis was used to determine the effect of age, gender, and some other factors on mortality. p<0.05 was considered as a statistically significant value. All statistical calculations were performed using SATA 14 software.

RESULTS

Demographic information and clinical characteristics of patients are given in Table 1. The mean age of the participants was 55.8±22.3 years. This value was significantly different between the groups (p=0.000), 54.2±22.3 in the survived group and 75.6±11.1 in the exitus group. There was a significant difference between the survivor and the exitus groups with respect to gender, and men had a higher death rate than women (p=0.008). The WBC was 9.4±31.6 in the survivor group and 12.6±7.04 in the exitus group, which was significantly higher than the exitus group (p=0.000). NEU, CRP, and NLR values were also significantly higher in the exitus group (p=0.000). PLR was a little higher in the exitus group, but this difference was not significant (p=0.42).

Table 1
Characteristics of the study population.

To determine the relationship between these biomarkers and COVID-19 fatality, the optimal cutoff values were calculated by ROC analysis, and the results are shown in Table 2. As shown in Table 2, the area under the curve (AUC) values for age, LYM, platelet, CRP, and NLR are 0.797, 0.424, 0.485, 0.778, and 0.729, respectively. LYM and platelet levels cannot be used as diagnostic biomarkers for patients’ risk of death because their AUC<0.50. However, Table 2 shows that CRP and NLR could be used as diagnostic biomarkers for COVID-19 fatality.

Table 2
Area under the curve values of age, lymphocyte, platelet, C-reactive protein, and neutrophil–lymphocyte ratio.

The Kaplan–Meier curve and the univariate Cox regression model were used to examine the factors that could lead to the death from COVID-19. The variables of NLR and PLR were included in univariate analyses to determine their effect on the death of patients with COVID-19. The analysis result shows that NLR can be considered an independent factor associated with the death of patients with COVID-19. However, PLR did not show any correlation with COVID-19 fatality.

The crude odds ratio (OR) was calculated through logistic regression analysis to evaluate the predictability of death due to COVID-19 by the investigated parameters (Table 3). Due to the age and gender effect on parameters, their effect was adjusted and presented in a separate column. The OR p-values of CRP, NLR, and PLR were 0.000, 0.004, and 0.02, respectively. As shown in Table 3, CRP, NLR, and PLR can effectively predict mortality by COVID-19 by considering age and gender. However, only the adjusted odds ratio (ORa) CRP was p>0.05, which means that only CRP could be used as an independent biomarker to predict death in COVID-19 patients, regardless of age and gender (p=0.000).

Table 3
The crude odds ratio and adjusted odds ratio for variables.

DISCUSSION

COVID-19, also commonly known as coronavirus, is an infectious disease caused by coronavirus (2019-nCoV) acute respiratory disease infections99 Henry BM, Oliveira MHS, Benoit S, Plebani M, Lippi G. Hematologic, biochemical and immune biomarker abnormalities associated with severe illness and mortality in coronavirus disease 2019 (COVID-19): a meta-analysis. Clin Chem Lab Med. 2020;58(7):1021-8. https://doi.org/10.1515/cclm-2020-0369
https://doi.org/10.1515/cclm-2020-0369...
,1010 Huang J, Cheng A, Lin S, Zhu Y, Chen G. Individualized prediction nomograms for disease progression in mild COVID-19. J Med Virol. 2020;92(10):2074-80. https://doi.org/10.1002/jmv.25969
https://doi.org/10.1002/jmv.25969...
. Our knowledge of this disease is incomplete and is developing. Also, coronaviruses are often known to combine mutations and openings, posing an ongoing challenge to our understanding and clinical management.

Apart from clinical symptoms, immunological features in patients can be warning signs of disease deterioration. This study showed that increased NLR in patients could sign the progression of pneumonia and an increased risk of death in patients with COVID-19. This finding was consistent with previous studies1111 Huang S, Huang M, Li X, Zhang T, Lu H. Significance of neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio for predicting clinical outcomes in COVID-19. medRxiv. 2020:20090431. https://doi.org/10.1101/2020.05.04.20090431
https://doi.org/10.1101/2020.05.04.20090...
1717 Zhu C, Yu S, Zhao H, Shen H, Yang J, Zhao L, et al. Neutrophil-to-lymphocyte ratio predicts the clearance of SARS-CoV-2 RNA in mild COVID-19 patients – a retrospective analysis from Dongxihu Fangcang Hospital in Wuhan, China. Clin Chem Lab Med. 2020;58(9):e167-70. https://doi.org/10.1515/cclm-2020-0733
https://doi.org/10.1515/cclm-2020-0733...
. The relationship between NLR and infectious diseases is well-known. An explanation for this relationship may be that the NEU is a part of leukocytes that arises from the venous system and is transmitted to the immune system1111 Huang S, Huang M, Li X, Zhang T, Lu H. Significance of neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio for predicting clinical outcomes in COVID-19. medRxiv. 2020:20090431. https://doi.org/10.1101/2020.05.04.20090431
https://doi.org/10.1101/2020.05.04.20090...
. NEU generates large amounts of reactive oxygen species and could save the cell from the virus by inducing DNA damage1212 Kerboua KE. NLR: A cost-effective nomogram to guide therapeutic interventions in COVID-19. Immunol Invest. 2021;50(1):92-100. https://doi.org/10.1080/08820139.2020.1773850
https://doi.org/10.1080/08820139.2020.17...
. Our results showed that increased NLR is a sign of COVID-19 progress and can lead to more severe disease and eventually death. In this study, a threshold of 3.3 was considered for NLR via the ROC, which showed that it could predict the severity of the disease well, and these results are consistent with the findings of other studies1313 Nalbant A, Kaya T, Varim C, Yaylaci S, Tamer A, Cinemre H. Can the neutrophil/lymphocyte ratio (NLR) have a role in the diagnosis of coronavirus 2019 disease (COVID-19)? Rev Assoc Med Bras (1992). 2020;66(6):746-51. https://doi.org/10.1590/1806-9282.66.6.746
https://doi.org/10.1590/1806-9282.66.6.7...
1515 Solomon IH, Normandin E, Bhattacharyya S, Mukerji SS, Keller K, Ali AS, et al. Neuropathological Features of Covid-19. N Engl J Med. 2020;383(10):989-92. https://doi.org/10.1056/NEJMc2019373
https://doi.org/10.1056/NEJMc2019373...
.

Another notable biomarker in our results was CRP, which is positively correlated with the level of inflammation in the body1818 Liu X, Sun X, Liu J, Kong P, Chen S, Zhan Y, et al. Preoperative C-Reactive Protein/Albumin Ratio predicts prognosis of patients after curative resection for gastric cancer. Transl Oncol. 2015;8(4):339-45. https://doi.org/10.1016/j.tranon.2015.06.006
https://doi.org/10.1016/j.tranon.2015.06...
. Studies have shown that factors such as gender, age, or physical condition of the patient do not impact the CRP concentration level1818 Liu X, Sun X, Liu J, Kong P, Chen S, Zhan Y, et al. Preoperative C-Reactive Protein/Albumin Ratio predicts prognosis of patients after curative resection for gastric cancer. Transl Oncol. 2015;8(4):339-45. https://doi.org/10.1016/j.tranon.2015.06.006
https://doi.org/10.1016/j.tranon.2015.06...
2020 Zhou B, She J, Wang Y, Ma X. Utility of ferritin, procalcitonin, and C-reactive protein in severe patients with coronavirus disease 2019. Novel Coronavirus Disease. 2020;Preprint (Version 1). https://doi.org/10.21203/rs.3.rs-18079/v1
https://doi.org/10.21203/rs.3.rs-18079/v...
, which is consistent with our findings. This study found that CRP could be used as an independent biomarker for COVID-19 fatality, regardless of age and gender. Previous studies have shown that CRP levels can be used for the early detection of patients with pneumonia, which had higher levels of CRP than others1818 Liu X, Sun X, Liu J, Kong P, Chen S, Zhan Y, et al. Preoperative C-Reactive Protein/Albumin Ratio predicts prognosis of patients after curative resection for gastric cancer. Transl Oncol. 2015;8(4):339-45. https://doi.org/10.1016/j.tranon.2015.06.006
https://doi.org/10.1016/j.tranon.2015.06...
,2121 Kermali M, Khalsa RK, Pillai K, Ismail Z, Harky A. The role of biomarkers in diagnosis of COVID-19 – a systematic review. Life Sci. 2020;254:117788. https://doi.org/10.1016/j.lfs.2020.117788
https://doi.org/10.1016/j.lfs.2020.11778...
. In line with these findings, in this study, CRP levels were significantly correlated with disease severity and patient death. This means that CRP levels can be considered a warning factor for the progression and mortality by COVID-19.

One of the limitations of this study is that the study data are taken from a single clinical research center, which may reduce the accuracy of conclusions due to demographic and local conditions. Future studies are needed to demonstrate the results of several clinical research centers with different demographic and influential clinical data.

CONCLUSIONS

Our results showed that NLR and CRP are significantly higher in patients who died from COVID-19, and they could be effective biomarkers in predicting COVID-19 mortality. Our results also showed that CRP could be used as an age- and gender-independent biomarker to predict disease progression and mortality.

  • Funding: none.

REFERENCES

  • 1
    Abrishami A, Khalili N, Dalili N, Khaleghnejad Tabari R, Farjad R, Samavat S, et al. Clinical and radiologic characteristics of COVID-19 in patients with CKD. Iran J Kidney Dis. 2020;14(4):267-77. PMID: 32655021
  • 2
    Cao X. COVID-19: immunopathology and its implications for therapy. Nat Rev Immunol. 2020;20(5):269-70. https://doi.org/10.1038/s41577-020-0308-3
    » https://doi.org/10.1038/s41577-020-0308-3
  • 3
    Asghar MS, Kazmi SJH, Khan NA, Akram M, Hassan M, Rasheed U, et al. Poor prognostic biochemical markers predicting fatalities caused by COVID-19: a retrospective observational study from a developing country. Cureus. 2020;12(8):e9575. https://doi.org/10.7759/cureus.9575
    » https://doi.org/10.7759/cureus.9575
  • 4
    Gemelli Against COVID-19 Post-Acute Care Study Group. Post-COVID-19 global health strategies: the need for an interdisciplinary approach. Aging Clin Exp Res. 2020;32(8):1613-20. https://doi.org/10.1007/s40520-020-01616-x
    » https://doi.org/10.1007/s40520-020-01616-x
  • 5
    Fauci AS, Lane HC, Redfield RR. Covid-19 – navigating the uncharted. N Engl J Med. 2020;382(13):1268-9. https://doi.org/10.1056/NEJMe2002387
    » https://doi.org/10.1056/NEJMe2002387
  • 6
    Chen L, Yu J, He W, Chen L, Yuan G, Dong F, et al. Risk factors for death in 1859 subjects with COVID-19. Leukemia. 2020;34(8):2173-83. https://doi.org/10.1038/s41375-020-0911-0
    » https://doi.org/10.1038/s41375-020-0911-0
  • 7
    Asghar MS, Kazmi SJH, Khan NA, Akram M, Khan SA, Rasheed U, et al. Clinical profiles, characteristics, and outcomes of the first 100 admitted COVID-19 patients in Pakistan: a single-center retrospective study in a tertiary care hospital of Karachi. Cureus. 2020;12(6):e8712. https://doi.org/10.7759/cureus.8712
    » https://doi.org/10.7759/cureus.8712
  • 8
    Eid M, Al-Kaisy M, Regeia W, Jiwa Khan H. The Prognostic accuracy of neutrophil-lymphocyte ratio in COVID-19 patients. Front Emerg Med. 2020; [cited on Oct. 22, 2021];5(1):e8. Available from: https://fem.tums.ac.ir/index.php/fem/article/view/472
    » https://fem.tums.ac.ir/index.php/fem/article/view/472
  • 9
    Henry BM, Oliveira MHS, Benoit S, Plebani M, Lippi G. Hematologic, biochemical and immune biomarker abnormalities associated with severe illness and mortality in coronavirus disease 2019 (COVID-19): a meta-analysis. Clin Chem Lab Med. 2020;58(7):1021-8. https://doi.org/10.1515/cclm-2020-0369
    » https://doi.org/10.1515/cclm-2020-0369
  • 10
    Huang J, Cheng A, Lin S, Zhu Y, Chen G. Individualized prediction nomograms for disease progression in mild COVID-19. J Med Virol. 2020;92(10):2074-80. https://doi.org/10.1002/jmv.25969
    » https://doi.org/10.1002/jmv.25969
  • 11
    Huang S, Huang M, Li X, Zhang T, Lu H. Significance of neutrophil-to-lymphocyte ratio, platelet-to-lymphocyte ratio for predicting clinical outcomes in COVID-19. medRxiv. 2020:20090431. https://doi.org/10.1101/2020.05.04.20090431
    » https://doi.org/10.1101/2020.05.04.20090431
  • 12
    Kerboua KE. NLR: A cost-effective nomogram to guide therapeutic interventions in COVID-19. Immunol Invest. 2021;50(1):92-100. https://doi.org/10.1080/08820139.2020.1773850
    » https://doi.org/10.1080/08820139.2020.1773850
  • 13
    Nalbant A, Kaya T, Varim C, Yaylaci S, Tamer A, Cinemre H. Can the neutrophil/lymphocyte ratio (NLR) have a role in the diagnosis of coronavirus 2019 disease (COVID-19)? Rev Assoc Med Bras (1992). 2020;66(6):746-51. https://doi.org/10.1590/1806-9282.66.6.746
    » https://doi.org/10.1590/1806-9282.66.6.746
  • 14
    Simadibrata DM, Calvin J, Wijaya AD, Ibrahim NAA. Neutrophil-to-lymphocyte ratio on admission to predict the severity and mortality of COVID-19 patients: a meta-analysis. Am J Emerg Med. 2021;42:60-9. https://doi.org/10.1016/j.ajem.2021.01.006
    » https://doi.org/10.1016/j.ajem.2021.01.006
  • 15
    Solomon IH, Normandin E, Bhattacharyya S, Mukerji SS, Keller K, Ali AS, et al. Neuropathological Features of Covid-19. N Engl J Med. 2020;383(10):989-92. https://doi.org/10.1056/NEJMc2019373
    » https://doi.org/10.1056/NEJMc2019373
  • 16
    Tjendra Y, Al Mana AF, Espejo AP, Akgun Y, Millan NC, Gomez-Fernandez C, et al. Predicting disease severity and outcome in COVID-19 patients: a review of multiple biomarkers. Arch Pathol Lab Med. 2020;144(12):1465-74. https://doi.org/10.5858/arpa.2020-0471-SA
    » https://doi.org/10.5858/arpa.2020-0471-SA
  • 17
    Zhu C, Yu S, Zhao H, Shen H, Yang J, Zhao L, et al. Neutrophil-to-lymphocyte ratio predicts the clearance of SARS-CoV-2 RNA in mild COVID-19 patients – a retrospective analysis from Dongxihu Fangcang Hospital in Wuhan, China. Clin Chem Lab Med. 2020;58(9):e167-70. https://doi.org/10.1515/cclm-2020-0733
    » https://doi.org/10.1515/cclm-2020-0733
  • 18
    Liu X, Sun X, Liu J, Kong P, Chen S, Zhan Y, et al. Preoperative C-Reactive Protein/Albumin Ratio predicts prognosis of patients after curative resection for gastric cancer. Transl Oncol. 2015;8(4):339-45. https://doi.org/10.1016/j.tranon.2015.06.006
    » https://doi.org/10.1016/j.tranon.2015.06.006
  • 19
    Mikami T, Miyashita H, Yamada T, Harrington M, Steinberg D, Dunn A, Siau E, et al. Risk factors for mortality in patients with COVID-19 in New York City. J Gen Intern Med. 2021;36(1):17-26. https://doi.org/10.1007/s11606-020-05983-z
    » https://doi.org/10.1007/s11606-020-05983-z
  • 20
    Zhou B, She J, Wang Y, Ma X. Utility of ferritin, procalcitonin, and C-reactive protein in severe patients with coronavirus disease 2019. Novel Coronavirus Disease. 2020;Preprint (Version 1). https://doi.org/10.21203/rs.3.rs-18079/v1
    » https://doi.org/10.21203/rs.3.rs-18079/v1
  • 21
    Kermali M, Khalsa RK, Pillai K, Ismail Z, Harky A. The role of biomarkers in diagnosis of COVID-19 – a systematic review. Life Sci. 2020;254:117788. https://doi.org/10.1016/j.lfs.2020.117788
    » https://doi.org/10.1016/j.lfs.2020.117788

Publication Dates

  • Publication in this collection
    26 Nov 2021
  • Date of issue
    Oct 2021

History

  • Received
    18 July 2021
  • Accepted
    14 Aug 2021
Associação Médica Brasileira R. São Carlos do Pinhal, 324, 01333-903 São Paulo SP - Brazil, Tel: +55 11 3178-6800, Fax: +55 11 3178-6816 - São Paulo - SP - Brazil
E-mail: ramb@amb.org.br