SciELO - Scientific Electronic Library Online

 
vol.49 issue6High seroconversion rates in Trypanosoma cruzi chronic infection treated with benznidazole in people under 16 years in GuatemalaNumber of leprosy reactions during treatment: clinical correlations and laboratory diagnosis author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

  • On index processCited by Google
  • Have no similar articlesSimilars in SciELO
  • On index processSimilars in Google

Share


Revista da Sociedade Brasileira de Medicina Tropical

Print version ISSN 0037-8682On-line version ISSN 1678-9849

Rev. Soc. Bras. Med. Trop. vol.49 no.6 Uberaba Nov./Dec. 2016

http://dx.doi.org/10.1590/0037-8682-0258-2016 

Major Articles

Predicting death from kala-azar: construction, development, and validation of a score set and accompanying software

Dorcas Lamounier Costa1  2 

Regina Lunardi Rocha3 

Eldo de Brito Ferreira Chaves2 

Vivianny Gonçalves de Vasconcelos Batista2 

Henrique Lamounier Costa4 

Carlos Henrique Nery Costa2  5 

1Departamento Materno-Infantil, Universidade Federal do Piauí, Teresina, Piauí, Brazil.

2Instituto de Doenças Tropicais Natan Portella, Teresina, Piauí, Brazil.

3Departamento de Pediatria, Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil.

4Departamento de Engenharia de Produção, Universidade Federal do Piauí, Teresina, Piauí, Brazil.

5Departamento de Medicina Comunitária, Universidade Federal do Piauí, Teresina, Piauí, Brazil.

Abstract

INTRODUCTION

Early identification of patients at higher risk of progressing to severe disease and death is crucial for implementing therapeutic and preventive measures; this could reduce the morbidity and mortality from kala-azar. We describe a score set composed of four scales in addition to software for quick assessment of the probability of death from kala-azar at the point of care.

METHODS:

Data from 883 patients diagnosed between September 2005 and August 2008 were used to derive the score set, and data from 1,031 patients diagnosed between September 2008 and November 2013 were used to validate the models. Stepwise logistic regression analyses were used to derive the optimal multivariate prediction models. Model performance was assessed by its discriminatory accuracy. A computational specialist system (Kala-Cal(r)) was developed to speed up the calculation of the probability of death based on clinical scores.

RESULTS:

The clinical prediction score showed high discrimination (area under the curve [AUC] 0.90) for distinguishing death from survival for children ≤2 years old. Performance improved after adding laboratory variables (AUC 0.93). The clinical score showed equivalent discrimination (AUC 0.89) for older children and adults, which also improved after including laboratory data (AUC 0.92). The score set also showed a high, although lower, discrimination when applied to the validation cohort.

CONCLUSIONS:

This score set and Kala-Cal(r) software may help identify individuals with the greatest probability of death. The associated software may speed up the calculation of the probability of death based on clinical scores and assist physicians in decision-making.

Keywords: Kala-azar; Visceral leishmaniasis; Prediction; Mortality; Software

INTRODUCTION

Kala-azar, or visceral leishmaniasis (VL), is the severest infection caused by protozoa of the Leishmania genus and is usually fatal when not treated1 The disease is endemic in 98 countries and three territories, in five continents. Case-fatality rates are 5-20% in endemic areas2) (3. Death is usually associated with systemic inflammation, hemorrhage, and bacterial infections, even with readily available hemoderivatives and antibiotics4) (5) (6.

Approximately 0.2-0.4 million VL cases occur annually7. Most infections are asymptomatic; clinical presentations of symptomatic infections range from mild to life-threatening8. The main symptom is prolonged fever, sometimes followed by hemorrhage, cough, diarrhea, or wasting. Physical examination reveals anemia and splenomegaly in almost all patients. Many also present with petechiae, jaundice, pulmonary rales, edema, and evidence of bacterial infections9. Typical laboratorial results are pancytopenia, elevated erythrocyte sedimentation rate, elevated C-reactive protein, hypoalbuminemia, and hypergammaglobulinemia. Some patients have elevated liver enzymes and kidney involvement10) (11. Diagnosis is based on serological tests and demonstration of parasites in bone marrow or spleen through direct examination, culture, or polymerase chain reaction12.

Two different Leishmania species transmitted through the bite of several sand fly species are responsible for the disease: Leishmania donovani species, in the Northeast of the Indian subcontinent and East Africa (majority of cases), and the zoonotic Leishmania infantum species, in Central Asia, the Middle East, the Mediterranean area, West Africa, and Central and South America13. Both are important opportunistic infections among patients with acquired immunodeficiency syndrome (AIDS)14) (15) (16 and have similar clinical presentations, but L. donovani may lead to post-kala-azar dermal leishmaniasis, or infectious skin lesions17) (18.

Since the 1980s, the disease has been transmitted in larger cities and spread throughout Brazil and the South Cone. Despite medical advances, mortality has been increasing in Brazil19, to levels similar to those among refugees in Sudan2) (20, despite free drug therapy. Moreover, in Brazil, the rate of kala-azar-related mortality remains unchanged 10 years after implementing a national guideline for the treatment of severely ill patients21. The major criticism of this guideline is the weak scientific evidence to support recommendations for treatment of complications. Policy failure is also the result of delayed recognition of the severest forms of the disease, lack of identification of subtle complications, and lack of proper treatment for some of the severest complications. Finally, only a few clinical trials of kala-azar treatment included death as an outcome, and none investigated kala-azar caused by L. infantum; one of the studies, conducted in Ethiopia, showed only higher mortality with antimonial therapy than with miltefosine therapy22. Since miltefosine is not licensed in Brazil, these results have little impact on the national kala-azar-related mortality rate.

Age, disease length, bacterial infections, bleeding, and immunodeficiency seem to be the most important predictors of poor prognosis in studies in Africa2) (23) (24) (25) and Brazil6) (26) (27) (28. Patients with kala-azar at the highest risk of death are those with the clinical symptoms, signs, and laboratory data of more severe disease; early identification of these patients may reduce mortality by enabling referrals to more complex medical attention. Outpatients could be referred to hospitals for amphotericin B, intravenous antibiotics, and blood derivatives, and patients in hospital wards could be transferred to intensive care units (ICUs), which might help reduce mortality. Moreover, quantitatively ascertaining the risk of death may help allocate patients in clinical studies, with quality improvement initiatives, and for benchmarking purposes26. Scoring systems may only indirectly reduce mortality by improving scientific knowledge and refining the quality of care. Although scoring systems are not recommended to assess individual patients, they may help identify changes in health status earlier, for medical interventions27. However, the combination of scoring and solid clinical knowledge, fundamental pathophysiological understanding, and available therapies might enlighten medical decisions and reduce the probability of death for individual patients.

The risk factors for death in patients with kala-azar by L. infantum have been identified, and prognostic models based on clinical scores have been developed5) (6) (25) (28. In a recent meta-analysis, jaundice, thrombocytopenia, hemorrhage, human immunodeficiency virus-1 (HIV-1) co-infection, diarrhea, age <5 years or >40-50 years, severe neutropenia, dyspnea, and bacterial infections were important risk factors for VL-related death. Edema and low hemoglobin concentration were also associated with unfavorable outcomes29. However, most prognostic studies did not validate the scores. Finally, although some systems are simple, they may not be fully recalled by physicians during their overcharged medical routines. Therefore, in the present study, we developed a validated score available through easy-to-access software for the rapid assessment of severity and evolution, to help reduce mortality by identifying individuals at a higher risk of death.

METHODS

Study population

A prospective, hospital-based study was conducted at the Institute of Tropical Diseases Natan Portella, in Teresina, the capital of Piauí State in Brazil. Inclusion criteria were the presence of fever and splenomegaly with laboratorial evidence of Leishmania infection: positive bone marrow smear or culture, indirect immunofluorescent test (IFI-Leishmaniose Humana. Bio-Manguinhos) with titers ≥1:80, or positive rK39 antigen-based immunochromatographic test. The prospective, derivation cohort to build the score set included patients diagnosed between September 1, 2005 and August 31, 2008. The first validation cohort included patients diagnosed between September 1, 2008 and July 31, 2009; the second validation cohort included patients diagnosed between August 1, 2009 and November 31, 2013.

Clinical definitions

Since kala-azar is an infectious, inflammatory, and febrile disease with overlapping or ambiguous clinical manifestations, operational definitions of the clinical manifestations were created. Hemorrhage was defined as any clinically spontaneous apparent bleeding such as petechiae, hematoma, hematuria, epistaxis, hemoptysis, gum or gastrointestinal bleeding, and persistent bleeding at venipuncture, intramuscular injection, or bone marrow puncture sites. Bacterial infection was defined as positive Gram stain or culture; presence of >10 leucocytes per field in the urine; typical skin infections; cough associated with dyspnea, rales, or signs of respiratory distress as well as other clinical signs of bacterial pneumonia; or radiological or tomographic images typical of infection, more commonly chest radiography suggestive of pneumonia. At least one of the following signs indicated bacterial sepsis: hyperventilation (respiratory rate for age or PaCO2 <32mmHg), low tissue perfusion (PaO2/FiO2 <250), venous oxygen saturation <70%, cyanosis, or organ dysfunction characterized by lactic acidosis, oliguria, and altered consciousness30. Kidney failure was defined as a glomerular filtration rate (GFR) <60 mL/min/m2 as calculated using the Cockcorft-Gault method for adults and children >12 years old or the Schwartz method for children ≤12 years old31. However, creatinine level was used instead of GFR because serum creatinine levels >1.5mg/dL were similar to GFR <60 mL/min/m2 in a preliminary analysis and are easier to assess. Finally, death was attributed to kala-azar only if the patient died before or during treatment. Patients who died presumably from drug toxicities were excluded.

Data collection

Clinical and laboratory evaluations were performed at hospital admission. Each patient from the derivation cohort was evaluated daily by a single and experienced author (Costa DL) or eventually, by a trained physician, until hospital discharge. All data were entered into a computerized database. Non-specific laboratory data included complete blood count and biochemistry, chest radiograph, urinary sediment, and HIV-1 serology. Blood culture, urine culture, or cultures from other secretions were provided before antibiotic administration or for suspected bacterial infection. The study outcome was registered as discharge or death.

Statistical analysis and score set

Statistical analysis was performed using Stata/SE(r) 10.0 for Windows (College Station, Texas, USA). Chi-square or Fisher's exact tests were used to analyze dichotomous variables. Spearman's correlation tests were used to analyze correlations among continuous variables with sparse data, while Pearson's tests were used for variables with normal distributions. The differences between independent variables with normal distributions were compared using Student's t-tests and without normal distributions using Wilcoxon and Kruskal-Wallis tests. The relationships among variables were measured using the relative risk (RR) or odds ratio (OR) with 95% confidence intervals (CIs). Multiple comparisons were conducted using the Bonferroni adjustment method, with Dunn's procedure. In this case, p-values refer to variables that remained statistically significant after adjustment.

Model derivation

To determine the significant prognostic variables, logistic regression analysis was performed, using backward elimination; only variables with a p < 0.05 were retained in the model. A variance inflation factor (VIF) was calculated after the regression analysis to assess co-linearity. The following continuous variables were converted into dichotomic variables: age, hemoglobin level, leukocyte count, platelet count, aspartate transferase (AST) level, alanine transferase (ALT) level, and glomerular filtration rate. All dichotomic and statistically significant (p < 0.20) variables that were not co-linear in the multivariate analysis were entered in the model. The final regression equation in the multivariate logistic models was used to determine the predicted probabilities of death. The weight of each variable was assumed as the ratio between the modules of the coefficient of the analyzed variable and of the smallest coefficient among the variables of the logistic regression model, which was rounded to the closest integer32. Finally, the score of each patient with a respective probability of death was calculated by adding the weight of each variable in the final model. Sensitivity, specificity, and positive and negative predictive values were calculated for each score. The score for which sensitivity and specificity were the highest was selected. Receiver operator characteristic (ROC) curves were constructed using a series of cut points from both the derivation and validation sets. Statistical significance of both ROC curves was analyzed by calculating the area under the ROC curve (AUC) and 95% CI33. The goodness-of-fit of the logistic regression model was tested using the Hosmer-Lemeshow test.

Model validation

During the validation procedure, the risk factors for death differed for children ≤ 2 years old and for older patients. New, separate models were built for infants and older individuals in the derivation cohort and were tested again with the validation cohort, showing better performance. Two models were then rebuilt for each age group: one addressing only clinical variables collected at the first medical evaluation and another incorporating the laboratory test results at hospital admission.

Thereafter, the predictive model was applied to two subsequent validation cohorts to check the probability of death based on the derivation models. Initially, the model was applied to the two cohorts divided only by age (children ≤ 2 years old and older children and adults). Then, subgroups were constructed: age, 2-year intervals in children <10 years old and 5-year intervals in older patients; origin, rural versus urban and smaller versus larger cities; period of admission, quartiles according to the year of diagnosis; and presence of HIV-1 co-infection. The ability to distinguish between death and survival was assessed for all the models using the AUC.

Sensitivity analysis

Sensitivity analysis was performed in the derivation cohort to assess the impact of extreme weighting of each variable. The model was reloaded using the extreme 5% and 95% values of the 95% CI coefficient for each of the variables retained in the final logistic regression model. The discriminatory accuracy (between death and survival) was assessed for all models through the AUC.

Software development

The software Kala-Cal(r) was developed to estimate the probability of death of patients with kala-azar, according to the clinical scores in the present study. The software is a special system developed using Java (prototype phase) as the programming language and PHP (release phase), and the paradigm was that of an object-oriented programming language. The human-computer system interface was designed to minimize the number and complexity of interactions between the user and program. Therefore, we developed a system optimized for productivity, applet availability, many browsers and any operational system, anywhere, and any cell phone and without the explicit necessity of a computer. The screen consists of five radio buttons allowing the selection of the age group and two buttons to choose between the clinical and clinical-laboratorial models. The subsequent screen shows checkboxes with the previously selected clinical and laboratory data, accordingly to the final selected model. The last screen shows the final score along with the estimated probability of death in a graph. For the testing, a classical cycle was used: prototypes were generated and tested, giving way to new requirement specifications, which were then implemented, generating new prototypes until a viable model was reached. The final result was an easy-to-use and light app, in which it is possible to obtain a quantitative estimation of prognosis, with a minimum task sequence of four clicks and three screens. The equation for the estimation of the probability of death was found through non-linear regression analysis at www.xuru.org and chosen based on the residual sum of squares value and shape. The applet can be accessed at http://sbmt.org.br/kalacal/.

Ethicals considerations

The Research Ethics Committee at the Federal University of Piauí granted ethical approval (number 0116/2005). Each participant or his legal representative signed the informed consent form.

RESULTS

Characteristics of the study population

Between September 2005 and November 2013, 1,914 patientswere admitted at the hospital for kala-azar. Based on the inclusion criteria, 883 patients were enrolled for the derivation model, 449 patients for the first validation cohort, and 582 patients for the second validation cohort.

Characteristics of the derivation cohort

Most patients were men [553 (62.6%)] and children ≤5 years old [463 (52.4%)], primarily ≤2 years old [314 (35.6%)]. The mean time from the start of symptoms until diagnosis was 74.8 days. HIV-1 serology was positive in 71 (10.9%) of the 652 screened patients. Sixty-six (7.5%) patients died. Figure 1 shows the age distribution considering mortality, which was two times higher in children ≤1 year old (10.5%) than in children >1 year old but ≤2 years old (5%), and five times higher than that in children 2 years old to ≤15 years old (2%). Mortality was highest among patients >40 years old (16.8%), almost two times as high as it was among people 15-40 years old (9.7%) and statistically significantly higher than in patients ≤2 years old (RR = 5.4; p = 0.008), >20 years old (RR = 6.9; p < 0.001), or 2-20 years old (RR = 11.23; p < 0.001). Mortality among patients with HIV-1 was 16.7%, more than two times higher than that among patients without HIV-1 (7%) (RR = 2.3; p = 0.03). There was no difference in mortality by sex or origin (urban or rural). Shock syndrome was the most prevalent clinical phenomena in the 24 hours preceding death, affecting 52 (79%) of the 66 patients in the derivation cohort who died. Because bleeding occurred in 43 (65%) patients, it was considered the most important cause of shock and death. Proven bacterial infections were present in only 27% of the patients. Other complications preceding death were seizures, respiratory insufficiency, acute kidney failure, pancreatitis, and cardiac arrest.

FIGURE 1 Mortality from kala-azar according to age group among 883 patients in Teresina, Brazil 

Table 1 shows the clinical manifestations and laboratory results associated with death in infants and older patients in the univariate analysis. Variables associated with a higher risk of death among infants than among older children and adults were somnolence, jaundice, and bleeding. Conversely, variables associated with a higher risk of mortality among older children and adults than among infants were sepsis syndrome and kidney failure. Skin infection was associated with death only among infants, while urinary infection and neutropenia were associated with death only among older individuals.

TABLE 1 Clinical manifestations and laboratory results associated to death among children under two years old and among children over this age and adults with kala-azar from the derivation cohort: univariate analysis. Teresina, Brazil. 

95% CI: 95% confidence interval; AST: aspartate aminotransferase; ALT: alanine aminotransferase; HIV: human immunodeficiency virus. 1Patients with HIV were excluded from the analysis when analyzing interstitial pneumonia. 2Glomerular filtration rate < 60mL/min/m2.

Table 2 and Table 3 show the clinical and laboratorial findings related to mortality in the multivariate analysis, for both infants and older patients. For infants, age, bleeding, edema, jaundice, and dyspnea remained in the final regression model, but jaundice was substituted by serum AST level in the clinical-laboratorial model. For older patients, age, bleeding, jaundice, edema, vomiting, dyspnea, and HIV-1 and bacterial co-infections composed the clinical multivariate model, while leukopenia, thrombocytopenia, and renal failure were added to the clinical-laboratorial model.

TABLE 2 Risk of death associated with clinical findings at hospital admission and laboratory results among children under two years-old with kala-azar: univariate and multivariate analysis. 

95% CI: 95% confidence interval; AST: aspartate aminotransferase; ALT: alanine aminotransferase. 1Coefficient.

TABLE 3 Risk of death associated with clinical findings at hospital admission and laboratory results among individuals over two years-old with kala-azar: univariate and multivariate analysis. 

95% CI: 95% confidence interval; HIV-1: human immunodeficiency virus; AST: aspartate aminotransferase; ALT: alanine aminotransferase. 1Coefficient. 2Glomerular filtration rate < 60mL/min/m2 or creatinine above the levels for age.

Prognostic models

Table 4 shows the composition and weighting of the scoring systems for the clinical and laboratorial models for infants and older individuals. In all four models, the regression lines of the scores and probability of death for use in the Kala-Cal(r) software for mortality prediction were slightly exponential (Figure 2).

TABLE 4 Prognostic models for predicting death by kala-azar built by summing up clinical and clinical plus laboratory variables, weighed by the force of statistical association in Teresina, Brazil. 

AST: aspartate aminotransferase; ALT: alanine aminotransferase. 1Glomerular filtration rate <60mL/min/m2 or creatinine above the levels for age.

FIGURE 2 Predicted probability of death from kala-azar according to scores derived from the logistic regression analyses for: (A): Infants using the model with clinical data. (B): Infants using the model with both clinical and laboratory data. (C): Children aged >2 years and adults using the model with clinical data, and (D): Children aged >2 years and adults using the model with both clinical and laboratory data. Fitted lines were smoothed with the use of best-fit equations obtained in www.xuru.com. 

Derivation model for children ≤2 years old

The best set of predictor variables based solely on clinical information for infants were age <1 year, bleeding, edema, jaundice, and dyspnea (Table 4). This model had a maximum score of 9 points. The observed probability of death was proportional to the score, ranging from 0.01 to 0.80, generating a ROC curve with an AUC of 0.90 (95% CI 0.84-0.97), with appropriate goodness-of-fit (Pearson = 6.12; p = 0.52). Maximum sensitivity and specificity were achieved with a score of 4 points (Youden's index [J] = 0.69). When the laboratory variables were added, the final model included age <1 year, edema, bleeding, dyspnea, and increased aminotransferase levels (AST or ALT >100U/L). The maximum score was 11, generating an AUC of 0.93 (95% CI 0.88-0. 98), with appropriate goodness-of-fit (Pearson = 6.48; p = 0.69). Maximum sensitivity and specificity were once again achieved with a score of 4 points (J = 0. 76).

Derivation models: patients >2 years old

For patients >2 years old, the variables that fit the clinical model were the presence of bleeding, HIV-1 co-infection, edema, vomiting, jaundice, dyspnea, bacterial infection, and age >15 years. The maximum score was 13 points, generating an AUC of 0.89 (95% CI 0.84-0.93) with appropriate goodness-of-fitness (Pearson = 6.78; p = 0.66). A score of 4 points had the best cut-off point (J = 0.62). When laboratorial data were included, the variable set included HIV-1 co-infection, jaundice, dyspnea, bacterial infection, leukopenia (<1,500 leukocytes/mm3), thrombocytopenia (<50,000 platelets/mm3), and low GFR. The maximum score was 10 points, generating an AUC of 0.92 (95% CI 0.88-0.96) with an appropriate goodness-of-fit (Pearson=1.93; p=0.93). The best cut-off was 3 points (J = 0.70).

Characteristics of the validation cohorts

Table 5 shows the comparison of the baseline characteristics between the derivation cohort and the two validation cohorts. Compared with validation cohort 1, the derivation cohort had more infants, more women, and the following signs of an increased probability of death: higher proportion of diarrhea, vomiting, bleeding at venipuncture sites, and hepatosplenomegaly and lower leukocyte and neutrophil counts. However, the derivation cohort had signs of more benign disease than validation cohort 1 (less severe anemia, less severe dyspnea, less kidney failure, higher hemoglobin levels, lower creatinine levels, and fewer patients infected with HIV-1), which resulted in lower mortality.

TABLE 5 Baseline characteristics of 883 participants with kala-azar from the derivation sample, 49 from the first validation cohort, and 582 from the second validation cohort. 

95% CI: 95% confidence interval; IQ: interquartile; AST: aspartate aminotransferase; ALT: alanine aminotransferase; HIV: human immunodeficiency vírus. *Values are numbers (percentages) of participants unless stated otherwise. **p-values refer to the comparison of validation cohorts with derivation cohort.

Compared with validation cohort 2, the derivation cohort had more number of younger individuals and female participants. The only evidence that could indicate an increased probability of death in the derivation cohort was a slightly higher proportion of patients with splenomegaly or kidney failure. In contrast, the disease was more benign, based on the following clinically statistically significant differences: less severe anemia, less severe dyspnea, lower leukocyte and neutrophil counts, lower AST level, higher albumin level, higher globulin, and lower proportion of patients with HIV-1 infection. However, mortality was similar in the two cohorts. Overall, patients in the derivation cohort had a more benign course of disease than those in both validation cohorts.

Validation

Table 6 shows the performance of the scoring sets in the two validation cohorts as compared with the AUC of the derivation cohort. The clinical scoring model among infants in both validation cohorts had slightly the same performance as the derivation cohort (AUC = 0.87 and 0.86, respectively). The addition of laboratory results reduced the scoring performance in validation cohort 1 (AUC = 0.83), with a wider confidence interval. This excellent performance was sustained in validation cohort 2.

TABLE 6 Performance of the score sets in the validation cohorts as compared to the derivation cohort. 

AUC: area under the curve; 95% CI: 95% confidence interval. 1Youden index = (sensitivity+specificity-1).

However, the scoring system for older children and adults had a noticeable reduction in performance in validation cohort 1 in both models without (AUC = 0.75) and with (AUC = 0.79) laboratory results. However, in validation cohort 2, the performance was good in both models without (AUC = 0.88) and with (AUC = 0.71) laboratory results. Sensitivity decreased substantially for infants in validation cohort 2, and specificity decreased for older patients in validation cohort 1.

Sensitivity analysis

The AUCs of the score set were ≥0.81 for urban or rural origin, large or smaller cities, capital or other cities, different years of hospital admission, and presence of co-infection and HIV-1, and there was no statistical difference in the AUCs generated in the validation cohort than in the two validation cohorts.

When the regression models were tested with the smallest and largest 95% CI coefficients obtained from the separate logistic regression analysis of both age groups, the AUCs remained between 0.85 and 0.88 when the variables were considered individually. However, when the smallest coefficients were used simultaneously for all model variables, the AUC of the clinical model for infants decreased to 0.75 but remained 0.88 for those aged >2 years.

DISCUSSION

The present dataset and discussion might contribute to the reduction of mortality related with kala-azar, given the potentially better understanding of a patient's risk of death based on the scoring system. Only age and pre-existing HIV-1 infection were associated with death in patients with kala-azar and therefore cannot be changed with therapy. For infants, age <1 year contributed only ~10% of the score set. However, for older individuals, age had a stronger impact, accounting for 23% of the total clinical score, but with no effect on the clinical-laboratory model. In other prognostic scoring models, age was included only in the model by Coura-Vital et al.6, but only for patients >60 years old. Other prognostic studies (not of scoring models) also revealed the importance of younger age and that old age is a strong risk factor for kala-azar-related death2) (5) (19) (28) (34). One possible explanation is the higher concentrations of inflammatory and anti-inflammatory cytokines in these age groups35) (36) (37) (38) (39) (40.

HIV-1 infection is also a strong antecedent risk factor in both the adult clinical and laboratory score sets. It was also included in the scoring model developed by Coura-Vital et al6 but not those developed by Sampaio28 and Werneck24, due to age restrictions and earlier data collection, respectively, before HIV-1 co-infection became a major problem for patients with kala-azar. The higher mortality for kala-azar in HIV-1-infected patients is likely affected by the lower cluster of differentiation 4 (CD4) count with kala-azar itself, increasing the vulnerability to other co-infections41, and to the more intense systemic inflammation when compared with patients without HIV-1 infection, as demonstrated previously35. Therefore, addressing other co-infections and systemic inflammation are important steps in the treatment of these patients.

Collectively, the unmodifiable factors of age and HIV-1 co-infection accounted for almost 40% of the clinical scores for adults. As an example, a kala-azar patient aged >40 years with HIV-1 infection would have an approximate 20% chance of death without any other sign of severity and, due to the exponential nature of the regression curve between scores and chance of death, would have a rapidly increasing probability of death with the addition of any other clinical sign.

Bleeding from >3 sites was the most significant independent clinical sign related with death among infants, corresponding to almost 50% of the total clinical score and almost 40% of the total clinical and laboratorial scores, and also conferred a high risk among older people. Therefore, progressive bleeding should be considered an urgent risk for infants with kala-azar, who should be treated at the highest level of medical intervention. For adults, the impact of bleeding on mortality was slightly lower, at approximately 20%, and was replaced by thrombocytopenia in the clinical and laboratory scoring. Indeed, hemorrhage occurred 24 hours before death in most patients in the derivation cohort, and has been identified as a major risk factor for death in most studies2) (3) (5) (22) (24( (25) (28) (42. Therefore, understanding the nature and origin of bleeding is crucial for risk stratification for referral and complication-driven medical interventions. Previous studies have shown that disseminated intravascular coagulation (DIC) followed by consumption coagulopathy are the most important underlying phenomena for bleeding in patients with kala-azar5) (42) (43; nevertheless, liver involvement, as indicated by jaundice, is another independent risk factor, both of all immediate causes of death, but also of dying specifically from bleeding. Apparently, both factors act synergistically to hamper hemostasis and sustain bleeding. Furthermore, DIC in kala-azar is clearly linked to systemic inflammation, particularly to serum γ-interferon and interleukin (IL)-635.

Thrombocytopenia is a very characteristic, but non-specific, element of kala-azar. Since thrombocytopenia is, similar to other markers of DIC such as fibrin degradation products and d-dimer, strongly correlated with γ-interferon and IL-635, the conclusion would be that thrombocytopenia in kala-azar is mostly a result of consumption by DIC. However, the usual large splenomegaly, typical of kala-azar, is an obvious competing explanation for thrombocytopenia. The question, therefore, of greatest importance is how much a platelet transfusion for patients with kala-azar, thrombocytopenia, and hemorrhage would be beneficial. Currently, the Brazilian recommendations follow the international recommendation for thrombocytopenia, which is to perform prophylactic transfusion for bone marrow failure if platelets are <10,000/mm3 or 20,000/mm3 in patients with DIC who are bleeding44) (45) (46. Therefore, severely ill patients with kala-azar and thrombocytopenia with hemorrhagic manifestations and evidence of DIC, such as high concentrations of fibrin degradation products or d-dimer, together with a prolonged prothrombin time, should be treated by focusing on the DIC itself. Unfortunately, the lack of evidence-based treatment for DIC is another hurdle for patient survival. Heparin use is not supported by evidence and must be avoided in such a complex situation. Two new candidates for the treatment of DIC are thrombomodulin and the older tranexamic acid but still lack evidence of efficacy47) (48. Finally, patients with hemorrhage and jaundice should have additional and proper treatment with vitamin K. When facing such uncertainty, clinical trials with these drugs or new candidates should be urgently performed in patients with kala-azar and hemorrhage.

In addition to these bleeding phenomena, edema, dyspnea, jaundice, and vomiting are important signs and symptoms that predict death. Similar to hemorrhage, they were more relevant for children than for older children and adults and a consequence of systemic inflammation35. When present, patients may need supplementary symptomatic therapy such as intravenous albumin due to the typical low serum albumin concentration, as well as respiratory support and antiemetics; however, the basic cause of inflammation will remain unaffected. Likewise, increased AST and creatinine concentrations are also part of the systemic and sustained inflammation. Because bacterial infection accounted only for ~10% of the total scoring for adults, the independent risk factors for death from kala-azar in Brazil are causally linked to systemic inflammation. However, and unfortunately, there is no effective recommendation for treating systemic inflammation49) (50.

Given the barrier imposed by systemic inflammation, the software application is limited for treatment decisions. However, hints are available. At the lowest number of points, the probability of death increases slowly, but then increases sharply at a certain point. This may indicate that milder cases worsen additively, but an exponential increase occurs after a certain point, suggesting interactivity between different organs or systems, which reflects the systemic nature of severe kala-azar inflammation, evolving to progressive multiple organ dysfunction, followed at the end by death. The complexity of the many mediators and the multiple organ involvement indicate that referral to a higher level of medical attention for organ protection and life support measures are critical decisions while waiting for the slow action of specific anti-Leishmania therapy and the prolonged inflammation period48) (49. Possible rules of thumb for model use at bedside would be infant referral from primary care to hospital care when the child's risk of death is greater than the expected 10% mortality. Similarly, referral to ICUs may be considered when the probability of death is >20%, due to the exponential growth of chance of death at this point, which increases to >40% if patients have one more point after mortality reaches the range of 20-30%. Older individuals should be referred to hospitals if their score reaches 5 points. However, if they are older than 40 years and have HIV-1 infection, they should be treated at the hospital with just a single clinical risk factor. If they have ≥6 points, they would be admitted to the ICU. The most important laboratory tests that would signify a medical decision to transfer to ICUs for patients older than 40 years with HIV-1 infection or with clinical complications, are AST >100UK/L in infants, or leukocyte count <1,500/µL, platelet count <50,000/µL, and creatinine level >1.5mg/dL. Multiple organ failure is suspected in infants with 6 clinical points or 7 clinical-laboratory points and for older people with 7 clinical points, corresponding to the exponential inflection of the curves. On the clinical and laboratory scoring scale for infants, reaching just 5 points indicates a very high risk of death. In these cases, after the available therapies for L. infantum or sepsis, the only alternatives would be organ protection.

At least three scoring systems have already been developed for predicting the risk of death from kala-azar. An earlier model, also developed by our research group in Teresina with a small sample at the same hospital where the present study was conducted26, had an AUC of 0.88. Another, developed for children in Recife by Sampaio et al28, had an AUC of 0.90. Both studies lacked a validation cohort. Finally, the system developed by Coura-Vital et al6 for all patients registered in Brazil had AUCs of 0.80 for the derivation cohort and 0.78 for a sub-sample that composed the validation cohort. When these models were tested with the derivation cohort in the present study, the AUCs were 0.75, 0.87, and 0.77, respectively. Therefore, our models provided the most accurate clinical score sets for children and adults. There was an overlap in the age groups between the study by Sampaio et al28 and the present study; the models used by Sampaio et al performed better with children 1-15 years old than with our cohort of patients aged >2 years. This was likely due to the presence of HIV-1 infection in our cohort. Indeed, patients infected with HIV-1 had a lower performance in the model, although in the range of the 95% CI of the AUC of the whole cohort, possibly due to the lower specificity generated by the competitive causes of death from HIV-1 co-infection. Additionally, and contrary to other studies, our data were collected prospectively, with a single medical doctor examining all data from the derivation cohort. Indeed, the first validation cohort was dissimilar to the derivation cohort, particularly regarding the contribution of infancy and HIV-1 co-infection, which may have weakened the model performances. An epidemic of HIV-1/L. infantum co-infection detected at the hospital occurred during the data collection period for the validation cohort 141. The final advantage of this scoring system is the model's availability as an applet, which is accessible on the Internet through cell phones, easing the utilization of the model at primary care or the bedside for quick assessment of the probability of death and facilitating medical decisions. Therefore, the score set developed in this is study was the most accurate, robust, and easy to use for predicting death and may contribute to better classification of patients in research projects and improved health services, helping to reduce the mortality from kala-azar.

However, the present study had limitations. First, it was restricted to a single hospital, which may constrain its external validity. Certainly, the model should be tested and calibrated in other scenarios in which both L. infantum and L. donovani are transmitted. Also, as the study was performed after the national recommendation for broad and early use of antibiotics in 200551, bacterial infections may have been under-represented and might have made the score set less accurate in areas where early antibiotic prescription for patients with severe disease is not routine. In addition, the full definition of bacterial infections could not be used for patients with kala-azar. Bacterial sepsis and VL share clinical signs such as fever, bleeding, lung opacities, leukopenia, and thrombocytopenia. For example, if the definition of sepsis52 was used for all patients, 863 (97.7%) of the patients would have been classified as having sepsis. Likewise, the diagnoses of pneumonia and urinary tract infection may have been confounded by kala-azar pneumonitis and nephritis in the absence of positive cultures10) (11. Therefore, the criteria for defining sepsis, pneumonia, and urinary tract infection were modified in order to increase the specificity of the operational definition. Therefore, less severe presentations of pneumonia, sepsis without organ dysfunction, hypoperfusion or hypotension, and urinary infection may have not been detected 53) (54) (55. A final comment on the absence of anemia as a risk factor in the score set is required, because it was described as a component of another score system and as a prognostic factor6) (24. In the present study, anemia was identified as a risk factor in the univariate analysis but, as with other risk factors, is also affected by systemic inflammation35. Therefore, anemia was omitted owing to collinearity with another variable that had a stronger association with death.

In conclusion, an easily accessible prognostic system available through a computer program for quick identification of patients of all ages with a greater probability of dying from VL was developed based on clinical data collected from three large in-hospital prospective cohorts. The first cohort (2005-2008) provided the data to derive the prognostic model set, while the second (2008-2009) and third (2009-2013) cohorts enabled validation of the performance of the testing model. The computerized system accessible via cell phone allowed quick assessment of the risk of death at the point of care. By rapidly identifying the patients with the highest probability of death, further referral to higher levels of medical attention such as hospitalization and ICUs could be timelier. Furthermore, the present study may help with the precise allocation of patients to clinical trials as well as to benchmark medical services dedicated to kala-azar and other tropical and neglected diseases. Finally, this study can help clinical decisions by comparing the measured scores to the logical, intuitive, and qualitative clinical evaluation of individual patients and may reduce mortality from kala-azar.

ACKNOWLEDGMENTS

We thank Natan Portella Tropical Diseases Institute and medical staff for making available all of their structure for the study. We are deeply grateful to Daniela Moura for helping with the patient evaluation and to Maria Lúcia Nascimento, Christiany M. de Castro, George Luís Costa, and José Mendes de Sousa for their support in gathering hospital data.

REFERENCES

1. Desjeux P. Leishmaniasis: current situation and new perspectives. Comp Immunol Microbiol Infect Dis 2004; 27:305-318. [ Links ]

2. Collin S, Davidson R, Ritmeijer K, Keus K, Melaku Y, Kipngetich S, et al. Conflict and kala-azar: determinants of adverse outcomes of kala-azar among patients in southern Sudan. Clin Infect Dis 2004; 38:612-619. [ Links ]

3. Rey LC, Martins CV, Ribeiro HB, Lima AAM. Leishmaniose visceral americana (calazar) em crianças hospitalizadas de área endêmica. J Pediatr 2005; 81:73-78. [ Links ]

4. Oliveira ALL, Paniago AMM, Dorval MEC, Oshiro ET, Leal CR, Sanches M, et al. Foco emergente de leishmaniose visceral em Mato Grosso do Sul. Rev Soc Bras Med Trop 2006; 39:446-450. [ Links ]

5. Braga AS, Toledo Junior AC, Rabello A. Factors of poor prognosis of visceral leishmaniasis among children under 12 years of age. A retrospective monocentric study in Belo Horizonte, State of Minas Gerais, Brazil, 2001-2005. Rev Soc Bras Med Trop 2013; 46:55-59. [ Links ]

6. Coura-Vital W, Araujo VE, Reis IA, Amancio FF, Reis AB, Carneiro M. Prognostic factors and scoring system for death from visceral leishmaniasis: an historical cohort study in Brazil. PLoS Negl Trop Dis 2014; 8:e3374. [ Links ]

7. Alvar J, Vélez ID, Bern C, Herrero M, Desjeux P, Cano J, et al. Leishmaniasis worldwide and global estimates of its incidence. PloS one 2012; 7:e35671. [ Links ]

8. Bañuls AL, Bastien P, Pomares C, Arevalo J, Fisa R, Hide M. Clinical pleiomorphism in human leishmaniases, with special mention of asymptomatic infection. Clin Microbiol Infec 2011; 17:1451-1461. [ Links ]

9. Zijlstra EE, Siddig Ali M, El-Hassan AM, El-Toum IA, Satti M, Ghalib HW, et al. Kala-azar in displaced people from southern Sudan: epidemiological, clinical and therapeutic findings. Trans R Soc Trop Med Hyg 1991; 85:365-369. [ Links ]

10. Dutra M, Martinelli R, de Carvalho EM, Rodrigues LE, Brito E, Rocha H. Renal involvement in visceral leishmaniasis. Am J Kidney Dis 1985; 6:22-27. [ Links ]

11. Duarte MI, Silva MR, Goto H, Nicodemo EL, Amato Neto V. Interstitial nephritis in human kala-azar. Trans R Soc Trop Med Hyg 1983; 77:531-537. [ Links ]

12. Sundar S, Singh RK, Bimal SK, Gidwani K, Mishra A, Maurya R, et al. Comparative evaluation of parasitology and serological tests in the diagnosis of visceral leishmaniasis in India: a phase III diagnostic accuracy study. Trop Med Int Health 2007; 12:284-289. [ Links ]

13. Ready PD. Epidemiology of visceral leishmaniasis. Clin Epidemiol 2014; 6:147-154. [ Links ]

14. van Griensven J, Carrillo E, López-Vélez R, Lynen L, Moreno J. Leishmaniasis in immunosuppressed individuals. Clin Microbiol Infec 2014; 20:286-299. [ Links ]

15. Alvar J, Aparicio P, Aseffa A, Den Boer M, Cañavate C, Dedet JP, et al. The relationship between leishmaniasis and AIDS: the second 10 years. ‎Clin Microbiol Rev 2008; 21:334-359. [ Links ]

16. Lindoso JA, Cota GF, da Cruz AM, Goto H, Maia-Elkhoury AN, Romero GAS, et al. Visceral leishmaniasis and HIV coinfection in Latin America. PLoS Negl Trop Dis 2014; 8:e3136. [ Links ]

17. Verma N, Bimal S, Das Rabi VN, Pandey K, Singh D, Lal CS, et al. Clinicopathological and immunological changes in Indian post kala-azar dermal leishmaniasis (PKDL) cases in relation to treatment: a retrospective study. Biomed Res Int 2015; 2015:ID 745062. [ Links ]

18. Zijlstra EE, Musa AM, Khalil EA, el-Hassan IM, el-Hassan AM. Post-kala-azar dermal leishmaniasis. Lancet Infect Dis 2003; 3:87-98. [ Links ]

19. Martins-Melo FR, Lima MS, Ramos Jr AN, Alencar CH, Heukelbach J. Mortality and case fatality due to visceral leishmaniasis in Brazil: a nationwide analysis of epidemiology, trends and spatial patterns. PloS one 2014; 9:e93770. [ Links ]

20. Seaman J, Mercer AJ, Sondorp HE, Herwaldt BL. Epidemic visceral leishmaniasis in southern Sudan: treatment of severely debilitated patients under wartime conditions and with limited resources. Ann Intern Med 1996; 124:664-672. [ Links ]

21. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de Vigilância Epidemiológica. Leishmaniose visceral: recomendações clínicas para redução da letalidade. Brasília: Ministério da Saúde; 2011. 51p. [ Links ]

22. Ritmeijer K, Dejenie A, Assefa Y, Hundie TB, Mesure J, Boots G, et al. A comparison of miltefosine and sodium stibogluconate for treatment of visceral leishmaniasis in an Ethiopian population with high prevalence of HIV infection. Clin Infec Dis 2006; 43:357-364. [ Links ]

23. Seaman J, Pryce D, Sondorp HE, Moody A, Bryceson AD, Davidson RN. Epidemic visceral leishmaniasis in Sudan: a randomized trial of aminosidine plus sodium stibogluconate versus sodium stibogluconate alone. J Infect Dis 1993; 168:715-720. [ Links ]

24. Abdelmoula MS, M'Hamdi Z, Amri F, Tebib N, Ben Turkia H, Ben Dridi MF. La leishmaniose viscerale chez l'enfant: facteurs pronostiques. Tunis Med 2003; 8:535-539. [ Links ]

25. Mueller Y, Mbulamberi DB, Odermatt P, Hoffmann A, Loutan L, Chappuis F. Risk factors for in-hospital mortality of visceral leishmaniasis patients in eastern Uganda. Trop Med Int Health 2009; 14:910-917. [ Links ]

26. Werneck GL, Batista MS, Gomes JR, Costa DL, Costa CH. Prognostic factors for death from visceral leishmaniasis in Teresina, Brazil. Infection 2003; 31:174-177. [ Links ]

27. de Araujo VEM, Morais MHF, Reis IA, Rabello A, Carneiro M. Early clinical manifestations associated with death from visceral leishmaniasis. PLoS Negl Trop Dis 2012; 6:e1511. [ Links ]

28. Sampaio MJAQ, Cavalcanti NV, Alves JGB, Fernandes Filho MJC, Correia JB. Risk factors for death in children with visceral leishmaniasis. PLoS Negl Trop Dis 2010; 4:e877. [ Links ]

29. Belo VS, Struchiner CJ, Barbosa DS, Nascimento BW, Horta MA, da Silva ES, et al. Risk factors for adverse prognosis and death in American visceral leishmaniasis: a meta-analysis. PLoS Negl Trop Dis 2014; 8:e2982. [ Links ]

30. Schlichting D, McCollam JS. Recognizing and managing severe sepsis: a common and deadly threat. South Med J 2007; 100:594-600. [ Links ]

31. Landry DW, Bazari H. Approach to the patient with renal disease. Goldman L, Schafer AI, editors. Goldman's Cecil Medicine. 24th edition. Chapter 116. Philadelphia, PA: Saunders Elsevier; 2012. p. 728-736. [ Links ]

32. Barquet N, Domingo P, Cayla JA, Gonzalez-Martin J, Rodrigo C, Fernandez-Viladrich P, et al. Prognostic factors in meningococcal disease. Development of a bedside predictive model and scoring system. Barcelona Meningococcal Disease Surveillance Group. JAMA 1997; 278:491-496. [ Links ]

33. Hosmer DW, Lemeshow S. Asseting the fit of the model. Wiley-Interscience, editor. Applied Logistic Regression. 1st. edition. Chapter 5. New York: John Wiley & Sons; 1989. p. 135-175. [ Links ]

34. Queiroz MJA, Alves JGB, Correia JB. Leishmaniose visceral: características clínico-epidemiológicas em crianças de área endêmica. J Pediatr 2004; 80:141-146. [ Links ]

35. Costa DL, Rocha RL, Carvalho RM, Lima-Neto AS, Harhay MO, Costa CH, et al. Serum cytokines associated with severity and complications of kala-azar. Pathog Glob Health 2013; 107:78-87. [ Links ]

36. Ansari NA, Saluja S, Salotra P. Elevated levels of interferon-gamma, interleukin-10, and interleukin-6 during active disease in Indian kala azar. Clin Immunol 2006; 119:339-345. [ Links ]

37. Groux H, Cottrez F, Rouleau M, Mauze S, Antonenko S, Hurst S, et al. A transgenic model to analyze the immunoregulatory role of IL-10 secreted by antigen-presenting cells. J Immunol. 1999; 162:1723-1729. [ Links ]

38. Alves CF, de Amorim IF, Moura EP, Ribeiro RR, Alves CF, Michalick MS, et al. Expression of IFN-gamma, TNF-alpha, IL-10 and TGF-beta in lymph nodes associates with parasite load and clinical form of disease in dogs naturally infected with Leishmania (Leishmania) chagasi. Vet Immunol Immunopathol 2009; 128:349-358. [ Links ]

39. Costa CH, Werneck GL, Costa DL, Holanda TA, Aguiar GB, Carvalho AS, et al. Is severe visceral leishmaniasis a systemic inflammatory response syndrome? A case control study. Rev Soc Bras Med Trop 2010; 43:386-392. [ Links ]

40. Silva JM, Zacarias DA, de Figueirêdo LC, Soares MR, Ishikawa EA, Costa DL, et al. Bone marrow parasite burden among patients with New World kala-azar is associated with disease severity. Am J Trop Med Hyg 2014; 90:621-626. [ Links ]

41. Lima IP, Müller MC, Holanda TA, Harhay M, Costa CH, Costa DL. Human immunodeficiency virus/Leishmania infantum in the first foci of urban American visceral leishmaniasis: clinical presentation from 1994 to 2010. Rev Soc Bras Med Trop 2013; 46:156-160. [ Links ]

42. Lindoso JA, Cruz LL, Spinola RM, Fortaleza CM, Nogueira PA, Madalosso G. Fatores associados à leishmaniose visceral grave. Rev Soc Bras Med Trop 2006; 39 (supl 3):133-134. [ Links ]

43. Lomtadze ML, Khochava MA, Shalamberidze IA, Kharaishvili VI, Vorob'eva EO. Study of intravascular coagulation activation markers in patients with visceral leishmaniasis. Georgian Med News 2005; 124-125:47-50. [ Links ]

44. Ministério da Saúde. Secretaria de Atenção à Saúde. Departamento de Atenção Hospitalar. E de Urgência. Guia para uso de hemocomponentes. Brasília: Ministério da Saúde; 2010. 140p. [ Links ]

45. National Guideline Clearinghouse (NGC). Guideline for platelet transfusion thresholds for pediatric hematology/oncology patients. Rockville MD: Agency for Healthcare Research and Quality (AHRQ); 2011. 55p. Available from https://www.guideline.gov/content.aspx?id=34608&search=platelet+transfusionLinks ]

46. National Guideline Clearinghouse (NGC) - Blood transfusion. Rockville MD: Agency for Healthcare Research and Quality (AHRQ) ; 2011. 90p. Available from http://www.guideline.gov/content.aspx?id=39340&search=thrombocytopenia+bleeding. [ Links ]

47. Martí-Carvajal AJ, Simancas D, Cardona AF. Treatment for disseminated intravascular coagulation in patients with acute and chronic leukemia. Cochrane Database Syst Rev 2015; 6:CD008562. [ Links ]

48. Wardrop D, Estcourt LJ, Brunskill SJ, Doree C, Trivella M, Stanworth S, et al. Antifibrinolytics (lysine analogues) for the prevention of bleeding in patients with haematological disorders. Cochrane Database Syst Rev 2013; 7:CD009733. [ Links ]

49. Balk RA. Systemic inflammatory response syndrome (SIRS): where did it come from and is it still relevant today? Virulence 2014; 5:20-26. [ Links ]

50. Binkowska AM, Michalak G, Slotwinski R. Current views on the mechanisms of immune responses to trauma and infection. Cent Eur J Immunol 2015; 40:206-216. [ Links ]

51. Ministério da Saúde. Secretaria de Vigilância em Saúde. Departamento de Vigilância Epidemiológica. Manual de vigilância e controle da leishmaniose visceral. Brasília: Ministério da Saúde; 2003. 120p. [ Links ]

52. Bone RC, Balk RA, Cerra FB, Dellinger RP, Fein AM, Knaus WA, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992; 101:1644-1655. [ Links ]

53. Andrade TM, Carvalho EM, Rocha H. Bacterial infections in patients with visceral leishmaniasis. J Infect Dis 1990; 162:1354-1359. [ Links ]

54. Garcés JM, Tomás S, Rubiés-Prat J, Gimeno JL, Drobnic L. Bacterial infection as a presenting manifestation of visceral leishmaniasis. Rev Infect Dis 1990; 12:518-519. [ Links ]

55. Guerreiro J, Ribeiro S, Carvalho EM, Badaro R, Rocha H. Infecção bacteriana em pacientes portadores de leishmaniose visceral. Mem Inst Oswaldo Cruz 1985; 80:447-452. [ Links ]

This work was funded by the National Counsel of Technological and Scientific Development [Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)] of Brazil, grants 142046/2005-1 and 554797/2006-2.

Received: August 17, 2016; Accepted: November 09, 2016

Corresponding author : Dra. Dorcas Lamounier Costa. e-mail: dorcas.lc@gmail.com

The authors declare that they have no conflict of interest

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License