Evaluation of the performance of three serological tests for diagnosis of Leishmania infantum infection in dogs using latent class analysis

of the performance diagnosis svs@unizar.es Abstract Canine leishmaniasis (CanL) is a disease caused by Leishmania infantum . Serological methods are the most common diagnostic techniques used for the diagnosis of the CanL. The objective of our study was to estimate the sensitivity and specificity of one in-house ELISA kit (ELISA UNIZAR) and three commercially available serological tests (MEGACOR Diagnostik GmbH) including an immunochromatographic rapid test ( FASTest LEISH®), an immunofluorescent antibody test (MegaFLUO LEISH®) and an enzyme-linked immunosorbent assay (MegaELISA LEISH®), using latent class models in a Bayesian analysis. Two hundred fifteen serum samples were included. The highest sensitivity was achieved for FAST est LEISH® (99.38%), ELISA UNIZAR (99.37%), MegaFLUO LEISH® (99.36%) followed by MegaELISA LEISH® (98.49%). The best specificity was obtained by FAST est LEISH® (98.43%), followed by ELISA UNIZAR (97.50%), whilst MegaFLUO LEISH® and MegaELISA LEISH® obtained the lower specificity (91.94% and 91.93%, respectively). The results of present study indicate that the immunochromatographic rapid test evaluated FAST est LEISH® show similar levels of sensitivity and specificity to the quantitative commercial tests. Among quantitative serological tests, sensitivity and specificity were similar considering ELISA or IFAT substrate buffer (Thermo were added (100 µL per well) and developed for 20±5 min at room temperature in the dark. The reaction was terminated by adding 100 µL of 2.5 M H2SO4 to each well. Absorbance values were read at 492 nm (reference wavelength) in an automatic microELISA reader (ELISA Reader Labsystems Multiskan, Midland, Canada). Each plate included serum samples from a dog infected with L. infantum as confirmed by cytological examination as a positive control (calibrator) and serum samples from a healthy, non infected dog from the blood donor program as a negative control. The same calibrator serum sample was used for all assays, and the plates with an interassay variation greater than 10% were tested again. All samples and controls were analyzed in duplicate. The ELISA UNIZAR test was performed by a different researcher who had no knowledge of the rapid test, MegaFLUO LEISH® results and MegaELISA LEISH® results. The results were quantified as ELISA UNIT (EU) compared to a positive control serum sample used as a calibrator that was arbitrarily set to 100 EU. The cutoff value was set to 30 EU (mean+4 standard deviations of values from 70 apparently healthy dogs from a non-endemic area and that were not included in this study). Sera with an EU ≥ 200 were classified as high positive, with an EU ≥ 100 and < 200 as moderate positive, and with an EU > 30 and < 100 as low positive. Various confirmatory methods for evaluating L. infantum infection exist, but their performances differ significantly.


Introduction
CanL is a vector-borne disease caused by Leishmania infantum, which dogs are considered the main domestic reservoir for human infection (Dantas-Torres, 2007). This infection is transmitted by the bite of infected female sand flies from the genera Phlebotomus in the Old World or Lutzomyia in the New World (Moreno & Alvar, 2002). In the Mediterranean basin, an estimation of 2.5 million dogs are infected by the parasite including subclinically infected dogs and sick dogs exhibiting clinical signs and/or clinicopathological abnormalities (Solano-Gallego et al., 2009). Cases of subclinical infections, defined as a situation in which Leishmania infection is confirmed but clinical signs and/or clinicopathological abnormalities are not present have been documented in all areas where CanL is endemic. It is no longer believed that absolutely every infected dog will inevitably develop clinical leishmaniasis (Solano-Gallego et al., 2009). In fact, a small proportion (prevalence of the disease ranges between 3 and 10%) of the dogs infected with L. infantum in endemic regions will develop the disease following infection . The evolution of this chronic infection depends on the cell-mediated immune response from the host against the parasite, cellular protective immunity is associated with activation of the macrophages by different cytokines, particularly interferon-gamma, tumor necrosis factor-alpha and interleukin-2. By contrast, disease susceptibility, by releasing a mixed T helper (Th1 and Th2) lymphocyte cytokines, tend to promote non-protective antibody formation and correlate with a diminution or absent cell-mediated immunity (Hosein et al., 2017). In this sense, the number of circulating CD4+ cells and the CD4+/CD8+ ratio drop during the disease (Esch et al., 2013). In the case of CD8+, these T cells are responsible for lysis of the macrophages infected with L. infantum and their activation increases the synthesis of INF-γ and TNF-α (Barbiéri, 2006). Different techniques to confirm L. infantum infection are available including parasitological methods with the direct identification and observation of the parasite such as cytology, histology, specific immunohistochemistry and the parasite culture; molecular methods to detect parasitic nucleic acids (DNA and RNA) by polymerase chain reaction (PCR) such as conventional PCR, nested PCR and quantitative PCR; and serological methods based on detecting specific antibodies response against L. infantum (Maia & Campino, 2018). Serology is the preferred diagnostic method for CanL considering information provided by the World Organisation for Animal Health (OIE, 2016). Among serological methods, enzyme-linked immunosorbent assay (ELISA), indirect immunofluorescence antibody test (IFAT), and the immunochromatographic rapid test (ICT) represent the most frequent methods used for the detection of infection in dogs (Maia & Campino, 2008;Bourdeau et al., 2014).
For L. infantum infection there is no perfect diagnosis but the diagnosis must be appropriate for each situation. The gold standard test does not necessarily have 100% sensitivity (Se) and 100% specificity (Sp) but it is the most sensitive and specific test for diagnosing that infectious agent. In absence of a reference test, the Latent Class Analysis (LCA) allows the estimation of sensitivity and specificity of two or more tests without assuming the true antibody status of the population under study, avoiding the bias connected to the use of an imperfect test (Branscum et al., 2005). The aim of this study was the statistically evaluation of the Se and Sp of one in-house ELISA test in comparison with three different commercial serologic tests in two groups of canine serum samples, one from non-endemic areas, the other from endemic areas, assuming significant different epidemiological conditions and seroprevalences between the two groups by a latent class models in a Bayesian analysis.

Study population and serum samples
A total of 215 serum samples from dogs admitted to the Veterinary Teaching Hospital of the University of Zaragoza (Spain) and from three different clinics from United Kingdom were used in this study. Serum samples were collected during the period from January 2017 to December 2018 and conserved at -25 °C until analyzed.
With the aim to evaluate the performances of the tests by means of Bayesian analysis, samples were stratified into two populations with different levels of prevalence: canine sera from a non-endemic area (group 1) and from an endemic area (group 2).
One hundred and ninety-five serum samples (group 2) came from The Ebro´s Valley, a Spanish region where CanL is endemic, and the remaining serum samples (group 1) were residual samples taken from dogs during a routine annual health check-up in the United Kingdom, a non-endemic area. Within group 2, each dog was classified in different subgroups (non-infected dogs, healthy seronegative dogs, infected seropositive dogs, clinically sick dogs and finally dogs with serological positive result to other pathogens) according to the clinical information sent with the sample to the laboratory for diagnostic purpose. The subgroups were not considered for the statistical evaluation. Serum samples selection have reflected different clinical settings in veterinary practice from diagnosis of L. infantum infection in healthy dogs to clinically sick dogs, based on clinical evaluation, routine red blood cell count, clinical chemistry, urinalysis and serum protein electrophoresis. The serology status was also routinely recorded by means of an in-house ELISA (Solano-Gallego et al., 2009), but this test had the only goal to assure the selection of a heterogeneous population and did not influence the statistical analysis.
In the case of dogs with serological positive result to other pathogens (cross-reaction group), serology was used together with a molecular test to detect L. infantum infection performed in a private laboratory. None of the samples used in the study came from dogs previously vaccinated with any of the two vaccines available in Spain to prevent CanL .
Clinical and epidemiological data were considered for the selection and samples were collected for the sole intention of determining a diagnosis. Ethical approval was not needed, but owners were requested to sign an informed consent. The study was reported to the Bioethical Committee of the University of Zaragoza and conducted in accordance with the European (2010/63/UE) and national (RD1201/2005) directives on animal experimentation. This study did not require official or institutional ethical approval.
Non-endemic canine sera (group 1) Twenty serum samples obtained from The United Kingdom were included. These dogs had never traveled to an endemic area and they had neither clinical signs nor laboratory abnormalities detected by routine red blood cell count, clinical chemistry, urinalysis and a serum protein electrophoresis.

Endemic canine sera (group 2)
A total of 195 sera were collected from dogs living in a leishmaniasis endemic area. To obtain a heterogeneous group of samples, some subgroups were considered: clinically-ill infected dogs, dogs infected by some other pathogens and seemingly healthy dogs that resulted seropositive or seronegative. The heterogeneity was based mainly on clinical evaluation and routine red blood cell count, clinical chemistry, urinalysis and serum protein electrophoresis. The serology status was also recorded, based on ELISA UNIZAR, but this classification did not have any influence on statistical evaluation, based only on the two groups 1 (non-endemic) and 2 (endemic).
Sixty-five serum samples from seronegative healthy dogs were obtained from the Veterinary Teaching Hospital -Zaragoza Veterinary Faculty (University of Zaragoza, Spain). These samples came from dogs that were taken to Hospital for a screening L. infantum infection purpose by quantitative serology. Clinical information showed absence of clinical signs during physical examination and no laboratory findings based on routine red blood cell count and clinical chemistry, serum protein electrophoresis and urine analysis.
Twenty-four sera samples from dogs evaluated out of seasonal activity of sand flies with a low positive result to the ELISA UNIZAR were included. Clinical information showed absence of clinical signs during physical examination and no laboratory findings being classified as seropositive asymptomatic infected dogs. All dogs were negative to L. infantum infection using a quantitative PCR in blood.

ELISA UNIZAR technique
Prior to performing the in-house ELISA, L. infantum antigen (strain MHOM/FR/78/LEM 75 belonging to L. infantum zimodeme MON-1) was obtained from parasite culture. For the ELISA UNIZAR, the crude antigen was adjusted to a concentration of 20 µg/mL with sterile phosphate buffered saline (PBS). Each plate was coated lightly with 100 µL/well of the 20 µg/mL antigen solution in 0.1 M carbonate/bicarbonate buffer (pH 9.6) and incubated overnight at 4 °C. Plates were then frozen and stored at -20 °C. One hundred microliters of dog serum, diluted 1:800 in phosphate buffered saline containing 0.05% Tween 20 (PBST) and 1% dry skimmed milk (PBST-M) were added to each well. The plates were incubated for 1 hour (h) at 37 °C in a moist chamber. After washing the plates three times with PBST for 3 minutes (min) followed by one wash with PBS for 1 min, 100 µL of Protein A conjugated to horseradish peroxidase (Thermo Fisher Scientific, Waltham, Massachusetts, USA) diluted 1:20000 in PBST-M was added to each well. The plates were incubated for 1 h at 37 °C in a moist chamber, followed by washes with PBST and PBS as described above. The substrate solution (ortho-phenylene-diamine) and stable peroxide substrate buffer (Thermo Fisher Scientific, Waltham, Massachusetts, USA) were added (100 µL per well) and developed for 20±5 min at room temperature in the dark. The reaction was terminated by adding 100 µL of 2.5 M H2SO4 to each well. Absorbance values were read at 492 nm (reference wavelength) in an automatic microELISA reader (ELISA Reader Labsystems Multiskan, Midland, Canada). Each plate included serum samples from a dog infected with L. infantum as confirmed by cytological examination as a positive control (calibrator) and serum samples from a healthy, non infected dog from the blood donor program as a negative control. The same calibrator serum sample was used for all assays, and the plates with an interassay variation greater than 10% were tested again. All samples and controls were analyzed in duplicate. The ELISA UNIZAR test was performed by a different researcher who had no knowledge of the rapid test, MegaFLUO LEISH® results and MegaELISA LEISH® results. The results were quantified as ELISA UNIT (EU) compared to a positive control serum sample used as a calibrator that was arbitrarily set to 100 EU. The cutoff value was set to 30 EU (mean+4 standard deviations of values from 70 apparently healthy dogs from a non-endemic area and that were not included in this study). Sera with an EU ≥ 200 were classified as high positive, with an EU ≥ 100 and < 200 as moderate positive, and with an EU > 30 and < 100 as low positive.

MegaFLUO LEISH®
The commercial IFAT test (MegaFLUO LEISH®, MEGACOR Diagnostik GmbH, Hörbranz, Austria) was performed as described in the instructions supplied with the test kit. Slides were examined under a fluorescence microscope (Leica DM750 RH; Leica Microsystems, Wetzlar, Germany) at 400× magnification and each well was compared to the fluorescence pattern observed in the positive and negative controls. All samples were examined by two different investigators. If discrepancies arose between results, a third observer participated. They were blinded to the results of the other serological tests.

MegaELISA LEISH®
The MegaELISA LEISH® test (MEGACOR Diagnostik GmbH, Hörbranz, Austria) is a quantitative indirect ELISA for the detection of anti-Leishmania infantum antibodies in the canine serum sample following instructions supplied with the test kit. The MegaELISA LEISH® test was performed by a different researcher without knowledge of the FASTest LEISH® test, the commercial MegaFLUO LEISH® and the ELISA UNIZAR results.

FASTest LEISH®
The ICT FASTest LEISH® test (MEGACOR Diagnostik GmbH, Hörbranz, Austria) is a qualitative serological test developed to detect canine antibodies against L. infantum based on immunochromatographic technology. All tests were stored at room temperature and were performed as described in the instructions supplied with the test kit. The examiner was blinded to the results of the quantitative serological tests.

Statistical analysis
The Bayesian version (Branscum et al., 2005) of the LCA introduced by Hui & Walter (1980) was adopted for evaluate the accuracy of the four diagnostic tests, given the absence of a gold standard. A four tests -two populations model was built in order to estimate the Se and Sp of each diagnostic test. The model assumptions in the Hui & Walter (1980) version were (1) the different true prevalences of the two populations, (2) the Se and Sp of the tests were constant across subpopulations and (3) the tests were conditionally independent given the true infection status. Accounting for the first assumption, the model was run using endemic versus non-endemic area as subgroups. Additionally, different hypotheses about the dependence among tests were considered in the model building: (Toft et al., 2005) the conditional independence among all the tests, given the infection status (i.e. presence of antibodies against L. infantum), MOD 1 in Table 1; the conditional covariance between the ELISA UNIZAR and MegaELISA LEISH® tests and conditional independence among FASTest LEISH® and MegaFLUO LEISH® compared to ELISA UNIZAR and MegaELISA LEISH®, given the disease status (MOD 2). For both models, uninformative priors for the test accuracy and prevalences were used. All analyses were carried out using Markov chain Monte Carlo techniques and implemented in WinBUGS software. Posterior inferences were based on 50,000 iterations, after a burn-in of 10,000 iterations. Convergence was assessed by running multiple chains from dispersed starting values, observing autocorrelation among samplings and investigating the Brooks-Gelman-Rubin convergence statistic (Brooks & Gelman, 1998). The Deviance Information Criterion (DIC) was used as measure of the model fitting (smaller value is better). The median of the posterior distributions was used as an estimate for the parameters of interest; the 2.5 and 97.5% points were used as estimates of the 95% credibility intervals (95% Posterior credibility interval, 95% PCI). To test Se and Sp between tests, the Bayesian posterior probabilities (POPR) were calculated and used as hypothesis testing, like in traditional statistical methods.

Results
The cross-tabulated counts of the raw test results are reported in the Table 2 for endemic area and non-endemic area including one of the 16 different test patterns (e.g. combination of results of the four serological tests detected in each group of dogs). 195 samples out of 215 (90.7%) showed concordant results for all tests included.
The posterior estimates (median and 95% PCI) are presented in Table 1. Looking at Deviance Information Criterion (DIC) (smaller is better), it seems that the model with conditional independence among all tests (MOD 1) would be the preferable one between the two models under evaluation. However, there were no detectable differences in posterior estimates from the two models. The results highlighted a very low prevalence for non-endemic area (3.25%) compared to the medium/high in the endemic area (55.85%). All tests showed high values of Se, near to 100% for FASTest LEISH® (99.38%), ELISA UNIZAR (99.37%) and MegaFLUO LEISH® (99.36%), followed by MegaELISA LEISH® (98.49%). With regard to the Sp, FASTest LEISH® (98.43%) and ELISA UNIZAR (97.50%) showed significantly better performances (POPR < 0.05) compared to MegaFLUO LEISH® (91.94%) and MegaELISA LEISH® (91.93%).

Discussion
The absence of a gold standard technique is a recurrent situation in clinical practice and diagnostic research studies including confirmatory techniques for L. infantum infection in dogs (Rodríguez-Cortés et al., 2010), cats (Persichetti et al., 2017), ferrets (Giner et al., 2020) and humans (Galluzzi et al., 2018). Although it is usual, the application of two or more tests combination based on different principles such as molecular, serological or other parasitological techniques when there is no reference standard. Nevertheless, classical validation is based on the use of a reference test such as IFAT technique to evaluate the other index test (OIE, 2016). The estimates of Se and Sp of the index test are biased because the reference test could have high Se and high Sp but not necessarily equal to 100%. This latter bias could be resolved using statistical validation based on latent class models in a Bayesian analysis. Despite the large number of articles focused on confirmatory techniques validation for diagnosis of leishmaniasis, very few studies using a Bayesian statistical approach have been published in dogs (Adel et al., 2016), cats (Persichetti et al., 2017) and other animals such as lagomorphs (De la Cruz et al., 2016).
In the study, Se value was equal for MegaFLUO LEISH®, FASTest LEISH® and the ELISA UNIZAR, obtaining a Se approximately equal to 99.40%, whilst MegaELISA LEISH® was the commercial test with less sensitivity in comparison to the others but its sensitivity was high (98.49%). In contrast, the highest Sp was obtained by the FASTest LEISH® (98.43%), in comparison to the quantitative tests: ELISA UNIZAR (97.50%), followed by MegaELISA LEISH® and MegaFLUO LEISH® (91.9%).
Various confirmatory methods for evaluating L. infantum infection exist, but their performances differ significantly. In vitro parasite cultures are laborious, require special facilities, and are limited to research. In contrast, histology, immunohistochemistry, molecular methods and quantitative serological methods are frequently applied in clinical practice, but samples often need to be sent to a specialized laboratory to perform these techniques (Miró et al., 2008).
The group of dogs coming from the endemic area was heterogeneously composed by clinically-ill dogs and apparently healthy dogs, as it is well known that the diagnosis in clinically-ill subjects is easy to obtain with perfect performances by means of any diagnostic test. Among the seropositive asymptomatic infected dogs, some had a previous serological diagnosis of positivity for Leishmania, some other did not. In any case, the classification into subgroups did not have any weight in the Bayesian evaluation. The apparently healthy seropositive dogs and clinically-ill infected dogs had different antibody levels ranging from 30 to 372 EU, according to the ELISA UNIZAR and in the case of MegaELISA LEISH® antibody levels ranging from 1 to 84 MEGACOR EU. In both ELISAs the antigen used is sonicated promastigote protein. However, small differences between the same serological ELISA technique were detected between ELISA UNIZAR and MegaELISA LEISH®. Conversely, antibody titers detected by MegaFLUO LEISH® ranged from 1:20 to > 1:12800. The main differences between the ELISA and IFAT techniques are the type of antigen used and the technical method performed to obtain the results. For IFAT, the entire parasite is present on the slide, and the evaluation is performed with a fluorescence microscope. In contrast, the ELISA technique uses different types of antigens, and the results are obtained by measuring the absorbance with an ELISA plate reader. In terms of interpreting the results, IFAT is subjective and depends on the operator's experience, even when two different experienced observers examine the samples. Intrinsic analytical variability of each quantitative serological technique may explain the differences between results obtained in the present study. In comparison to other studies where ELISA techniques showed better performances compared with in-house IFAT technique, in our case the Se and Sp of the MegaFLUO LEISH®, were similar to the other tests analyzed in the present study.
In-clinic tests, cytological examinations and rapid serological tests are the most rapid and cheap methods for detecting Leishmania infection in dogs. Microscopic examinations of samples, including bone marrow, lymph nodes, skin lesions and body fluids, have been used. Bone marrow and lymph nodes are considered the best samples to confirm an infection, but microscopy may not be as sensitive and is also time consuming due to the low numbers and randomly distributed parasites (Saridomichelakis et al., 2005;Moreira et al., 2007).
In Europe, there are a limited number of commercially available ICTs for CanL with diagnostic performance published such as SNAP® Canine Leishmania Antibody Test, Speed Leish K®, INGEZIM® LEISHMACROM and WITNESS® Leishmania (Rodríguez-Cortés et al., 2010). Diagnostic performance of ICTs shows high specificity and variable sensitivity from low to moderate degree depending on the test evaluated. In our study, the Sp of FASTest LEISH® was slightly lower (98.4%) compared to the Sp (100.0%) of other ICTs. On the other hand, the Se of FASTest LEISH® was clearly higher in comparison to other tests such as WITNESS® Leishmania with a low Se (58.0%) or INGEZIM® LEISHMACROM with a moderate Se (75.0%). Nevertheless, the absence of a common framework makes a correct comparison difficult in relation to the standardization of performance between ICTs.
When a test is available to use in a clinical setting, scientific information that is not obtained from the manufacturer should be collected. A recent study provided additional information about diagnostic measures of other rapid tests that was not included in the technical details (Solano-Gallego et al., 2014). The study (Solano-Gallego et al., 2014) showed important variations in the Se parameter between the technical set up and the rapid test results. A Se of 63.6% was obtained for the Speed Leish K® test, which was not consistent with the higher Se indicated in the manufacturer's instructions. However, the test performance of FASTest LEISH® obtained in the present study was similar to previously reported diagnostic performance results supplied with the kit.
Differences in diagnostic performance are described between quantitative serological tests and ICTs evaluated simultaneously with a better performance in favour of quantitative serological tests. In this study, Se and Sp were similar between FASTest LEISH® and the ELISA UNIZAR used but this situation was not similar in comparison to the Sp value detected between FASTest LEISH® (0.984) and the two quantitative commercial tests (0.919). This FASTest LEISH® may be a valid alternative in the absence of a quantitative test as a screening test.
One of the great difficulties in the evaluation of diagnostic tests is the lack of standardization of the clinical classification of the animal with respect to L. infantum infection. Different classifications and stages of disease have been proposed in CanL. However, there are notable differences when it comes to how a dog is classified following one or another clinical classification. This added difficulty still complicates the definition of sample selection and the possible bias can be established (Meléndez-Lazo et al., 2018). In the present study the Bayesian method, based on groups selected only on the basis of endemic/not endemic area, avoided this possible bias.
Sample selection should be included representative and different stage of infection status regarding Leishmania infection that can be found when serological tests are evaluated for detecting L. infantum infection. In our study, all three serological tests discriminate samples provided from clinically-ill infected dogs and samples from nonendemic areas. The main problem from the point of view of the application of serological techniques in a clinical context is the Se of the serology is lowest early in Leishmania infection but high with progressive infection, and this situation is not always included in the evaluation of serological techniques making it crucial that this type of samples should be tested. In our study, for a better characterization of this type of dog, an additional confirmatory technique including a quantitative PCR in blood was performed. Although blood is not the best samples selection for detecting parasitic DNA in comparison to bone marrow or lymph node samples, the procedure to obtain these two different samples is painful and difficult to be accepted by the owner when the dog is apparently healthy without clinical signs and laboratory alterations.
A potential limitation of similar studies could be associated with the evaluation of the cross reactivity phenomenon that it is described commonly in serological techniques (Krawczak et al., 2015). We have evaluated an entire gamma of serum samples with a serological positive result to most frequent pathogens present in our geographical area. The inclusion of serum samples positive to other type of Leishmania species or Trypanosoma cruzi would be desirable but our region like other regions of the European Mediterranean basin, L. infantum is the common parasite and these geographical regions are free of T. cruzi. The analysis of serum samples from different regions where other pathogens are present can confirm the potential use of these commercial in other regions aside from Europe. In any case, as above clarified, in this study the Bayesian approach overtakes the meaning of the real status of every single sample, having the only goal to estimate the performances of the tests.

Conclusions
Latent class analysis provides a useable alternative to traditional test evaluations for serological tests for L. infantum infection in dogs, overtaking the needing of a gold standard test or samples with a known serological status. Using LCA we have estimated the sensitivity and specificity of the three different serological tests used in the European Mediterranean area to detect anti-Leishmania antibodies. The results of present study indicate that the immunochromatographic rapid test evaluated FASTest LEISH® shows similar levels of sensitivity and specificity to the quantitative commercial tests included.