Rapid Tests for Diagnosing Syphilis: Validation in an Std Clinic in the Amazon Region, Brazil

Correct, early diagnosis and treatment of syphilis are essential for its control. Traditional diagnostic tests depend on specialized equipment , installations, and human resources. In the search for quick, simple tests, a project was conducted on the validation and reproducibil-ity of four different tests, previously assessed by WHO reference laboratories. The study also verified the operational characteristics and acceptance by patients and health professionals. Samples obtained at an STD clinic were from 541 and 248 patients with 51 and 52 positive results according to FTA-Abs (gold standard) in studies 1 and 2, respectively. The sensitivity varied from 84 to 96%, specificity was greater than 98%, and PPV was > 90%. Reproducibility was > 97% and kappa index 0.94, comparing the results obtained by different health workers. The tests took less than 20 minutes to perform, and more than 90% of patients agreed to wait up to two hours for the results. The tests presented the necessary requirements for use in diagnosis of syphilis, thus providing an additional option for controlling this disease.


Introduction
Syphilis is one of the primary causes of adverse events during pregnancy, aside from being one of the most prevalent sexually transmitted diseases (STD) 1 .It has a global distribution, but it affects developing countries in Africa, Asia, Latin America, and the Caribbean with greater intensity.The World Health Organization (WHO) estimated that in 1999 there were 12 million new cases of syphilis of which more than 75% of them reported in poor countries, with a tendency of continuous increasing in the last years 2,3,4,5,6,7 .
Fetal deaths and morbidity through congenital syphilis can be prevented if identified and treated adequately in the mothers infected before the beginning of the third trimester.Nevertheless, unfortunately, the majority of pregnant women infected by syphilis are asymptomatic and it is only possible to identify them if they are included in programs of prevention and control and perform serological tests.
In the case of adults, the algorithms designed to manage genital ulcer syndrome include syphilis as one of the most probable causes, making possible a proper treatment of this pathology.Meanwhile, most of the time, patients are totally asymptomatic and, not being diagnosed correctly, are not treated, and have important complications related to gestation and childbirth, facilitating the sexual transmission of HIV 8,9 .
Many developed countries have established tracking activities in their syphilis control programs.They have been using tests called nontreponemal tests, like the Venereal Disease Research Laboratory (VDRL) and the Rapid Plasma Reagent test (RPR).Aside from being easy to carry out, their low cost, and their relatively rapid results, they cannot be applied in all the country's primary health care units in virtue of the fact they require refrigeration or other laboratory facilities like electricity, or a centrifuge, for example.Together with these factors, when these tests are employed, principally on pregnant women, up to 28% percent of positive results are biologically false reactions and.even in ideal conditions require additional tests of greater specificity (treponemal tests, such as the fluorescent treponemal antibody absorption test-FTA-Abs, the microhemagglutination test and hemoagglutination test for Treponema pallidum antibodies-TPHA, etc.) 1,10 .
In Brazil, there has been an investment in the scale-up of the primary health care network by means of the implementation of the Family Health Program (FHP) and the work of community health agents, and an ample network of diagnostic laboratories has been installed.Even so, the distribution of these services is unequal, and reflects the different realities of regions and subregions.
For these reason, the Special Programme for Research and Training in Tropical Disease (TDR)/Sexually Transmitted Diseases Diagnostics Initiative (SDI) of the WHO stimulates the search for new tests for diagnosing syphilis that comply with the necessary requisites: rapid results (less than 15 minutes), ease of use by professionals who work directly with the patients, not requiring the resources of traditional laboratories, being stable at room temperature, possessing good sensitivity and specificity, and low cost.Thus they can be used on large scale in primary health care facilities in developing countries to adequately identify and treat the greatest number of infected people.
More than twenty commercially available rapid tests exist 11,12,13,14 .The SDI program first selected six of the most promising ones for evaluation of their performance, utilizing serum banks in eight countries (the United States, Russia, China, Sri Lanka, Tanzania, Gambia, South Africa, and Haiti).In this research 789 samples were used, and sensitivity values of 85-98%, and a specificity of 93-98% were obtained, when they were compared with test of hemogglutination and agglutination in particles against T. pallidum (TPHA and TPPA) as a gold standard 15 .
Afterwards, in the year 2003, four different sites in Asia (China), Africa (South Africa), and America (Haiti and Brazil) were selected to realize and validate their operational characteristics and acceptance by the patients and health professionals that work with the care of patients living with STD.For this stage, four rapid tests were selected, fundamentally based on their common characteristics, of using whole blood samples, serum or plasma, and not requiring refrigeration.The validation of the four rapid tests at a specialized clinic in the city of Manaus, Amazonas State, Brazil, is discussed in this article.
The objective of this study was to evaluate the operational characteristics (validation and reproducibility) of four diagnostic tests proposed, under the denomination of "rapid tests for syphilis".Also evaluated were the feasibility and acceptability of their use by health professionals who work directly in caring for cases of suspected syphilis or other STD, as well as for the possibility of this clientele to get the results of the test before terminating the consultation.

Comparative validation research of four rapid treponemic tests for the diagnosis of syphilis
Whole blood samples from patients who presented themselves in a consecutive manner at a clinic specialized in STD in Manaus were used, and serum samples from the same patients in the laboratory.The FTA-Abs test was used as a "gold standard".
During the research all the patients were treated based on the results of routine clinical exams (VDRL and FTA-Abs).
The validation stage activities were carried out at four different sites: Asia (China), Africa (South Africa), and America (Haiti and Brazil).The following phases were carried out at a specialized clinic in Manaus: • The investigation was divided into two stages, because the tests were delivered at different times.In the first part the Syphicheck-WB and the SD Biolina Syphilis tests were validated (study period 1).Afterwards the tests of VisiTect Syphilis and Determine Syphilis TP were completed (study period 2).
• The participating patients were recruited from among those who presented themselves at the specialized STD clinic in a voluntary and consecutive manner, starting on March 1, 2003, being randomly selected for study 1, and starting in on January 15, 2004 for study 2.
• All patients signed an Informed Consent Form stating that their participation in the research was voluntary.
• Criteria for inclusion were men and women over 18 years old with no previous history of syphilis, while criteria for exclusion were those under 18 years old and/or with previous history of syphilis or positive serology for syphilis.
• Two teams were prepared for the execution of the exams: (a) professionals who work directly providing health care in clinics (nurses and paramedics), and (b) biochemists and regular field laboratory technicians.
• From each participant in the studies 10ml of venous blood was drawn, in vacutainer tubes.From this total 1ml was immediately used for evaluating the rapid tests in the clinic, while the rest was sent to the laboratory for centrifuging and the execution of the same rapid tests by the laboratory team, in the same way that is done when carrying out routine tests (VDRL, serum anti-HIV etc.) and the gold standard exam (FTA-Abs).An aliquot of 2ml was stored in a freezer at -70ºC for future tests and quality control to be carried out by the reference laboratories of SDI/ WHO in all samples that tested positive, and 10% of those that tested negative.
• Identification, epidemiological data, as well as test results from each patient were recorded on a form and double entered into a database using the Epi Info 6.4 software (CDC; Centers for Disease Control and Prevention, Atlanta, U.S.A.).
• The sample size was determined through multicentric protocols, according to each site characteristics, and had to incorporate a number of patients large enough to reach fifty positive patients using the gold standard test (FTA-Abs).In this study project, the sample was 541 people for study 1, and 248 for study 2. A possible explanation for the difference in sample size was the fact that the period when study 2 was being carried out coincided with an increment of sex workers seeking treatment.
• The samples of studies 1 and 2 were analyzed to verify if they were able to minimize the random error.It is known that the appropriate sample size for evaluating a specific test is determined by the formula as follows 16 : N = Z 2 [p (1-P)]/D 2 and this applied to the syphilis prevalence obtained in the patient group (9.4% and 21.1%) has enough power to arrive at a maximum acceptable error of 3% and 5% in each of the samples, respectively.
• For validation of the tests, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated with their respective confidence interval of 95% (95%CI).To evaluate the reproducibility of each test, global agreement rates and kappa indices were employed.To investigate the existence of statistically significant differences between the validation results for each test, chi-square tests for proportional differences were carried out (with Yates correction).
• To get opinions about the operational qualities of each test an opinion questionnaire was created and filled out by the 13 professionals responsible for the execution of rapid test evaluations.

Results
The validation of the tests was carried out in two different time stages in distinct samples of patients, each of them used to evaluate a pair of tests.The comparison of the validation results, therefore, cannot be extended to the four tests together and only can be referred to by separate pairs, denominated studies 1 and 2. The same professionals worked on all four tests,minimizing statistical bias.
In the first study to validate the tests Syphicheck-WB and SD Bioline Syphilis, a total of 541 consecutive patients were studied during the period from March 1 through June 14, 2003 until 51 cases had tested positive for FTA-Abs.In the second study to validate the tests VisiTect Syphilis and Determine Syphilis TP a total of 248 consecutive patients were studied from January 15 through June 25, 2004 until 52 cases had tested positive for FTA-Abs.
The principal characteristics of the patients in the two samples studied are presented in Tables 1 and 2.
The epidemiological variables of the patients included in the samples under analysis and the laboratory results of the two studies were recorded on registration forms and in the laboratory ledgers.
The reproducibility of the tests, when done by the project teams (patient care team and laboratory team) were evaluated through the percentage of agreement and the kappa coefficient.For the purpose of this study when these indicators had values of more than 0.80 the reproducibility was considered good and when they were greater than 90% and 0.90, as excellent.
Table 2 shows the prevalences of syphilis according to the principal reasons for patient consultation.In the two groups studied, suspicion of latent syphilis and patients directed to the service under suspicion of secondary syphilis presented the highest prevalence values: 66.6% and 66.6%, 76.9% and 62.5%, respectively.Altogether, all the groups included had high or moderately syphilis prevalence rates, 9.4% in the first period and 21% in the second, as was to be expected at a STD specialized clinic.
The performance of the four tests, relative to their principal validation indicators are shown in Table 3.However, as was explained earlier, these results can only be analyzed independently in relation to the two pairs that were analyzed each time (study 1 and study 2).
In both the studies, the four rapid tests were compared with the FTA-Abs and the VDRL (routinely employed in health service).The tests carried out by the professionals working in the clinic were compared to those carried out in the laboratory by professionals more experienced in this type of work (Table 3).
The validation of sensitivity compared to FTA-Abs showed that in study 1 SD Bioline Syphilis (88.2% and 90%) had slightly higher values than Syphicheck-WB (84.3% and 90%) whether they were performed by the clinical professionals or by the laboratory professionals.However the 95CI% intervals of both tests overlap and, therefore, it cannot be stated that the sensitivity of one test is higher than the sensitivity of the second one.The specificity of the two rapid tests was similar and, in general, with very high values, higher than 98% and the VPP showed a slightly better performance of Syphicheck-WB, but both with a very similar 95%CI.
The analysis of reproducibility or reliability was carried out through a comparison of the results obtained for each test when it was executed by clinic professionals and when it was executed by the laboratory team, employing for this the overall agreement indicators, or the percentage of agreement or the kappa index.
The reproducibility achieved by the SD Bioline Syphilis and Syphicheck-WB proved excellent when executed by both teams, with agreement values of 99% and kappa index > 0.95% for both tests (Table 4).
In study 2, the VisiTect Syphilis test showed sensitivity values of 96.2% and 96.2%, specificities of 98.5% and 98.5%, and VPP of 94.3% and 94.3%, respectively, when carried out by the health care or the laboratory team.In both cases results were higher to those obtained with Determine Syphilis TP that had sensitivities of 88.5% and 88.5%, specificities of 97.9% and 97.9%, and VPPs of 92% and 92% when carried out by both teams.The 95%CI for the sensitivity value of VisiTect Syphilis when carried out by the laboratory team was found to be between 85.7 and 99.3, surpassing that of Determine Syphilis TP with between 75.9 and 95.2, respectively.A chi-square test for proportion differences p = 0.002, confirmed that the VisiTect Syphilis was more sensitive than the Determine Syphilis TP.
The reproducibility in the second study was excellent for both tests, with similar results when carried out by both the health care and laboratory teams.
The possibility of patients being able to wait for exam results at the clinic was high: 92.1% in the first study, 97% in the second, and, in total, for both studies, of 93.7% (739/789).When asked how long they were willing to wait, 100% of the participant in both studies agreed to wait up to 30 minutes, 59.1% in up to an hour, and 33% up to two hours, however as the average wait for test results was 15 minutes (between 10 and 20 minutes), it was confirmed that more than 90% of the participants in both studies were willing to wait for their results (Table 5).
The results of the opinion questionnaire applied to those professionals performing the tests showed that instruction comprehension, manageability, and results interpretation were rated 100% easy, or very easy, for all four tests.The speed of obtaining results with different executors was always less than 15 minutes, for SD Bioline Syphilis (100%), Syphicheck-WB (75%), VisiTect Syphilis (89%), and Determine Syphilis TP (78%).

Discussion
When compared with FTA-Abs, the four rapid tests showed sensibility, specificity and VPP performance superior to that obtained using VDRL, which is the technique most used in the routine tracking of syphilis in Brazil, nevertheless the specificity of the four tests was very similar to that of VDRL.
In general it can be said that the sensitivity, specificity, and VPP of the four tests, when dealing with patients who had high syphilis prevalence rates, was satisfactory, as well as its reproducibility when comparing the results obtained by the health care team with those of the laboratory team, giving evidence that all four tests are easily executed.
In study 1, the SD Bioline Syphilis presented a sensitivity (90.2%) higher to that of Syphicheck-WB (88.2%).This difference, however, was not statistically significant, suggesting that the performance of both of them, when dealing with high prevalence syphilis cases, was similar.
The reproducibility, measured by employing indicators of overall agreement and with the kappa index, was also very similar for both teams, demonstrating that it is possible to use them, independent of the team of professionals carrying them out.
In study 2, the VisiTect Syphilis test (96.2%)had greater sensitivity, statistically significant (p = 0.002), when compared with that of Determine Syphilis TP (88.5%).The specificity values (98.5% and 97.5%, respectively) were high, and similar, in both of the tests.The reproducibility was excellent (of 100%) possibly in virtue of the fact that the professionals had become more experienced and trained with the new technology after study 1.Despite the fact that VisiTect Syphilis, of the four tests studied, showed the greatest sensitivity, it is not possible to be certain that it is better than SD Bioline Syphilis and Syphicheck-WB as the samples employed for validation were not the same.
When comparing the results of these tests by the laboratory professionals with those of the first SDI 11 study, the following performances for each of the four tests researched were observed.

SD Bioline Syphilis
In Manaus, the sensitivity of (90.2%) was below the weighted mean for the eight SDI reference laboratories (95%).Only one of them, from Nanjin (China), was slightly lower (89%).Its specificity in the Brazilian site was found to be (99.4%),higher than the mean of the SDI studies (94.9%), and only the studies carried out in Port-au-Prince (Haiti) and Colombo (Sri Lanka) obtained 100%.

Syphicheck-WB
The point sensitivity found in Manaus (88.2%) was higher than the weighted results of the eight studies (84.5%), and only one of them, the Gambian study, with 96%, was higher.The specificity (99.6%) was also slightly superior to that pondered in the eight studies (97.7%), although four of them had higher values than those achieved in the Brazilian city.

VisiTect Syphilis
The mean point value obtained in Manaus (96.2%) was much higher than the mean of the SDI studies (85%), and was also higher than those obtained in the eight studies; on the other hand, the specificities were very similar (98.5% and 98%).

Determine Syphilis TP
In Manaus, the value obtained (88.5%) was lower than the mean of the SDI study (97.2%),only being similar to that obtained in Birmingham (U.K).The specificity was higher in the Brazilian city (97.9%), compared to the SDI mean (94.19%).
The Determine Syphilis TP was also recently evaluated in a joint study carried out by the CDC, the Pan American Health Organization/WHO, and the Instituto Evandro Chagas e Instituto Oswaldo Cruz [Evandro Chagas Institute and Oswaldo Cruz Institute] in Brazil 17 , being interpreted by three different observers.The sensitivity in relation to the gold standard employed (TPHA) varied between 95.6% and 98%, which were higher than those encountered in Manaus.The specificity, on the other hand, varied between 95.7% and 97.3%, a similar gradient to the one made evident by the present research.
Another study of Determine Syphilis TP, carried out in São Paulo, Brazil (Instituto Adolfo Lutz/Adolfo Lutz Institute), found a sensitivity of 93.6%, a specificity of 92.5%, and a VPP of 95.2% when compared to FTA-Abs and the TPHA 18 .
A more pronounced difference was observed between the results of different validations carried out in reference to the VisiTect Syphilis test which showed a higher sensitivity in Manaus when compared to the results obtained in the eight SDI laboratories.
The prevalence of positive cases in the sample submitted in the first SDI study was higher (around 50%) than that of the one studied in Manaus (20.1%), which may explain the different findings.It will be necessary, however, to collect additional information about its performance in populations with low prevalence (3% or less), to confirm that this explanation is satisfactory.
In the peer-reviewed international bibliography, only studies about Determine Syphilis TP were found, and it seems to be, of the four tests studied, the best known and studied.The studies carried out by more than dozen different observers demonstrate a sensitivity that oscillates between 88% (Birmingham and Manaus) and 100% (Moscow [Russia], Colombo, and Gambia) and that, combined with a high specificity, varying from 88% (Gambia) to 100% (Moscow), and excellent reproducibility (Manaus and Rio de Janeiro), defines it as an excellent test.Meanwhile, in the Manaus study, when compared with the VisiTect Syphilis test, this last one had a stistically significant higher sensivity, possibly due to variations of temperature and humidity which are very high in the Brazilian city, possibly causing alterations in the strip of Determine Syphilis TP (the only one of the four tests which uses such strips).This hypothesis should be verified in the future.
Another element that should always be remembered is that a positive result of any one of these rapid tests (as also occurs with other treponemic tests) does not necessarily mean a recent or active infection.This is a negative factor for their use as a tracking test in areas of high prevalence where there are many people that have already had syphilis and have been treated and cured of it and will still test positive with rapid tests.This implies the additional use of the tests currently used, VDRL or RPR, with titration, to avoid unnecessary treatment.
Finally, it was shown that more than 90% of the participants in both studies were willing to wait up to 30 minutes, a large enough time to get results in all four of the The health professionals responsible for their execution considered that all of the four tests were relatively easy to execute and interpret.

Conclusion
To sum up, the tests validated presented quite high sensitivity, specificity and PPV.Easily manipulated by health professionals, they had high acceptability among both the patients and the health professionals that participated in the studies.The study demonstrated that it is possible to guarantee treatment to people on their first contact with the health system.These results lead the authors to believe in the necessity of the incorporation of these rapid tests as one more tool in the fight against syphilis, emphasizing their utilization in hard-to-reach populations.

Sífilis; Doenças Sexualmente Transmissíveis; Reprodutibilidade dos Testes; Diagnóstico
ContributorsA. S. Benzaken participated in the elaboration of the project, the execution coordination of it, and the writing and editing of the article.E. G. Garcia contributed to the epidemiological data analysis.J. C. G. Sardinha and J. C. Dutra Junior participated in the execution of the project and the writing and editing of the article.R. Peeling collaborated on the final discussion of the article.

Table 1
Characteristics of the sample employed for the validation of rapid tests at a specialized STD clinic in.Manaus, Amazonas

Table 2
Cases studied and number and percentage of positives for FTA-Abs according to the main reason for visiting a specialized STD clinic in Manaus, Amazonas State, Brazil.

Table 3
Performance of each one of the rapid tests for syphilis in comparison with FTA-Abs at a specialized STD clinic in Manaus, Amazonas State, Brazil.

Table 4
Reproducibility of rapid tests for syphilis carried out in a specialized STD clinic in Manaus, Amazonas State, Brazil.
A: ambulatory team; L: laboratory team.

Table 5
Acceptability and willingness of patients to wait for results of rapid tests at a specialized STD clinic in Manaus, Amazonas State, Brazil.