Diagnostic accuracy of a SARS-CoV-2 rapid test and optimal time for seropositivity according to the onset of symptoms

Acurácia diagnóstica de um teste rápido para SARS-CoV-2 e tempo ótimo para a soropositividade a partir do início dos sintomas

Precisión diagnóstica del test rápido SARS-CoV-2 y período óptimo para la seropositividad según la aparición de síntomas

Caroline Nespolo de David Fernanda Hammes Varela Ivaine Tais Sauthier Sartor Márcia Polese-Bonatto Ingrid Rodrigues Fernandes Gabriela Oliveira Zavaglia Luciane Beatriz Kern Charles Francisco Ferreira Gisele Alsina Nader Bastos Paulo Márcio Pitrez Walquiria Aparecida Ferreira de Almeida Victor Bertollo Gomes Porto Alexandre Prehn Zavascki Renato Tetelbom Stein Marcelo Comerlato Scotta About the authors

Abstracts

Point-of-care serological tests for SARS-CoV-2 have been used for COVID-19 diagnosis. However, their accuracy over time regarding the onset of symptoms is not fully understood. We aimed to assess the accuracy of a point-of-care lateral flow immunoassay (LFI). Subjects, aged over 18 years, presenting clinical symptoms suggestive of acute SARS-CoV-2 infection were tested once by both nasopharyngeal and oropharyngeal RT-PCR and LFI. The accuracy of LFI was assessed in periodic intervals of three days in relation to the onset of symptoms. The optimal cut-off point was defined as the number of days required to achieve the best sensitivity and specificity. This cut-off point was also used to compare LFI accuracy according to participants’ status: outpatient or hospitalized. In total, 959 patients were included, 379 (39.52%) tested positive for SARS-CoV-2 with RT-PCR, and 272 (28.36%) tested positive with LFI. LFI best performance was achieved after 10 days of the onset of symptoms, with sensitivity and specificity of 84.9% (95%CI: 79.8-89.1) and 94.4% (95%CI: 91.0-96.8), respectively. Although the specificity was similar (94.6% vs. 88.9%, p = 0.051), the sensitivity was higher in hospitalized patients than in outpatients (91.7% vs. 82.1%, p = 0.032) after 10 days of the onset of symptoms. Best sensitivity of point-of-care LFI was found 10 days after the onset of symptoms which may limit its use in acute care. Specificity remained high regardless of the number of days since the onset of symptoms.

Keywords:
SARS-CoV-2; COVID-19 Testing; Point-of-Care Testing; Immunoassay; Sensitivity and Specificity


Os testes sorológicos no local de atendimento (point-of-care) para a infecção pelo SARS-CoV-2 têm sidos utilizados para o diagnóstico da COVID-19. Entretanto, não está plenamente elucidada a acurácia dos testes ao longo do tempo em relação ao início dos sintomas. Nosso objetivo foi de avaliar a acurácia, no local de atendimento, do imunoensaio de fluxo lateral (LFI). Pacientes com ≥ 18 anos de idade que apresentavam sintomas clínicos sugestivos de infecção aguda pelo SARS-CoV-2 foram testados uma vez com RT-PCR da nasofaringe e orofaringe, além do LFI. A acurácia do LFI foi avaliada com intervalos periódicos de 3 dias a partir do início dos sintomas. O ponto de corte ótimo foi definido como o número necessário de dias para atingir a melhor sensibilidade e especificidade. Esse ponto foi utilizado também para comparar a acurácia do LFI de acordo com a situação do paciente (ambulatorial ou hospitalizado). Foram incluídos 959 pacientes, dos quais 379 (39,52%) testaram positivos para SARS-CoV-2 pelo RT-PCR e 272 (28,36%) pelo LFI. Foi atingido o melhor desempenho para o LFI com 10 dias a partir do início dos sintomas, com sensibilidade e especificidade de 84,9% (IC95%: 79,8-89,1) e 94,4% (IC95%: 91,0-96,8), respectivamente. Embora a especificidade não tenha sido diferente entre os grupos de pacientes (94,6% vs. 88,9%, p = 0,051), a sensibilidade foi mais alta nos pacientes hospitalizados que nos ambulatoriais (91,7% vs. 82,1%, p = 0,032) no dia 10 depois do início dos sintomas. A melhor sensibilidade do LFI no local de atendimento ocorre 10 dias depois do início dos sintomas, o que pode limitar seu uso no atendimento agudo. A especificidade permanece alta, independentemente do número de dias desde o início dos sintomas.

Palavras-chave:
SARS-CoV-2; Teste para COVID-19; Testes Imediatos; Imunoensaio; Sensibilidade e Especificidade


Los puestos de atención para pruebas serológicas del SARS-CoV-2 han sido usado para la diagnosis de la COVID-19. No obstante, su precisión a lo largo del tiempo, en lo que respecta a la aparición de los síntomas, no se ha comprendido completamente. Nuestro objetivo fue evaluar la precisión de un puesto de atención de inmunoanálisis de flujo lateral (LFI). Se hizo pruebas a individuos ≥ 18 años, presentando síntomas clínicos compatibles con una infección aguda de SARS-CoV-2, tanto vía nasofaríngea y orofaríngea RT-PCR, como LFI. La precisión de LFI fue evaluada en intervalos periódicos de 3 días con respecto a la aparición de los síntomas. El punto óptimo de corte se definió como el número de días requerido para alcanzar la mejor sensibilidad y especificidad. Este punto también se usó para comparar la precisión del LFI, según el estatus de los participantes: ambulatorios u hospitalizados. Se incluyeron a 959 pacientes, 379 (39,52%) dieron positivo en las pruebas de SARS-CoV-2 RT-PCR, y 272 (28,36%) fueron positivos en los LFI. Se alcanzó el mejor rendimiento de los LFI tras 10 días de la aparición de los síntomas, con una sensibilidad y especificidad de un 84,9% (IC95%: 79,8-89,1) y 94,4% (IC95%: 91,0-96,8), respectivamente. A pesar de que la especificidad no fue diferente (94,6% vs. 88,9%, p = 0,051), la sensibilidad fue mayor en pacientes hospitalizados que en los ambulatorios (91,7% vs. 82,1%, p = 0,032) tras 10 días desde la aparición de los síntomas. La mejor sensibilidad LFI del puesto de cuidado se produce tras 10 días de la aparición de los síntomas, lo que quizás limite su uso en el cuidado de urgencias. La especificidad permanece alta independientemente del número de días desde la aparición de los síntomas.

Palabras-clave:
SARS-CoV-2; Prueba de COVID-19; Pruebas en el Punto de Atención; Inmunoensayo; Sensibilidad y Especificidad


Introduction

Efforts to combat the ongoing COVID-19 pandemic are currently still dependent on non-pharmacological measures, such as prompt diagnosis and social distancing 11. Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med 2020; 382:727-33.,22. Lyu W, Wehby GL. Comparison of estimated rates of coronavirus disease 2019 (COVID-19) in border counties in Iowa without a stay-at-home order and border counties in Illinois with a stay-at-home order. JAMA Netw Open 2020; 3:e2011102.,33. Hsiang S, Allen D, Annan-Phan S, Bell K, Bolliger I, Chong T, et al. The effect of large-scale anti-contagion policies on the COVID-19 pandemic. Nature 2020; 584:262-7.. For this reason, the widespread availability of accurate and rapid tests is essential to control the SARS-CoV-2 spread 44. Cheng MP, Papenburg J, Desjardins M, Kanjilal S, Quach C, Libman M, et al. Diagnostic testing for severe acute respiratory syndrome-related coronavirus-2. Ann Intern Med 2020; 172:726-34.. Although molecular biology tests are considered the gold standard for the diagnosis of acute infections, false negatives may occur 55. Long DR, Gombar S, Hogan CA, Greninger AL, O'Reilly-Shah V, Bryson-Cahn C, et al. Occurrence and timing of subsequent severe acute respiratory syndrome coronavirus 2 reverse-transcription polymerase chain reaction positivity among initially negative patients. Clin Infect Dis 2021; 72:323-6.. Serological tests are not yet completely validated and the timing for best seropositivity is not fully understood 66. Bastos ML, Tavaziva G, Abidi SK, Campbell JR, Haraoui L-P, Johnston JC, et al. Diagnostic accuracy of serological tests for COVID-19: systematic review and meta-analysis. BMJ 2020; 370:m2516., limiting its use in acute care. Moreover, molecular biology and traditional serological tests demand laboratory resources and are not always available during pandemic-overloaded laboratories. In this scenario, the knowledge about the accuracy of the qualitative antibody point-of-care-style lateral flow immunoassays (LFI) for COVID-19 diagnosis may help to rationalize decisions at public health level.

Since the beginning of the pandemic, LFI tests have been given fast-track approval for use, but lacking validation in clinical practice 66. Bastos ML, Tavaziva G, Abidi SK, Campbell JR, Haraoui L-P, Johnston JC, et al. Diagnostic accuracy of serological tests for COVID-19: systematic review and meta-analysis. BMJ 2020; 370:m2516.,77. Castro R, Luz PM, Wakimoto MD, Veloso VG, Grinsztejn B, Perazzo H. COVID-19: a meta-analysis of diagnostic test accuracy of commercial assays registered in Brazil. Braz J Infect Dis 2020; 24:180-7.. In Brazil, more than 20 LFI tests were approved between March and April of 2020, period of the study design and ethical approval 88. Agência Nacional de Vigilância Sanitária. Consultas. https://consultas.anvisa.gov.br/#/saude/q/?nomeTecnico=coronav%C3%ADrus (accessed on 14/May/2021).
https://consultas.anvisa.gov.br/#/saude/...
,99. Agência Nacional de Vigilância Sanitária. Testes para Covid-19: perguntas e respostas. https://www.gov.br/anvisa/pt-br/assuntos/noticias-anvisa/2020/testes-para-covid-19-perguntas-e-respostas (accessed on 14/May/2021).
https://www.gov.br/anvisa/pt-br/assuntos...
. Despite clinical studies currently published, it is important to test LFI accuracy in specific age groups, to evaluate different levels of disease severity, and the best time for its use from a disease onset baseline 1010. Chen S-Y, Lee Y-L, Lin Y-C, Lee N-Y, Liao C-H, Hung Y-P, et al. Multicenter evaluation of two chemiluminescence and three lateral flow immunoassays for the diagnosis of COVID-19 and assessment of antibody dynamic responses to SARS-CoV-2 in Taiwan. Emerg Microbes Infect 2020; 9:2157-68.,1111. Imai K, Tabata S, Ikeda M, Noguchi S, Kitagawa Y, Matuoka M, et al. Clinical evaluation of an immunochromatographic IgM/IgG antibody assay and chest computed tomography for the diagnosis of COVID-19. J Clin Virol 2020; 128:104393.,1212. Costa SF, Buss L, Espinoza EPS, Vieira JM, de Oliveira da Silva LC, de Souza RM, et al. Performance of a qualitative rapid chromatographic immunoassay to diagnose COVID-19 in patients in a middle-income country. J Clin Virol 2020; 131:104592.,1313. Li Z, Yi Y, Luo X, Xiong N, Liu Y, Li S, et al. Development and clinical application of a rapid IgM-IgG combined antibody test for SARS-CoV-2 infection diagnosis. J Med Virol 2020; 92:1518-24.,1414. Serrano MM, Rodríguez DN, Palop NT, Arenas RO, Córdoba MM, Mochón MDO, et al. Comparison of commercial lateral flow immunoassays and ELISA for SARS-CoV-2 antibody detection. J Clin Virol 2020; 129:104529.,1515. Charpentier C, Ichou H, Damond F, Bouvet E, Chaix M-L, Ferré V, et al. Performance evaluation of two SARS-CoV-2 IgG/IgM rapid tests (Covid-Presto and NG-Test) and one IgG automated immunoassay (Abbott). J Clin Virol 2020; 132:104618.,1616. Kohmer N, Westhaus S, Rühl C, Ciesek S, Rabenau HF. Clinical performance of different SARS-CoV-2 IgG antibody tests. J Med Virol 2020; 92:2243-7.,1717. Paiva KJ, Grisson RD, Chan PA, Huard RC, Caliendo AM, Lonks JR, et al. Validation and performance comparison of three SARS-CoV-2 antibody assays. J Med Virol 2021; 93:916-23.,1818. Guedez-López GV, Alguacil-Guillén M, González-Donapetry P, Bloise I, Tornero-Marin C, González-García J, et al. Evaluation of three immunochromatographic tests for rapid detection of antibodies against SARS-CoV-2. Eur J Clin Microbiol Infect Dis 2020; 39:2289-97.,1919. Moshe M, Daunt A, Flower B, Simmons B, Brown JC, Frise R, et al. SARS-CoV-2 lateral flow assays for possible use in national covid-19 seroprevalence surveys (React 2): diagnostic accuracy study. BMJ 2021; 372:n423..

We aimed to assess the accuracy of a LFI test (Wondfo Biotech Co.; https://en.wondfo.com.cn/) and to evaluate the optimal time for seropositivity according to the onset of symptoms. We hypothesize that the sensibility will be higher if performed later on from the onset of symptoms, due to the usual immune response time. We have also compared the test accuracy in relation to outpatients and hospitalized COVID-19 subjects.

Materials and methods

This is a cross-sectional multicenter observational study with data collected from two hospitals, in Porto Alegre, Rio Grande do Sul State, Brazil. From May to October 2020, a convenience sample of adult subjects (aged ≥ 18 years) presenting signs or symptoms suggestive of COVID-19 (cough, fever, or sore throat) were assessed within 14 days of the onset of symptoms onset. Hospitalized individuals were eligible if they presented the same inclusion criteria as outpatients and if they had less than 48 hours of hospitalization. The exclusion criteria was failure to collect a RT-PCR for SARS-CoV-2. The STARD checklist was followed for the study development 2020. Korevaar DA, Cohen JF, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, et al. Updating standards for reporting diagnostic accuracy: the development of STARD 2015. Res Integr Peer Rev 2016; 1:7..

The presence of antibodies was assessed with the Wondfo SARS-CoV-2 antibody test (Wondfo Biotech Co.), by finger-prick blood samples. This test detects immunoglobulins (Ig) G and M isotypes specific to SARS-CoV-2 receptor binding domain of the spike protein in a lateral flow assay. Two drops of blood from a pinprick are sufficient to detect the antibodies. The assay reagent consists of colloidal gold particles coated with recombinant SARS-CoV-2 receptor-binding domain (RBD; according to the manufacturer’s specifications). If colloidal gold complexes are present, they are captured by the specific antibodies against human IgM and/or IgG included in the test (T) line in the kit’s window, leading to the appearance of a dark-colored line. Samples without SARS-CoV-2-reactive antibodies will not display this line. Valid tests are identified by a positive control line (C) in the same window. If this control line was not visible, the test was deemed invalid and the patient was retested. Two members of the research team assessed the results, if there was discordance between researchers the test was repeated.

At inclusion, all patients were subjected to one RT-PCR test to detect SARS-CoV-2, and it was considered the reference standard diagnosis test. Both SARS-CoV-2 nasopharyngeal and oropharyngeal swabs were collected simultaneously in all participants and allocated in the same transport media with saline solution and RNAlater, RNA Stabilization Solution (catalog number AM7021). A total of 60µL of RNA were extracted from 400µL of a respiratory specimen using MagMax Viral/Pathogenic Nucleic Acid Isolation (Applied Biosystems; https://www.thermofisher.com/br/en/home/brands/applied-biosystems.html) and performed using the KingFisher Duo Prime System platform (Thermo Fisher Scientific Inc.; https://www.thermofisher.com). Total reaction volume was 10µL, 5µL with TaqPathTM 1-Step RT-qPCR Master Mix, CG (catalog number A15299; AppliedBiosystems) and TaqManTM 2019-nCoV Assay Kit v1 (catalog number A47532), and 5µL of RNA. Five microliters of TaqManTM 2019-nCoV Control Kit v1 (catalog number A47533) were used as control (200 copies/µL). RT-PCR was performed on a QuantStudio 5 (Thermo Fisher Scientific Inc.). Results were analyzed with QuantStudio Design & Analysis Software v1.5.1 (https://www.thermofisher.com/br/en/home/global/forms/life-science/quantstudio-3-5-software.html). A detected SARS-CoV-2 RT-PCR test was defined as amplification with less than 37 cycle threshold (Ct), if one detected over 37 Ct the sample where repeated, if two more tests were over 37 Ct, or undetected, the sample was considered negative for SARS-CoV-2.

Up to June 30th, RT-PCR and LFI sample collection was performed, at enrollment for all participants. After an interim analysis, we found that 63.25% (253/400) of the participants were included before the seventh day after the onset of symptoms, with an overall low sensitivity in this group. Due to this finding and aiming to explore the accuracy of LFI in participants with longer clinical courses, the protocol of testing procedures was changed after July 1st, for participants with less than seven days of clinical onset. In this group, RT-PCR was collected at inclusion, but LFI was scheduled 14 days later. For individuals included between 7 and 14 days after the onset of symptoms, both tests were made simultaneously throughout the whole study period. Participants were contacted by telephone call one day before the scheduled date and a second call was done for those who did not attend. All testing was performed with a standardized protocol by the trained study team. The laboratory team was blind to the LFI test results until the final analysis. However, when LFI was not performed at the time of inclusion, the researchers who performed the LFI had access to the RT-PCR test results.

Means and standard deviations were calculated for continuous variables and percentages for the categorical variables to describe patients’ characteristics. Diagnostic accuracy was calculated using the epiR package (version 1.0-15) 2121. Stevenson M, Sergeant E, Nunes T, Heuer C, Marshall J, Sanchez J, et al. epiR: tools for the analysis of epidemiological data. https://CRAN.R-project.org/package=epiR (accessed on 11/Feb/2021).
https://CRAN.R-project.org/package=epiR...
to estimate sensitivity and specificity values. Sensitivity was characterized as the number of positive LFI test results divided by the number of patients with COVID-19 diagnosis confirmed by RT-PCR. Specificity was calculated as the number of negative LFI results divided by the number of participants with symptoms suggestive of COVID-19 (cough, fever, or sore throat) who had a negative RT-PCR. Exact method was used to calculate 95% confidence intervals for all estimates. To calculate the power for the diagnostic test based on sensitivity, the “power.diagnostic.test” function was used from MKmisc R package (version 1.2) 2222. Kohl M. MKmisc: miscellaneous functions from M. Kohl. https://CRAN.R-project.org/package=MKmisc (accessed on 14/May/2021).
https://CRAN.R-project.org/package=MKmis...
, considering total participants with LFI cut-off point of 10 days.

To explore the optimal time for seropositivity, days of the onset of symptoms were counted backwards from the time of LFI testing, and sensitivity and specificity were assessed in periodic intervals of 3 days (1-3, 4-6, 7-9, 10-12, 13-15, 16-18, and ≥ 19 days of symptoms). The point in time in which sensitivity values stabilized was used as a cut-off point to compare the accuracy according to whether the participant was hospitalized (at enrollment or during the following 28 days) or remained as an outpatient during this period.

A sensitivity analysis was performed excluding participants with clinical conditions that compromised the immune system, such as type 1 or 2 diabetes mellitus, previous organ transplant, cancer diagnosis or those who had chemotherapy in the two weeks prior to enrollment. Participants without either LFI or RT-PCR results were excluded from the analysis. The analyses were performed in R version 3.6.323 (https://www.r-project.org/), and the sensitivity and specificity proportions were compared using MedCalc software (https://www.medcalc.org/).

The study was performed in accordance with the Resolution n. 466/2012 of the Brazilian National Health Council and the Declaration of Helsinki and Good Clinical Practice Guidelines, after approval by the Institutional Review Board of the Moinhos de Vento Hospital (n. 30749720.4.1001.5330). All participants included in this study provided written informed consent.

Results

A total of 1,359 subjects were screened for the study, and 400 were excluded (103 did not meet inclusion criteria, 57 did not consent to participate, 238 had unpaired LFI and RT-PCR results, and 2 withdrew consent), as shown in Figure 1. A total of 959 participants were included and, in 555 (57.87%) of these individuals, samples were collected for both RT-PCR and LFI simultaneously. SARS-CoV-2 infection was confirmed by RT-PCR in 379 (39.52%), while 272 (28.36%) had positive LFI results. The median of days of symptoms were different between the participants in which LFI was performed simultaneously and those whose test was scheduled later (5.0 [IQR 2.0-8.0] vs. 3.0 [IQR 2.0-4.0] days, p< 0.001). The median time between RT-PCR and LFI was 11 days (IQR 5.0-17.0, range 0-40 days).

Figure 1
Subject’s flowchart.

In total, 16 (1.67%) participants had negative RT-PCR and positive LFI results. Among these, 14 subjects were included in the study after 1-4 days of the onset of symptoms, while other two participants were included between 10-11 days (all RT-PCR sample collection occurred at the inclusion and all LFI tests were performed between 8 to 21 days after onset of symptoms). In 123 (32.45%) of 379 participants the results were positive for RT-PCR and negative for LFI, and in most of these situations (n = 91) the individuals had been tested with LFI before the 10-day cut-off point.

Among 959 individuals with complete data, 136 were included during hospitalization and 823 as outpatients, however, 25 of them presented complications that led to hospitalization up to 28 days after study inclusion and were thus re-classified as hospitalized. Accordingly, the data was analyzed with 798 (83.21%) outpatients and 161 (16.79%) hospitalized patients. Among those hospitalized, 27 (2.81%) participants were admitted to the intensive care unit. The most prevalent symptoms reported were headache (83.73%), myalgia (83.73%), cough (79.14%), malaise (76.43%), coryza (64.96%), and sore throat (63.71%). Table 1 shows demographic and clinical characteristics.

Table 1
Demographic and clinical characteristics of included subjects. Porto Alegre, Rio Grande do Sul State, Brazil.

The rate of positive LFI increased with the course of the disease, during testing among individuals with confirmed COVID-19 acute infection (positive RT-PCR), as shown in Figure 2. This increase in the test sensitivity stabilized after 10 days, remaining above 80% (Figure 2a), whereas specificity remained stable, above 92%, throughout the observed time period (Figure 2b). The power from the diagnostic test considering the sensitivity of the LFI test ≥ 10 days was 92%.

Figure 2
Lateral flow immunoassay (LFI) sensitivity and specificity according to days after onset of symptoms *.

To test the accuracy of LFI regarding disease severity, a comparison between hospitalized and outpatients was performed considering an optimal time for diagnosis at a cut-off point of 10 days (Table 2). The sensitivity was higher in hospitalized patients than in outpatients (91.7% [95%CI: 82.7-96.9] vs. 82.1% [95%CI: 75.7-87.4], p = 0.032). The specificity did not differ between the hospitalized participants and outpatients (88.9% [95%CI: 51.8-99.7] vs. 94.6% [95%CI: 92.1-96.9], p = 0.051). The analysis - excluding participants with clinical conditions that compromise the immune system - did not show differences in LFI sensitivity (84.8% [95%CI: 79.4-89.3], p = 0.140) and specificity (94.5% [95%CI: 91.1-96.9], p = 0.964), considering the cut-off ≥ 10 days after symptoms (Supplementary Material; http://cadernos.ensp.fiocruz.br/static//arquivo/suppl-e00069921_3495.pdf).

Table 2
Sensitivity and specificity at the cut-off point of 10 days after onset of symptoms, according to the severity of the patient’s clinical condition.

Discussion

This is one of the largest study samples testing optimal time for seropositivity and the accuracy of a point-of-care LFI in a prospective study during the COVID-19 pandemic in Brazil. According to our results, optimal time for seropositivity of this test occurs over 10 days after the onset of clinical signs or symptoms, in accordance with studies performed with similar tests 1616. Kohmer N, Westhaus S, Rühl C, Ciesek S, Rabenau HF. Clinical performance of different SARS-CoV-2 IgG antibody tests. J Med Virol 2020; 92:2243-7.,2323. Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Spijker R, Taylor-Phillips S, et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev 2020; (6):CD013652.,2424. Maine GN, Lao KM, Krishnan SM, Afolayan-Oloye O, Fatemi S, Kumar S, et al. Longitudinal characterization of the IgM and IgG humoral response in symptomatic COVID-19 patients using the Abbott Architect. J Clin Virol 2020; 133:104663.,2525. Whitman JD, Hiatt J, Mowery CT, Shy BR, Yu R, Yamamoto TN, et al. Evaluation of SARS-CoV-2 serology assays reveals a range of test performance. Nat Biotechnol 2020; 38:1174-83.. The specificity is above 92% regardless of the days of symptoms, although the sensitivity does not go over 85% for tests performed over 10 days of symptoms.

As aforementioned, most serological tests, including LFI tests, were validated early on in the beginning of the pandemic using laboratory-based data due to the urgency of a public health crisis 66. Bastos ML, Tavaziva G, Abidi SK, Campbell JR, Haraoui L-P, Johnston JC, et al. Diagnostic accuracy of serological tests for COVID-19: systematic review and meta-analysis. BMJ 2020; 370:m2516.,77. Castro R, Luz PM, Wakimoto MD, Veloso VG, Grinsztejn B, Perazzo H. COVID-19: a meta-analysis of diagnostic test accuracy of commercial assays registered in Brazil. Braz J Infect Dis 2020; 24:180-7.. A comparison between different tests showed a superiority of the Wondfo test 2626. Almeida SM, Spalanzani RN, Nogueira MB, Sanada B, Cavalli BM, Rotta I, et al. Rapid serological tests for SARS-CoV-2: diagnostic performance of 4 commercial assays. Med Princ Pract 2021; 30:385-94.. In some studies, several tests were evaluated in the same samples, and the performance increased with time after the onset of disease, with positive results above 80% for all studied tests over 20 days of infection. The same test which was used in this study has previously shown better positivity at later time points, while maintaining specificity over 95% 1010. Chen S-Y, Lee Y-L, Lin Y-C, Lee N-Y, Liao C-H, Hung Y-P, et al. Multicenter evaluation of two chemiluminescence and three lateral flow immunoassays for the diagnosis of COVID-19 and assessment of antibody dynamic responses to SARS-CoV-2 in Taiwan. Emerg Microbes Infect 2020; 9:2157-68.,1212. Costa SF, Buss L, Espinoza EPS, Vieira JM, de Oliveira da Silva LC, de Souza RM, et al. Performance of a qualitative rapid chromatographic immunoassay to diagnose COVID-19 in patients in a middle-income country. J Clin Virol 2020; 131:104592.,2525. Whitman JD, Hiatt J, Mowery CT, Shy BR, Yu R, Yamamoto TN, et al. Evaluation of SARS-CoV-2 serology assays reveals a range of test performance. Nat Biotechnol 2020; 38:1174-83.. However, the Wondfo test has been described with a poorer sensitivity when compared to serum serologic tests 2727. Dos Santos VA, Rafael MM, Sabino EC, Duarte AJS. Sensitivity of the Wondfo One Step COVID-19 test using serum samples. Clinics (São Paulo) 2020; 75:e2013.,2828. Borges LP, Oliveira MGBD, Souza DRV, Santos KAD, Heimfarth L. Can the Wondfo(r) SARS-CoV-2 IgM/IgG antibodies be used as a rapid diagnostic test? Archives of Biotechnology and Biomedicine 2020; 4:13-7.. Our study shows similar results, with a sensitivity of 84.9%, and 95.3% for tests performed at or over 10 days and ≥ 19 days of infection, respectively. The higher values of sensitivity found for hospitalized patients could be associated with increased antibody load, typically found in more severe disease clinical presentation 2424. Maine GN, Lao KM, Krishnan SM, Afolayan-Oloye O, Fatemi S, Kumar S, et al. Longitudinal characterization of the IgM and IgG humoral response in symptomatic COVID-19 patients using the Abbott Architect. J Clin Virol 2020; 133:104663.,2525. Whitman JD, Hiatt J, Mowery CT, Shy BR, Yu R, Yamamoto TN, et al. Evaluation of SARS-CoV-2 serology assays reveals a range of test performance. Nat Biotechnol 2020; 38:1174-83.. However, our study was not powered to assess such differences.

It is well known that RT-PCR testing - which has a central role during the first days of COVID-19 - may lead to false negative results 55. Long DR, Gombar S, Hogan CA, Greninger AL, O'Reilly-Shah V, Bryson-Cahn C, et al. Occurrence and timing of subsequent severe acute respiratory syndrome coronavirus 2 reverse-transcription polymerase chain reaction positivity among initially negative patients. Clin Infect Dis 2021; 72:323-6.,2929. Woloshin S, Patel N, Kesselheim AS. False negative tests for SARS-CoV-2 infection: challenges and implications. N Engl J Med 2020; 383:e38.,3030. Abreu MC, Choquet C, Petit H, Bouzid D, Damond F, Marot S, et al. SARS-CoV-2 IGM and IGG rapid serologic test for the diagnosis of COVID-19 in the emergency department. J Infect 2020; 81:816-46.. In our study, we observed 16 (1.67%) cases of negative RT-PCR with positive LFI results, which could be related to a false negative RT-PCR due to premature sample collection.

Our study has a few noteworthy limitations. Firstly, RT-PCR can lead to false negative results and its use as the reference test may overestimate sensitivity and underestimate the specificity of the LFI. Nonetheless, a specificity value of over 92% was maintained consistently throughout the study. Secondly, unlike other studies assessing several possible LFI tests, we tested only one type of LFI. However, a higher number of patients were prospectively enrolled compared to many previous studies 1010. Chen S-Y, Lee Y-L, Lin Y-C, Lee N-Y, Liao C-H, Hung Y-P, et al. Multicenter evaluation of two chemiluminescence and three lateral flow immunoassays for the diagnosis of COVID-19 and assessment of antibody dynamic responses to SARS-CoV-2 in Taiwan. Emerg Microbes Infect 2020; 9:2157-68.,1111. Imai K, Tabata S, Ikeda M, Noguchi S, Kitagawa Y, Matuoka M, et al. Clinical evaluation of an immunochromatographic IgM/IgG antibody assay and chest computed tomography for the diagnosis of COVID-19. J Clin Virol 2020; 128:104393.,1212. Costa SF, Buss L, Espinoza EPS, Vieira JM, de Oliveira da Silva LC, de Souza RM, et al. Performance of a qualitative rapid chromatographic immunoassay to diagnose COVID-19 in patients in a middle-income country. J Clin Virol 2020; 131:104592.,1313. Li Z, Yi Y, Luo X, Xiong N, Liu Y, Li S, et al. Development and clinical application of a rapid IgM-IgG combined antibody test for SARS-CoV-2 infection diagnosis. J Med Virol 2020; 92:1518-24.,1414. Serrano MM, Rodríguez DN, Palop NT, Arenas RO, Córdoba MM, Mochón MDO, et al. Comparison of commercial lateral flow immunoassays and ELISA for SARS-CoV-2 antibody detection. J Clin Virol 2020; 129:104529.,1616. Kohmer N, Westhaus S, Rühl C, Ciesek S, Rabenau HF. Clinical performance of different SARS-CoV-2 IgG antibody tests. J Med Virol 2020; 92:2243-7.,1818. Guedez-López GV, Alguacil-Guillén M, González-Donapetry P, Bloise I, Tornero-Marin C, González-García J, et al. Evaluation of three immunochromatographic tests for rapid detection of antibodies against SARS-CoV-2. Eur J Clin Microbiol Infect Dis 2020; 39:2289-97.,2525. Whitman JD, Hiatt J, Mowery CT, Shy BR, Yu R, Yamamoto TN, et al. Evaluation of SARS-CoV-2 serology assays reveals a range of test performance. Nat Biotechnol 2020; 38:1174-83., making it possible to perform analysis considering severity and response in relation to the onset of symptoms. Thirdly, researchers who performed the scheduled LFI were unblinded for RT-PCR results. Yet, two researchers evaluated the results and the tests were repeated if there was disagreement. Also, LFI performed in samples obtained with finger-prick may have lower sensitivity compared to serum samples 3131. Flower B, Brown JC, Simmons B, Moshe M, Frise R, Penn R, et al. Clinical and laboratory evaluation of SARS-CoV-2 lateral flow assays for use in a national COVID-19 seroprevalence survey. Thorax 2020; 75:1082-8.. Fifthly, as participants were enrolled either as hospitalized or at ERs, the performance of LFI in individuals with milder disease who do not seek healthcare at the hospital may be underrepresented. Finally, the protocol was adapted regarding the timing of sample collection for LFI after an interim analysis. However, despite leading to a more complex methodology, it allowed us to explore the accuracy in several different periods during the disease course.

LFIs devices similar to the one we tested may be helpful for epidemiologic purposes. For clinical purposes, a negative result cannot exclude the diagnosis. They can instead indicate the need for RT-PCR testing in a stepwise protocol. Performing serologic testing for COVID-19 may be used as a complementary strategy to minimize misdiagnosis since both molecular biology and conventional serologic tests are time and resource-consuming. LFI testing can be performed to verify possible RT-PCR false negatives, whenever molecular biology re-analysis is not readily available, or the ideal test moment has already passed.

Conclusions

The LFI Wondfo presented high specificity (> 92%) for SARS-CoV-2 infection in a large population sample. Sensitivity higher than 80% occurred 10 days after the onset of symptoms, which may limit its use in acute care. However, this test is an option especially for epidemiological objectives, when considering these limitations. It can be useful to evaluate clusters of cases, since a COVID-19 early diagnosis may allow confirmation of subsequent cases by using epidemiological criteria. Further studies are necessary to assess the long-term performance of this test, as it is not known how long antibodies remain positive after an acute infection, especially regarding these LFI tests.

Acknowledgments

We thank the Scientific Committee of the Research Support Nucleus (NAP) of the Moinhos de Vento Hospital for technical-scientific consultancy. We thank the inclusion personnel, laboratory team, and site staff from Moinhos de Vento Hospital and from Restinga e Extremo Sul Hospital. Brazilian Ministry of Health through the Unified Health System’s Support Program for Institutional Development.

References

  • 1
    Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, et al. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med 2020; 382:727-33.
  • 2
    Lyu W, Wehby GL. Comparison of estimated rates of coronavirus disease 2019 (COVID-19) in border counties in Iowa without a stay-at-home order and border counties in Illinois with a stay-at-home order. JAMA Netw Open 2020; 3:e2011102.
  • 3
    Hsiang S, Allen D, Annan-Phan S, Bell K, Bolliger I, Chong T, et al. The effect of large-scale anti-contagion policies on the COVID-19 pandemic. Nature 2020; 584:262-7.
  • 4
    Cheng MP, Papenburg J, Desjardins M, Kanjilal S, Quach C, Libman M, et al. Diagnostic testing for severe acute respiratory syndrome-related coronavirus-2. Ann Intern Med 2020; 172:726-34.
  • 5
    Long DR, Gombar S, Hogan CA, Greninger AL, O'Reilly-Shah V, Bryson-Cahn C, et al. Occurrence and timing of subsequent severe acute respiratory syndrome coronavirus 2 reverse-transcription polymerase chain reaction positivity among initially negative patients. Clin Infect Dis 2021; 72:323-6.
  • 6
    Bastos ML, Tavaziva G, Abidi SK, Campbell JR, Haraoui L-P, Johnston JC, et al. Diagnostic accuracy of serological tests for COVID-19: systematic review and meta-analysis. BMJ 2020; 370:m2516.
  • 7
    Castro R, Luz PM, Wakimoto MD, Veloso VG, Grinsztejn B, Perazzo H. COVID-19: a meta-analysis of diagnostic test accuracy of commercial assays registered in Brazil. Braz J Infect Dis 2020; 24:180-7.
  • 8
    Agência Nacional de Vigilância Sanitária. Consultas. https://consultas.anvisa.gov.br/#/saude/q/?nomeTecnico=coronav%C3%ADrus (accessed on 14/May/2021).
    » https://consultas.anvisa.gov.br/#/saude/q/?nomeTecnico=coronav%C3%ADrus
  • 9
    Agência Nacional de Vigilância Sanitária. Testes para Covid-19: perguntas e respostas. https://www.gov.br/anvisa/pt-br/assuntos/noticias-anvisa/2020/testes-para-covid-19-perguntas-e-respostas (accessed on 14/May/2021).
    » https://www.gov.br/anvisa/pt-br/assuntos/noticias-anvisa/2020/testes-para-covid-19-perguntas-e-respostas
  • 10
    Chen S-Y, Lee Y-L, Lin Y-C, Lee N-Y, Liao C-H, Hung Y-P, et al. Multicenter evaluation of two chemiluminescence and three lateral flow immunoassays for the diagnosis of COVID-19 and assessment of antibody dynamic responses to SARS-CoV-2 in Taiwan. Emerg Microbes Infect 2020; 9:2157-68.
  • 11
    Imai K, Tabata S, Ikeda M, Noguchi S, Kitagawa Y, Matuoka M, et al. Clinical evaluation of an immunochromatographic IgM/IgG antibody assay and chest computed tomography for the diagnosis of COVID-19. J Clin Virol 2020; 128:104393.
  • 12
    Costa SF, Buss L, Espinoza EPS, Vieira JM, de Oliveira da Silva LC, de Souza RM, et al. Performance of a qualitative rapid chromatographic immunoassay to diagnose COVID-19 in patients in a middle-income country. J Clin Virol 2020; 131:104592.
  • 13
    Li Z, Yi Y, Luo X, Xiong N, Liu Y, Li S, et al. Development and clinical application of a rapid IgM-IgG combined antibody test for SARS-CoV-2 infection diagnosis. J Med Virol 2020; 92:1518-24.
  • 14
    Serrano MM, Rodríguez DN, Palop NT, Arenas RO, Córdoba MM, Mochón MDO, et al. Comparison of commercial lateral flow immunoassays and ELISA for SARS-CoV-2 antibody detection. J Clin Virol 2020; 129:104529.
  • 15
    Charpentier C, Ichou H, Damond F, Bouvet E, Chaix M-L, Ferré V, et al. Performance evaluation of two SARS-CoV-2 IgG/IgM rapid tests (Covid-Presto and NG-Test) and one IgG automated immunoassay (Abbott). J Clin Virol 2020; 132:104618.
  • 16
    Kohmer N, Westhaus S, Rühl C, Ciesek S, Rabenau HF. Clinical performance of different SARS-CoV-2 IgG antibody tests. J Med Virol 2020; 92:2243-7.
  • 17
    Paiva KJ, Grisson RD, Chan PA, Huard RC, Caliendo AM, Lonks JR, et al. Validation and performance comparison of three SARS-CoV-2 antibody assays. J Med Virol 2021; 93:916-23.
  • 18
    Guedez-López GV, Alguacil-Guillén M, González-Donapetry P, Bloise I, Tornero-Marin C, González-García J, et al. Evaluation of three immunochromatographic tests for rapid detection of antibodies against SARS-CoV-2. Eur J Clin Microbiol Infect Dis 2020; 39:2289-97.
  • 19
    Moshe M, Daunt A, Flower B, Simmons B, Brown JC, Frise R, et al. SARS-CoV-2 lateral flow assays for possible use in national covid-19 seroprevalence surveys (React 2): diagnostic accuracy study. BMJ 2021; 372:n423.
  • 20
    Korevaar DA, Cohen JF, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, et al. Updating standards for reporting diagnostic accuracy: the development of STARD 2015. Res Integr Peer Rev 2016; 1:7.
  • 21
    Stevenson M, Sergeant E, Nunes T, Heuer C, Marshall J, Sanchez J, et al. epiR: tools for the analysis of epidemiological data. https://CRAN.R-project.org/package=epiR (accessed on 11/Feb/2021).
    » https://CRAN.R-project.org/package=epiR
  • 22
    Kohl M. MKmisc: miscellaneous functions from M. Kohl. https://CRAN.R-project.org/package=MKmisc (accessed on 14/May/2021).
    » https://CRAN.R-project.org/package=MKmisc
  • 23
    Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Spijker R, Taylor-Phillips S, et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev 2020; (6):CD013652.
  • 24
    Maine GN, Lao KM, Krishnan SM, Afolayan-Oloye O, Fatemi S, Kumar S, et al. Longitudinal characterization of the IgM and IgG humoral response in symptomatic COVID-19 patients using the Abbott Architect. J Clin Virol 2020; 133:104663.
  • 25
    Whitman JD, Hiatt J, Mowery CT, Shy BR, Yu R, Yamamoto TN, et al. Evaluation of SARS-CoV-2 serology assays reveals a range of test performance. Nat Biotechnol 2020; 38:1174-83.
  • 26
    Almeida SM, Spalanzani RN, Nogueira MB, Sanada B, Cavalli BM, Rotta I, et al. Rapid serological tests for SARS-CoV-2: diagnostic performance of 4 commercial assays. Med Princ Pract 2021; 30:385-94.
  • 27
    Dos Santos VA, Rafael MM, Sabino EC, Duarte AJS. Sensitivity of the Wondfo One Step COVID-19 test using serum samples. Clinics (São Paulo) 2020; 75:e2013.
  • 28
    Borges LP, Oliveira MGBD, Souza DRV, Santos KAD, Heimfarth L. Can the Wondfo(r) SARS-CoV-2 IgM/IgG antibodies be used as a rapid diagnostic test? Archives of Biotechnology and Biomedicine 2020; 4:13-7.
  • 29
    Woloshin S, Patel N, Kesselheim AS. False negative tests for SARS-CoV-2 infection: challenges and implications. N Engl J Med 2020; 383:e38.
  • 30
    Abreu MC, Choquet C, Petit H, Bouzid D, Damond F, Marot S, et al. SARS-CoV-2 IGM and IGG rapid serologic test for the diagnosis of COVID-19 in the emergency department. J Infect 2020; 81:816-46.
  • 31
    Flower B, Brown JC, Simmons B, Moshe M, Frise R, Penn R, et al. Clinical and laboratory evaluation of SARS-CoV-2 lateral flow assays for use in a national COVID-19 seroprevalence survey. Thorax 2020; 75:1082-8.

Publication Dates

  • Publication in this collection
    12 Jan 2022
  • Date of issue
    2022

History

  • Received
    15 Mar 2021
  • Reviewed
    18 May 2021
  • Accepted
    02 July 2021
Escola Nacional de Saúde Pública Sergio Arouca, Fundação Oswaldo Cruz Rua Leopoldo Bulhões, 1480 , 21041-210 Rio de Janeiro RJ Brazil, Tel.:+55 21 2598-2511, Fax: +55 21 2598-2737 / +55 21 2598-2514 - Rio de Janeiro - RJ - Brazil
E-mail: cadernos@ensp.fiocruz.br