Analytical assessment of clinical sensitivity and specificities of pharmaceutical rapid SARS-CoV-2 detection nasopharyngeal swab testing kits in Pakistan

Avaliação analítica da sensibilidade e especificidades clínicas de kits de teste de swab nasofaríngeo para detecção rápida de SARS-CoV-2 no Paquistão

Abstract

Despite of the global unity against COVID-19 pandemic, the threat of SARS-CoV-2 variants on the lives of human being is still not over. SARS-CoV-2 pandemic has urged the need of rapid viral detection at earliest. To cope with gradually expanding scenario of SARS-CoV-2, accurate diagnosis is extremely crucial factor which should be noticed by international health organizations. Limited research followed by sporadic marketing of SARS-CoV-2 rapid pharmaceutical detection kits raises critical questions against quality assurance and quality control measures. Herein we aimed to interrogate effectivity and specificity analysis of SARS-CoV-2 pharmaceutical rapid detection kits (nasopharyngeal swab based) using conventional gold standard triple target real-time polymerase chain reaction (USFDA approved). A cross-sectional study was conducted over 1500 suspected SARS-CoV-2 patients. 100 real time-PCR confirmed patients were evaluated for pharmaceutical RDT kits based upon nasopharyngeal swab based kits. The SARS-CoV-2 nasopharyngeal swab based rapid diagnostic kit (NSP RDTs) analysis showed 78% reactivity. Among real time PCR confirmed negative subjects, 49.3% represented false positivity. The positive predictive analysis revealed 67.82%, while negative predictive values were 64.40%. The NSP RDTs showed limited sensitivities and specificities as compared to gold standard real time PCR. Valid and authentic detection of SARS-CoV-2 is deemed necessary for accurate COVID-19 surveillance across the globe. Current study highlights the potential consequences of inadequate detection of SARS-CoV-2 and emerging novel mutants, compromising vaccine preventable diseases. Current study emphasizes need to wake higher authorities including strategic organizations for designing adequate measures to prevent future SARS-CoV-2 epidemics.

Keywords:
SARS-CoV-2; COVID-19; Pakistan; Nasopharyngeal Swab; rapid testing

Resumo

Apesar da unidade global contra a pandemia de COVID-19, a ameaça das variantes do SARS-CoV-2 na vida do ser humano ainda não acabou. A pandemia de SARS-CoV-2 alertou para a necessidade de detecção viral rápida o mais cedo possível. Para lidar com o cenário de expansão gradual do SARS-CoV-2, o diagnóstico preciso é um fator extremamente crucial que deve ser observado pelas organizações internacionais de saúde. Pesquisa limitada seguida de comercialização esporádica de kits de detecção farmacêutica rápida de SARS-CoV-2 levanta questões críticas contra a garantia de qualidade e as medidas de controle de qualidade. Aqui, objetivamos questionar a eficácia e a análise de especificidade dos kits farmacêuticos de detecção rápida de SARS-CoV-2 (baseados em swab nasofaríngeo) usando a técnica de reação em cadeia da polimerase em tempo real padrão-ouro convencional (aprovado pelo USFDA). Um estudo transversal foi realizado em 1.500 pacientes suspeitos de SARS-CoV-2. Cem pacientes confirmados por PCR em tempo real foram avaliados para kits farmacêuticos de RDT baseados em kits baseados em swabs nasofaríngeas. A análise do kit de diagnóstico rápido baseado em swab nasofaríngeo SARS-CoV-2 (NSP RDTs) mostrou 78% de reatividade. Entre os indivíduos negativos confirmados por PCR em tempo real, 49,3% representaram falsa positividade. A análise preditiva positiva revelou 67,82%, enquanto os valores preditivos negativos foram de 64,40%. Os NSP RDTs mostraram sensibilidades e especificidades limitadas em comparação com a PCR em tempo real padrão-ouro. A detecção válida e autêntica do SARS-CoV-2 é considerada necessária para a vigilância precisa de COVID-19 em todo o mundo. O estudo atual destaca as possíveis consequências da detecção inadequada de SARS-CoV-2 e novos mutantes emergentes, comprometendo doenças evitáveis por vacina. O estudo atual enfatiza a necessidade de alertar autoridades superiores, incluindo organizações estratégicas, para projetar medidas adequadas para prevenir futuras epidemias de SARS-CoV-2.

Palavras-chave:
SARS-CoV-2; COVID-19; Paquistão; Swab Nasofaríngeo; teste rápido

1. Introduction

COVID-19 is caused by Severe acute respiratory syndrome (SARS) coronavirus 2 (SARS-CoV-2), the prototype virus of the family Coronaviridae which preferentially infects respiratory tract cells, but also affect other organs such as; brain, conjunctiva, heart, liver, lungs, kidneys and pharynx (Hui et al., 2020; Puelles et al., 2020). COVID-19, ‘CO’ stands for ‘corona,’ ‘VI’ for ‘virus,’ and ‘D’ for disease. Formerly, this disease was referred to as “2019 novel coronavirus” or “2019-nCoV”. The first case of pneumonia patient with unknown cause was officially reported on December 8, 2019. Several cases of pneumonia infected by 2019-nCoV were found in Wuhan, Hubei Province. World Health Organization announced its outbreak as 6^th public health emergency of international concern in January 2020, but two months later it was declared as pandemic. As a large viral family, the coronaviruses can cause major diseases such as colds, Middle East Respiratory Syndrome (MERS), and severe acute respiratory syndrome (SARS). 2019-nCoV is a kind of new coronavirus which has not been found in human before. It belongs to novel β-coronavirus with an envelope, round or oval particles and is often polymorphic, with a diameter of 60-140nm. WHO stated global number of COVID-19 positive cases have been mounted to 646,278,129 among which 6,636,700 deaths, 624,840,065 recoveries, 14,801,364 infected patients during November 2022 were reported worldwide and > 68.5% of the world population has received at least one dose of COVID-19 vaccine (World Health Organization, 2020a, b).

The SARS-CoV-2 contain four major structural proteins; nucleocapsid, matrix core protein, envelop, and glycoprotein spike surface. SARS-CoV-2 utilizes angiotensin‐converting enzyme 2 ACE2 receptor expressed on Alveolar type 2 progenitor (AT2) epithelial cells. The virus penetrates host cell through clathrin- and caveolae- independent endocytic pathways and via host cell directed network of G-protein-coupled receptors it may activate c-Jun N-terminal Kinase (JNK) and Janus Tyrosine Kinase (JAK)-Signal Transducer and Activator of Transcription (STAT) pathways, for enhanced viral replication (Singh et al., 2020).

Those who are infected by COVID-19 often have the symptoms of fever, weakness and dry cough. Few patients have the symptoms of nasal congestion, runny nose, sore throat and diarrhea. Severe patients often suffer from dyspnea and/or hypoxemia one week after onset, and very severe patients rapidly progress to acute respiratory distress syndrome, septic shock, intractable metabolic acidosis and coagulation disorders. Notably, patients with severe or critically ill patients may have moderate to low fever or even no obvious fever. Patients with mild symptoms just have slight fever and weakness without pneumonia. Patient without symptoms have been also reported. When cultured in vitro separately, 2019-nCoV can be found in human respiratory epithelial cells in about 96 hours; when cultured in Vero E6 and Huh-7 cell lines separately, about 6 days. It is known that COVID-19 is mainly transmitted through respiratory droplets and contact. Routes of transmission such as aerosol and digestive tract have yet to be investigated further (World Health Organization, 2020a, b; Singh et al., 2020).

Rapid pharmaceutical diagnostic tests are easy to use, cheaper and safe to use, but there are several potential concerns regarding validation and accurate performance of these diagnostic assays (Pallett et al., 2020). Previously, in comparison to gold-standard PCR positive patients, we showed the sensitivities of nasopharyngeal swab (52%) and saliva based (21%) SARS-CoV-2 antigen based rapid pharmaceutical diagnostic kits in Pakistan (Saeed et al., 2021). To validate further the accurate diagnosis of nasopharyngeal swab rapid pharmaceutical diagnostic kits we further evaluated COVID-19 antigen based kits.

2. Materials and Methods

A cross-sectional, among 1500 COVID-19 suspected subjects, was conducted from capital twin cities (Islamabad and Rawalpindi) of Pakistan. After comprehensive medical examination and history of SARS-CoV-2 suspected patients, the consent was obtained by trained counselor and specimens were sent for analysis at Islamabad Diagnostic Center (Specialized Center for COVID-19), Islamabad, Pakistan. The study was approved by institutional review board of IDC Pakistan. Study investigators had professional background and capacity related to diagnostic test evaluation. The investigators had a full understanding and knowledge of specific contents of the protocol and all indicators through training. The quality control of the IDC laboratory met the requirements of quality control of clinical laboratory to ensure the standardization of experimental operating procedures. The integrity and accuracy of the kits were verified using gold standard methods.

As standard, real time PCR confirmed positive cases were pre-selected for diagnostic kit evaluation. The samples were examined against SARS-CoV-2 via Real-Time PCR assay (Roche, USA) after RNA extraction through Auto pure 32 Zybio China, using standard Primerdesign. The assay included positive control template and RNA internal extraction control. USFDA approved Triple target genes (Sarbecovirus E gene, SARS-CoV-2 N gene, and SARS-CoV-2 RNA-dependent RNA polymerase) were used, along with Seegene kit (#RP10244Y Allplex™ 2019-nCoV Assay, Seegene South Korea) based real time PCR. The detection limit was (100) copies/ml. Positive samples with exponential growth curve and Ct value < 40 were considered as SARS-CoV-2 positive patients. To examine the accuracy of SARS-CoV-2 antigen detection from NSP samples, the colloidal gold labeled SARS-CoV-2 N protein monoclonal antibody based immunochromatographic rapid pharmaceutical diagnostic kit (Lepu Medical, China 20CG2704X) tests were performed and evaluated via gold-standard real time PCR results. After collection, the samples were put in the treatment fluid, stored at 2~8°C and tested within 24h. The samples cannot be placed at room temperature for a long period of time. Discordant results were either excluded or reconfirmed. Whole procedures were performed in accordance to standard manufacturer protocols. The sensitivity, specificity, positive predictive values, and negative predictive values were calculated using gold standard real time PCR results.

3. Results

About 1500 COVID-19 suspected patients were enrolled this study and tested for SARS CoV-2 using gold standard real time PCR based COVID-19 antigen detection kits at Islamabad Diagnostic, Center G8 Markaz, Pakistan. We selected 100 real time PCR positive subjects for evaluation of nasopharyngeal swab based SARS-CoV-2 antigen detection kits. The accuracy of NSP detection kit was determined by triple target USFDA approved Seegene kit. Among selected subjects, 60% were male while 40% were females. 4% of these subjects were children below the age of 18 years. Mean age was 42 (ranging 3-72). In comparison to gold standard, real time PCR based COVID-19 detection kit, the NSP based rapid pharmaceutical diagnostic test showed 78 % reactivity, while remaining 21% showed false negative results. Further validation, was performed to examine the false positivity among PCR negative subjects. The data revealed that 49.3% of the subjects showed false positive results. The sensitivity and specificity were 78% and 50.66%, while the positive predictive value and negative predictive vales of the assays were 67.82% and 64.40% respectively Table 1. Cycle threshold (Ct) values of 100 real time PCR positive samples taken at different days was also maintained. The kit showed zero cross reactivity against Adenovirus, Cytomegalovirus, Epstein-Barr virus, human Coronavirus OC43, human Metapneumovirus, Influenza A virus, Influenza B virus, Measels virus, Mumps virus, Mycoplasma pneumonia, Norovirus, Respiratory Syncytial virus, Rotavirus, and Varicella Zosyer virus.

Table 1
Evaluation of diagnostic accuracy of nasopharyngeal swab based SARS-CoV-2 rapid diagnostic kits in Pakistan.

Samples	Ct (Av)	Real-Time PCR	Age (Av)	NSP-RDT	Fever	Cough	Breathing problem
Male (n=60)	27.7	Positive	40.02 (19-72)	Reactive (n=48)	Positive	Positive	Positive
				N-reactive (n=12)
Female (n=40)	28.8	Positive	44.1 (20-66)	Reactive (n=31)	Positive	Positive	Positive
				N-reactive (n=9)
Children (n=4)	30.6	Positive	9.75 (3-17)	Reactive (n=4)	Positive	Positive	Negative
				N-reactive (n=0)
Total (n=100)	29	Positive	42 (3-72)	Sensitivity (78%)	Positive	Positive	Positive
				Specificity (50.66%)
				Positive predictive value (67.82%)
				Negative predictive value (64.40%)

• Abbreviations: Av refers to average; Ct refers to Cycle Threshold; PCR refers to Polymerase Chain Reaction; NSP-RDT refers to Nasopharyngeal Rapid Diagnostic Test.

4. Discussion

Previously we evaluated SARS-CoV-2 rapid pharmaceutical diagnostic test in Pakistan and demonstrated that RDT based diagnostic kits need to be investigated before commercialization of the products (Saeed et al., 2021). Despite of limited sensitivities and specificities several SARS-CoV-2 RDT based kits are being utilized by many diagnostic centers across Pakistan which needs to be re-monitored on priority basis to estimate accurate COVID-19 trends among several populations in Pakistan.

During first wave of COVID-19 in Pakistan, people vigilantly implemented standard operating procedures and successfully combat SARS-CoV-2. However, during second wave of COVID-19 in Pakistan, there were several unknown reasons possibly one of them would be ineffective testing strategy or poor diagnostic means which bring misunderstandings during the initial screenings. Several RDT based diagnostic tests were used with no reference to gold standard real time PCR based detection.

Since WHO recommendations indicate that SARS-CoV-2 RDTs tests should have minimum sensitivity of ≥ 80% and specificity of ≥ 97% (World Health Organization, 2020a). During the second wave of COVID-19 we aimed to re-investigate new COVID-19 testing RDT diagnostic kits. In continuation of our previous study (Saeed et al., 2021), current study indicated that in comparison to gold standard, real time PCR based COVID-19 detection kit, the nasal RDT showed improved 78 % sensitivity, while 21% showed false negative results. Nasopharyngeal and oropharyngeal swabs of 82 real time PCR confirmed patients revealed sensitivity of 93.9% via SARS-CoV-2 antigen test kits (Porte et al., 2020). Similar study from China, reported 68% sensitivity in 208 real time PCR confirmed nasopharyngeal swab samples (Diao et al., 2020). While 83.43% sensitivity was reported by SARS-CoV-2 RDT kits in Kuwait (Altawalah et al., 2020). Recently, from Pakistan nasal and saliva based RDTs revealed sensitivities of 52% and 21% respectively (Saeed et al., 2021). The NSP based RDTs kit showed zero cross reactivity against Human metapneumovirus, Mumps virus, Rotavirus, Adenovirus, Epstein-Barr virus, human Coronavirus OC43, Mycoplasma pneumonia Varicella Zosyer virus, Respiratory Syncytial virus, Cytomegalovirus, Measels virus, Influenza viruses and Norovirus. The current study revealed that real time PCR analysis from study enrolled subjects (during second COVID-19 wave) revealed that prevalence rate of COVID-19 during the period of November in Islamabad/Rawalpindi region of Pakistan was 6.67%, which is 22 times higher than COVID-19 prevalence during October 2020 (Saeed et al., 2021).

5. Conclusion

Accurate and timely diagnosis of SARS-CoV-2 can further prevent progression of COVID-19, but low standard rapid diagnostic kits in low income countries due to lack of resources, are providing misinformation. Current study provides consolidate analysis of SARS-CoV-2 nasopharyngeal swab rapid testing kits in Pakistan. And highlights the actual sensitivity, specificities, positive predictive values and negative predictive values of the nasopharyngeal swab rapid testing kits. Current study is torch bearer for strategic organizations for policy making against low standardized commercially available kits due to which misdiagnosis of COVID-19 take place and newly emerging strains of SARS-CoV-2 are sporadically disseminating across the globe. This study is critical for international public health organizations to devise adequate prevention strategies against COVID-19. Current study is more reliable, as same nasal samples were used for comparison of real time PCR and RDTs, without possible distribution error from other specimen. The kit manufacturer analysis showed 90% sensitivity of nasal RDTs among Chinese population. However possibly due to different epidemiological conditions or interracial factors as compare to China, the findings of present study are not consistent with Pakistani population, which warrants further in-depth investigations on RDT in Pakistan. The rapid pharmaceutical tests results in current study were not satisfactory, and combination-test algorithm should be employed for accurate diagnosis of SARS-CoV-2 during second COVID-19 wave in Pakistan.

References

• ALTAWALAH, H., ALHURAISH, F., ALKANDARI, W.A. and EZZIKOURI, S., 2020. Saliva specimens for detection of severe acute respiratory syndrome coronavirus 2 in Kuwait: a cross-sectional study. Journal of Clinical Virology, vol. 132, pp. 104652. http://dx.doi.org/10.1016/j.jcv.2020.104652 PMid:33053493.
  » http://dx.doi.org/10.1016/j.jcv.2020.104652
• DIAO, B., WEN, K., CHEN, J., LIU, Y., YUAN, Z., HAN, C., CHEN, J., PAN, Y., CHEN, L., DAN, Y., WANG, J., CHEN, Y., DENG, G., ZHOU, H. and WU, Y. (2020). Diagnosis of acute respiratory syndrome coronavirus 2 infection by detection of nucleocapsid protein. http://dx.doi.org/10.1101/2020.03.07.20032524
  » http://dx.doi.org/10.1101/2020.03.07.20032524
• HUI, K.P.Y., CHEUNG, M.C., PERERA, R.A.P.M., NG, K.C., BUI, C.H.T., HO, J.C.W., NG, M.M.T., KUOK, D.I.T., SHIH, K.C., TSAO, S.W., POON, L.L.M., PEIRIS, M., NICHOLLS, J.M. and CHAN, M.C.W., 2020. Tropism, replication competence, and innate immune responses of the coronavirus SARS-CoV-2 in human respiratory tract and conjunctiva: an analysis in ex-vivo and in-vitro cultures. The Lancet. Respiratory Medicine, vol. 8, no. 7, pp. 687-695. http://dx.doi.org/10.1016/S2213-2600(20)30193-4 PMid:32386571.
  » http://dx.doi.org/10.1016/S2213-2600(20)30193-4
• PALLETT, S.J.C., RAYMENT, M., PATEL, A., FITZGERALD-SMITH, S.A.M., DENNY, S.J., CHARANI, E., MAI, A.L., GILMOUR, K.C., HATCHER, J., SCOTT, C., RANDELL, P., MUGHAL, N., JONES, R., MOORE, L.S.P. and DAVIES, G.W., 2020. Point-of-care serological assays for delayed SARS-CoV-2 case identification among health-care workers in the UK: a prospective multicentre cohort study. The Lancet. Respiratory Medicine, vol. 8, no. 9, pp. 885-894. http://dx.doi.org/10.1016/S2213-2600(20)30315-5 PMid:32717210.
  » http://dx.doi.org/10.1016/S2213-2600(20)30315-5
• PORTE, L., LEGARRAGA, P., VOLLRATH, V., AGUILERA, X., MUNITA, J.M., ARAOS, R., PIZARRO, G., VIAL, P., IRURETAGOYENA, M., DITTRICH, S. and WEITZEL, T., 2020. Evaluation of a novel antigen-based rapid detection test for the diagnosis of SARS-CoV-2 in respiratory samples. International Journal of Infectious Diseases, vol. 99, pp. 328-333. http://dx.doi.org/10.1016/j.ijid.2020.05.098 PMid:32497809.
  » http://dx.doi.org/10.1016/j.ijid.2020.05.098
• PUELLES, V.G., LÜTGEHETMANN, M., LINDENMEYER, M.T., SPERHAKE, J.P., WONG, M.N., ALLWEISS, L., CHILLA, S., HEINEMANN, A., WANNER, N., LIU, S., BRAUN, F., LU, S., PFEFFERLE, S., SCHRÖDER, A.S., EDLER, C., GROSS, O., GLATZEL, M., WICHMANN, D., WIECH, T., KLUGE, S., PUESCHEL, K., AEPFELBACHER, M. and HUBER, T.B., 2020. Multiorgan and renal tropism of SARS-CoV-2. The New England Journal of Medicine, vol. 383, no. 6, pp. 590-592. http://dx.doi.org/10.1056/NEJMc2011400 PMid:32402155.
  » http://dx.doi.org/10.1056/NEJMc2011400
• SAEED, U., UPPAL, S.R., PIRACHA, Z.Z., RASHEED, A., AFTAB, Z., ZAHEER, H. and UPPAL, R., 2021. Evaluation of SARS-CoV-2 antigen-based rapid diagnostic kits in Pakistan: formulation of COVID-19 national testing strategy. Virology Journal, vol. 18, no. 1, pp. 34. http://dx.doi.org/10.1186/s12985-021-01505-3 PMid:33581714.
  » http://dx.doi.org/10.1186/s12985-021-01505-3
• SINGH, Y., GUPTA, G., SATIJA, S., PABREJA, K., CHELLAPPAN, D.K. and DUA, K., 2020. COVID-19 transmission through host cell directed network of GPCR. Drug Development Research, vol. 81, no. 6, pp. 647-649. http://dx.doi.org/10.1002/ddr.21674 PMid:32329083.
  » http://dx.doi.org/10.1002/ddr.21674
• WORLD HEALTH ORGANIZATION - WHO. 2020a [viewed 1 July 2022]. Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassay [online]. Available from: https://www.who.int/publications/i/item/antigen-detection-in-the-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays
  » https://www.who.int/publications/i/item/antigen-detection-in-the-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays
• WORLD HEALTH ORGANIZATION - WHO, 2020b [viewed 1 July 2022]. Coronavirus disease 2019 (COVID-2019): situation report — 51 [online]. Available from: https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200311-sitrep-51-covid-19.pdf
  » https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200311-sitrep-51-covid-19.pdf

Publication Dates

• Publication in this collection
  04 Mar 2024
• Date of issue
  2024

History

• Received
  01 July 2022
• Accepted
  28 Oct 2022