Sensitivity and specificity in prevalence studies: The importance of considering uncertainty

Serological surveys, such as EPICOVID19 (¹), are important to monitor the evolution of COVID-19 in a population. In this letter, we discuss how to best estimate its prevalence. It is well known that the naive estimator of prevalence that consists of counting how many individuals tested positive ignores the possibility of test errors and may therefore substantially bias the conclusions. The often-used Rogan-Gladen estimator (²) is an alternative that provides corrected confidence intervals based on sensitivity/specificity values. However, this estimator has two main issues: (i) it often yields negative estimates of prevalence, and (ii) it assumes that the precision of the test is known with certainty, which is never the case; sensitivity/specificity are estimated from data. In this letter we focus on (ii) and demonstrate that taking the uncertainty regarding the precision of the test into account provides a different perspective for serological surveys. Our illustrative example is based on ENE-COVID (³), which investigates the prevalence of COVID-19 in Spain.

Since we do not have access to the exact numbers, we assume that among the 61,075 individuals in the survey, 3,054 (5%) tested positive on the point-of-care test. We use the sensitivity/specificity values provided in the paper: 82.1% (69.6%-91.1%) and 100.0% (96.5%-100.0%), respectively. For the sake of simplicity, we ignore sampling weights. Figure 1 shows 95% confidence intervals using different approaches. The naive estimate (i.e., the proportion of individuals that tested positive) has a small interval with no intersection with the Rogan-Gladen estimate, which is also short. On the other hand, the Bayesian interval that takes this uncertainty (⁴) into account is much wider; it contains points that are consistent with very different stages of the evolution of the epidemic. It is evident that the interval should be wide; it is not possible to recover the prevalence of the disease from data about the proportion of positive tests alone. Indeed, the statistical model is not identifiable (⁵). We conclude that uncertainties must also be transparently reported to subsidize decisions properly. An app that performs the analyses presented on new data can be found at https://rizbicki.shinyapps.io/tests/.

Figure 1
Confidence (Naive and Rogan-Gladen) and credible (Bayesian) intervals for prevalence of COVID-19.

REFERENCES

• ¹
  Hallal PC, Barros FC, Silveira MF, Barros AJD, Dellagostin OA, Pellanda LC, et al. EPICOVID19 protocol: repeated serological surveys on SARS-CoV-2 antibodies in Brazil. Cien Saude Colet. 2020;25(9):3573-38. https://doi.org/10.1590/1413-81232020259.25532020
  » https://doi.org/10.1590/1413-81232020259.25532020
• ²
  Rogan WJ, Gladen B. Estimating prevalence from the results of a screening test. Am J Epidemiol. 1978;107(1):71-6. https://doi.org/10.1093/oxfordjournals.aje.a112510
  » https://doi.org/10.1093/oxfordjournals.aje.a112510
• ³
  Pollán M, Pérez-Gómez B, Pastor-Barriuso R, Oteo J, Hernán MA, Pérez-Olmeda M, et al. Prevalence of SARS-CoV-2 in Spain (ENE-COVID): a nationwide, population-based seroepidemiological study. Lancet. 2020;396(10250):535-44. https://doi.org/10.1016/S0140-6736(20)31483-5
  » https://doi.org/10.1016/S0140-6736(20)31483-5
• ⁴
  Gelman A, Carpenter B. Bayesian analysis of tests with unknown specificity and sensitivity. medRxiv. 2020 Jan 1.
• ⁵
  Wechsler S, Izbicki R, Esteves LG. A Bayesian look at nonidentifiability: A simple example The American Statistician. 2013;67(2):90-93.

Publication Dates

• Publication in this collection
  09 Dec 2020
• Date of issue
  2020