Comparative analysis of the accuracy of urinary hCG tests in vitro

Abstracts

OBJETIVO: Identificar se os pontos de corte de sensibilidade anunciados por três testes de gravidez na urina são compatíveis com os anunciados pelo fabricante e descrever os seus desempenhos diagnósticos. MÉTODOS: A urina de um voluntário masculino foi usada para diluir β-hCG recombinante em concentrações definidas de 0; 6,25; 12,5; 25; 50; e 100 mUI/mL. As amostras foram codificadas e cegamente analisadas para a positividade em três diferentes lotes dos testes hCG Strip Test Plus®, BioEasy® e Visitect Pregnancy®. O tamanho da amostra foi calculado para um erro alfa de 5%, com um poder de 99%. RESULTADOS: As três marcas apresentaram sensibilidade de 100% na detecção do β-hCG nos três lotes analisados, com 100% de valor preditivo negativo, usando somente controles negativos e amostras com concentrações iguais ou superiores ao limite do teste (n = 180/marca). A acurácia dos testes foi 83% (BioEasy®), 84% (Visitect®) e 91% (Strip Test Plus®). O Strip Test Plus® apresentou o melhor desempenho para a razão de probabilidade positiva (52,5), enquanto que o produto Visitect® teve a melhor razão de probabilidade negativa (zero). CONCLUSÃO: Os três produtos analisados têm a sensibilidade dos pontos de corte anunciados. O produto Strip Test Plus® tem o melhor desempenho para identificar concentrações urinárias de β-hCG > 12,5 mUI/mL, consequentemente, confirmando gravidez, enquanto que o Visitect® tem o melhor desempenho para descartar a presença de β-hCG na urina (probabilidade pós-teste negativo: zero).

Testes de gravidez; urina; diagnóstico in vitro; gonadotropina coriônica

ORIGINAL ARTICLE

Comparative analysis of the accuracy of urinary hCG tests in vitro

Gisele Silva de Moraes^I; Rafael do Amaral Cristovam^I; Ricardo Francalacci Savaris^II

^IMedical Students; Medical School, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, RS, Brazil

^IIPhD in Medical Sciences; Adjunct Professor, UFRGS; Professor, Service of Gynecology and Obstetrics, Hospital de Clínicas de Porto Alegre, Porto Alegre, RS, Brazil

Correspondence to

SUMMARY

OBJECTIVE: To identify whether cutoff for sensitivity advertised by three pregnancy tests in urine are compatible to those reported by the manufacturer and to describe their diagnostic performance.

METHODS: The urine of a male volunteer was used to dilute recombinant β-hCG at defined concentrations of 0, 6.25, 12.5, 25, 50, and 100 mIU/mL. The tubes containing each of the concentrations were coded and blindly assessed for positivity in three different lots of hCG tests: Strip Test Plus^®, BioEasy^®, and Visitect Pregnancy^®. The sample size was calculated for an alpha error of 5%, with a power of 99%.

RESULTS: All three brands, in their three lots analyzed, had 100% of sensitivity for detecting β-hCG, with 100% negative predictive value, using only negative controls and samples with concentrations equal or higher than the test cutoff (n = 180/brand). The accuracy of the tests was 83% (BioEasy^®), 84%(Visitect^®) and 91% (Strip Test Plus^®). Strip Test Plus^® had the best positive likelihood ratio (52.5), while Visitect^® had the best negative likelihood ratio (zero).

CONCLUSION: The three brands have adequate sensitivity for the advertised cutoffs. The Strip Test Plus^® test had the best performance to identify urinary concentrations of β-hCG > 12.5 mIU/mL, and consequently, to confirm pregnancy, while Visitect^® had the best performance to exclude β-hCG in urine (negative post-test probability: zero).

Keywords: Pregnancy tests; urine; reagent kits, in vitro diagnosis; chorionic gonadotropin.

Introduction

Urine pregnancy tests have been widely used and accepted as the first step in detecting early pregnancy in gynecological emergencies¹, as in addition to being rapid², the current qualitative pregnancy tests in serum or urine have similar sensitivity³. Since their introduction in the market in 1975, the number of available brands has significantly increased, with the production of monoclonal antibodies against the β subunit of hCG and the use of enzyme immunoassays that solved the problems of using radioimmunoassay. Initially, these tests needed 2 hours to show the result.

In the 1980s, the immunochromatic tests to detect hCG in urine appeared, and in the 1990s, the generation of the "dipstick" test emerged⁴. Currently, manufacturers of these products advertise sensitivity of 25 mIU/mL, and accuracy between 97 and 99%³. This sensitivity to detect urinary hCG is not affected by the dilution of urine⁵, unless tests with sensitivity > 200 mIU/mL are used⁶. However, international studies have shown that these parameters advertised by the manufacturers are not reproduced by independent researchers^7,8. Cervinski et al.⁹ demonstrated that some types of pregnancy tests in urine have better analytical sensitivity when compared to others. Tomlinson et al.¹⁰ compared the accuracy of six pregnancy tests and verified that the method that used a digital system to give the result had significantly better accuracy compared to the other methods.

In Brazil, there are more than 100 types of tests to detect the presence of pregnancy in urine; some manufacturers give unclear information about product accuracy, such as for instance, "this test has 99.4% of agreement with results obtained with the use of other immunological pregnancy tests previously qualified by current clinical practice"¹¹.

In Brazil, the In-Vitro Diagnostic Product Team (GEVIT / GGTPS) of the National Health Surveillance Agency (ANVISA) says "the methods used for product qualification have not yet been determined or listed by ANVISA, but each manufacturer determines, among the methods recognized in international procedures, those that best suit their needs". Given that the data submitted for the products come from the manufacturers themselves, it would be important, compared to what is done in the international literature, to verify whether the advertised accuracy can be reproduced by independent investigators.

Moreover, a search for law cases related to pregnancy tests at JusBrasil site (www.jusbrasil.com.br) reveals that there are 1,639 civil cases related to false-positive or false-negative results (search performed on 04/12/2011, keywords: pregnancy test). Similar cases have been described in the literature^12,13. The importance of this study is the precise information on the accuracy of the product and on their threshold sensitivity values for the detection of hCG, so that doctors can understand test results and medical experts can have foundations in legal proceedings. The objectives of this research are to identify whether the cutoff sensitivity of three products used for the diagnosis of urinary hCG are consistent with those announced by the manufacturers, as well as to describe their diagnostic performance (sensitivity, specificity, positive and negative predictive values, accuracy, positive and negative likelihood ratios), with known concentrations of recombinant β-hCG diluted in urine as reference standards.

Methods

Analyzed products

Three brands of immunochromatographic pregnancy tests in urine available in the Brazilian market were obtained: hCG Strip Test Plus^® (InLab Diagnostica, SP, Brazil), β-hCG S&U One Step BioEasy^® (Acon Biotech Hangzhou, China) and Visitect Pregnancy^® (Omega Diagnostics, Scotland, UK). The tests were obtained through the same public notice made by Hospital de Clínicas de Porto Alegre (HCPA), specifically for this study, asking the manufacturers to send three different lots of the same brand, whereas they did not know that their products would be undergoing analysis. There was no preference for any specific brand, except for Strip Test Plus^®, as it is the standard test used in HCPA. All lots were within the expiry date.

The Strip Test Plus^®test lots consisted of a dipstick with sensitivity of 25 mIU/mL for the diagnosis of pregnancy. According to the manufacturer, the test has a 99.4% concordance with results obtained with other tests¹¹. Test reading was carried out five minutes after immersing the test strip in the urine sample.

The BioEasy^® test lots consisted of a dipstick from sensitivity of 25 mIU/mL. The manufacturer advertises sensitivity and specificity of 100% (95% CI: 0.95 to 1) and accuracy of 100% (95% CI: 0.98 to 1) for the detection of β-hCG in the urine, which was evaluated with n = 159¹⁴. The test results were verified five minutes after immersion of the test strips for twenty seconds in the urine samples.

The Visitect Pregnancy^® test lots consisted of a strip and a dropper for each individual test. The manufacturer advertises that the test sensitivity is from 10 mIU/mL, its specificity does not undergo cross-reaction and its accuracy shows good correlation when performed in parallel with other pregnancy tests on the market, using known positive and negative controls¹⁵. The test was performed by placing two drops of each urine sample with an individual dropper and the result was verified after five minutes.

Sample preparation

A male urine sample was used as diluent. The recombinant β-hCG concentration at a concentration of 5,000,000 IU/mL (Choragon Ferring GmbH, Kiel, Germany) was diluted in 49 mL of urine, resulting in a working solution at a concentration of 100 mIU/mL. From this solution, serial 50, 12.5 and 6.25 mIU/mL dilutions were made, in a final volume of 10 mL. In one of the test tubes, 10 mL of pure urine was prepared for analysis as a negative control. During the dilutions, materials were not reused to prevent contamination.

Urine and recombinant β-hCG were stored up to 72 hours under refrigeration at 4ºC, and were not frozen. The test tubes at concentrations 100; 50; 25; 12.5; 6.25 and 0 mIU/mL were codified and randomly prepared for analysis by one investigator (GSM), who did not participate in the test reading.

Test performance and reading

One of the authors (RAC), blinded to the concentrations in the tubes, performed the tests and carried out the readings according to the manufacturers' instructions, determining them as positive, when there were two distinct markings on the test; negative when the test line was absent; when the positive test line was not clear, the test was considered indeterminate. Tests without the control line were discarded. For readings, indeterminate results were considered positive, as they could mean positivity below the cutoff point recommended by the manufacturer.

The readings were performed consecutively, without interruption, under the same conditions of temperature and pressure to prevent measurement bias.

Sample size

The sampling was based on similar studies, published in the international literature⁷ and by the Food and Drug Administration of the United States¹⁶ and considering an alpha error of 0.05; a 0.99 power to detect an increase of 81% among controls and cases. These parameters established 10 different replicates for each concentration (n = 6) for each lot (n = 3 lots), in the three analyzed brands (n = 10 x 6 x 3 = 180 analysis for each brand).

Statistical analysis and ethical aspects

Data were analyzed using a contingency table for diagnostic test evaluation parameters, using the software GrahPad InStat, release 3.00 for Mac (GraphPad Software, San Diego California USA, www.graphpad.com) and the diagnostic test calculator (http://araw.mede.uic.edu/cgi-bin/testcalc.pl) for a confidence interval of 95%.

The comparison of accuracy between the three products and the lots was assessed using two-way ANOVA (product and lot), and considering significant a p < 0.05. For performance analysis, we considered three different scenarios: (1) product performance with negative concentrations and concentrations > the cutoff recommended by the manufacturer, (2) product performance with all concentrations, having as positive values the cutoffs established by the manufacturer, (3) product performance with all concentrations, but having 12.5 mIU/ mL, as positive value (equivalent to pregnancy), as suggested by Butler et al.⁸

Scenario 1 represents the manufacturers' data; scenario 2, the clinical practice, which considers the value of 10 or 25 mIU/mL a pregnancy diagnosis, and scenario 3, an intermediate situation, where the diagnosis of pregnancy is suggested from 12 mIU/mL, as this is the minimum concentration found in pregnancies with 4 weeks of gestational age⁸. This study was approved by the Research Ethics Committee of the Research and Postgraduate Group of Hospital de Clínicas de Porto Alegre, under # 09-301.

Results

All tests showed positive internal control and it was not necessary to discard any test. The three brands identified the minimum values specified by the manufacturers, if indeterminate results were considered positive. The individual data of each lot are shown in Table 1. Some strips were positive for hCG even at lower concentrations than the threshold specified by the manufacturer (Table 1). Table 2 represents scenario 1, product performance using only the negative controls (zero) and values > product cutoff, without intermediate values. Scenario 2 is shown in Table 3, which describes product performance to identify any concentration of β-hCG in the urine, having the cutoffs of each product as positive tests.

Table 4 represents scenario 3, which shows product performance where values > 12 mIU/mL are considered positive. The performance of the three brands was analyzed considering the three different scenarios. Statistical analysis with 2-way ANOVA showed no significant difference between brands and lots (evaluated factors), with accuracy as the variable to be analyzed (Tables 5 and 6).Lots, as well as the different types of tests, had no effect on test accuracy, with a cutoff value of β-hCG > 6.25 mIU/mL.

Discussion

The three analyzed brands use the principle of immunochromatography with antigen-antibody reaction and have very similar instructions. These products vaguely inform the consumer about test accuracy. For instance, Strip Test Plus^® stated that it has "an agreement of 99.4% with the results obtained when using other tests," but gives no further information. The accuracy of the tests is different between lots of the same test, and lower than that advertised by the manufacturer, as shown in Table 5. These differences, however, are not statistically significant and do not influence test results (Table 6). BioEasy^®, on the other hand, gives accurate information about product sensitivity, specificity and accuracy, but does not inform which hCG concentrations were analyzed.

The accuracy found for BioEasy^®, in the present study, is out of the 95% CI established by the manufacturer. These differences may be due to the β-hCG cutoff (6.26 mIU/mL) that was used, which represents the clinical practice scenario. If we consider scenario 1, which reflects the manufacturers' data, where there are only β-hCG concentrations > the cutoff of the product and the negative control, two brands (Strip Test Plus^® and BioEasy^®) show specificity, sensitivity and accuracy of 100%, as advertised by the manufacturer (Table 2). The brand Visitect^® showed an indeterminate result for 5 negative samples, which reduced its specificity and its accuracy was 94% (Table 2).

Indeterminate tests (weakly positive) were seen in all samples and lots of the three products. None of the manufacturers distinguishes the results in which the coloration or marks of positivity was not clear. We verified that the majority of indeterminate results were seen at β-hCG concentrations below the product threshold (Table 1). Consequently, we considered the test as positive. Therefore, the results of the manufacturers are compatible with our findings within scenario 1; however, this does not apply to clinical practice.

Table 3 shows scenario 2, where we used any presence of hCG in the urine as a positive value. In scenario 2, the performance of each product deteriorated. The accuracy between the products and their lots ranged between 68% and 100%. This scenario is what the clinician sees in everyday practice. The clinician considers the diagnosis of pregnancy is confirmed when the test result is positive for β-hCG levels > 25 mIU/mL, and below this threshold, the patient is considered non-pregnant. A more detailed analysis can verify that when there is a positive result for Strip Test Plus^® and BioEasy^®, the clinician will be sure that there is β-hCG level > 25 mIU/mL in 100% (95% CI: 0.96 to 1) of the cases, confirming the pregnancy. On the other hand, with a negative result, this certainty decreases to 41% (95% CI: 0.29 to 0.52) for the Strip Test Plus^®, as there may be very early pregnancies, and consequently, with β-hCG levels < 25 mIU/mL. These observations may explain the discrepancies reported in the literature^12,13.

For the clinician, the question to be answered is: which test should be used to diagnose or rule out pregnancy? To answer this, Butler et al⁷. reported that the concentration of hCG in urine at 4 weeks of gestation ranges from 12 to 2,438 IU/L, or 12 to 2,438 mIU/mL. Therefore, scenario 3 is represented in Table 4, where values > 12.5 mIU/mL are considered positive, and those < 6.25 mIU/mL, negative. For the clinician, the most important value of an auxiliary test is the likelihood ratio¹⁴. Together with these data, one associates the pre-test odds, i.e., the prevalence of cases of pregnancy in the setting where the test is performed. A recent study published by our group showed that the prevalence of pregnant women who were treated at the gynecological emergency department, without a pregnancy diagnosis, is 18.5%¹⁷.

Considering these data, we verify that Strip Test Plus^®, when positive, has a probability of 92.3% that the urinary β-hCG is > 12.5 mIU/mL, and therefore we conclude that the patient is pregnant; as for Visitect^®, when it shows a negative result, the chance of having a β-hCG > 12.5 mIU/mL is zero, therefore ruling out the diagnosis of pregnancy. [These values were obtained using the formula: positive post-test probability = post-test odds/(post-test odds +1), where post-test odds = pretest odds x likelihood ratio; pretest odds = prevalence/(1-prevalence)]¹⁵. A negative result of the Strip Test Plus^® indicates that there is a 2.9% chance of pregnancy and the test did not recognize the pregnancy because the β-hCG concentration is approximately 12 mIU/mL in a 4-week gestation⁷.

A positive aspect of this study is the use of recombinant β-hCG at concentrations known as the gold standard. This substitutes the measurements of β-hCG by radioimmunoassay or enzyme immunoassay. Additionally, with pre-defined concentrations of β-hCG, it was possible to identify whether the products identified the lower limits of sensitivity specified by the manufacturer, that is, 10 mIU/mL and 25 mIU/mL.

The negative results, at urinary concentrations of 6.25 and 12.5 mIU/mL of β-hCG, as well as positive results in negative controls, may partly explain the situations of false positive and false negative results in clinical practice¹⁶. Another explanation, in addition to the limits below the cutoff established by the product manufacturer, is the possibility that the monoclonal antibody against the regular hCG that is found in the kit does not react against hyperglycosilated hCG (H-hCG), which comprehends 61% of all immunoreactive hCGs in the first four weeks of gestation⁷.

Conclusion

In conclusion, all three products analyzed recognize β-hCG in the urine at sensitivity cutoffs announced by manufacturers, if the indeterminate cases are considered positive. Some lots of three brands identified values below the cutoff advertised in the product package insert. Strip Test Plus^® has the best accuracy in identifying urinary concentrations of β-hCG, with a positive likelihood ratio of 52.5, while Visitect^® has the best performance to rule out a pregnancy, with a negative likelihood ratio of 0.

References

• 1. Minaretzis D, Tsionou C, Tsandoulas E. Clinical evaluation of a sensitive urine pregnancy test in the gynecological emergencies. Eur J Obstet Gynecol Reprod Biol 1991;39:123-6.
• 2. Lazarenko GC, Dobson C, Enokson R, Brant R. Accuracy and speed of urine pregnancy tests done in the emergency department: a pros-pective study. CJEM 2001;3:292-5.
• 3. Chard T. Pregnancy tests: a review. Hum Reprod 1992;7:701-10.
• 4. Haarburger D, Pillay TS. Historical perspectives in diagnostic clinical pathology: development of the pregnancy test. J Clin Pathol 2011;64(6):546-8.
• 5. Neinstein L, Harvey F. Effect of low urine specific gravity on pregnancy testing. J Am Coll Health 1998;47:138-9.
• 6. komi A, Matthews M, Kuan AM, Henson G. The effect of physio-logical urine dilution on pregnancy test results in complicated early pregnancies. Br J Obstet Gynaecol 1998;105:462-5.
• 7. Cole LA, Sutton-Riley JM, Khanlian SA, Borkovskaya M, Rayburn BB, Rayburn WF. Sensitivity of over-the-counter pregnancy tests: comparison of utility and marketing messages. J Am Pharm Assoc 2005;45:608-15.
• 8. Butler SA, Khanlian SA, Cole LA. Detection of early pregnancy forms of human chorionic gonadotropin by home pregnancy test devices. Clin Chem 2001;47:2131-6.
• 9. Cervinski MA, Lockwood CM, Ferguson AM, Odem RR, Stenman UH, Alfthan H et al. Qualitative point-of-care and over-the-counter urine hCG devices differentially detect the hCG variants of early pregnancy. Clin Chim Acta 2009;406:81-5.
• 10. Tomlinson C, Marshall J, Ellis JE. Comparison of accuracy and certainty of results of six home pregnancy tests available over-the-counter. Curr Med Res Opin 2008;24:1645-9.
• 11. Alamar Tecno Científico Ltda. HCG Strip Test Plus, bula do produto. Disponível em: http://www.interlabdist.com.br/dados/produtos/bula/doc/15487490742feb212a.pdf
• 12. Er TK, Chiang CH, Cheng BH, Hong FJ, Lee CP, Gines MA. False-positive urine pregnancy test in a woman with adenomysosis. Am J Emerg Med 2009;27:1019.e5-7.
• 13. Kalinski MA, Guss DA. Hemorrhagic shock from a ruptured ectopic pregnancy in a patient with a negative urine pregnancy test result. Ann Emerg Med 2002;40:102-5.
• 14. Bioeasy Diagnóstica Ltda. HCG one step ultra bioeasy. Disponível em: http://www.bioeasy.com.br/arquivos/Produtos/59/hcg_10muiml__c__e__t__30_07_2010.pdf
• ¹⁵
  Omega diagnostics - Visitect Pregnancy bula. Disponível em: http://www.biosys.com.br/data/jpf_kovbio_product/422/visitecpregnancy2.pdf
• 16. USA-FDA. Human chorionic-gonadotropin (hCG) 510(k)s. 2000. Disponível em: http://www.accessdata.fda.gov/cdrh_docs/reviews/K071930.pdf
• 17. Savaris RF, Moraes GS, Cristovam RA. Reviewing the guidelines from the Brazilian Ministry of Health for diagnosis of early pregnancy. Rev Assoc Med Bras 2010;56:622-4.

Publication Dates

• Publication in this collection
  14 Oct 2011
• Date of issue
  Oct 2011

History

• Accepted
  15 June 2011
• Received
  24 Jan 2011