SciELO - Scientific Electronic Library Online

 
vol.24 issue11The feminization of dentistry and the perceptions of public service users about gender issues in oral healthOral Health Impact Profile: need and use of dental prostheses among Northeast Brazilian independent-living elderly author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


Ciência & Saúde Coletiva

Print version ISSN 1413-8123On-line version ISSN 1678-4561

Ciênc. saúde coletiva vol.24 no.11 Rio de Janeiro Nov. 2019  Epub Oct 28, 2019

http://dx.doi.org/10.1590/1413-812320182411.01012018 

FREE THEMES

The sensitivity, specificity, and agreement of a point of care method: an assessment of the diagnostic accuracy

Sensibilidade, especificidade e concordância do método point of care: um estudo de precisão diagnóstica

Erika da Silva Maciel1 
http://orcid.org/0000-0002-9836-7665

Fernando Rodrigues Peixoto Quaresma2 
http://orcid.org/0000-0001-8407-0310

Francisco Winter dos Santos Figueiredo3 
http://orcid.org/0000-0002-9377-6443

Jonathan Souza Sarraf4 
http://orcid.org/0000-0001-9477-3618

Taynah Putty Luis4 
http://orcid.org/0000-0002-0380-5731

Luis Fernando Castagnino Sesti5 
http://orcid.org/0000-0002-4322-9353

Fernando Luiz Affonso Fonseca6 
http://orcid.org/0000-0003-1223-1589

Fernando Adami3 
http://orcid.org/0000-0003-1918-3364

1Universidade Federal do Tocantins. Av. Lourdes Solino s/n, Setor Universitário. 77650-000 Miracema do Tocantins, TO, Brasil. erikasmaciel@uft.edu.br

2Curso de Enfermagem, Universidade Federal do Tocantins. Palmas, TO, Brasil.

3Faculdade de Medicina do ABC. Santo André, SP, Brasil.

4Oncológica do Brasil. Belém, PA, Brasil.

5Universidade Luterana do Brasil. Palmas, TO, Brasil.

6Universidade Federal de São Paulo. Diadema, SP, Brasil.


Abstract

The use of new technologies can improve screening in communities with difficult access to health. This article aims to evaluate the sensitivity, specificity, and agreement of a point of care test in comparison to laboratory methods for the determination of glucose (GLI), triglyceride (TG), and total cholesterol (TC) concentrations. This prospective study used data from the remaining adult population of quilombolas in Brazil. Laboratory tests using conventional methods for the analysis of venipuncture samples were used as a standard method to measure the concentrations of GLI (mg/dL), TG (mg/dL), and TC (mg/dL) and compared to the metered dose from the collection of fingertip capillary blood (point of care). Contingency tables (2x2) were used to estimate the sensitivity and specificity of the methods. Lin and Bland & Altman coefficients were used to statistically assess agreement, the level of significance was 5%. There was substantial agreement between the methods for measuring TG and poor agreement for of TC and GLI. Analysis of the Bland & Altman coefficients revealed that the fingertip method did not produce good measures. The point of care method did not offer a good ability to measure compared to that of the reference laboratory method.

Key words Point of care; Laboratory tests; Technology Assessment

Resumo

O uso de novas tecnologias pode melhorar o screening em comunidades de difícil acesso à saúde. O objetivo deste artigo é avaliar a sensibilidade, especificidade e concordância do teste de point of care em comparação com método laboratorial para dosagem de Glicose (GLI), Triglicerídeo (TG) e Colesterol total (CT). Estudo prospectivo com dados de população de adultos remanescentes de quilombolas no Brasil. Exames laboratoriais convencionais para análise foram obtidos por venopunção, utilizados como método padrão para mensuração das concentrações de GLI (mg/dL), TG (mg/dL) e CT (mg/dL) e comparados a mensuração por meio de técnica de ponta de dedo (point of care). Tabelas de contingência (2x2) foram utilizadas para estimar sensibilidade e especificidade dos métodos e o coeficiente de Lin e análises de Bland & Altman foram métodos de concordância com nível de significância de 5%. Houve concordância substancial entre os métodos para mensuração de TG e fraca concordância para mensuração de CT e GLI. Os coeficientes de Bland & Altman indicam que o método de ponta de dedo não apresentou boa mensuração. O método point of care não apresentou boa capacidade de mensuração de Glicose, Triglicerídeo e Colesterol total tendo como referência o método laboratorial.

Palavra-chave Point of care Systems; Testes Laboratoriais; Avaliação da tecnologia biomédica

Introduction

Advances in laboratory tests for patient diagnosis and follow-up have been implemented in recent years, but the centralization of physical and professional infrastructure in more developed areas limits access to exams in less accessible regions1.

Limited access to health care in populations located outside of large centers, such as rural populations, quilombolas2, riverside, and sertanejos, has been one associated with the high prevalence of health problems in these populations3,4. Lack of basic sanitation, geographic distance, and low socioeconomic status do not favor the availability of healthcare human resources; thus, the health monitoring network is fragile at various levels of health care. This difficulty in accessing health care influences disease prevention, health promotion, diagnostic testing, medicine availability5, treatment administration, post-treatment monitoring2, and chronic disease screening.

Amongst laboratory tests, blood tests are performed to help diagnose diseases such as diabetes, dyslipidemias, and parasitic and bacterial infections, among others6,7, as well as for health monitoring.

Fingertip blood collected for glycemic evaluation is already performed for follow-up purposes in patients with diabetes mellitus; there is a certain reliability for this purpose, but not for the diagnosis of this condition8. Thus, there is a gap in the reliability of this method for screening for changes in triglyceride, glucose, and total cholesterol levels9.

This method is convenient and rapid and its sensitivity and specificity have been tested and approved for epidemiological studies in difficult to reach places10. However, the point-of-care method has been criticized for the accuracy of diagnosis11, with recommendations for additional studies to determine their accuracy and applicability12.

In populations such as quilombolas and riverside communities, the laboratory methods tests become complex and other methods of measurement are used in an attempt to optimize care. The point-of-care method may provide high sensitivity and specificity and good reliability for the measurement of triglyceride, glucose, and total cholesterol compared to laboratory tests.

Thus, the objective of the present study was to evaluate the sensitivity, specificity, and agreement of the fingertip test compared to those of the laboratory measurements of glucose, triglycerides, and total cholesterol in a quilombola community.

Materials and methods

Study design

In accordance with the prior planning, this prospective study collected fasting, venous puncture samples. Following laboratory examinations, measurements were also performed using the point-of-care equipment.

Participants

The study participants were residents of a quilombola rural community located in the Southeastern region of the state of Tocantins. The eligibility criteria were:

  • - Over 18 years of age of both sexes;

  • - A resident of the community;

  • - Provided voluntary consent to participate in the research;

Test methods

Index test

The Accutrend Roche Plus® device (enzymatic method) was used to measure capillary parameters. (registration ANVISA: 10287410740). A drop of blood was collected by puncture from the tip of the finger for each tested parameter using a specific reagent strip. The readings were performed immediately after blood collection.

Reference standard

Laboratory tests using conventional enzymatic-colorimetric methods and venipuncture samples were performed to measure the concentrations of glucose (mg/dL), triglycerides (mg/dL), and total cholesterol (mg/dL). Two tubes were collected by venipuncture containing fluoride to evaluate glucose and with a separator gel to obtain blood serum to allow the measurement of cholesterol and triglyceride concentrations. The samples were placed in isothermal cases and transported to the University Laboratory of Clinical Analysis for testing, followed good laboratory practices; we used registered reagents (ROCHE®).

The same classification criteria were used for the index and reference tests.

For the diagnosis of a change in glucose concentration, the normal 8-hour concentration was defined as 126 mg/dL, according to American Diabetes Association guidelines13. The evaluation of total cholesterol and triglyceride levels adopted the cutoff values of 200 and 150 mg/dL, respectively, according to the reference values recommended by the V Brazilian Directive on Dyslipidemia and Prevention of Atherosclerosis14.

Analysis

Contingency (2x2) tables were used to calculate the sensitivity and specificity of the classification methods. To analyze the agreement between the methods, we used Lin’s correlation, Spearman’s correlation, and Bland & Altman’s concordance coefficients. The level of significance was 5%. Stata® was used to perform these analyses (Stata Corp., College Station, USA) 11.0.

Ethical aspects

The following project was approved by the Committee of Ethics in Research with Human Beings.

Results

Participants

All residents of the quilombola community who met the eligibility criteria were invited to participate voluntarily. Since there are no baseline studies on the use of point of care kits in quilombola communities, the sample size calculation was not performed and the number of individuals living in the communities during the study determined the sample size. A survey previously conducted with community leaders estimated the total population of 146 adults and elderly, but only 31 volunteers participated in the study. On the date scheduled for data collection, there were 31 potentially eligible adults. (Figure 1)

Figure 1 Participant flowchart. 

Only one of these individuals did not participate in the study because they were not fasting at the time of collection. Thus, blood samples were collected from 30 participants with a mean age of 47 years (SD = 18.28). All participants had been residents of a rural quilombola community for more than one year.

Test results

The sensitivities and specificities of the point-of-care method in relation to the laboratory method for the diagnosis of glucose concentration > 126 mg/dL, total cholesterol level > 200 mg/dL, and triglyceride level > 150 mg/dL were 25% and 95.45%, 100% and 69.23%, and 100% and 80%, respectively (Table 1).

Table 1 Sensitivity and specificity of laboratory analysis (gold standard) and fingertip point-of-care methods 

Evaluated parameter (mg/dL) Sensitivity (%) Specificity (%)
Glucose 25 95.45
Cholesterol 100 69.23
Triglycerides 100 80

The concordance, as assessed by Lin, Spearman correlation, and Bland & Altman concordance coefficients, are shown in Table 2 and plotted in Figures 2, 3, and 4.

Table 2 Lin's concordance, Spearman's correlation, and Bland & Altman's concordance coefficients of laboratory and fingertip methods for the measurement of triglyceride, glucose, and cholesterol levels. 

Variables Lin's coefficient (95%CI) Bland &Altman (95%CI)
p* Spearman p**
Triglycerides 0.91 (0.84; 0.98) <0.001 0.4 <0.001 5.35 (-44.16; 54,86)
Glucose 0.63 (0.44; 0.82) <0.001 0.78 <0.001 -22.50 (-70.20; 25.10)
Total cholesterol 0.86 (0.74; 0.98) <0.001 0.89 <0.001 8.70 ( -18.64; 36.16)

*Lin's coefficient;

**Spearman's correlation; 95%CI: 95% confidence interval

Figure 2 Lin (A) and Bland & Altman (B) correlation coefficients between laboratory and fingertip methods for the measurement of triglyceride levels. 

Figure 3 Lin (A) and Bland & Altman (B) correlation coefficients of laboratory and fingertip methods for glucose measurement. 

Figure 4 Lin (A) and Bland & Altman (B) correlation coefficients of laboratory and fingertip methods for cholesterol measurement. 

There was substantial agreement between the methods for the measurement of triglyceride concentration (Lin coefficient = 0.91, 95% confidence interval [CI] 0.84-0.98), and poor agreement for measuring cholesterol and glucose (Lin coefficient = 0, 86, 95% CI 0.74-0.98, and 0.63, 95% CI 0.44-0.82, respectively). However, analysis based on the Bland & Altman coefficients, suggested that the point-of-care method was not a good procedure for evaluating triglyceride (Bland & Altman = 5.35, 95% CI: -22.50, 95% CI: -70.20, 25.10) or total cholesterol (Bland & Altman = 8.70, 95% CI -18.64; 36.16) concentrations (Table 2).

Discussion

When evaluating the sensitivity, specificity, and agreement of the fingertip test in comparison to those of laboratory measurements of glucose, triglycerides, and total cholesterol concentrations in the quilombola community, we found a significant correlation, but comparison of the classification of the results based on the changes in concentrations can result in confusion and, therefore, increased errors in the use of the point-of-care method.

The sensitivity of the point-of-care method showed good results for the measurement of triglycerides and total cholesterol concentrations. As for specificity, the result was satisfactory only for fasting blood glucose assessment, where the test could correctly diagnose approximately 96% of the cases.

Despite few studies on fingertip devices, there is some agreement with the potential for error in the independent measurement of the component to be evaluated (glucose, total cholesterol, or triglycerides)9,15, as we observed in the present study.

When comparing glycemic rates based on laboratory techniques, similar results were obtained with handheld glucometers, even different brands and models, with close agreement and statistically nonsignificant differences, although the characteristics of these devices, such as fast return times, small sample volumes, and ease of handling are indisputable16.

The device used in the present study for the analysis of cholesterol levels has shown good accuracy compared to other brands and models. Since dyslipidemia is a risk factor for cardiovascular disease, it is recommended to screen populations; thus, the use of fingertip devices appears to be an appropriate alternative15.

The characteristics that may influence measurements include pH variations, blood oxygen, hematocrit, changes in microcirculation, and vasopressor therapy in addition to external factors, which include room temperature and incorrect handling of reagent tape. These elements alone or combined can significantly impact the accuracy of the devices17.

Fingertip testing can accelerate clinical management and increase the sensitivity of cardiovascular disease screening campaigns in populations with limited resources1. This is a low-cost test for cholesterol and triglyceride assay, which may serve as an alternative strategy for conducting epidemiological studies and screening for cardiovascular diseases in these populations, as these indicators are directly related to cardiovascular events as well as monitoring fasting blood glucose values10,18.

Point-of-care devices offer a practical option in epidemiological studies, which allows access in places where conventional techniques are limited10,19. Since these devices are used in situations with limited resources, the benefits need to outweigh the costs20 in addition to increasing convenience without sacrificing data reliability.

Cardiovascular disease is the leading cause of death worldwide, with 80% of cardiovascular events occurring in low- and middle-income countries; thus, reliable data on the prevalence of risk factors in developing countries could be obtained from door-to-door epidemiological studies using automatic devices2.

Similarly, the prevalence of diabetes has increased in recent years; monitoring is necessary to maintain glycemic control within an acceptable range, both in the hospital and outpatient settings. The accuracy of the measurements plays an important role in treatment decisions for glycemic control17.

A systematic review of published studies on point-of-care tests in limited resource settings observed that there is no clearly defined metric for the clinical utility of these tests in different pathologies20. As there is a risk of error in the diagnosis, especially regarding the critical values, both high and low, that compromise the proper use of the devices9, the high probability of a measurement error and subsequent unsuitable treatment should be considered when using this technology21,22.

Based on the concordance methods proposed by Bland & Altman23 that compare methods and the variation in the observed results, the fingertip test did not appear to be a good predictor for triglyceride, fasting glucose, and total cholesterol indicators in the present study. However, the results of the present study should be interpreted with caution since, although there was statistical agreement between the methods, there were also some limitations.

The main limitation, in this case, was in establishing how acceptable measurement variations are in comparison with the reference method, since these variations may lead to errors in treatment and diagnosis17. To minimize the potential bias of the point-of-care method, some researchers have suggested that studies consider retesting with the device9,21.

In addition, the reduced sample size, despite recruiting the entire resident adult population, is a limitation of the study and may have been responsible for observed variations, limiting statistical inference24.

Thus, in the present study, there did not appear to be a good correlation between the evaluated methods; the accuracy of the fingertip device was similar to the trends reported in the literature and suggest the need for more evidence on this topic4-6.

Conclusion

In this study, the point-of-care method did not show a good capacity to measure glucose, triglyceride, and total cholesterol levels compared to the laboratory method in this study population. This result implies the potential for improper classification when using this method. Caution is advised in the use of the point-of-care methods in population and epidemiological studies. Additional studies with larger samples are necessary to demonstrate their usefulness.

Acknowledgments

Financial support by Programa Pesquisa para o SUS - PPSUS/TO FAPT-TO/SESAU-TO/MS-DECIT/CNPq - Fomento Programa Regular FAPESP.

References

1 Drain PK, Hyle EP, Noubary F, Freedberg KA, Wilson D, Bishai WR, Rodriguez W, Bassett IV. Diagnostic point-of-care tests in resource-limited settings. Lancet Infect Dis 2014;14(3):239-249. [ Links ]

2 Marques AS, Freitas DA, Leão CDA, Oliveira SKM, Pereira MM, Caldeira AP. Atenção Primária e saúde materno-infantil: a percepção de cuidadores em uma comunidade rural quilombola. Ciên Saude Colet 2014;19(2):365-371. [ Links ]

3 Oliveira SKM, Caldeira AP. Fatores de risco para doenças crônicas não transmissíveis em quilombolas do norte de Minas Gerais. Cad Saude Colet 2016;24(4):420-427. [ Links ]

4 Bezerra VM, Medeiros DS, Gomes KO, Souzas R, Giatti L, Steffens AP, Kochergin CN, Souza CL, Moura CS, Soares DA, Santos LRCS, Cardoso LGV, Martins PC, Neves OSC, Guimarães MDC. Inquérito de Saúde em Comunidades Quilombolas de Vitória da Conquista, Bahia, Brasil (Projeto COMQUISTA): aspectos metodológicos e análise descritiva. Ciên Saude Colet 2014;19(6):1835-1847. [ Links ]

5 Medeiros DS, Moura CS, Guimarães MDC, Acurcio FA. Medication use by the "quilombola" population: A survey in Southwestern Bahia, Brazil. Rev Saude Publica 2013;47(5):905-913. [ Links ]

6 Nicoll D, Lu C, Pignone M, McPhee S. Manual de exames diagnósticos [livro na Internet]. 2013 [acessado 24 Jun 2017]. Disponível em: https://books.google.com.br/books?hl=pt-BR&lr=&id=Kxk7AgAAQBAJ&oi=fnd&pg=PA1&dq=Manual+de+Exames+Diagnósticos.&ots=PUGmhG4LSf&sig=0SEFSj67Yh_R0xipQLfPmKGE3AcLinks ]

7 Sacher R, McPherson R. Interpretação clínica dos exames laboratoriais [Internet]. 2002 [acessado 24 Jun 2017]. Disponível em: https://scholar.google.com.br/scholar?q=interpretação+Clínica+dos+Exames+Laboratoriais&btnG=&hl=pt-BR&as_sdt=0%2C5Links ]

8 Maciel TES. Correlação de medidas de glicosímetros e dosagem laboratorial [dissertação na Internet]. Curitiba: Universidade Tecnológica Federal do Paraná; 2014 [acessado 24 Jun 2017]. Disponível em: http://repositorio.utfpr.edu.br:8080/jspui/handle/1/971Links ]

9 Schifman RB, Howanitz PJ, Souers RJ. Point-of-Care Glucose Critical Values: A Q-probes study involving 50 health care facilities and 2349 critical results. Arch Pathol Lab Med 2016;140(2):119-124. [ Links ]

10 Rapi S, Bazzini C, Tozzetti C, Sbolci V, Modesti PA. Point-of-care testing of cholesterol and triglycerides for epidemiologic studies: evaluation of the multicare-in system. Transl Res 2009;153(2):71-76. [ Links ]

11 Dale RA, Jensen LH, Krantz MJ. Comparison of two point-of-care lipid analyzers for use in global cardiovascular risk assessments. Ann Pharmacother 2008;42(5):633-639. [ Links ]

12 Plüddemann A, Thompson M. Point-of-care testing for the analysis of lipid panels: primary care diagnostic technology update. Br J Gen Pract 2012;62(596):224-226. [ Links ]

13 American Diabetes Association (ADA). Standarts of Medical Care in Diabetes. Diabetes Care [periódico na Internet]. 2005 [acessado 24 Jun 2017];28(Supl. 1):S4-S36. Disponível em: https://care.diabetesjournals.org/content/28/suppl_1/s4.full-text.pdfLinks ]

14 Xavier HT, Izar MC, Faria Neto JR, Assad MH, Rocha VZ, Sposito AC, Fonseca FA, Santos JE, Santos RD, Bertolami MC, Faludi AA, Martinez TLR, Diament J, Guimarães A, Forti NA, Moriguchi E, Chagas ACP, Coelho OR, Ramires JAF. V Diretriz Brasileira de dislipidemias e prevenção da aterosclerose. Arq Bras Cardiol [periódico na Internet]. 2013 [acessado 24 Jun 2017];101(4 Supl. 1):1-22. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0066-782X2013004100001&lng=en&nrm=iso&tlng=ptLinks ]

15 Haggerty L, Tran D. Cholesterol point-of-care testing for community pharmacies: a review of the current literature. J Pharm Pract 2017;30(4):451-458. [ Links ]

16 Rajendran R, Rayman G. Point-of-care blood glucose testing for diabetes care in hospitalized patients: an evidence-based review. J Diabetes Sci Technol 2014;8(6):1081-1090. [ Links ]

17 Rebel A, Rice MA, Fahy BG. Accuracy of point-of-care glucose measurements. J Diabetes Sci Technol 2012;6(2):396-411. [ Links ]

18 Scafoglieri A, Tresignie J, Provyn S, Pieter J, Ivan C. Reproducibility, accuracy and concordance of Accutrend® Plus for measuring circulating lipid concentration in adults. Biochemia Medica 2012;22(1):100-108. [ Links ]

19 Hu J, Wang SQ, Wang L, Li F, Pingguan-Murphy B, Lu TJ, Xu F. Advances in paper-based point-of-care diagnostics. Biosens Bioelectron 2014;54:585-597. [ Links ]

20 Drain PK, Hyle EP, Noubary F, Freedberg KA, Wilson D, Bishai WR, Rodriguez W, Bassett IV. Diagnostic point-of-care tests in resource-limited settings. Lancet Infect Dis 2014;14(3):239-249. [ Links ]

21 Schifman RB, Nguyen TT, Page ST. Reliability of point-of-care capillary blood glucose measurements in the critical value range. Arch Pathol Lab Med 2014;138(7):962-966. [ Links ]

22 Lonjaret L, Claverie V, Berard E, Riu-Poulenc B, Geeraerts T, Genestal M, Fourcade O. Relative accuracy of arterial and capillary glucose meter measurements in critically ill patients. Diabetes Metab 2012;38(3):230-235. [ Links ]

23 Bland JM, Altman DG. Comparing two methods of clinical measurement: a personal history. Int J Epidemiol 1995; 24(Supl. 1):S7-S14. [ Links ]

24 Figueiredo F, Adami F. Steps for the decision making based on the statistical analyses. Journal of Human Growth and Development 2017; 27 (3):350-352. [ Links ]

Received: August 27, 2017; Accepted: March 29, 2018; Published: March 31, 2018

Authors’ contributions

ES Maciel contributed in the generation of the topic, preparation of proposal, data acquisition, analyses, interpretation drafting and development of the manuscript. FRP Quaresma and FWS Figueiredo contributed to the study design, analysis, and interpretation of results. JS Sarraf and TP Luis contributed to study conception, and interpretation of results. LFC Sesti, FLA Fonseca and F Adami contributed to the interpretation of results and development of the manuscript. All authors read and approved the final manuscript.

Creative Commons License This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.