Performance of instruments aimed at detecting obstructive sleep apnea syndrome among individuals in Chile

TO THE EDITOR:

Obstructive sleep apnea syndrome (OSAS) is a common condition, affecting 5-20% of the adult population.¹ Validated diagnostic methods, including polysomnography and home sleep apnea testing (HSAT), are not available at every center, making them poorly accessible in most countries, including Chile.

Clinical prediction models are useful in order to evaluate the population at risk for OSAS.² However, it remains unclear which instrument or measure is most effective. It seems to depend on the population studied, and there are no surveys or clinical parameters that have been validated for use in Chile. We designed a study to evaluate the performance of the Snoring, Tiredness, Observed apnea, high blood Pressure, Body mass index, Age, Neck circumference, and Gender (STOP-Bang) questionnaire, the Epworth Sleepiness Scale,³ the snoring scale,⁴ the Sleep Apnea Clinical Scale (SACS),⁵ and determination of neck circumference in patients suspected of having OSAS in Chile.

Adult patients undergoing sleep studies were prospectively evaluated for clinical suspicion of OSAS and were referred for HSAT. Initially, we evaluated a population of such patients treated at a clinical hospital between 2013 and 2016. We subsequently evaluated a population of such patients treated at a private clinic between 2016 and 2018. Patients suspected of having a sleep disorder other than OSAS were excluded. Before conducting the diagnostic study, we applied the following instruments in all of the patients: the Epworth Sleepiness Scale³; the modified snoring scale⁶; the STOP-Bang questionnaire⁷; and the SACS,⁵ which is used in order to calculate the adjusted neck circumference (ANC).² The study was approved by the local research ethics committee, and all participating patients gave written informed consent.

To perform HSAT, we used a portable sleep monitoring device (Embletta; Natus Medical, Foster City, CA, USA). In accordance with the requirements of the American Academy of Sleep Medicine recommendations for level 3 sleep studies (those involving the use of portable equipment, performed in the home or elsewhere),⁸ the tests were performed in the home of the patient, who is given prior instruction. The HSAT results were analyzed manually by a pulmonologist. The patient was categorized as having OSAS if the apnea-hypopnea index (AHI) was ≥ 5 events/h and as having severe OSAS if the AHI was ≥ 30 events/h.

The results are expressed as mean ± standard deviation. Quantitative variables were analyzed by Student’s t-tests, and qualitative variables were compared by chi-square tests with confidence intervals (CIs). The result of the HSAT was used as the reference standard, normal HSAT results being used as the reference population.

Discrimination was evaluated by constructing ROC curves for each AHI cutoff point. We calculated the values for sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio. The ROC curves were used in order to evaluate tests that presented area under the curve (AUC) > 0.7. Data analysis and recording were performed with Excel 2011 software and with the Statistical Package for the Social Sciences, version 12.0 (SPSS Inc., Chicago, IL, USA). Values of p < 0.05 were considered statistically significant.

A total of 759 patients were included in the study: 520 in the clinical hospital group; and 239 in the private clinic group. In the clinical hospital group, the AUC for the risk of a diagnosis of OSAS, as determined with the various instruments, were as follows: STOP-Bang questionnaire (AUC = 0.77; CI: 0.70-0.84); SACS (AUC = 0.77; CI: 0.71-0.84); ANC determination (AUC = 0.79; CI: 0.72-0.85); snoring scale (AUC = 0.63; CI: 0.55-0.70); and Epworth Sleepiness Scale (AUC = 0.48; CI: 0.39-0.56). The risk of a diagnosis of severe OSAS was comparable for the SACS, STOP-Bang questionnaire, and ANC determination (AUC = 0.852, 0.837, and 0.863, respectively). As can be seen in Table 1, the best STOP-Bang cutoff score was ≥ 5 points, which had a sensitivity and specificity of 81.7% and 61.4%, respectively, for predicting an AHI ≥ 5 events/h and of 90.0% and 61.4%, respectively, for predicting an AHI ≥ 30 events/h. The best cutoff score on the SACS was ≥ 48 points, which had a sensitivity and specificity of 66.2% and 72.4%, respectively, for predicting a diagnosis of OSAS and of 77.4% and 72.4%, respectively, for predicting a diagnosis of severe OSAS. Finally, the best cutoff ANC was ≥ 40 cm, which had a sensitivity and specificity of 77.3% and 67.2%, respectively, for predicting a diagnosis of OSAS and of 87.5% and 67.2%, respectively, for predicting a diagnosis of severe OSAS. In our validation cohort (private clinic group), contingency tables showed that those cut-off points still had the best sensitivity and specificity for predicting a diagnosis of OSAS-STOP-Bang score ≥ 5: 78.1% and 72.1%, respectively; SACS score ≥ 48: 54.9% and 84.1%, respectively; and an ANC ≥ 40 cm: 81.5% and 59.1%, respectively-as well as for predicting a diagnosis of severe OSAS-STOP-Bang score ≥ 5: 92.1% and 72.1%, respectively; SACS score ≥ 48: 66.1% and 84.1%, respectively; and an ANC ≥ 40 cm: 87.1% and 59.1%, respectively.

Table 1
Performance of instruments employed for predicting an apnea-hypopnea index ≥ 5 events/h or ≥ 30 events/h in patients suspected of having obstructive sleep apnea syndrome in Chile.

Cutoff	Clinical hospital group						Private clinic group
	(n = 520)						(n = 239)
	Se	Sp	PPV	NPV	LR+	LR−	Se	Sp	PPV	NPV	LR+	LR−
	STOP-Bang (AHI ≥ 5 events/h)
≥ 1 pt	100.0	0.0	88.7	*	1.0	*	100.0	0.0	81.7	*	1.0	*
≥ 2 pts	99.8	1.8	88.8	50.0	1.02	0.13	99.5	2.3	82.0	50.0	1.02	0.22
≥ 3 pts	99.8	8.8	89.6	83.3	1.09	0.03	98.4	20.9	84.8	75.0	1.24	0.07
≥ 4 pts	96.0	22.8	90.7	41.9	1.24	0.18	92.7	41.9	87.7	56.3	1.59	0.17
≥ 5 pts	81.7	61.4	94.3	29.9	2.12	0.3	78.1	72.1	92.6	42.5	2.8	0.3
≥ 6 pts	55.0	80.7	95.7	18.6	2.85	0.56	46.9	88.4	94.7	27.1	4.03	0.6
≥ 7 pts	23.3	91.2	95.4	13.2	2.65	0.84	19.8	97.7	97.4	21.4	8.51	0.82
8 pts	5.1	100.0	100.0	11.9	*	0.95	4.7	100.0	100.0	19.0	*	0.95
	STOP-Bang (AHI ≥ 30 events/h)
≥ 1 pt	100.0	0.0	78.7	*	1.0	*	100.0	0.0	59.0	*	1.0	*
≥ 2 pts	100.0	1.8	78.9	100.0	10.2	0.0	100.0	2.3	59.6	100.0	1.02	0.0
≥ 3 pts	100.0	8.8	80.2	100.0	1.1	0.0	100.0	20.9	64.6	100.0	1.26	0.0
≥ 4 pts	99.0	22.8	82.5	86.7	1.28	0.04	100.0	41.9	71.3	100.0	1.72	0.0
≥ 5 pts	90.0	61.4	89.6	62.5	2.33	0.16	92.1	72.1	82.9	86.1	3.3	0.11
≥ 6 pts	69.0	80.7	92.9	41.4	3.58	0.38	55.6	88.4	87.5	57.6	4.78	0.5
≥ 7 pts	30.5	91.1	92.8	25.9	3.41	0.76	27	97.7	94.4	47.7	11.6	0.75
8 pts	5.7	100.0	100.0	22.4	*	0.94	9.5	100.0	100.0	43.0	*	0.9
	SACS (AHI ≥ 5 events/h)
38-42 pts	98.7	19.0	90.6	64.7	1.22	0.07	98.5	15.9	83.8	70.0	1.17	0.1
43-47 pts	91.6	39.7	92.4	37.1	1.52	0.21	89.2	50.0	88.8	51.2	1.78	0.22
≥ 48 pts	66.2	72.4	95.0	21.2	2.4	0.47	54.9	84.1	93.9	29.6	3.45	0.54
	SACS (AHI ≥ 30 events/h)
38-42 pts	100.0	19.0	82.5	100.0	1.23	0.0	100.0	15.9	62.6	100.0	119.0	0.0
43-47 pts	98.2	39.7	86.1	85.2	1.63	0.05	98.4	50.0	73.5	95.7	1.97	0.03
≥ 48 pts	77.4	72.4	91.4	45.7	2.8	0.31	66.1	84.1	85.4	63.8	4.16	0.4
	Neck circumference (AHI ≥ 5 events/h)
38 cm	86.8	46.6	92.8	30.7	1.62	0.28	92.3	34.1	86.1	50.0	1.4	0.23
39 cm	83.1	60.3	94.3	31.0	2.1	0.28	87.7	40.9	86.8	42.9	148.0	0.3
40 cm	77.3	67.2	94.9	27.1	2.36	0.34	81.5	59.1	89.8	41.9	1.99	0.31
41 cm	68.0	75.9	95.7	22.9	2.82	0.42	69.2	68.2	90.6	33.3	2.18	0.45
42 cm	56.5	84.5	96.7	19.6	3.64	0.51	55.4	77.3	91.5	28.1	2.44	0.58
43 cm	45.0	87.9	96.7	16.7	3.73	0.63	42.7	88.6	94.3	26.2	3.76	0.65
44 cm	36.6	94.8	98.3	15.8	7.07	0.67	31.3	95.5	96.8	23.9	6.88	0.72
≥ 45 cm	28.4	96.6	98.5	14.5	8.22	0.74	21.5	100.0	100.0	22.3	*	0.78
	Neck circumference (AHI ≥ 30 events/h)
38 cm	94.6	46.6	87.1	69.2	1.77	0.12	93.5	34.1	66.7	78.9	1.42	0.19
39 cm	92.3	60.3	89.9	67.3	2.33	0.13	90.3	40.9	68.3	75.0	1.53	0.24
40 cm	87.8	67.2	91.1	59.1	2.68	0.18	87.1	59.1	75.0	76.5	2.13	0.22
41 cm	78.7	75.9	92.6	48.4	3.26	0.28	77.4	68.2	77.4	68.2	2.43	0.33
42 cm	67.0	84.5	94.3	40.2	4.32	0.39	59.7	77.3	78.7	57.6	263	0.52
43 cm	57.0	87.9	94.7	34.9	4.72	0.49	54.8	88.6	87.2	58.2	4.83	0.51
44 cm	48.0	94.8	97.2	32.4	9.27	0.55	41.9	95.3	92.9	53.2	9.02	0.61
≥ 45 cm	38.9	96.6	97.7	293	11.29	0.63	29.0	100.0	100.0	50.0	*	0.71

• Se: sensitivity; Sp: specificity; PPV: positive predictive value; NPV: negative predictive value; LR+: positive likelihood ratio; LR−: negative likelihood ratio; STOP-Bang: Snoring, Tiredness, Observed apnea, high blood Pressure, Body mass index, Age, Neck circumference, and Gender (questionnaire); AHI: apnea-hypopnea index; pt(s): point(s); and SACS: Sleep Apnea Clinical Scale. *Not estimable.

In the present study, the STOP-Bang questionnaire showed the best performance in a population at risk for OSAS, similar to what was reported in a study conducted in Brazil.⁹ In a systematic review and meta-analysis, a STOP-Bang cutoff score ≥ 3 had an AUC of 0.72 (with a sensitivity and specificity of 90% and 49%, respectively) for predicting an AHI ≥ 5 events/h.¹⁰ For predicting an AHI ≥ 30 events/h, the authors found the sensitivity and specificity of that same cut-off score to be 96% and 25%, respectively. However, when they applied a STOP-Bang cutoff score ≥ 5, the predictive performance of the questionnaire was similar to that found in the present study.

The SACS is easy to use, and a score of < 43 makes the presence of OSAS less likely. It can thus help clinicians exclude patients with very low probability of presenting OSAS. In addition, the simplicity of the index, which evaluates only four domains (daily functioning, social interactions, emotional functioning, and symptoms), makes it very attractive to use. A score ≥ 48 has a sensitivity and specificity of 72% and 84%, respectively, for the diagnosis of OSAS; therefore, patients meeting that criterion could be studied by HSAT.²

The ANC is the most useful anthropometric measure for the study of OSAS, more useful than weight and body mass index. Even when compared with nocturnal oximetry parameters, ANC has been shown to be better,¹¹ with an OR of 3.72 (CI: 2.2-6.31) when an ANC cut-off ≥ 41 cm is applied. In the present study, the ANC with the best sensitivity and specificity was 40 cm. This measurement is very useful in men who know their shirt size, because wearing a shirt with a ≥ 17-inch neck translates to having an ANC ≥ 43 cm, which has a sensitivity of 45% and a specificity of 88% for the diagnosis of OSAS.

Other questionnaires showed unclear utility in clinical practice. First, the Epworth Sleepiness Scale had the lowest sensitivity of the instruments used in predicting OSAS. Second, for the selected targets of ≥ 5 and ≥ 30 events/h, the snoring scale presented AUCs of 0.63 and 0.69, respectively. Another candidate, the Berlin questionnaire, was not tested, because of its documented poor performance in the Chilean population.¹²

In conclusion, among individuals suspected of having OSAS in Chile, the STOP-Bang questionnaire and the SACS, by virtue of their high sensitivity, may help clinicians identify those who do not require further study. In contrast, individuals with a STOP-Bang score ≥ 5, a SACS score ≥ 48, or an ANC ≥ 43 cm are likely to present OSAS, requiring prioritization of their diagnosis and treatment.

REFERENCES

• ¹
  Durán J, Esnaola S, Rubio R, Iztueta A. Obstructive sleep apnea-hypopnea and related clinical features in a population-based sample of subjects aged 30 to 70 yr. Am J Respir Crit Care Med. 2001;163(3 Pt 1):685-9. https://doi.org/10.1164/ajrccm.163.3.2005065
  » https://doi.org/10.1164/ajrccm.163.3.2005065
• ²
  Flemons WW. Clinical practice. Obstructive sleep apnea. N Engl J Med. 2002;347(7):498-504. https://doi.org/10.1056/NEJMcp012849
  » https://doi.org/10.1056/NEJMcp012849
• ³
  Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep. 1991;14(6):540-5. https://doi.org/10.1093/sleep/14.6.540
  » https://doi.org/10.1093/sleep/14.6.540
• ⁴
  Thornton AT, Singh P, Ruehland WR, Rochford PD. AASM criteria for scoring respiratory events: interaction between apnea sensor and hypopnea definition. Sleep. 2012;35(3):425-32. https://doi.org/10.5665/sleep.1710
  » https://doi.org/10.5665/sleep.1710
• ⁵
  Flemons WW, Whitelaw WA, Brant R, Remmers JE. Likelihood ratios for a sleep apnea clinical prediction rule. Am J Respir Crit Care Med. 1994;150(5 Pt 1):1279-85. https://doi.org/10.1164/ajrccm.150.5.7952553
  » https://doi.org/10.1164/ajrccm.150.5.7952553
• ⁶
  Richard W, Kox D, den Herder C, Laman M, van Tinteren H, de Vries N. The role of sleep position in obstructive sleep apnea syndrome. Eur Arch Otorhinolaryngol. 2006;263(10):946-50. https://doi.org/10.1007/s00405-006-0090-2
  » https://doi.org/10.1007/s00405-006-0090-2
• ⁷
  Chung F, Yegneswaran B, Liao P, Chung SA, Vairavanathan S, Islam S, et al. STOP questionnaire: a tool to screen patients for obstructive sleep apnea. Anesthesiology. 2008;108(5):812-21. https://doi.org/10.1097/ALN.0b013e31816d83e4
  » https://doi.org/10.1097/ALN.0b013e31816d83e4
• ⁸
  Rosen IM, Kirsch DB, Carden KA, Malhotra RK, Ramar K, Aurora RN, et al. Clinical Use of a Home Sleep Apnea Test: An Updated American Academy of Sleep Medicine Position Statement. J Clin Sleep Med. 2018;14(12):2075-2077. https://doi.org/10.5664/jcsm.7540
  » https://doi.org/10.5664/jcsm.7540
• ⁹
  Duarte RLM, Fonseca LBM, Magalhães-da-Silveira FJ, Silveira EAD, Rabahi MF. Validation of the STOP-Bang questionnaire as a means of screening for obstructive sleep apnea in adults in Brazil. J Bras Pneumol. 2017;43(6):456-463. https://doi.org/10.1590/s1806-37562017000000139
  » https://doi.org/10.1590/s1806-37562017000000139
• ¹⁰
  Nagappa M, Liao P, Wong J, Auckley D, Ramachandran SK, Memtsoudis S, et al. Validation of the STOP-Bang Questionnaire as a Screening Tool for Obstructive Sleep Apnea among Different Populations: A Systematic Review and Meta-Analysis. PLoS One. 2015;10(12):e0143697. https://doi.org/10.1371/journal.pone.0143697
  » https://doi.org/10.1371/journal.pone.0143697
• ¹¹
  Saldías PF, Jorquera AJ, Díaz PO. Predictive value of clinical features and nocturnal oximetry for the detection of obstructive sleep apnea syndrome [Article in Spanish]. Rev Med Chil. 2010;138(8):941-50. https://doi.org/S0034-98872010000800001
  » https://doi.org/S0034-98872010000800001
• ¹²
  Jorquera J, Freire M, Muñoz P, Saldías F. Utilidad de los índices clínicos y oximétricos en el diagnóstico de apnea obstructiva del sueño moderada a severa. Rev Chil Enferm Respir. 2002;18(4):307.

Publication Dates

• Publication in this collection
  13 Dec 2019
• Date of issue
  2020