Validation of the Brazilian version of the Halitosis Associated Life-Quality Test (HALT)

Abstract

The present study aimed to validate (cross-culturally adapt and test psychometric properties) the Brazilian version of the Halitosis Associated Life-Quality Test (HALT). A process of translation and cross-cultural adaptation was conducted by a group of dental researchers. The first draft of the Brazilian Portuguese version was pre-tested on a sample of 33 individuals leading up to the final version of the questionnaire. The Brazilian version of the HALT (B-HALT) was applied to 100 individuals with halitosis (organoleptic score ≥ 2) and 100 individuals without halitosis (organoleptic score < 2). Exploratory factor analysis (EFA) was performed to evaluate the dimensionality of B-HALT. Cronbach’s alpha (α) and interclass correlation coefficient (ICC) were used to measure its reliability. For convergent validity, Spearman’s correlation was conducted between the B-HALT and the organoleptic scores. The discriminant validity was evaluated through the Mann-Whitney and Kruskal-Wallis tests. EFA confirmed the unidimensionality of B-HALT, which has also demonstrated excellent internal consistency (α = 0.96) and test-retest reliability (ICC = 0.93). There was a positive correlation between B-HALT and organoleptic scores (r = 0.33; p < 0.001). B-HALT was able to discriminate between the groups with and without halitosis measured by the organoleptic method (p < 0.001) and self-reported halitosis (p < 0.001). B-HALT has demonstrated to be a reliable and valid tool to evaluate the oral health-related quality of life associated to halitosis in Brazilian adults.

Halitosis; Quality of Life; Validation Study

Introduction

Halitosis is a general term to describe the malodor emanating from the oral cavity, being perceptible by others, and many times creating personal discomfort and embarrassment. ¹ In most patients with persistent halitosis (around 85%), the malodor is originated in the oral cavity, ² , ³ mainly by the white coating of the tongue, gingivitis, and/or periodontitis. ⁴ Yet, causes non-related to the mouth are less frequent and include upper and lower respiratory infections, gastrointestinal tract conditions, carcinoma, and some systemic diseases. ⁵ , ⁶

Self-reported oral health-related quality of life (OHRQoL) tools aim to capture the subjective aspect of oral health, taking into consideration the individual’s own perception of their conditions, limitations, and quality of life. ⁷ , ⁸ Patient-reported outcome measures (PROMs) are validated and standardized questionnaires used to evaluate individuals’ opinion and investigate self-perceived health condition, level of compromise, incapacity and health-related quality of life. ⁹ PROMs are import tools that could indicate OHRQoL from patients’ experience data as well as direct changes to how healthcare is organized and offered. ⁷

Although halitosis can impact social communication, ¹⁰ , ¹¹ the relationship between halitosis and OHRQoL has rarely been evaluated. ¹² Indeed, some studies evaluating the impact of halitosis in OHRQoL were performed by using generic tools. ¹² - ¹⁴ To capture specific aspects of halitosis, assess its impact on OHRQoL, and measure the effectiveness of treatment from the patient’s perspective, Kizhner et al. ¹⁵ proposed a questionnaire named Halitosis Associated Life-Quality Test (HALT). The tool is a condition-specific questionnaire to assess the relationship of oral malodor with OHRQoL, and it attained adequate psychometric properties ¹⁵ . The HALT has been cross-culturally adapted and validated to a Chinese version, obtaining satisfactory psychometric properties. ¹⁶

Because of the lack of a validated Brazilian instrument that measures the impact of halitosis in self-perceived oral health-related quality of life considering the social-cultural reality in Brazil, the aim of the present study was to cross-culturally adapt the HALT to Brazilian Portuguese, and test the Brazilian version’s (B-HALT) validity and reliability.

Methodology

Description of Halitosis Associated Life-Quality Test (HALT)

The Halitosis Associated Life-Quality Test (HALT) questionnaire was developed originally in English in 2011 as a tool to evaluate the quality of life related to oral malodor (halitosis), and it attained good psychometric properties. ¹⁵ The HALT is a specific instrument for the comprehensive evaluation of physical, social, and psychosocial negative impacts of halitosis in adults. ¹⁶

The HALT is comprised of 20 items rated with a 5-point scale, ranging from 0 to 5, where 0 corresponds to “no problem” and 5 to “as bad as it can be”. The final score is calculated by the sum of the scores from the 20 items, varying from 0 to 100. Higher scores indicate a greater impact of halitosis in OHRQoL.

Development of the Brazilian version of the HALT (B-HALT)

The process of translation and cross-cultural adaptation was conducted by a group of experienced researchers, following the guidelines by Beaton et al. ¹⁷ ( Figure ).

Figure
Flowchart of cross-cultural adaptation and validation of the instrument-HALT.

At first, two professors from the dental school that were fluent in English and had Brazilian Portuguese as their mother language performed independent translations of the original instrument from English to Brazilian Portuguese. The professors did not have previous knowledge of the HALT. Then, a panel of specialists compared the two translated versions, disagreements were solved consensually, and a single Portuguese version of the HALT was obtained.

Next, the new version was retro-translated by two professional translators that were fluent in Brazilian Portuguese and had English as their mother language. Again, these professionals did not have previous knowledge of the original version of the HALT. The retro-translations were compared to the original instrument by the same panel of experts. There was little difference between the 2 versions regarding synonymous words, and a retro-translation was sent to the original author for his approval (Dr. Victor Kizhner). No alterations were suggested.

Then, a pre-test of the Brazilian Portuguese version was performed on a convenience sample of 33 individuals that attended the dental clinics at the School of Dentistry of the Federal University of Minas Gerais (UFMG). These individuals did not participate in the psychometric properties’ tests. Each participant was questioned about difficulties in filling the preliminary tool, and understanding the meaning of each question, their answers, or instructions.

The panel of specialists later discussed the results and developed the final version of the B-HALT. All steps of the previously mentioned cross-cultural adaptation were followed to guarantee that the B-HALT achieved conceptual equivalence and semantics on its 20 items. ¹⁸ , ¹⁹

Evaluation of psychometric properties of the B-HALT

In order to test the psychometric properties of the B-HALT, the questionnaire was applied to volunteers attending the dental clinics at the School of Dentistry of UFMG, Belo Horizonte, capital of Minas Gerais, in the southeast region of Brazil. The sample size calculation was based on Anthonie et al. ²⁰ who recommend 2 to 20 individuals per item, and an absolute minimum of 100 to 250 individuals. Since HALT has 20 items, and considering 5 individuals per item, 100 individuals were defined as the minimum sample.

Data collection took place from April to November of 2018. The volunteers included staff, professors, students, and patients under treatment in the dental clinics, with at least 18 years of age. People having Brazilian Portuguese as a second language, being visually impaired or with hearing loss, being illiterate or intoxicated by alcohol or drugs during the organoleptic evaluation were excluded. A calibrated professional (JOS) (Kappa = 0.84) evaluated the malodor of all eligible individuals by the organoleptic method. Individuals were instructed to remain with their mouths closed, breathing solely through their noses for 3 minutes. After this time, they were instructed to exhale through a paper tube, at a 10-cm distance from the examiner’s nose. The oral malodor intensity was registered following the 5-point scale stipulated in Rosenberg et al. ²¹ , ²² in which 0 is absence of malodor, 1 is questionable malodor, 2 is slight malodor, 3 is moderate malodor, 4 is strong malodor, and 5 is intense malodor. Individuals were diagnosed as having halitosis when their organoleptic score was of ≥ 2. ²³ , ²⁴

Clear and detailed instructions were given to all participants regarding the filing in of B-HALT, and it was performed without the presence of a researcher to minimize bias. Questionnaires were excluded if not completely answered or if individuals could not be reached within a 7 to 10-day period of its first application (for test-retest). Moreover, participants filled a form containing demographic and socioeconomic questions, as well as general and oral health questions.

For reliability evaluation, only individuals with halitosis were selected. For convergent and discriminant validity, the instrument was applied to all participants. The B-HALT was applied twice within a 7 to 10-day period in order to measure its stability. In its second application, a dichotomous question was included: “Have you studied about the topic during the interval between the first and second application of the questionnaire?”, so as to exclude bias.

Ethical principles

The protocol of this study was approved by Ethics Research Committee from the Universidade Federal de Minas Gerais (CAAE 06404819.7.0000.5149). All participants were informed about the aim of study and signed an informed written consent before data collection, following the established principles from the Resolution 466 in the Brazilian National Council of 12/12/2012.

Statistical analysis

The reliability of the B-HALT was evaluated by its internal consistency (Cronbach’s alpha) and test-retest reliability (interclass correlation coefficient - ICC). Values ≥ 0.70 for Cronbach’s alpha were considered acceptable. ICC indicated a weak correlation if ≤0.40, moderate if 0.41–0.60, good if 0.61–0.80, and excellent if 0.81–1.00. ²⁵

The exploratory factor analysis (EFA) was performed to evaluate the dimensionality of B-HALT. Data collection adequacy was verified by the measurement of Kayser-Meyer-Olkin (KMO) (> 0.50), as well as Barlett’s test for sphericity (p < 0.05). No item was excluded. Promax rotation method was performed since there were correlations between the items of the questionnaire.

Since B-HALT scores were not normally distributed, the Spearman’s correlation test was used to evaluate the convergent validity between B-HALT scores and the scores generated by the organoleptic method (gold standard for halitosis diagnosis). The Mann-Whitney and Kruskall-Wallis tests were used for the discriminant analysis. The B-HALT total score was compared between specific subgroups, including ethnicity (Caucasian, Afro-Caucasian, Black), sex (male and female), schooling (< 8 years, 8–12 years, ≥ 12 years), body mass index (BMI) (underweight, normal weight, overweight, class I obesity, class II obesity, class III obesity), organoleptic (score ≥ 2) and (score < 2), marital status (with and without partner), self-reported halitosis (yes or no), smoking (yes or no), family income (≤ 5 or > 5 Brazilian minimum wages (US$ 1.285,00) (BMW), and alcohol consumption (yes or no).

All analyses were performed using Statistical Package for the Social Sciences (SPSS for Windows, version 25.0, IBM Inc., Armonk, EUA.). Significance level was established at 5%.

Results

Adaptation to Brazilian Portuguese

The evaluations conducted by the panel of specialists demonstrated idiomatic equivalencies between the original and the two translations. The evaluation of semantics equivalency was performed with comparison of the retro-translated version and the original questionnaire. The panel also suggested some alterations for better understanding, such as starting the items with a verb, as in the original #2 item: “Frequent sinusitis” to “Having frequent sinusitis” and some words from the original English version were substituted, such as “embarrassed” to “ashamed”, for clearer meaning in Portuguese. In addition, the response scale was considered adequate and without inconsistencies.

Participants’ characteristics during the validation study

The validation study was performed with 200 individuals (117 females, 83 males), ranging from 18-78 years old (41.34 ± 15.31 years). A total of 76 (38.0%) participants self-reported having halitosis, 48 (24.0%) had a family income above 5 BMW, and 9.0% were smokers. All individuals fully answered the 20 items of the B-HALT. The evaluation of halitosis by the organoleptic method had minimum value of 0 and a maximum of 5, with 50% of the sample not presenting halitosis (organoleptic score < 2) ( Table 1 ).

Table 1
Sociodemographic characteristics of the sample (n = 200).

Variables	n	(%)
Ethnicity
Caucasian	69	34.50
Afro-Caucasian	70	35.00
Black	61	30.50
Sex
Female	116	58.00
Male	84	42.00
Schooling
< 8 years	26	13.00
8–12 years	62	31.00
≥ 12 years	112	56.00
Body mass index (BMI)
Underweight	7	3.50
Normal weight	86	43.00
Overweight	70	35.00
Obesity Class I	24	12.00
Obesity Class II	10	5.00
Obesity Class III	3	1.50
Halitosis diagnostics
Without halitosis	100	50
With halitosis	100	50
Marital status
With partner	89	44.50
Without partner	111	55.50
Self-reported halitosis
No	124	62
Yes	76	38
Smoking
No	182	91.00
Yes	18	9.00
Family income
≤ 5 BMW	152	76.00%
Brazilian minimun wages (BMW)
> 5 BMW	48	24.00
Alcohol consumption
No	112	56.00
Yes	88	44.00

Psychometric properties of B-HALT

All data collection procedures were concluded without any loss. The average time to fill the B-HALT was 10 minutes. The final score of the B-HALT for all 200 individuals ranged from 0 to 98, with an average of 22.6 ± 27.2 (median of 9.0).

The reliability was only calculated for the 100 participants with halitosis, once B-HALT is a condition-specific instrument. It presented excellent reliability results, for both internal consistency (Cronbach’s alpha = 0.96) and test-retest reliability (ICC = 0.93; 95%CI = 0.89–095). These results were above recommended levels. ²⁵

EFA prerequisites for the 100 participants with halitosis (organoleptic score ≥2) were reached (KMO = 0.91; Barlett’s Test of Sphericity p < 0.001). ²⁶ , ²⁷ The solution of one single factor explained 62.7% of the total variance, and all items presented factor loading > 0.40, indicating that the B-HALT is a unidimensional instrument. Table 2 presents the factor loading of each item measured by EFA. ²⁸

Table 2
Exploratory factor analysis (n = 100) results.

Description	Items	Factor loading
Breathing mainly through the mouth	Q1	0.485
Having frequent tonsillitis	Q2	0.633
Having frequent sinusitis	Q3	0.652
Feeling worried or embarrassed due to my breath	Q4	0.771
Feeling terrible or tense due to my bad breath	Q5	0.835
Having difficulties in chewing or to restrict some food because of my bad breath	Q6	0.754
Notice change in taste	Q7	0.776
Having problems when speaking (covering my mouth) due to my bad breath	Q8	0.882
Having my appearance affected because of my bad breath	Q9	0.848
Being depressed because of my bad breath	Q10	0.861
Having problems concentrating due to my bad breath	Q11	0.789
Feeling ashamed of my bad breath	Q12	0.818
Spending extra time because of my bad breath (e.g., chewing more gum, brushing my teeth more frequently, etc.)	Q13	0.789
Talking to people at a distance due to my bad breath	Q14	0.835
Avoid leaving the house because of my bad breath	Q15	0.837
Having communication problems because of my bad breath	Q16	0.858
Being mentioned by others because of my bad breath	Q17	0.840
Having financial loss because of my bad breath	Q18	0.801
Having social or personal losses because of my bad breath	Q19	0.836
Having reduced life satisfaction because of my bad breath	Q20	0.845

For the validity tests, a sample of 200 individuals with or without halitosis was used. The instrument presented convergent validity, since there was a positive significant correlation between the final scores of B-HALT and the organoleptic score (r = 0.33; p < 0.001). B-HALT also presented discriminant validity considering different groups in relation to the presence of halitosis diagnosed by the organoleptic method (p < 0.001) and self-reported halitosis (p < 0.001) ( Table 3 ).

Table 3
Discriminant validity: comparison of B-HALT total score among specific groups (n = 200).

Group	n	Average	Standard deviation	Median	Min	Max	p-value
Halitosis diagnostics*
Without halitosis	100	15.12	23.390	5.00	0	95	< 0.001
With halitosis	100	30.11	28.692	20.00	0	98
Self-reported halitosis*
No	124	14.52	24.507	5.00	0	98	< 0.001
Yes	76	35.83	26.236	31.50	0	98

• *Mann-Whitney test; **Kruskal-Wallis test. Results significant at the 5% level.

Discussion

The HALT questionnaire, originally written in English, was evaluated as a tool for measuring the oral health-related quality of life associated to halitosis. ¹⁵ The cross-cultural adaptation to Brazilian Portuguese and the validation of the B-HALT were conducted by following guidelines. ¹⁸ The HALT has also been previously adapted and validated into the Chinese ¹⁶ and Polish languages. ²⁹ These studies have concluded that the HALT is an efficacious instrument to evaluate the impact of halitosis in oral health-related quality of life.

The panel of specialists of the present study concluded that the translation/retro-translation pairs achieved adequate equivalency of the semantic aspects of the original instrument. By an audit process, the panel verified that all recommended steps were followed. In order to do so, all reports, translations, and the final questionnaire (after cross-cultural adaptation) were revised by the same panel. The panel did not have the responsibility to alter any content, and by following this process, an adequate translation was reached. ³⁰ The final version of the B-HALT was also evaluated and approved by the original author.

In the present study, Cronbach’s alpha showed the excellent internal consistency of the questionnaire, above the minimal recommended level. In the original HALT, ¹⁵ Cronbach’s alpha was of 0.93. The stability of B-HALT, measured by test-retest reliability, considered that the individuals’ oral health condition remained unaltered between the two applications, showing excellent test-retest reliability, and attaining a higher ICC value than that obtained in the original English version validation. ¹⁵ These findings indicate that the B-HALT is a reliable and stable instrument to evaluate the impact of halitosis in OHRQoL.

The clinical evaluation of halitosis is usually an objective examination and it is based on smelling patients’ odors (organoleptic method). ³¹ The advantages of this score are its low cost and no equipment, and a wide range of detectable odors. As disadvantages, the extreme subjectivity of test, the lack of quantification, nose saturation, and low reproducibility could be mentioned. ³¹ Even so, it is considered the gold standard in the diagnosis of halitosis.

The organoleptic method was used to evaluate halitosis to measure the tool’s convergent validity, once B-HALT tests the negative impact of halitosis in oral health-related quality of life. This study confirmed that individuals with higher B-HALT scores have more severe halitosis. The average total score was significantly higher for individuals that self-reported having halitosis than those that did not; this difference should be seen as evidence of discriminant validity.

Halitosis is an unpleasant odor that emanates from the oral cavity, ³¹ affecting the wellbeing of individuals, diminishing their self-confidence, increasing their insecurities regarding social and intimate relationships, ¹⁰ and interfering on their oral health-related quality of life. The impact of halitosis on the individuals’ OHRQoL measured by this instrument can be used as a parameter to decide on the beginning of treatment or even its efficacy. In addition, PROMs are clinical tools used to capture patients’ perceptions about specific aspects of their oral health. ⁷ The B-HALT, by being a condition-specific questionnaire, was found to be a useful method for dental professionals to observe changes in oral health through time.

In conclusion, the B-HALT has shown to be a valid and reliable instrument, easy to be filled and understandable. It could be used as an isolated tool or combined with other indicators diagnosis or to measure therapeutic efficacy or outcomes.

Acknowledgments

This study was supported by grants from the Coordination for the Improvement of Higher Education Personnel (CAPES), Brasil. The authors would like to thank Carolina de Castro Martins, Dhelfelson Willya Douglas de Oliveira, Rafael Paschoal Esteves Lima, and Ricardo Alves de Mesquita for the valuable contributions through the process of cross-cultural adaptation.

References

• ¹
  Rösing CK, Loesche W. Halitosis: an overview of epidemiology, etiology and clinical management. Braz Oral Res. 2011; Sep/Oct;25(5):466-71. https://doi.org/10.1590/s1806-83242011000500015
  » https://doi.org/10.1590/s1806-83242011000500015
• ²
  Broek AM, Feenstra L, Baat C. A review of the current literature on management of halitosis. Oral Dis. 2008 Jan;14(1):30-9. https://doi.org/10.1111/j.1601-0825.2006.01350.x PMID:18173446
  » https://doi.org/10.1111/j.1601-0825.2006.01350.x
• ³
  Eldarrat AH. Influence of oral health and lifestyle on oral malodour. Int Dent J. 2011 Feb;61(1):47-51. https://doi.org/10.1111/j.1875-595X.2011.00010.x PMID:21382034
  » https://doi.org/10.1111/j.1875-595X.2011.00010.x
• ⁴
  Quirynen M, Dadamio J, Van den Velde S, De Smit M, Dekeyser C, Van Tornout M, et al. Characteristics of 2000 patients who visited a halitosis clinic. J Clin Periodontol. 2009 Nov;36(11):970-5. https://doi.org/10.1111/j.1600-051X.2009.01478.x PMID:19811581
  » https://doi.org/10.1111/j.1600-051X.2009.01478.x
• ⁵
  Tangerman A, Winkel EG. Extra-oral halitosis: an overview. J Breath Res. 2010 Mar;4(1):017003. https://doi.org/10.1088/1752-7155/4/1/017003 PMID:21386205
  » https://doi.org/10.1088/1752-7155/4/1/017003
• ⁶
  Kapoor U, Sharma G, Juneja M, Nagpal A. Halitosis: current concepts on etiology, diagnosis and management. Eur J Dent. 2016 Apr-Jun;10(2):292-300. https://doi.org/10.4103/1305-7456.178294 PMID:27095913
  » https://doi.org/10.4103/1305-7456.178294
• ⁷
  Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013 Jan;346 jan28 1:f167. https://doi.org/10.1136/bmj.f167 PMID:23358487
  » https://doi.org/10.1136/bmj.f167
• ⁸
  Locker D, Allen F. What do measures of ‘oral health-related quality of life’ measure? Community Dent Oral Epidemiol. 2007 Dec;35(6):401-11. https://doi.org/10.1111/j.1600-0528.2007.00418.x PMID:18039281
  » https://doi.org/10.1111/j.1600-0528.2007.00418.x
• ⁹
  Davidson M., Keating J. Patient-reported outcome measures (PROMs): how should I interpret reports of measurement properties? A practical guide for clinicians and researchers who are not biostatisticians. Br J Sports Med. 2014 May;48(9):792-6. https://doi.org/10.1136/bjsports-2012-091704
  » https://doi.org/10.1136/bjsports-2012-091704
• ¹⁰
  McKeown L. Social relations and breath odour. Int J Dent Hyg. 2003 Nov;1(4):213-7. https://doi.org/10.1034/j.1601-5037.2003.00056.x
  » https://doi.org/10.1034/j.1601-5037.2003.00056.x
• ¹¹
  Elias MS, Ferriani MGC. Historical and social aspects of halitosis. Rev Lat Am Enfermagem. 2006 Sep/Oct;14(5):821-3. https://doi.org/10.1590/s0104-11692006000500026
  » https://doi.org/10.1590/s0104-11692006000500026
• ¹²
  Lu HX, Chen XL, Wong M, Zhu C, Ye W. Oral health impact of halitosis in Chinese adults. Int J Dent Hyg. 2017 Nov;15(4):e85-92. https://doi.org/10.1111/idh.12242
  » https://doi.org/10.1111/idh.12242
• ¹³
  Kishi M, Abe A, Yonemitsu M. Relationship between the SF-36 questionnaire and patient’s satisfaction following halitosis therapy. Oral Dis. 2005;11(s1 Suppl 1):89-91. https://doi.org/10.1111/j.1601-0825.2005.01102.x PMID:15752110
  » https://doi.org/10.1111/j.1601-0825.2005.01102.x
• ¹⁴
  Buunk-Werkhoven Y, Dijkstra-le Clercq M, Verheggen-Udding E, Jong N, Spreen M. Halitosis and oral health-related quality of life: a case report. Int J Dent Hyg. 2012 Feb;10(1):3-8. https://doi.org/10.1111/j.1601-5037.2011.00512.x PMID:21689373
  » https://doi.org/10.1111/j.1601-5037.2011.00512.x
• ¹⁵
  Kizhner V, Xu D, Krespi YP. A new tool measuring oral malodor quality of life. Eur Arch Oto-Rhino-Laryngology; 2011 Aug;268(8):1227-32. https://doi.org/10.1007/s00405-011-1518-x
  » https://doi.org/10.1007/s00405-011-1518-x
• ¹⁶
  He SL, Wang JH, Wang MH, Deng YM. Validation of the Chinese version of the Halitosis Associated Life-quality Test (HALT) questionnaire. Oral Dis. 2012 Oct;18(7):707-12. https://doi.org/10.1111/j.1601-0825.2012.01929.x
  » https://doi.org/10.1111/j.1601-0825.2012.01929.x
• ¹⁷
  Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000 Dec;25(24):3186-91. https://doi.org/10.1097/00007632-200012150-00014
  » https://doi.org/10.1097/00007632-200012150-00014
• ¹⁸
  Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993 Dec;46(12):1417-32. https://doi.org/10.1016/0895-4356(93)90142-N
  » https://doi.org/10.1016/0895-4356(93)90142-N
• ¹⁹
  Herdman M, Fox-Rushby J, Badia X. ‘Equivalence’ and the translation and adaptation of health-related quality of life questionnaires. Qual Life Res. 1997 Apr;6(3):237-47. https://doi.org/10.1023/A:1026410721664
  » https://doi.org/10.1023/A:1026410721664
• ²⁰
  Anthoine E, Moret L, Regnault A, Sébille V, Hardouin JB. Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures. Health Qual Life Outcomes. 2014 Dec;12(1):176. https://doi.org/10.1186/s12955-014-0176-2
  » https://doi.org/10.1186/s12955-014-0176-2
• ²¹
  Rosenberg M, Kulkarni GV, Bosy A, McCulloch CA. Reproducibility and sensitivity of oral malodor measurements with a portable sulphide monitor. J Dent Res. 1991a Nov;70(11):1436-40. https://doi.org/10.1177/00220345910700110801
  » https://doi.org/10.1177/00220345910700110801
• ²²
  Rosenberg M, Septon I, Eli I, Bar-Ness R, Gelernter I, Brenner S, et al. Halitosis Measurement by an Industrial Sulphide Monitor. J Periodontol. 1991b Aug;62(8):487-9. https://doi.org/10.1902/jop.1991.62.8.487
  » https://doi.org/10.1902/jop.1991.62.8.487
• ²³
  Murata T, Yamaga T, Iida T, Miyazaki H, Yaegaki K. Classification and examination of halitosis. Int Dent J. 2002 Jun;52(S5P1 Suppl 3):181-6. https://doi.org/10.1002/j.1875-595X.2002.tb00921.x
  » https://doi.org/10.1002/j.1875-595X.2002.tb00921.x
• ²⁴
  Dupim Souza AC, Franco CF, Pataro AL, Guerra T, Costa FO, Costa JE. Halitosis in obese patients and those undergoing bariatric surgery. Surg Obes Relat Dis. 2013 Mar-Apr;9(2):315-21. https://doi.org/10.1016/j.soard.2011.10.020
  » https://doi.org/10.1016/j.soard.2011.10.020
• ²⁵
  Li L, Zeng L, Lin ZJ, Cazzell M, Liu H. Tutorial on use of intraclass correlation coefficients for assessing intertest reliability and its application in functional near-infrared spectroscopy-based brain imaging. J Biomed Opt. 2015 May;20(5):50801. https://doi.org/10.1117/1.JBO.20.5.050801
  » https://doi.org/10.1117/1.JBO.20.5.050801
• ²⁶
  Hair JF, Anderson RE. Mulitivariate data analysis with readings. 2nd ed. New York: Macmillan; 1987.
• ²⁷
  Kaiser HF, Rice J. Little Jiffy, Mark IV. Education and Psychological Measurement. 1974 Apr;34(1):111-7. https://doi.org/10.1177/001316447403400115
  » https://doi.org/10.1177/001316447403400115
• ²⁸
  Nunnally JE. Bernstein IH. Psychometric theory. New York: Mc Graw-Hill; 1974.
• ²⁹
  Dudzik A, Chomyszyn-Gajewska M, Łazarz-Bartyzel K. An evaluation of halitosis using oral chromatm data manager, organoleptic scores and patients’ subjective opinions. J Int Oral Heal. 2015;7(3):6-11.
• ³⁰
  Porter SR, Scully C. Oral malodour (halitosis). BMJ. 2006 Sep;333(7569):632-5. https://doi.org/10.1136/bmj.38954.631968.AE
  » https://doi.org/10.1136/bmj.38954.631968.AE
• ³¹
  Tonzetich J. Production and origin of oral malodor: a review of mechanisms and methods of analysis. J Periodontol. 1977 Jan;48(1):13-20. https://doi.org/10.1902/jop.1977.48.1.13
  » https://doi.org/10.1902/jop.1977.48.1.13

Publication Dates

• Publication in this collection
  17 Aug 2020
• Date of issue
  2020

History

• Received
  19 Sept 2019
• Accepted
  16 Dec 2019
• Reviewed
  18 July 2020