Validity of periodontitis screening questions in a Brazilian adult population-based study

Population-based studies assessing self-reported periodontal questions in low-income countries are lacking, and therefore we aimed to assess the accuracy of self-reported periodontal items in Brazil. One thousand one hundred and forty adults from Florianópolis, Brazil, had their periodontium clinically examined, and responded to the following self-reported items on periodontal conditions: Question (Q)1, Do you have any wobbly teeth?; Q2, Do your gums usually bleed?; and Q3, Has your dentist ever told you that you have gum disease? Periodontitis was defined as: a. ≥ 6.0 mm periodontal pocket and ≥ 4.0 mm clinical attachment loss in the same tooth, in at least one tooth (PD1); or b. ≥ 6.0 mm periodontal pocket and ≥ 4.0 mm clinical attachment loss, not necessarily in the same tooth (PD2). Sensitivity (SN) and specificity (SP) were calculated, and analyses were stratified by socioeconomic status and time since last dental visit. Scores were generated in order to determine the accuracy of the whole set of items. Receiver operating characteristic (ROC) curves were plotted. Prevalence of clinically diagnosed periodontitis was 2.6% (95%CI = 1.7–4.0%) for PD1 and 3.8% (95%CI = 2.7–5.3%) for PD2. Prevalence of self-reported periodontitis varied between 2.7 (Q2) and 22.0% (Q3). SN and SP ranged between 0.0–60.0% and 73.3–98.6%, respectively; Q1 showed the highest accuracy (140.8%) followed by Q3 (140.0%). The combined score of the three self-reported items did not improve accuracy estimates; the areas under the ROC curves were 0.70 and 0.68 for PD1 and PD2, respectively. The accuracy of self-reported items was low, and further studies are needed in order to develop valid and reliable periodontitis screening questions for population-based studies.


Introduction
Data collection for population health surveys may be obtained through various approaches including clinical examinations, face-to-face interviews, self-administered questionnaires or a combination of these techniques 1 .Usually referred as the gold standard assessment to estimate disease occurrence in populations, clinical examinations present some important disadvantages, such as being time-consuming, implying higher costs, imposing a substantial burden on participants, and being associated with higher refusal rates 2,3 .Therefore, clinical exams are not commonly used in large Declaration of Interests: The authors certify that they have no commercial or associative interest that represents a conflict of interest in connection with the manuscript.clinical measures.Therefore, the objectives of this study are: a. to estimate the prevalence of self-reported and clinically diagnosed periodontitis in adults aged 22-61 years from a population-based study in Brazil; and b. to determine the accuracy of self-reported items (individually and as a set) on periodontal signs and symptoms in the total sample, as well as in groups with different levels of education, income and time since the last dental visit.To the authors's knowledge, there are no population-based studies which have assessed self-reported periodontal questions outside high-income countries.

Methodology
The present study is part of the ongoing EpiFloripa Study, a population-based cohort study, which started in 2009, designed to investigate the health and living conditions of adults aged 20-59 years at baseline, from Florianópolis, Southern Brazil.The following parameters were considered in our sample size estimation: a. unknow outcome prevalence (50%); b. the target population as the 249,530 individuals from the city of Florianópolis, aged 20-59 years; c. a sample error of 3.5 percentage points; and d. a design effect of 2, due to the cluster sampling design.Furthermore, the sample size was increased by 10% to compensate losses and refusals.A final sample of 2,016 individuals was estimated, of which 1,720 participated in the first phase of the study 16 .
A two-stage sample selection was adopted.First, 60 of the 420 census tracts in the urban area of the city were selected, according to the average monthly income of the household head -i.e. six census tracts in each income decile were included in the study.The selected census tracts were visited by the fieldwork team and all occupied homes were checked and their residents added in order to ascertain the number of eligible residents in the selected clusters.Given that the number of households ranged from 61 to 810, some census tracts were merged to reduce the variability in the number of households in each.Finally, 63 census tracts were included in the study, totaling 16,755 eligible households, of which 1,134 were selected.On average, 32 adults were selected in each census tract.All adults aged 20 to 59 years, living in the selected households were eligible for this study 16 .
In the second phase, three years later, 1,222 subjects were investigated, of whom 1,140 underwent dental examination.Disabled individuals and subjects unable to take part in the interview due to a specific physical or mental condition, were excluded.
A questionnaire was used to collect data on gender, educational attainment, monthly household income and time since the last dental visit..The question on gingival bleeding was dichotomized as Yes, when the response "Always" was selected and No, whenever any of the remaining categories was selected.Clinical examinations included the assessment of periodontal conditions -gingival bleeding, pocket depth and clinical attachment loss -and were performed in the participant's home, followed by face-to-face interviews.
Periodontitis was defined according to pocket depth and clinical attachment loss 17 .Six sites (mesio-buccal, mid-buccal, disto-buccal, mesio-lingual, mid-lingual, disto-lingual) on all teeth in one maxillary and one mandibular randomly selected quadrant 18,19 were examined using a periodontal ball probe -World Health Organization (WHO) probe -according to the WHO Oral health surveys recommended guidelines 20 .Shallow periodontal pocket was defined as a probing depth between 4.0 mm and 5.5 mm; and deep periodontal pocket as, at least, 6.0 mm.Clinical attachment loss was categorized as: a. 0.0 to 3.0 mm; b. 4.0 to 5.0 mm; c. 6.0 to 8.0 mm; d. 9.0 to 11.0 mm; or e. 12.0 mm or more, based on the inherent periodontal ball probe intervals of measurement.Periodontitis was clinically defined according to two different criteria: deep periodontal pocket and clinical attachment loss of 4.0 mm or more in the same tooth, in at least one tooth (PD1); or deep periodontal pocket and clinical attachment loss of 4.0 mm or more, not necessarily in the same tooth 21 (PD2).All clinical periodontitis measures were defined previously to statistical analysis.
Eight dentists were subjected to rigorous training and standardization prior to the fieldwork, with 20 non-participant adults, following a protocol described elsewhere 22 .The questionnaire was pre-tested in the same group of adults.The intra-and inter-examiner reliability were assessed with simple and weighted Kappa statistics, where appropriate.
Data analysis included descriptive statistics of the sample according to socioeconomic, demographic and oral health-related characteristics.Furthermore, the prevalence of periodontitis and its 95% confidence interval (95%CI) were estimated, following the abovementioned diagnostic criteria.Frequencies for the self-reported items were calculated for the total sample and for each of the studied strata.
In order to determine the accuracy of the whole set of items on periodontal conditions, three different scores were generated.The first one was constructed by adding up all the items -given that each item was scored on a 0-1 (no/yes) scale, the final score ranged between 0-3.The second and third scores were derived by means of multiple logistic regression equations.Each of the two clinically defined periodontitis status was predicted from the self-reported items on periodontal conditions, according to the following general equation 'Y = β0 + β1X1 + β2X2 + β3X3', where Y is one of the clinically defined periodontal status, β 0 is a constant, and β 1 , β 2 , and β 3 are the "weights" for the self-reported items on periodontal conditions (mobility, bleeding and diagnosis), which are represented by X 1 , X 2 , and X 3 , respectively.Each item's score was multiplied by its respective weight, and these were then added to achieve a final score to be included in the receiver operating characteristic (ROC) models.In summary, one weighted equation was generated for each of the clinically defined outcomes.However, preliminary analyses showed that the equations did not improve the predictive accuracy of the referred scores.Therefore, only unweighted scores were used in the analysis and in the construction of all the graphs detailed below.
Sensitivity (SN), specificity (SP), and their 95% CI were calculated for each self-reported question and for the abovementioned scores, taking the clinical exam as the reference for the total sample and for the stratified analysis.Stratification was done for schooling (< 11, or ≥ 12 years of study), income (< R$3,225.00 or ≥ R$3,225.00 -which is the sample median and corresponded to US$1,897.00 at the time of data collection), and time since the last dental visit (less than a year, or one year or more).Nonoverlapping 95% CIs were considered indicative of statistically significant differences among SN and SP estimates.Finally, we plotted two ROC curves for unweighted scores of self-reported periodontal items and estimated the areas under each ROC curve.Analyses were carried out using Stata v.13.1, taking into account the complex sampling design (clustering and weighting).
The Ethics Committee in Human Research of the Federal University of Santa Catarina approved the project on February 28 th 2011.All participants in the study signed the informed consent form after the procedures had been fully explained.

Results
A total of 1,140 individuals were investigated.The inter-and intra-examiner Kappa values ranged from 0.60 to 0.95 for the combination of periodontal pocket and clinical attachment loss measurements.Sample characteristics are displayed in Table 1.The majority of the interviewees were female (56.3%) and visited a dentist less than a year before the interview (63.9%); most of them had 12 years of schooling or more (44.7%),and half of the sample (50.2%) had a monthly household income up to R$3,225.00 (US$1,897.00).
Table 2 shows the prevalence of periodontal conditions according to the two clinical criteria for periodontitis (PD1 and PD2), and to each self-reported periodontal health item in the total sample, and stratified by schooling, income and time since last dental visit.The highest prevalence of clinically assessed periodontitis was identified when PD2 was used, and the highest prevalence of self-reported periodontitis was found when 'self-reported diagnosis' was used, followed by 'self-reported mobility'.In general, the most educated people, and those with a higher income presented a lower prevalence of periodontitis.Specifically, higher frequencies of adverse periodontal conditions were observed among participants with lower schooling (PD2, 'self-reported mobility' and 'self-reported bleeding'), and lower family income ('self-reported mobility'), whereas respondents who visited a dentist more than a year before the survey were more likely to show a lower prevalence of 'self-reported diagnosis' of periodontal problems.
Tables 3 and 4 exhibit total SN values and values according to stratification.For the total sample, the highest SN values were found for 'self-reported diagnosis' and the lowest, for 'self-reported bleeding'.Strata-specific values according to schooling, income and time since last dental visit were not significantly different when compared to each other.The highest SN values were identified for 'self-reported diagnosis' among participants who visited a dentist less than a year before the survey, those with lower levels of education and higher income (60.0%,59.1% and 53.3%, respectively).Differences between SN values for the different clinical criteria of periodontitis were not statistically significant.The SP values are also presented in Tables 3 and 4. For the total sample, the highest SP values were found for 'self-reported bleeding' and the lowest for 'self-reported diagnosis', for both clinical definition of periodontitis.Except for 'self-reported diagnosis', stratified analyses showed, in general, higher SP values for most educated and wealthier people, and for those who visited a dentist less than a year before the survey.Significantly higher SP values were observed for 'self-reported mobility' in individuals with more years of formal education, as well as for 'self-reported diagnosis' for those who visited the dentist in the previous 12 months.Figures 1 and 2 show the ROC curves for prevalence of PD1 and PD2, respectively, taking the unweighted score into account.For PD1, the area under the ROC curve was 0.70 and for PD2, 0.68.
The combined use of the three self-reported items did not improve accuracy estimates.
The prevalence of periodontitis based on two self-reported items ('mobility' and 'diagnosis') was higher than that found clinically (PD1 and PD2), reflecting an inconsistency between results.'Self-reported diagnosis' showed the highest prevalence for the total sample, which was considerably higher than the values found according to clinical criteria.This may have happened because the question about 'self-reported diagnosis' of periodontitis is too broad, including all disease levels, from mild to severe periodontal conditions.Therefore, more individuals were identified as having adverse periodontal conditions with this item, rendering higher SN values when compared to the remaining self-reported items.It is possible that if the question had been "Has your dentist ever told you that you have a severe periodontal Table 4. Sensitivity, specificity and their 95% confidence intervals for the second clinical definition of periodontitis (≥ 6.0 mm periodontal pocket and clinical attachment loss of 4.0 mm or more, not necessarily in the same tooth).

Self-reported diagnosis
Self-reported mobility

Self-reported bleeding
Self   problem?", mild cases would have been ignored and SN values would have been lower.Although previous studies 2, 13,23,24 have applied similar questions, they have not provided prevalence estimates, making comparisons difficult.SN values of the self-reported items ('mobility', 'bleeding' and 'diagnosis') varied significantly across socioeconomic strata, in contrast to what was found for the SP values.Furthermore, the combination of the self-reported items into a single score did not significantly improve these values.For the schooling strata, the highest SN values were found for those with lower educational level.The lowest family income level showed slightly lower SN values than those in the lowest educational level; on the other hand, the highest family income level showed higher SN values than those in the highest educational levels.For time since last dental visit, the highest SN value was found for the combination of PD2 and 'self-reported diagnosis', in accordance with the family income strata.SP values were similar among the different strata.Since SP and SN values are inversely correlated, the lowest SP values were found for 'self-reported diagnosis'.
Our findings are not directly comparable to those from other studies, given the distinct methodologies (periodontal examination protocol and definition of periodontitis) and sample characteristics (age, socioeconomic characteristics and access to dental services).It is noteworthy, however, that 'self-reported mobility' presented similar results to those reported by Gilbert and Nuttall 2 in a study carried out in the United Kingdom, and lower diagnostic values than those by a study from Japan, by Yamamoto et al. 23 .'Self-reported bleeding' presented lower SN and SP values than those found in a study from southeast Brazil 24 ; SN and SP values for 'self-reported diagnosis' were similar to those from Yamamoto et al. 23 and higher than those described by Gilbert and Nuttall 2 and Dietrich et al. 13 (carried out in Germany).The interviewee's age in these studies ranged from 19 to 80 years.
A recently published study 31 on the validity of self-reported periodontal questions in a New Zealand cohort had different findings from this paper.A higher prevalence of periodontal disease was found due to the higher prevalence of smoking, among other reasons, when compared to this study 32 .Furthermore, they applied the gold standard clinical examination (full-mouth periodontal examinations, three sites per tooth) instead of partial-mouth examination, which may underestimate the prevalence of periodontal disease 18 .
Generally, in clinical settings the diagnosis of periodontitis is based on a clinical examination with full-mouth periodontal probing and, in some cases, radiographic examination 3,12 .The complexity of this approach makes it unfeasible for multi-thematic population surveys.Thus, in order to overcome this practical issue, different partial-mouth periodontal examination (PMPE) protocols for recording and monitoring periodontitis have been proposed since the late 1950s 25 .An alternative method to the PMPE is the examination of six sites per tooth, of all teeth from one maxillary and one mandibular randomly selected quadrants, called the "diagonal quadrants six-sites protocol".This was the protocol selected for our study, as it provides an accurate estimate of periodontitis prevalence, severity and extent, and its use reduces costs and examination time 26 .
This study presents some strengths, such as: the adoption of a large and representative sample of all social strata from Florianópolis, southern Brazil; the examiners achieved adequate diagnostic reliabilities 27 and were unaware of the research questions, minimizing observer bias; finally, we analyzed SN and SP values of self-reported items according to schooling and income levels, as well as regarding dental visiting patterns.
On the other hand, this study also has some limitations: (i) the items on periodontal conditions were not previously validated -a Brazilian study 12 investigated the conditions of interest, although it used questions extracted from USA population-based studies, or from studies with a different context than ours; (ii) test-retest reliability of the questions was not performed; and (iii) the present investigation adopted a periodontitis criterion different than other validation studies.However, this was done for operational reasons, including limited time and resources.Nevertheless, it is important to mention that there is no universal or consensual criteria to define periodontitis; the WHO probe provides categorical measures -instead of discrete ones -of periodontal pocket and clinical attachment loss.Furthermore, these are the criteria that have been used in nation-wide oral epidemiological studies in Brazil.
A self-reported item is considered valid when the sum of its SN and SP is 160% or more 28,29 .Since the highest accuracy value found in our study was 140.8%, none of the self-reported items can be considered as valid or accurate.The area under the ROC curve lower than 0.70 is poor, as values between 0.7 to 0.9 are considered useful and higher than 0.9, excellent 30 .It is well documented in the literature that the use of partial periodontal evaluation protocols may underestimate the prevalence of periodontal disease 18 , which might have occurred in this study.However, as an inherent characteristic of the test 33 , the prevalence of an outcome affects the predictive values -diseases with higher prevalence will yield higher predictive values -which is important in the clinical setting, but does not affect the accuracy.

Conclusions
The three self-reported questions used in this study were not found to be accurate.Additionally, the results presented might not be generalizable to wider populations, given that relatively young adults, in which the prevalence of periodontitis is low, took part in the study.Future studies should be carried out in order to assess the validity of different self-reported items on signs and symptoms of periodontitis.Furthermore, other surveys using the same questions as those employed in this study, and maybe additional items, should be undertaken in different populations, with other socioeconomic and cultural backgrounds, with a purpose to develop a valid and reliable questionnaire that could be used as a screening tool for periodontitis in different populations.

Figure 2 .
Figure 2. Receiver operating characteristic (ROC) curve for the three unweighted self-reported items on periodontal conditions, considering PD2 as the gold standard.

Figure 1 .
Figure 1.Receiver operating characteristic (ROC) curve for the three unweighted self-reported items on periodontal conditions, considering PD1 as the gold standard.
* This variable has three missing values; ** This variable has twenty-two missing values.*** R$3,225.00 corresponded to approximately US$ 1,897.00 at the time of data collection.****This variable has twelve missing values.
; Self-reported mobility: Q1.Do you have any wobbly teeth?Self-reported bleeding: Q2.Do your gums usually bleed?Self-reported diagnosis: Q3.Has your dentist ever told you that you have gum disease?

Table 3 .
Sensitivity, specificity and 95%CI for the first clinical definition of periodontitis (≥ 6.0 mm periodontal pocket and clinical attachment loss of 4.0 mm or more in the same tooth, in one or more teeth).
Self-reported mobility: Q1.Do you have any wobbly teeth?Yes/no; Self-reported bleeding: Q2.Do your gums usually bleed?Yes, always/any other response; Self-reported diagnosis: Q3.Has your dentist ever told you that you have gum disease?Yes/no.
Do you have any wobbly teeth?Yes/no; Self-reported bleeding: Q2.Do your gums usually bleed?Yes, always/any other response; Self-reported diagnosis: Q3.Has your dentist ever told you that you have gum disease?Yes/no.
PD2: second definition of clinically diagnosed periodontitis (≥ 6.0 mm periodontal pocket and ≥ 4.0 mm clinical attachment loss, not necessarily in the same tooth).