Development of a self-report questionnaire designed for population-based surveillance of gingivitis in adolescents: assessment of content validity and reliability

Abstract The major infectious diseases in Chile encompass the periodontal diseases, with a combined prevalence that rises up to 90% of the population. Thus, the population-based surveillance of periodontal diseases plays a central role for assessing their prevalence and for planning, implementing, and evaluating preventive and control programs. Self-report questionnaires have been proposed for the surveillance of periodontal diseases in adult populations world-wide. Objective This study aimed to develop and assess the content validity and reliability of a cognitively adapted self-report questionnaire designed for surveillance of gingivitis in adolescents. Material and Methods Ten predetermined self-report questions evaluating early signs and symptoms of gingivitis were preliminary assessed by a panel of clinical experts. Eight questions were selected and cognitively tested in 20 adolescents aged 12 to 18 years from Santiago de Chile. The questionnaire was then conducted and answered by 178 Chilean adolescents. Internal consistency was measured using the Cronbach’s alpha and temporal stability was calculated using the Kappa-index. Results A reliable final self-report questionnaire consisting of 5 questions was obtained, with a total Cronbach’s alpha of 0.73 and a Kappa-index ranging from 0.41 to 0.77 between the different questions. Conclusions The proposed questionnaire is reliable, with an acceptable internal consistency and a temporal stability from moderate to substantial, and it is promising for estimating the prevalence of gingivitis in adolescents.


Introduction
Periodontal disease is the most common chronic inflammatory disease detected in humans, affecting nearly 98% of adults over 65 years in Chile 11 . It is a major public health problem due to its high prevalence, its consequences in terms of social, psychological, and economic impacts on individuals, communities, and health services, and its potential prevention and management in terms of alleviation or cure of the disease 2 .
Comprehensive periodontal examination with fulldata registration is the gold standard method designed to detect early signs of periodontal disease, allowing us to prevent the destructive forms of the disease; however, its clinical application is expensive and time consuming 9 . Therefore, one must have inexpensive, safe, and easy-to-perform alternative tools to simplify the periodontal data-collection process, which allow the feasibility of the surveillance of the periodontal diseases.
Self-reporting is frequently used as a recording and surveillance method in several pathological conditions and diseases 3,19,20,23,30 . In adults, selfreport questionnaires have been proposed as an epidemiologic tool to analyze the prevalence and incidence of periodontitis 10,21,27 . In fact, a periodontal self-reporting with a set of eight questions has been proposed by the Division of Oral Health at the National

Center for Chronic Disease Prevention and Health
Promotion and, after its validation, it was applied in different populations 6,8,10,18,24 . With the purpose of evaluating tools that allow the surveillance of periodontal disease at its early stage in younger populations, we analyze the content validity and reliability of a new self-report questionnaire designed to detect gingivitis in adolescents.

Experimental design
This cross-sectional study was conducted among Chilean adolescents (

Preliminary item selection
An initial self-report questionnaire was developed by selecting items related to gingivitis, after reviewing the relevant literature on existing self-reported periodontal measures. Criteria for selecting the selfreport items included: having a recognized association with gingival inflammation as a risk indicator or factor; or having face validity as being associated with early signs and symptoms of periodontal disease. A total of 10 items were identified, written in English, and then adapted and translated into Latin-American Spanish, using a backward-forward translational method by two bilingual periodontists (questionnaire version 1).

Item validation
The validation of the content of each item was performed by a consensus panel of rater experts.
This panel consisted of two periodontists, a PhD in public health, a master in pediatric dentistry, and a general dentist expert in adolescents, all of them University Professors and selected considering at least 5 years of clinical expertise, national and international presentations, and research on the referred phenomenon of interest. The analyzed parameters were coherence, relevance, and clarity of each item, as well as whether answer options were appropriate and sufficient. For each parameter, a score of 1 to 9-points was assigned, being 9 very important, 6 important, and 1 unimportant, and when the judges disagreed or an average <5-points was obtained, the item was discarded. After panel consensus and recommendations, questionnaire version 2 was obtained and then pilot tested.

Questionnaire cognitive evaluation
Pilot test was conducted using 20 adolescent respondents from Santiago de Chile, randomly selected. An answer percentage was calculated and then each answer was discussed considering how the participant understood and processed each question in a focus-group consisting of 10 of these adolescents.
A discussion of approximately 60 minutes was conducted, audio-recorded, and transcribed, and data were analyzed by qualitative thematic analysis, in line with the framework approach. This process resulted in Development of a self-report questionnaire designed for population-based surveillance of gingivitis in adolescents: assessment of content validity and reliability 8 cognitively tested questions, which were pooled in a new questionnaire (questionnaire version 3).

Content validity
The questionnaire version 1 was then evaluated by an expert panel that agreed (>90%) in rating the 10 items as important or critical in the parameters of adequacy, clarity, consistency, and relevance.
Following the experts' written suggestions, items were revised, 2 questions were excluded, and one question was added, obtaining questionnaire version 2 ( Figure 1).

Cognitive evaluation of the instrument
Twenty adolescents answered questionnaire version 2 and each question had a response rate >95%. Content analysis of focus group was made and the cognitively adapted questions were included in questionnaire version 3.

Q1; How would you rate your gum health?, Q2;
How would you rate your toothbrushing?, and Q3; During toothbrushing, do your gums bleed? No problems were identified with these questions. The adolescents said that "diseased gums" are those that bleed during toothbrushing, smell bad, and are red.
They associate a good brushing with brushing all the tooth surfaces, leaving them with a smooth texture: "to pass the tongue over them and feel them soft." Therefore, these items remained in questionnaire version 3.
Q4; In the last month, have you noticed your gums reddish and/or swollen? and Q5; Have you noticed your teeth unaligned or crooked?. The differences in the way respondents answered depended, mostly, on whether they had seen a dentist in the recent past. One of the participants told he did not know "which is the normal color" for gums and, for another participant, a reddish gum is a gum that bleeds. They reported that Q5 is a subjective question, whose answer is determined by personal expectations; some used the comparison with their peers, a dentist's opinion, and/or self-examination to answer. The group of researchers, however, decided to keep both items.
Q6; When you have visited your dentist, have you ever had a deep cleaning? The term "cleaning" seemed confusing; they associate it with caries removal, whitening, or any other activity performed by a dentist and not necessarily related to periodontal treatment.
Therefore, the group decided to remove this question for version 3 of the instrument.  Q10 If your answer was YES, did you visit him because of a problem? Si tu respuesta fue SI, ¿asististe por algún problema?
Which? It is considered confusing to ask for "a problem"; it was necessary to specify the reason for the visit (control, emergency, etc.).
The question was removed.
The inclusion of a question about brushing frequency was suggested.
Q9 How many times a day do you brush your teeth? ¿Cuántas veces al día te cepillas los dientes? Less than once a day/ Once a day/ Twice a day/ Three or more times a day.
items. As seen in Table 3   Always/Often/Sometimes/Never FQ4. During the last month, have you noticed your gums reddish and/ or swollen? ¿En el último mes has notado tus encías rojizas e hinchadas?
Always/Often/Sometimes/Never FQ5. How many times a day do you brush your teeth? ¿Cuántas veces al día te cepillas los dientes?
Less than once a day/once a day/twice a day/three or more times a day from moderate to substantial. In future studies, we recommend researchers to assess its criterion validity and include questions about risk indicators for periodontal disease, to obtain a questionnaire that allows to estimate the prevalence of gingivitis and to evaluate preventive campaigns.