Comparison of Face-To-Face Interview and Telephone Methods of Administration on the Ecohis Scores

Abstract

The aim of this study was to compare the administration of the Brazilian version of Early Childhood Oral Health Impact Scale (ECOHIS) questionnaire, applied by face-to-face and telephone format. A randomized sample of 76 parents/guardians of children up to 6 years old was selected in a Pediatric Dentistry Clinic of the Federal University of Santa Maria, Brazil. Patients were randomly selected for 2 different groups, according to administration sequence: F-T (Face-to-face-Telephone) and T-F (Telephone-Face-to-face). Two interviewers administered the ECOHIS questionnaire with 2 week interval between the methods. The reliability between sections of the different methods of administration was assessed by Cronbach's alpha and Intraclass Correlation Coefficient (ICC). There were differences in means of the ECOHIS between face-to-face and telephone methods. However, the value Cronbach's alpha were between 0.94-0.96, and value ICC ranged from 0.91-0.93, proving to be acceptable values. The Bland-Altman plots confirmed the results of reliability tests, supporting the accuracy of the methods. Although there is a slight difference in the scores, the use of both mode of administration of the ECOHIS yields reliable data when single method is used from the beginning to the end of the study. The results provided evidence to use either Face-to-face or Telephone method of administration of the ECOHIS

Key Words:
administration method; oral health-related quality of life; face-to-face interview, telephone interview, children

Resumo

O objetivo deste estudo foi comparar a administração da versão brasileira do questionário Early Childhood Oral Health Impact Scale (ECOHIS), aplicado através de entrevista pessoal e por telefone. Uma amostra randomizada de 76 pais/responsáveis de crianças de até 6 anos de idade foi selecionada na Clínica de Odontopediatria da Universidade Federal de Santa Maria, Brasil. Os pacientes foram selecionados aleatoriamente para 2 grupos diferentes, de acordo com a sequência de administração: E-T (Entrevista Pessoal-Telefone) e T-E (Telefone-Entrevista Pessoal). Dois entrevistadores aplicaram o questionário ECOHIS com um intervalo de 2 semanas entre os métodos. A confiabilidade entre as seções dos diferentes métodos de administração foi avaliada pelo Alfa de Cronbach e Coeficiente de Correlação Intraclasse (ICC). Houve diferenças nas médias do ECOHIS entre os métodos de entrevista pessoal e telefone. No entanto, os valores de Alfa de Cronbach foram entre 0,94-0,96 e ICC entre 0,91-0,93, mostrando valores aceitáveis. O gráfico de Blant-Altman confirmou os resultados dos testes de confiabilidade, suportando a acurácia dos métodos. Embora exista uma pequena diferença nos escores, o uso de ambos os métodos de administração do ECOHIS produz dados confiáveis quando um único método é utilizado do início até o final do estudo. Os resultados forneceram evidências para usar tanto entrevista pessoal quanto por telefone como métodos de administração do ECOHIS.

Introduction

Oral health-related quality of life (OHRQoL) refers to the impact of oral health status on daily activities, quality of life, and well being, including both biological and psychosocial factors of the individual ^1,2. These are needed to communicate with health managers and contribute to the planning of oral health programs and provision of access to services ³.

Information on OHRQoL is frequently measured through self-administered questionnaires ⁴. The use of these questionnaires has advantages, including low cost, preserving the anonymity of participants, and reduction of bias that may occur from interactions with interviewers ⁵. In contrast, low response rates are a risk with this form of administration ^6,7. Studies comparing responses to different applications (i.e., self-administered questionnaire or interview) in adult populations have found no difference in scores ^8,9. However, self-administered questionnaires were found to be associated with incomplete answers and information loss ⁹. On the other hand, face-to-face interviews have been used in Brazil ¹⁰ and other countries ¹¹.

Moreover, the same instrument administered by telephone has been used in some studies ^6,12. This method can reduce costs involving logistics for application without influencing the responses obtained, thus facilitating data collection in surveys ^6,12. Another advantage is that the respondent is anonymous, thereby decreasing response bias in the presence of an interviewer ⁶.

The questionnaire Early Childhood Oral Health Impact Scale (ECOHIS) was developed to assess OHRQoL in 2- to 5-year-old children ¹³. It is completed by parents/caregivers, and can be used in surveys to discriminate children with or without experience of dental diseases ¹³. Furthermore, it is one of the few validated instruments for preschool-age children in Brazil (B-ECOHIS) ^14,15. The ECOHIS has been applied as a face-to-face interview, but there is no evidence that the telephone method influences its psychometric properties or response rate.

Thus, the aim of this study was to test the validity of the Brazilian version of the ECOHIS depending on whether it is administered by face to face or by telephone interview.

Material and Methods

The study was approved by the Research Ethics Committee of the Federal University of Santa Maria (Protocol number: 43675715.5.0000.5346). Parents/guardians consented to participate with their children after receiving a full explanation of the goals and methods.

Study participants were parents or guardians of children up to 6 years old seeking dental care in the Pediatric Dentistry Clinic of the Federal University of Santa Maria, Brazil, from March to October 2015. Children with no teeth were excluded from analysis. A pilot study with 10 parents was conducted to test the feasibility of the methodology ⁹; these were not included in the final sample. For the sample size calculation, we considered an effect size of 0.5, alpha value of 0.05, and a minimum power of 80%. The sample size required was 70 parents/guardians.

Patients were randomly selected for the two different groups. Groups were named F-T (Face-to-face-Telephone) and T-F (Telephone-Face-to-face), based on the format taken in the first week of the study. In the second administration, participants responded to the other application method. Each participant was taken to a nearby room to ensure confidentiality of data. First, a socioeconomic questionnaire was administered. For the F-T group, the B-ECOHIS was administered in this first meeting, and the participant was informed that s/he would be called after a 2-week interval. For the T-F group, after completing the socioeconomic questionnaire, the participant was informed that s/he would be called to answer the B-ECOHIS in the evening of the same day. They were also informed that the personal interview would be administered after a 2-week interval upon returning to the Pediatric Dentistry Clinic. The 2-week interval was used to avoid memory bias ⁸. In the telephone interview, patients did not have a copy of the questionnaire, and at the beginning of the face-to-face interview, they were verbally given the six possible responses for each question ¹⁶.

The study profile is presented in Figure 1. Participants were lost for not meeting on the phone or not returning to the clinic after the 2-week interval.

Figure 1
Participant flow and follow-up.

The B-ECOHIS assesses the OHRQoL of preschool-age children ¹⁷ with 13 items targeted to parents or guardians, including four questions about family function and parent distress, and the remainder about the child, including symptoms, functional limitations, psychological health, and self-image/social interaction. Each item is evaluated individually on a Likert scale of 5 points (0 = never, 1 = rarely, 2 = sometimes, 3 = often, 4 = very often, 5 = do not know). The total score ranges from 0 to 52. Higher scores represent lower OHRQoL. Items answered with number 5 ("do not know") were considered as missing data. For participants with up to two missing responses in the child section and one in the family section, a score was imputed as an average of the remaining items for that section ¹³. Individuals with more than two missing responses items were excluded from the sample. The time that the parent spent to answer the questionnaire was also collected and recorded in seconds.

Sociodemographic variables were collected through a structured questionnaire. Skin color was adopted by classification of race/ethnic groups according to the criteria established by the agency for demographic analysis (the Brazilian Institute of Geography and Statistics). Children were classified as "non-white" (children of African and mixed descent) and "white" (children of European descent) ¹⁸. There were no indigenous or oriental children. Educational level included fathers and mothers who had completed less than 8 years of formal instruction (<8 years), which in Brazil corresponds to primary school, those who had high school (8-11 years), and those with undergraduate or technical education (>11 years). Household income was measured in terms of the Brazilian minimum wage and was divided into tertiles of the sample distribution, comparing the poorest to the richest tertile. The questionnaire also included information on the child's dental visits and toothaches.

The data were analyzed using STATA 12.0 software (Stata Corporation, College Station, TX, USA). Differences in the response rate for the total scores and each domain were compared between the methods (face to face or telephone). Differences in overall scale and domains mean, and standard deviation were compared using the Wilcoxon rank test, with a significance level of 5%. The paired t-test was used to verify the mean and standard deviation of the time spent in the face-to-face or on the telephone interview. The reliability between the total and specifics domains of the B-ECOHIS according to the method of administration was assessed by Cronbach's alpha coefficient. Test-retest reliability was assessed with the Intraclass Correlation Coefficient (ICC), allowing us to assess the reproducibility of the questionnaires ¹². Reliability test scores >0.70 are considered acceptable ¹⁹. Lastly, the Bland-Altman plot was used to evaluate the correlation between the two methods of application, with its limit of agreement and confidence interval.

Results

A total of 76 children (51.3% male) were recruited for this study. The mean age was 3.96 years (SD 1.49). One participant was excluded in the data analysis due to answering more than two items as "do not know." The majority was white (88.2%), had visited the dentist (64.5%) and had no toothache (53.9%). Most of the parents had a high educational level, and 38.2% belonged to the poorest tertile of income. The time to apply the ECOHIS through face-to-face interview was 179.69 seconds (SD 67.60) and by telephone interview was 170.07 seconds (SD 77.56) (Table 1).

Table 1
Sample characteristics of preschool children, Santa Maria-Brazil, 2015

• BMW: Brazilian minimum wage; *Paired t test - p=0.62.

Table 2 shows the mean and standard deviation for ECOHIS administered by face-to-face and by telephone interview. The mean ECOHIS (overall scale) by face-to-face was 7.03 (SD 8.32) and by telephone was 6.17 (SD 7.59), p<0.01. The same differences occurred for the function domain of the child section (p<0.01), the total child section (p=0.01), and of the family section for the family function (p=0.04).

Table 2
ECOHIS (overall scale), child section, total child, family section and total family mean (SD) scores, according to the interview and telephone. Santa Maria, Brazil

• **Wilcoxon rank test.

Cronbach's alpha, test-retest reliability coefficients (ICC), and mean differences between administration methods for the total and specifics domains of ECOHIS are shown in Table 3. Cronbach's alpha ranged from 0.94 to 0.96, and ICCs ranged from 0.91 to 0.93, proving to be acceptable values. The Bland-Altman plots show the averages of the different face-to-face and telephone concentrated within the limits of agreement, demonstrating a high correlation (Fig. 2).

Table 3
Cronbach's alpha, test-retest reliability coefficients and mean differences between administration methods in the overall scale and specifics domains of ECOHIS. Santa Maria, Brazil

• ICC: intra-class correlation coefficient.

Figure 2
Bland-Altman plot considering mean different of interview and telephone in ECOHIS overall, limits of agreement.

Discussion

This was the first study to compare different application methods for the ECOHIS. We found that both the telephone and face-to-face methods can be used for the assessment of COHRQoL using the ECOHIS. The ICC and the Cronbach's alpha values confirmed that the responses were highly correlated (0.91-0.93 and 0.94-0.96, respectively).

A slight difference in the mean ECOHIS overall scale, total child section and family function domain were found. The average score of the total ECOHIS (7.03) for face-to-face interviews was higher than it was for telephone interviews (6.17), indicating that subjects reported poorer OHRQoL in face-to-face interviews. However, this difference may be due to the questionnaire, and not the individual or the method applied ²⁰.

The reliability values (ICC and Cronbach's alpha) were high and acceptable, similar to previous study evaluating the OHIP-49 questionnaire ²¹. Thus, even if the questionnaire has different responses, ranging from "never" to "very often," this variation occurred between the next options as "often and very often" or "almost never and sometimes". These variations in responses are normal and depend on the individual. Therefore, it is common for people to respond to similar questions differently ²⁰.

There were differences between the mean scores of some domains in the face-to-face interview (overall scale, child section-function, total child section and family section-function). It has been suggested that specific differences are more likely to be caused by the question itself and a general effect of the administration method. However, it is hard to explain why differences between methods occurred in a specific direction for a particular question ^6,22.

The Bland-Altman plots confirmed the results of reliability tests. The differences in mean scores for telephone and face-to-face interview were focused around limits of agreement, and this further supported the accuracy of the measurement.

In surveys or studies where there is dispersal of participants, researchers mix data collection methods (e.g., start with a personal interview and finish with phone) to reduce costs and facilitate data collection ²³. However, the results of this study show that using the same administration method -from the start to the end of a study- is reliable, and answers are equivalent ¹⁶. Thus, it is possible to obtain complete and reliable answers on the ECOHIS with both face-to-face and telephone methods.

Finally, no differences were found in terms of application time. Hawthorne (2003) ⁽⁶ showed that when answering questions by telephone, unless people expressed their opinions, and administration time was reduced. In contrast, Weinberger et al.(1996) ⁵ showed that face-to-face interviews decrease response time. There is still no consensus among studies comparing the time required for each method of administration. We believe the telephone method would facilitate the logistics of an epidemiological study.

There are some limitations of this study. Selection bias may have emerged, as we used a convenience sample of parents/guardians seeking dental treatment. Further studies with different populations should be performed to confirm these results. One could also argue that a possible effect of the dental visit may have influenced the results between the first and second application. However, we used a 2-week interval between the first and second administration, which may account for possible carry-over effects.

There was a mean difference in ECOHIS scores depending on administration method. Nevertheless, the values of Cronbach's alpha and ICC demonstrate low variability between both of them. The Blant-Altman plots also confirmed the results of reliability tests, supporting the accuracy of the methods. Although there is a slight difference in the scores the use of both mode of administration of the ECOHIS yields reliable data when single method is used from the beginning to the end of the study.

This study offers an important perspective to the scientific community to evaluate administration by telephone, which can facilitate lower costs and data collection.

Acknowledgements

We would like to thank for all the subjects and their families for their cooperation with this study. And, we also thank the Federal University of Santa Maria for supporting us. The authors declare that they have no conflict of interest.

References

• ¹
  Sischo, L; Broder, HL. Oral health-related quality of life: what, why, how, and future implications. J Dent Res 2011;90:1264-1270.
• ²
  Baker, SR; Mat, A; Robinson, PG. What psychosocial factors influence adolescents' oral health? J Dent Res 2010;89:1230-1235.
• ³
  Shamrany, MA. Oral health-related quality of life : a broader perspective. East Mediterr Health J 2006;12:894-901.
• ⁴
  Slade, GD. Assessing change in quality of life using the Oral Health Impact Profile. Community Dent Oral Epidemiol 1998;26:52-61.
• ⁵
  Weinberger, M; Oddone, EZ; Samsa, GP; Landsman, PB. Are health-related quality-of-life measures affected by the mode of administration? J Clin Epidemiol 1996;49:135-140.
• ⁶
  Hawthorne, G. The effect of different methods of collecting data: Mail, telephone and filter data collection issues in utility measurement. Qual Life Res 2003;12:1081-1088.
• ⁷
  Robinson, PG; Gibson, FA; Khan, FABW. A comparison of OHIP 14 and OIDP as interviews and questionnaires. Community Dent Heal2001;18:144.
• ⁸
  Sousa, PCB DE; Mendes, FM; Imparato, JCP; Ardenghi, TM. Differences in responses to the Oral Health Impact Profile (OHIP14) used as a questionnaire or in an interview. Braz Oral Res 2009;23:358-364.
• ⁹
  Cheung, YB; Goh, C; Thumboo, J; Khoo, K-S; Wee, J. Quality of life scores differed according to mode of administration in a review of three major oncology questionnaires. J Clin Epidemiol 2006;59:185-191.
• ¹⁰
  Abanto, J; Carvalho, TS; Mendes, FM; Wanderley, MT; Bönecker M; Raggio, DP. Impact of oral diseases and disorders on oral health-related quality of life of preschool children. Community Dent Oral Epidemiol2011;39:105-114.
• ¹¹
  Masumo, R; Bardsen, A; Mashoto, K; Åstrøm, AN. Prevalence and socio-behavioral influence of early childhood caries, ECC, and feeding habits among 6-36 months old children in Uganda and Tanzania. BMC Oral Health 2012;12:24.
• ¹²
  Schuck, P. Assessing reproducibility for interval data in healthrelated quality of life questionnaires: which coefficient should be used? Qual Life Res 2004;13:571-586.
• ¹³
  Pahel, BT; Rozier, RG; Slade, GD. Parental perceptions of children's oral health: the Early Childhood Oral Health Impact Scale (ECOHIS). Health Qual Life Outcomes 2007;5:6.
• ¹⁴
  Tesch, FC; Oliveira, BH; Leão, A. {Semantic equivalence of the Brazilian version of the Early Childhood Oral Health Impact Scale}. Cad Saude Publica 2008;24:1897-1909.
• ¹⁵
  Scarpelli, AC; Oliveira, BH; Tesch, FC; Leão, AT; Pordeus, I A; Paiva, SM. Psychometric properties of the Brazilian version of the Early Childhood Oral Health Impact Scale (B-ECOHIS). BMC Oral Health 2011;11:19.
• ¹⁶
  Desai, R; Durham, J; Wassell, RW; Preshaw, PM. Does the mode of administration of the Oral Health Impact Profile-49 affect the outcome score? Journal of Dent 2014;42:84-89.
• ¹⁷
  Martins-Júnior, PA; Ramos-Jorge, J; Paiva, SM; Ramos-Jorge, ML. Validations of the Brazilian version of the Early Childhood Oral Health Impact Scale (ECOHIS). Cad Saude Publica 2012;28:367-374.
• ¹⁸
  IBGE. Sample results - Work and Performance 2010{Latest access November 12, 2016}. Avaliable from: Avaliable from: http://biblioteca.ibge.gov.br/visualizacao/periodicos/1075/cd_2010_trabalho_rendimento_amostra.pdf
  » http://biblioteca.ibge.gov.br/visualizacao/periodicos/1075/cd_2010_trabalho_rendimento_amostra.pdf
• ¹⁹
  Revicki, DA; Osoba, D; Fairclough, D; Barofsky, I; Berzon, R; Leidy, NK; et al.. Recommendations on health-related quality of life research to support labeling and promotional claims in the United States. Qual Life Res 2000;9:887-900.
• ²⁰
  Dillman, DA; Sangster, RL; Tarnai, J; Rockwood, TH. Understanding differences in people's answers to telephone and mail surveys. New Dir Eval 1996;199:45-61.
• ²¹
  Reissmann, DR; John, MT; Schierz, O. Influence of administration method on oral health-related quality of life assessment using the Oral Health Impact Profile. Eur J Oral Sci 2011;119:73-78.
• ²²
  Galobardes, B; Sunyer, J; Anto, JM; Castellsague, J; Soriano, JB; Tobias, A. Effect of the method of administration, mail or telephone, on the validity and reliability of a respiratory health questionnaire. The Spanish centers of the European Asthma Study. J Clin Epidemiol 1998;51:875-881.
• ²³
  Brambilla, DJ; Mckinlay, SM. A comparison of responses to mailed questionnaires and telephone interviews in a mixed mode health survey. Am J Epidemiol 1987;126:962-971.

Publication Dates

• Publication in this collection
  Sep-Oct 2016

History

• Received
  14 June 2016
• Accepted
  02 Aug 2016