Validations of the Brazilian version of the Early Childhood Oral Health Impact Scale ( ECOHIS )

This study aimed to evaluate the psychometric properties of the Brazilian version of the Early Childhood Oral Health Impact Scale (ECOHIS). A total of 247 children aged 2 to 5 years and their parents/guardians participated in this study. A clinical oral examination was performed and the parents/guardians completed the ECOHIS questionnaire. Within a period of four-weeks, 20% of the participants repeated the ECOHIS questionnaire. Construct validity was determined using Spearman’s rank correlation. Discriminant validity, internal consistency and test-retest reliability were also evaluated. The children were divided into 2 groups: group 1 (with dental caries) and group 2 (caries-free). Children with caries showed higher mean ECOHIS scores than the caries-free children. The child impact section (p < 0.01), family impact section (p < 0.01) and total ECOHIS scores (p < 0.01) were significantly correlated with tooth decay. Cronbach’s alpha coefficients demonstrated satisfactory internal consistency. The Brazilian version of the ECOHIS is a valid instrument for assessing oral health-related quality of life in preschool children with Brazilian Portuguese-speaking primary caregivers. Preschool Child; Questionnaires; Oral Health; Quality of Life Introduction The assessment of quality of life has become an integral part of evaluating health programs. In recent years, several instruments used to improve and validate oral health-related quality of life (OHRQoL) in children and adolescents have emerged 1,2,3,4,5. The Early Childhood Oral Health Impact Scale (ECOHIS) was developed and validated in English in the United States to assess the impact of oral health problems and related treatment on the quality of life of preschool children (aged three to five years) and their families 6. Studies have determined that the ECOHIS is valid for assessing the impact of OHRQoL in children between zero and five years of age and their families in Canada 7. Due to the target age group, the ECOHIS questionnaire is filled out by parental proxy. Up to the age of five, children have difficulty in understanding basic health concepts, are incapable of adequately expressing themselves and tend to give exaggerated responses 8. Children’s selfconcept and health cognition is age dependent and results from continuous cognitive, emotional, social and language development 7. According to child developmental psychology, the age of six marks the beginning of abstract thinking and self-concept 9. Children begin to compare their physical features and personality traits to those of other children or to social norms. Their ability to make evaluative judgments regardARTIGO ARTICLE Martins-Júnior PA et al. 368 Cad. Saúde Pública, Rio de Janeiro, 28(2):367-374, fev, 2012 ing their appearance, quality of friendships and other people’s thoughts, emotions and behavior gradually develops throughout middle childhood (six to 10 years) 10. Preschool children may suffer from a number of oral health problems such as teething pain 11, eruption disturbances 12, early childhood caries 13 and dental trauma 14,15. A dental survey among Brazilian preschool children found that nearly 27% of children between 18 and 36 months of age have at least one primary tooth with caries and this proportion rises to nearly 60% among five-year-olds 16. This data illustrates the dimension of caries as a public health problem in Brazil. Measuring OHRQoL can make an important contribution through providing further data on this issue to help guide oral health policies and thereby contribute to the definition and prioritization of the socially appropriate use of resources. The cross-cultural adaptation of this instrument is necessary to enable the collection of comparable information in different cultures. The ECOHIS has already been tested and validated among children in the United States, Canada, China and Iran 6,7,17,18. It is imperative to evaluate the different language versions of this instrument to assure that each version exhibits psychometric properties similar to the original version and guarantee its effective utilization in cross-cultural comparisons 17. In Brazil, the ECOHIS has been translated, adapted and validated for use in children between two and five years of age 19,20. However, this instrument has yet to be evaluated with regard to certain psychometric properties, such as effect size measures which serve as a complement to the validation process. The aim of the present study was therefore to evaluate the psychometric properties (validity and reliability) of the Brazilian version of the ECOHIS.


Introduction
The assessment of quality of life has become an integral part of evaluating health programs.In recent years, several instruments used to improve and validate oral health-related quality of life (OHRQoL) in children and adolescents have emerged 1,2,3,4,5 .The Early Childhood Oral Health Impact Scale (ECOHIS) was developed and validated in English in the United States to assess the impact of oral health problems and related treatment on the quality of life of preschool children (aged three to five years) and their families 6 .Studies have determined that the ECOHIS is valid for assessing the impact of OHRQoL in children between zero and five years of age and their families in Canada 7 .
Due to the target age group, the ECOHIS questionnaire is filled out by parental proxy.Up to the age of five, children have difficulty in understanding basic health concepts, are incapable of adequately expressing themselves and tend to give exaggerated responses 8 .Children's selfconcept and health cognition is age dependent and results from continuous cognitive, emotional, social and language development 7 .According to child developmental psychology, the age of six marks the beginning of abstract thinking and self-concept 9 .Children begin to compare their physical features and personality traits to those of other children or to social norms.Their ability to make evaluative judgments regard-Cad.Saúde Pública, Rio de Janeiro, 28(2):367-374, fev, 2012 ing their appearance, quality of friendships and other people's thoughts, emotions and behavior gradually develops throughout middle childhood (six to 10 years) 10 .
Preschool children may suffer from a number of oral health problems such as teething pain 11 , eruption disturbances 12 , early childhood caries 13 and dental trauma 14,15 .A dental survey among Brazilian preschool children found that nearly 27% of children between 18 and 36 months of age have at least one primary tooth with caries and this proportion rises to nearly 60% among five-year-olds 16 .This data illustrates the dimension of caries as a public health problem in Brazil.Measuring OHRQoL can make an important contribution through providing further data on this issue to help guide oral health policies and thereby contribute to the definition and prioritization of the socially appropriate use of resources.
The cross-cultural adaptation of this instrument is necessary to enable the collection of comparable information in different cultures.The ECOHIS has already been tested and validated among children in the United States, Canada, China and Iran 6,7,17,18 .It is imperative to evaluate the different language versions of this instrument to assure that each version exhibits psychometric properties similar to the original version and guarantee its effective utilization in cross-cultural comparisons 17 .In Brazil, the ECOHIS has been translated, adapted and validated for use in children between two and five years of age 19,20 .However, this instrument has yet to be evaluated with regard to certain psychometric properties, such as effect size measures which serve as a complement to the validation process.
The aim of the present study was therefore to evaluate the psychometric properties (validity and reliability) of the Brazilian version of the ECOHIS.

Measure
The ECOHIS is an English language measure of oral health-related quality of life developed and validated in the United States to assess the impact of oral health problems and related treatment on the quality of life of preschool children (aged zero to five years) and their families 6 .The ECOHIS consists of 13 questions divided into two main parts: child impact section (part one) and family impact section (part two).The child impact section comprises of four subscales: child symptom, child function, child psychology and child self-image/social interaction.The family impact section contains two subscales: parental distress and family function.The questionnaire is scored using a simple five-point Likert scale with responses ranging from "never" to "very often" (equivalent to a score of 0 and 4, respectively).A total score ranging from zero to 52 is calculated as a simple sum of the responses with higher scores denoting a greater oral health impact and/or poorer OHRQoL.The different ranges of the subscale scores are as follows: child impact section: child symptom -one item, range 0 to 4; child function -four items, range 0 to 16; child psychology -two items, range 0 to 8; and child self-image/social interaction -two items, range 0 to 8. Family impact section: parental distress -two items, range 0 to 8; family function -two items range 0 to 8.

Translation into Brazilian Portuguese
To assess the use of the ECOHIS in Brazilian children, the questionnaires used in the present study were translated and adapted to Brazilian culture 19 .

Main study
A total of 247 children (boys and girls) aged two to five years were randomly selected from children living in the town of Diamantina, in the State of Minas Gerais, Brazil, participating in the National Child Vaccination Day program in the town's ten health centers (n = 10) in August 2010.The children were selected according to order of arrival in the health centre.In 2010, 2,113 children aged two to five years were living in the city of Diamantina and the vaccination program in Diamantina had previously shown constant uptake rates of over 90%.
A total of 250 children were randomly selected as no specific recommendations are given regarding the sample size required for discriminant and construct validity testing.A sample size of this magnitude is more than sufficient to detect correlation coefficients as low as 0.20 using a one-tailed test when the null hypothesis is r = 0, alpha = 0.05 and beta = 0.20.
To participate in the study children had to fulfill the following inclusion criteria: must be aged between two and five years; must not exhibit any serious underlying medical conditions, must not show the long-term use of medication, must not have any physical or learning disabilities; must be accompanied by a Brazilian Portuguese-speaking parent/guardian who lives with the child for at least 50% of the time.Parents of children who met these inclusion criteria were interviewed to complete the ECOHIS questionnaire and provide additional social demographic information.All parents agreed to participate in the study.The interviews were performed individually by trained interviewers who were unaware of the results of the clinical examinations.Interviewees that did not answer or responded "don't know" to one or more questions were excluded from the analysis.Based on recommended World Health Organization (WHO) criteria for the visual assessment of dental caries, children underwent a clinical oral examination carried out by 10 trained and calibrated examiners 22 .Caries experience was recorded using the d-DMFT index.Within a period of four-weeks after the first interview, 20% of the participants were randomly selected by lots and agreed to repeat the ECOHIS questionnaire process at the health care centre.The repeated interview was performed individually with each parent/guardian and the interviewer was unaware of the results of the first interview.
Prior to the study, a team made up of ten examiners participated in a training program which included intra-examiner and inter-examiner calibration exercises where minimum and maximum kappa values were agreed (0.81 and 0.86 respectively) between the examiners.A pilot study with 30 children and their parents/guardians was also performed to determine the applicability of the measure.
All parents that participated in this study signed an informed consent form.The study received approval from the ethics committee of the Vales do Jequitinhonha e Mucuri Federal University (Universidade Federal dos Vales de Jequitinhonha e Mucuri; nº.181/10).

Assessment of validity and reliability of the Brazilian version of the ECOHIS
Data was inputted and organized using the software SPSS version 17.0 (SPSS Inc., Chicago, United States).Construct validity of the Brazilian version of the ECOHIS was determined by correlating the ECOHIS scores with DMFT scores (Spearman's rank correlation).To evaluate discriminant validity, differences in ECOHIS scores between children with caries and those without caries were assessed using the Mann-Whitney U-test (nonparametric test equivalent to t-test).To estimate the effect size, statistics were calculated by dividing the mean ECOHIS scores by the standard deviation (SD).Effect size statistics of < 0.2 indicate a small, clinically meaningful magnitude of difference; effect size statistics of 0.2-0.7 indicate moderate difference; and effect size statistics of > 0.7 indicate a large difference 23 .Reliability was assessed in two ways: internal consistency and test-retest reliability.Internal consistency was assessed by determining the mean item correlation of the ECOHIS items using Cronbach's alpha statistic.Test-retest reliability was assessed by determining the level of agreement between the results of the first and repeated ECOHIS questionnaire using the intraclass correlation coefficient (ICC) in a one-way random effect parallel model.
Table 2 displays the responses to each item in the ECOHIS questionnaire.In the child impact section, "pain in the teeth, mouth or jaws" was the most frequently reported item by the parents (21.5%).The items "irritation or frustration" (12.6%), "difficulty drinking" (10.9%), "difficulty eating" (10.3%) and "trouble sleeping" (10.1%) were also frequently reported in this section.In the family impact section, the most frequently reported items were "felt guilty" (14.2%) and "been upset" (12.6%).
Discriminant validity was determined by comparing scores between the clinical groups.Variations in the mean ECOHIS and the two subscale scores (child impact and family impact sections) (p < 0.001) were apparent between the caries and caries-free groups (Table 3).Median values for the two subscale scores and total score were 0.0.Effect sizes were considered moderate for the total score and the two subscales.Children with caries had higher mean ECOHIS scores than the caries-free children, demonstrating that the instrument was capable of discriminating between clinically different groups.
The results of the correlations (Spearman's rank) between ECOHIS scores and caries experience revealed that the child impact section (r = 0.36, p < 0.01), family impact section (r = 0.39, p < 0.01) and total ECOHIS scores (r = 0.40, p < 0.01) were significantly correlated with tooth decay.The analysis of Cronbach's alpha coefficients revealed satisfactory internal consistency, with coefficients of 0.85 in the child impact section, 0.65 in the family impact section and 0.86 for the total ECOHIS score.A total of 50 parents/guardians repeated the Brazilian version of the ECOHIS questionnaire reporting no change in children's oral health status.The ICC values were 0.94 for the total score and child impact section and 0.88 for the family impact section (Table 4).

Discussion
The present study evaluated the psychometric properties of the Brazilian Portuguese version of the ECOHIS by determining its validity and reliability using discriminant validity, internal consistency and test-retest reliability.This process is imperative to assess the cross-cultural adaptation of this quality of life measure 24,25,26 .
Participants were selected from a community-based sample.In contrast with findings of previous studies 7,17 , the majority of the parents/ guardians reported no oral health impact on the quality of life of their children.This may be explained by the fact that the children which were part of these other studies were from clinically based convenience samples and therefore already exhibited some type of oral health problem, whereas the children in the present study were selected from the participants of a National Child Vaccination Day program and were not seeking dental treatment.This approach was adopted to achieve greater accuracy in the results 20 .As is the case in other ECOHIS validation studies 7,17,18 , the present study assessed each age group covered by this measure.
Considering that this study comprises a representative sample of children living in Diamantina, a 50.6% prevalence of dental caries in this group affirms that almost 1,070 preschool children from the town have one or more carious teeth.This represents a high prevalence of dental caries when compared to the findings of a dental survey carried out among Brazilian preschool children 16 .The implementation of oral health policies and oral health promotion among this age group is therefore essential to reduce the high prevalence of dental caries among this group.
The original version of the ECOHIS was developed to be a self-completed questionnaire.However, considering the low educational level of most Brazilians, it was decided to assess the psychometric properties of the Brazilian version of the ECOHIS by interviewing participants 19 .The effect of this method on completion rates and the actual results is not clear.Further studies should be carried out to assess the psychometric properties using both methods for completing the ECOHIS questionnaire.Significant differences in mean ECOHIS and subscale scores were found between the caries and caries-free groups.As found in other ECOHIS validation studies 17,20 and the original ECOHIS study 6 , high scores were obtained by children with dental caries.These findings demonstrate the ability of the Brazilian version of the ECOHIS to discriminate between these two groups of children.A number of studies also report that children with untreated dental caries can have difficulty chewing, sleeping and socializing and that caries can affect self-esteem, growth and weight gain 18,21,27,28,29 , so reducing quality of life.However, as public health programs do not cover younger children, oral health in this group is often overlooked, leaving many children with untreated dental caries and suffering from chronic pain and discomfort 30 .
It is interesting to note that the most common impacts reported by parents in the child impact section were practically the same as those found in studies of the original, French and Brazilian versions of the ECOHIS 6,7,20 .The most frequent responses in these three studies were "pain in the teeth, mouth or jaws", "irritation or frustration", "difficulty eating" and "trouble sleeping".An important finding of the present study is the fact that the effect size analysis demonstrated a moderate clinical significance in the difference between the groups with and without caries 31 .Publications in the field of psychology have demonstrated an increasing trend in the use of effect size measures and that such measures are considered fundamental to investigations of excellence 31,32 .However, the other validation of the B-ECOHIS did not evaluate the clinical significance of the study results 20 .
The results of the present study showed Cronbach's alpha coefficients of 0.85 for the child impact section, 0.65 for the family impact section and 0.86 for the overall scale, demonstrating good internal consistency for the Brazilian version of the ECOHIS, higher than the recommended value of 0.70 33 .Other ECOHIS validation studies report similar Cronbach alpha values 7,17,18,20 .The intra-class correlation value for the total score was 0.94, indicating an excellent level of agreement between the test-retest results 6 .This coefficient is higher than that reported in the Farsi, Chinese and original versions of the ECOHIS (0.82, 0.64 and 0.84, respectively) and similar to the ICC value of the French version (0.95).Another study of the Brazilian version of the ECOHIS did not present an ICC value for the total score, but the values for the child and family impact sections are similar to those revealed by the present study 20 .It should be noted that in the present study most of the responses to the items in the ECOHIS questionnaire were "never".This can be seen as a limitation of this study and may explain the good level of testretest agreement in comparison to other studies.The result regarding the internal consistency reliability of the family impact section is marginal, as found in the preliminary study of the Brazilian version of the ECOHIS 20 .This is due to the fact that the family impact section includes only four items 20 , while the child impact section contains nine.It has been demonstrated that the alpha value tends to be higher with measures that contain more items 34 .
In the present study, the ECOHIS was significantly but not strongly correlated with caries experience.Most studies have used global measures of oral health and actual disease status to examine the construct validity of the ECOHIS 6,7,18 .However, one study determined construct validity by correlating the ECOHIS scores with DMFT and DMFS scores 17 .It was stated above that clinical measures may not be a very appropriate means of assessment of construct validity and subjective measures should be used for this purpose.Therefore, the fact that data on overall well-being and oral health status of children as perceived by the parents was not collected is another limitation of this study.Although evidence shows that parents can provide valid reports regarding their preschool children's OHRQoL when dental conditions are observable 27 , the validity of these reports may be conditioned to caries experiences associated with pain symptoms.Clinical measures may not be strongly correlated with the d-DMFT, as untreated carious deciduous teeth can remain symptomless until shed 35 .

Conclusion
The adaptation and psychometric evaluation of the Brazilian Portuguese version of the ECOHIS suggested good construct validity, discriminant validity and internal consistency as well as acceptable test-retest reliability.The Brazilian Portuguese version of the ECOHIS is therefore a valid instrument for assessing oral health-related quality of life in preschool children with Brazilian Portuguese-speaking primary caregivers.It would be useful to evaluate the sensitivity and responsiveness of this measure using a longitudinal study in order to assess the effectiveness of oral healthcare interventions.

Contributors
All authors made an equal contribution to the production of this paper.

Table 1
Comparison of parent and child characteristics of caries and caries-free groups.

Table 2 Distribution
of Brazilian Early Childhood Oral Health Impact Scale (ECOHIS) responses in community-based sample (N = 247).

Table 3
Discriminant validity: comparison of mean Early Childhood oral Health Impact Scale (ECOHIS) scores and sub-scales by caries status.