Dental caries and quality of life of preschool children : discriminant validity of the ECOHIS

The  aim  of  the  present  study  was  to  confirm  the discriminant validity (obtained using traditional statistical methods) of the Early Childhood Oral Health Impact Scale (ECOHIS) between preschool children with and without caries (mean score) through an evaluation of the effect size. A systematic search of electronic databases and a manual search were performed for studies published up to December 2015 involving the use of the ECOHIS for the evaluation of the impact of dental caries on oral health–related quality of life (OHRQoL) among preschool children. Two independent raters performed the selection of the studies and data extraction. Only papers published in English and Spanish were selected. No restrictions were imposed regarding the year of publication. Twelve studies were included, and the magnitude of standardized differences between the means of the “without caries” and “with caries” groups was calculated using Cohen’s d. Most studies demonstrated a large magnitude in the difference between the  groups  evaluated.  The  estimate  of  the  effect  size  confirmed the discriminant validity of the ECOHIS obtained through traditional statistics. Thus, the magnitude of the difference should be  considered  an  important  analytical  tool  for  the  confirmation  of statistical findings regarding null hypotheses and demonstrates the clinical significance of these research results. Keyewords: Child, Preschool; Dental Caries; Quality of Life; Surveys and Questionnaires.


Introduction
Studies addressing oral health-related quality of life (OHRQoL) among preschool children using the Early Childhood Oral Health Impact Scale (ECOHIS) have demonstrated that dental caries exert an impact mainly on the "symptoms", "function" and "psychological" domains of the Child Impact portion of the scale as well as the "distress" domain of the Family Impact portion. 1,2,3,4,5,6,7Assessing the discriminant validity of the ECOHIS in groups of children with and without caries, such studies have used traditional statistical methods involving the testing of the null hypothesis.However, some studies have demonstrated that this method is limited, as it may sometimes mask the presence of an effect. 8,9In contrast, results from the analysis of the effect size are more precise and detailed, demonstrating the clinical significance of findings rather than merely focusing on statistical significance. 10Thus, the analysis of the effect size is recommended to overcome the limitation of traditional hypothesis testing and can be used to compare the results obtained with the two methods. 10he analysis of the effect size results i n information on the real significance of an effect of an adverse health condition or intervention in addition to the concept of statistical significance. 11tatistical significance depends considerably on the sample size, as large samples lead to a statistically significant result, even if the association between variables is of a small magnitude without clinical relevance.Real sig nificance is given by the description of the observed effect size, 12 which allows the reader to interpret the importance of the findings. 13Unlike hypothesis tests that either accept or reject differences between the groups studied, the analysis of the effect size furnishes information regarding the magnitude of the relationship found between the outcome and explanatory factors. 14This means that the effect size is useful in determining the practical or theoretical importance of an effect and the relative contribution of different factors or the same factor under different circumstances. 13Different effect size measures are available and several may be appropriate for a given dataset. 11he description of the effect size in scientific papers in the health field can be considered incipient.In papers published in the Journal of Experimental Psychology: General in 2009 and 2010, for example, the effect size of the findings was calculated in less than half of the studies.Moreover, nearly no interpretation of the effect size was offered in the studies that estimated this aspect. 13o confirm the discriminant validity of the ECOHIS obtained through traditional statistical methods, we evaluated the magnitude of the differences found in studies that have used this scale in groups of preschool children with and without dental caries.

Methodology
The inclusion criteria for the studies in the present investigation consisted of 1) the use of preschool children as the study subjects; 2) the use of the ECOHIS to determine OHRQoL; and 3) the use of dental caries (regardless of the index used) as the method of distinguishing groups within a study.
The ECOHIS is an OHRQoL assessment tool developed in the United States 1 for the evaluation of the impact of oral problems on the quality of life of preschool children (0 to 5 years of age) and their families.This scale has 13 items distributed between the Child Impact Section (CIS) and Family Impact Section (FIS).The CIS has four domains: symptoms, function, psychological aspects and self-image/social interaction.The FIS has two domains: family distress and family function.Each item has five response options regarding the frequency of an event in the life of the child: 0 = never, 1 = hardly ever, 2 = occasionally, 3 = often, 4 = very often and 5 = "I don't know" (which is not counted when the scores are summed).The score is calculated by the sum of the response codes of the items.The CIS and FIS scores range from 0 to 36 and 0 to 16, respectively.The total score ranges from 0 to 52, with higher scores indicating a greater impact on OHRQoL.The time of reference for the questions is the child's entire life, given the infrequent nature of oral problems/treatment and the young age of the children for which the scale was developed.
Electronic searches of the Latin American and Caribbean Health Sciences, Web of Science and Medline databases were performed for articles published in English and Spanish using the following key words: Child, Preschool (MeSH term) OR Preschool Child OR Children, Preschool OR Preschool Children AND Dental caries (MeSH term) OR Dental Decay OR Caries, Dental OR Decay, Dental AND ECOHIS.Correlate key words in Spanish were also used.A systematic search of electronic databases was performed for studies published up to December 2015.Additionally, a manual search was performed of the list of references of all studies pre-selected from the electronic search.
Two independent raters (M.C.F. and M.L.R.J.) performed the analysis of the titles and abstracts, followed by the complete reading of pre-selected articles.Disagreements between raters were resolved by consensus.In cases for which no consensus was reached, a third rater (L.S.M.) was consulted to decide the eligibility of the study.
The following data were extracted from each selected article: study design, type of study (validation or application), type of sampling, sample size and mean + standard deviation of ECOHIS score (outcome) of groups with and without caries as well as respective p-values.The inclusion of validation and application studies is justified by the fact that both designs present data on children with and without dental caries.The difference between the two is that the psychometric properties of the measure are evaluated in a validation study, but not in an application study, as this type of evaluation has previously been performed.The validation studies included in the present review revealed discriminant validity, meaning that the total ECOHIS score is capable of distinguishing children with and without dental caries.
The groups (children with and without caries) in the selected studies were compared using the effect size, which was determined through the mean + standard deviation values of the ECOHIS scores (outcome variable).The effect size was calculated using the model proposed by Cohen 8 for standardized differences between the means of two independent groups: Cohen's d = X 1 -X 2 /s', with s' = (S 1 2 + S 2 2 )/2 , for which s' is the combined standard deviation of both groups, s 1 is the standard deviation of the first group (without caries) and s 2 is the standard deviation of the second group (with caries).In the present study, X 1 was the mean of the first group and X 2 was the mean of the second group.Thus, one of the means of the distribution is subtracted from the other and the result is divided by the standard deviation common to both groups.The standardized effect size is calculated considering variability (given by the standard deviation) as well as the differences between means. 13Based on the interpretation criteria proposed by Cohen, 15 0.20 < d ≤ 0.40 indicates a small effect size, 0.40 < d < 0.80 indicates a moderate effect size and d ≥ 0.80 indicates a large effect size.

Results
Twenty-nine potentially relevant studies were identified from the electronic databases, 20 of which were pre-selected following the analysis of the title and abstract.Five of the seven studies identified through manual searches of the reference lists were also pre-selected.Thus, 25 pre-selected studies were submitted to full-text analysis, 12 of which (11 from the electronic search and one from the manual search) were included in the present investigation.Figure 1 displays the flowchart of the study selection process.Among the articles excluded in the pre-selection phase was an article that was published in Chinese.
Table 1 displays the effect size findings for validation studies with non-probabilistic sampling.The effect sizes for the studies by Pahel et al. 1 and Peker et al. 5 ranged from small to moderate when using a cutoff of one to three teeth with caries and large when a cutoff of four or more teeth with caries was used.The effect size was large for the three remaining studies.
Table 2 displays the effect size findings for validation studies with probabilistic sampling.The effect size was moderate for the study conducted by Scarpelli et al. 4 , who used a cutoff of one to three teeth with caries, and in the study conducted by Martins-Júnior et al. 7 , who used a cutoff of one or more teeth with caries.The effect size was large in the study by Scarpelli et al. 4 when a cutoff of four or more teeth with caries was used.
Table 3 shows the effect size findings for appl icat ion st udies w it h non-probabi l ist ic sampling.The effect size was moderate in the study that used a cutoff of one to five teeth with caries 16 as well as in the study that used a cutoff of one or more teeth with caries. 17The effect sizes were large when a cutoff of one or more teeth with caries and six or more teeth with caries was used in studies conducted by Lee et al. 18 and Abanto et al. 16 , respectively.
Table 4 displays the effect size findings for application studies with probabilistic sampling.The effect size was moderate or small in studies that used a cutoff of one to four teeth with caries, one or more teeth with caries and greater than four teeth with caries. 6,19The effect size was large for the study conducted by Ramos-Jorge et al. 20 , who dichotomized dental caries as zero and one or more teeth with caries.

Discussion
The importance of the effect size resides in highlighting the clinical significance of the results of studies.Large, clinically relevant, but non-significant effects may suggest the need for further studies with a greater test power 12,21 to furnish sufficiently strong evidence to prove that the differences are indeed statistically significant.Small effects that are clinically irrelevant but statistically significant because of a large sample size may warn researchers against Articles included after complete reading (n = 1) Articles excluded after complete reading (n = 4) Reasons for exclusion: • A single ECOHIS item was used in the study "Feeling of guilt of parents/family due to problems with teeth or dental treatment of child."• Four items from Child Impact Section of original ECOHIS were removed.
• Evaluated the effect of socio-demographic factors on clinical indicators (dental caries and gingivitis) and OHRQoL.
• Clinical indicator "dental caries" was used as the predictor variable of impact.
• Did not measure impact of caries on quality of life based on the mean ECOHIS score.Impact was measured based on the frequency of response categories for different ECOHIS items and using "decayed, missing and filled teeth" as the predictor of impact.
• Did not evaluate discriminant validity of ECOHIS based on the clinical indicator "dental caries".Study groups were with and without treatment need.

Reasons for exclusion:
• Aim of study was to evaluate the semantic equivalence between the items of the American and Brazilian versions of the ECOHIS.
• Study groups were related to the reasons for seeking care: Dental treatment and control evaluation or other reasons.
• Did not measure impact of caries on quality of life using mean ECOHIS score.Impact was measured based on frequency of response categories for different ECOHIS items and using "dental caries" as the predictor of impact.overvaluation of their statistical significance. 12,21Any estimate of the effect size should be evaluated within the context of the study.Thus, for any phenomenon under analysis, the effect size should be presented, along with the relevant conditions under which it was estimated. 13The type of sampling, sample size, floor effect rate for the sample, cutoff point of the independent variable and percentage of "I don't know" responses can exert an influence on the findings, which should be analyzed with caution by the investigators so as to neither underestimate nor overestimate the differences encountered.For all the studies analyzed here, comparisons were made between children with no caries and those with caries, although the cutoff points varied.The studies analyzed here established cutoff points  equal to or greater than one, one to three, one to four, greater than four, equal to or greater than four, one to five and equal to or greater than six teeth with caries.When the cutoff point was closer to zero, the magnitude of the difference was smaller, which leads one to reflect upon the minimum number of teeth with caries capable of exerting an impact on the quality of life of preschool children.In contrast to the comparison between a group without caries and another group with a number of teeth with caries more distant from zero. 1,5,16n a probabilistic sample, a statistically significant result, but with a moderate effect size, was found in the comparison of children without caries and those with one to three teeth with caries, whereas a large effect size was found in the comparison of children without caries and those with four or more teeth with caries. 4A moderate effect size was also found for a probabilistic sample that demonstrated a high degree of statistical significance with a cutoff point of one or more teeth with caries. 7In this situation, the cutoff point of the explanatory factor (dental caries) for an impact on quality of life likely affected the magnitude of the differences.
The aim of reporting the effect size is to enable the reader a better understanding of the importance of the findings of a given study.A larger effect size denotes a greater impact of the central variable of the study (explanatory factor) on the issue that is being analyzed (outcome).In the present study, caries was found to exert a real (as well as significant) influence on the OHRQoL of preschool children.The discriminant validity of the ECOHIS was confirmed based on the large effect size found for the majority of studies analyzed, 1,3,4,5,16,18,20,22 which means that this scale can be used to distinguish groups with different degrees of oral problems.
The size effect was moderate for two validation studies (one with non-probabilistic sampling and one with probabilistic sampling) and two application studies (one with non-probabilistic sampling and one with probabilistic sampling) that used cutoff points of one to three, 1,4 one to five 16 and more than four teeth with caries. 19Unlike the finding for the study conducted by Scarpelli et al. 4 with probabilistic sampling (field study), a large magnitude of the difference was found between a group of children without caries and another group with one to three teeth with caries in a preliminary study with non-probabilistic sampling, which suggests that the magnitude of differences in population-based samples is more discrete.In contrast, a validation study with the same type of sampling had significant findings but a small to negligible effect size (0.27 and 0.08), 5 despite the fact that the study used the same cutoff point for dental caries as that used in the previously cited investigation. 4The statistical significance combined with the small effect size in the study conducted by Peker et al. 5 demonstrates that not every statistically significant result has clinical relevance.
Moderate effect sizes were found for the studies conducted by Wong et al. 6 , Martins-Júnior et al. 7 and Guedes et al. 17 , in which the researchers compared ECOHIS scores between children without caries and those with one or more teeth with caries.Wong et al. 6 and Martins-Júnior et al. 7 used probabilistic sampling, whereas Guedes et al. 17 used non-probabilistic sampling.Although sampling differences can result in different effect sizes, the common cutoff point for caries in these studies likely explains the similar findings.
Regarding effect size values, Cohen 15 states that the use of reference values (given by the author himself) is recommended only when there is no basis of comparison, such as effects previously established within a field.This was the case in the present investigation, as the authors of most of the studies analyzed did not consider the effect size of their findings, with the exception of Lee et al. 3 , Martins-Júnior et al. 7 and López Ramos et al. 22 .When typically found values are on the order of 0.2, an effect size of 0.3 may indicate an important effect.In contrast, an effect size of 0.6 may not be considered important if the values typically found in the field of interest are on the order of 0.8. 14he value obtained through the calculation of the effect size can be understood as a probability.If the presupposition of normality of the data distribution is accepted, an interpretation of the effect size value can be made based on a Z distribution table.Such a table provides the proportions of a localized area beneath sections of the standardized normal curve, allowing the determination of what percentage of individuals in the experimental group will exhibit an effect greater than the mean of the control group in future experiments. 12,14Presuming normality in the data distribution in the validation study conducted by Scarpelli et al. 4 with a probabilistic sample for which the effect size regarding child impact was 1.11 in the comparison of children without caries and those with four or more teeth with caries.Can be expected that 87% of children with four or more teeth with caries will exceed the mean impact value of the group without caries in future studies, as based on the Z distribution.
The high floor effect rate observed in some of the studies selected exerted an influence on the moderate effect size found for certain comparisons.If most of the respondents answer "never" for questions on the ECOHIS, the groups studied will likely be similar with regard to the impact given by the mean score of the OHRQoL scale, which will obviously lead to a difference with a small magnitude.In the study by Pahel et al., 1 approximately half of the parents/caregivers reported no impact from caries on the quality of life of their children.The characteristics of dental caries in the sample of this study may explain the floor effect for the ECOHIS, as only 69 children had caries and 112 were caries free.In the study by Lee et al., 3 the floor effect for the child and family impact sections (22.6 and 33.0%, respectively) did not exert an influence on the effect size, which was large.The lower floor effect for this study can be related to the fact that the data came from a clinical convenience sample and therefore those individuals already exhibited some type of oral problem, although this is not a general rule, as in a study involving a convenience sample from a dental clinic wherein the floor effect rate was 40.1 and 59.9% for the child and family impact sections, respectively, 16 which is nearly twice the rate found for the study by Lee et al. 3 The considerable floor effect rate (56.7 and 64.7% for the child and family sections) for the study by Scarpelli et al. 4 with a non-probabilistic sample did not exert an influence on the effect size, which suggests the occurrence of a significant difference regarding the floor effect between the groups with and without dental caries (data not shown in the study).However, with a floor effect rate of 51.5 and 64.6% for the child and family impact sections, respectively, in the probabilistic sampling, the effect size was moderate for both sections between the group with one to three teeth with caries and the group without caries.The same occurred in the application study conducted by Wong et al. 6 with probabilistic sampling.Most of the parents/caregivers (60.6 to 89.8%) reported that neither their children nor families ever experienced the problems addressed by the ECOHIS.Two population-based studies found a high percentage of "never" responses. 7,19According to Martins-Júnior et al., 7 this may be explained by the fact that the children were recruited during a national child vaccination campaign and were therefore not seeking dental treatment, which could affect the impact measured by the OHRQoL scale.In the study by Kramer et al., 19 the floor effect rate was higher than 80%.According to the authors, this distribution of scores is a characteristic of population-based studies and is likely indicative of individuals with a genuinely low degree of problems.In these population-based studies, the floor effect was probably less discrepant between groups, which may have exerted an influence on the moderate effect size encountered.
As a subjective evaluation, the ECOHIS allows parents/caregivers to have greater awareness of the impact that oral problems have on their children and themselves.Indeed, a lack of awareness in the general public is the reason why the oral health of preschool children is often relegated to a position of secondary importance; thus, many children with untreated dental caries experience physical pain as well as difficulties chewing, sleeping and socializing, with possible effects on self-esteem, growth, weight gain 16,18,23,24 and quality of life.Besides the possibility of raising awareness on the part of parents/caregivers, the outcomes derived from the assessment of OHRQoL are considered better measures for communication with policy administrators than merely the use of quantitative measures, such as caries indices. 25Thus, considering the clinical significance of OHRQoL findings derived from the ECOHIS, this scale can be used to furnish more concrete data regarding the impact of oral problems on the daily lives of preschool children.
The use of the effect size in the quantitative description of study outcomes is an important complement to the description of results derived from hypothesis tests.It is not enough to identify that some findings were significant and others were not, as the value of a study goes beyond statistical significance.In the present investigation, the determination of the effect size demonstrated the clinical significance of dental caries, proving that this oral problem exerts an impact on the OHRQoL of preschool children.

Conclusion
The determination of the effect size confirmed the discriminant validity of the ECOHIS.For most of the studies analyzed, the effect size was moderate to large.This means that the ECOHIS furnishes valid findings regarding the OHRQoL of preschool children with caries through the reports of parents/caregivers.

Figure 1 .
Figure 1.Flowchart of the process used to select published studies for this analysis

Table 2 .
Comparison of ECOHIS validation studies with probabilistic sampling for groups of preschool children with and without caries.
a Value for "n" used in data analysis; b Mean and standard deviation (SD) of ECOHIS score; c Effect size for standardized difference in mean ECOHIS scores between groups with and without caries; d Effect size calculated in the original study

Table 1 .
Comparison of ECOHIS validation studies with non-probabilistic sampling for groups of preschool children with and without caries.
a Value for "n" used in data analysis; b Mean and standard deviation (SD) of ECOHIS score; c Effect size for standardized difference in mean ECOHIS scores between groups with and without caries; d Effect size calculated in the original study.

Table 3 .
Comparison of ECOHIS application studies with non-probabilistic sampling for groups of preschool children with and without caries.Value for "n" used in data analysis; b Mean and standard deviation (SD) of ECOHIS score; c Effect size for standardized difference in mean ECOHIS scores between groups with and without caries; d Decayed, missing and filled deciduous teeth (dmft); a dash indicates that a p-value was not reported in the original study. a

Table 4 .
Comparison of ECOHIS application studies with probabilistic sampling for groups of preschool children with and without caries.
a Value for "n" used in data analysis; b Mean and standard deviation (SD) of ECOHIS score; c Effect size for standardized difference in mean ECOHIS scores between groups with and without caries.