Validation of a measuring instrument for the perception of oral health in women

The aim of this study was to estimate the reliability, validity and factorial invariance of the Portuguese version of the Geriatric/ General Oral Health Assessment Index (GOHAI) as applied to Brazilian women. A total 701 women over 18 years of age participated in this study. Telephone interviews were conducted. We evaluated the construct-related validity through factorial, convergent and discriminant validity. We carried out a confirmatory factor analysis using the χ2/df, CFI, GFI and RMSEA indexes. The invariance of the model in a second independent sample was estimated by multi-group analysis and internal consistency using Cronbach’s alpha coefficient. Items 5 and 9 presented factor weights below the adequate value and were removed. The three-dimensional and unifactorial model presented an adequate fit. We observed strong factorial invariance of the models in two independent samples (three-factor: ρλ = 0.62; pCov = 0.89, one-factor: ρλ = 0.81; pCov = 0.68) and weak factorial invariance between users and nonusers of dental prosthetics (threefactor: ρλ = 0.55; pCov = 0.01, one-factor: ρλ = 0.51; pCov = 0.02). The convergent validity was suboptimal. Internal consistency was adequate. The GOHAI applied to the study sample showed adequate reliability, factorial validity and stability in independent samples and between users and nonusers of dental prosthetics in both the threedimensional and the unifactorial structures.


Introduction
The assessment of oral health and its impact on the quality of life of individuals is an important step in health care practices, and some measuring instruments have been proposed in the literature to perform this assessment.Among these, one can mention the Oral Health Impact Profile (OHIP) 1 and the Geriatric/General Oral Health Assessment Index (GOHAI). 2 The purpose of these instruments is to assess the impact of oral health on an individual's quality of life, measured according to the perception of oral health.
The GOHAI was initially proposed in English by Atchison and Dolan 2 for the U.S. population, but currently has versions adapted for different countries, namely Sweden, 3 Malaysia, 4 Saudi Arabia, 5,6 Germany, 7 France, 8 Romania, 9 China, 10 India, 11 Turkey, 12 Mexico 13 and Brazil. 14espite the several transcultural proposals submitted by different countries, only the studies conducted in Sweden, 3 Saudi Arabia, 5,6 India, 11 Declaration of Interests: The authors certify that they have no commercial or associative interest that represents a conflict of interest in connection with the manuscript.
Turkey 12 and Mexico 13 included the assessment of the oral health perception construct as evidence of factorial validity.However, even these studies did not perform confirmatory factor analysis, which should have been adopted, considering that the theoretical construct had been previously defined by Atchison and Dolan. 2 It should be noted that the confirmatory factor analysis, based on the theoretical construct defined by the scale's authors, is required to evaluate the validity of the data gathered.Although this analysis strategy is still unusual in the medical literature, psychometrics states that it is essential in order to clarify the metric characteristics of the scales in different samples, because it is the only way to assess the quality of the information obtained.
Knowing the widespread use of GOHAI in Brazilian studies, and given the lack of information on its psychometric qualities, this study was designed to estimate the reliability, validity and factorial invariance of the Portuguese version of the Geriatric/General Oral Health Assessment Index (GOHAI), as applied to Brazilian women.

Methodology Study and sample design
This is a cross-sectional study with a probabilistic sampling design conducted in two stages.First, the participants were stratified according to the census sector, and then the sample units were randomly and systematically selected from the phone book.
The inclusion criteria were: living in the city of Araraquara (SP-Brazil), having a telephone landline, being female and over 18 years of age.
The minimum sample size to be used in validation studies is estimated by standard formulas for analyzing structural models 15 that take into consideration the model's degree of freedom, a of 5%, and a test power of at least 80%, which resulted in a sample size estimation of 144 subjects for this study.However, since the objective was to study the psychometric qualities of the GOHAI in a normative female population, we decided to use a sample large enough to capture the population's variability adequately.Thus, we chose to use a representative sample of residents from the city of Araraquara -SP, female, and over 18 years of age.Accordingly to the 2010 Census there was a total population of 162,912 residents over the age of 18 years, 50% female, in Araraquara, SP.Assuming an α = 5%, a relative error margin of 7.5% resulted in a minimum sample size of 681 people.Assuming a 10% rate of refusal to participate, the sample size was corrected to n = 757.

Measuring instrument
The instrument used was the Portuguese version of the Geriatric/General Oral Health Assessment Index (GOHAI).The questionnaire consists of 12 items distributed in one dimension, but the items composing the instrument were drawn up considering three aspects, namely, "physical/functional," "psychosocial/psychological," and "pain/discomfort."In this study, we tested a three-factor structure, preserving the theoretical distinction of items.The answers were given in a five-point type-Likert scale. 2 In this Portuguese version, three of the 12 items were inverted in relation to the remaining items.
To characterize the sample, data was gathered on age, marital status (single, married, widowed, divorced), use of dental prosthetics (yes or no) and type (fixed partial prosthesis, removable partial prosthesis or complete denture), education level and economic level.The level of education and economic development were estimated according to the Brazilian Economic Classification Criteria. 16

Procedures
Given the evidence presented in the literature 17 and the simplicity of the application of the instrument used, we decided to conduct the survey by telephone interviews.
Whenever a phone call was answered, the researcher presented himself and read the Informed and Free Consent Form for Research Subjects.Only people who agreed to its terms participated in the study.Participants were selected from the phone book of the city of Araraquara, using a systematic sampling process.The phone calls lasted approximately 10 minutes and were performed by only one interviewer calibrated in a pilot study (k =0.89).For calibration, the researcher applied the GOHAI on two separate occasions one week apart.Sixty women participated in the pilot study.Telephone calls were made until a total of 757 calls were reached.Thus, we performed 810 calls, of which 757 were answered, and, of these, 56 people refused to participate in the study.

Analysis of the psychometric characteristics
The psychometric sensitivity of each item of the GOHAI was assessed through its measures of shape.Absolute values of skewness (Sk) and kurtosis (Ku) higher than 3 and 7, respectively, were considered indicative of severe deviation from the normal distribution. 18he factorial validity of the three-dimensional model for the total sample was estimated by confirmatory factor analysis, using the qui-square statistic over degrees of freedom (χ 2 /df), the comparative fit index (CFI), the goodness of fit index (GFI) and the root mean square error of approximation (RMSEA), as the goodness of fit indices.The fit of the model to the data was considered adequate when factorial item weights of λ ≥ 0.50, χ 2 /df ≤ 4.0, CFI and GFI ≥ 0.90 and RMSEA ≤ 0.10 were reached. 18tems with λ < 0.40 were removed from the model, as well as those related to more than one factor, as indicated by the modification indices estimated by the Lagrange Multipliers (LM > 11, p < 0.001).The comparison between the initial model (Model 0-M0) and that obtained after removal of the items (Model 1-M1) was performed using indices based on the information theory (Akaike Information Criterion-AIC, Browne-Cudeck Criterion-BCC, and Bayesian Information Criterion-BIC).The best model was that which had the lowest values in one or more of these indices. 18he stability of the factor model was assessed in two steps and evaluated using cross-validation.In the first step, the sample was divided into two parts, one containing 70% of the sample (n = 490), called "test sample" (Model 2-M2), and the other, 30% (n = 211), named "sample validation" (Model 3-M3).A multigroup analysis with the chi-square difference test (Δχ 2 ) was performed to test the invariance of the model in the two subsamples.When the hypothesis of factorial invariance of the factorial weights (weak invariance) was accepted, invariance of covariance (strong invariance) analysis was performed. 18e same procedure was adopted in a second analysis, but now the stability of the model was tested on users (n = 274) (Model 4-M4) and nonusers (n = 427) (Model 5-M5) of dental prosthetics.
For models 1-5, convergent validity, discriminant validity and internal consistency were calculated.Convergent validity was estimated by the average variance extracted (AVE) and composite reliability (CR). 19Values of AVE ≥ 0.50 and CR ≥ 0.70 were considered indicative of convergent validity. 19Discriminant validity was accepted when AVE i and AVE j for factors i and j was greater than the squared correlations between these two factors (r ij 2).The internal consistency of the factors was estimated with the standardized Cronbach's alpha coefficient (a), and was considered appropriate when a ≥ 0.70. 20nalyses were conducted using IBM SPSS ® Amos (v.20, SPSS Inc., Chicago, USA) and IBM SPSS ® Statistics (v.21 SPSS Inc., Chicago, USA).
First-order correlations between factors were calculated on the total sample data.Values close to 1 were considered suggestive of a second-order hierarchical model.The standardized regression weights (β) from the proposed second-order factor to the firstorder factors were estimated in order to study the second-order hierarchical model.However, given the strong and highly significant correlation values between first-order factors, we also evaluated the goodness of fit of a unifactorial structural model, and validity and reliability were estimated following the same procedure reported above.

Ethical Aspects
The present study was approved by the Ethics Research Committee of the School of Pharmaceutical Sciences -UNESP (protocol: 52/2009).

Results
The study included 701 women, showing an adherence rate of 92.6%, thus indicating the representativeness of the sample in the study population.The mean age of the participants was 44.4 (SD=16.3)years.Table 1 presents the distribution of participants in regard to sociodemographic and clinical characteristics.
As observed, most participants were married, belonged to social class C and were users of some type of dental prosthesis.
In all dimensions, the items had skewness (Sk = |0.42-1.92|)and kurtosis (Ku = |0.30-2.36|)values close to those presented in the normal distribution, therefore, posing no considerable sensitivity or normality issues.
The information-theory-derived indices support the better fit of the adjusted model (M1) to the data, as compared to the initial model (M0).All models M1 to M5 presented an adequate goodness of fit to the data.The reliability (CR, α) was adequate for the physical/functional and psychosocial/psychological dimensions in all models.The convergent validity (AVE) had below adequate values for all models, but the 3 factors presented discriminant validity (AVE≥r 2 ) for two of the three factors.
Judging from the strong correlations observed between factors, we evaluated the goodness of fit of a second-order factorial model.Figure 1 shows the second-order hierarchical factor of "Perception of oral health" of GOHAI, fitted to the total sample.We can confirm through the standardized regression weights that all dimensions of GOHAI are strongly and significantly (p < 0.01) related to the central "Perception of oral health" construct.
Owing to the low convergent validity of the three factors, the lack of discriminant validity between D1 and the other factors (Table 1), and the strong relation between the three factors and the central construct (Figure 1), from a strictly psychometric perspective,   Figure 1.Second-order hierarchical model adjusted to the total sample (χ 2 /df = 3.66, CFI = 0.97, GFI = 0.97, RMSEA = 0.06) for the Geriatric/ General Oral Health Assessment Index (GOHAI) the GOHAI would produce results with greater validity and reliability if it were scored as a unifactorial construct.Thus, Table 2 shows the goodness of fit of a unifactorial construct in the different subsamples under study (models M1 u to M5 u ).
All models from M1 u to M5 u presented an adequate fit to the data.However, taking into account the indices based on the information theory, we noted that there was no relevant difference between the one-factor and the three-factor models (Table 1 and Table 2).The reliability (CR, a) was adequate, and convergent validity was below adequate for all models.

Discussion
As far as we know, this is the first study that has ever evaluated the psychometric properties of the data gathered with the Geriatric/General Oral Health Assessment Index (GOHAI), applied to Brazilian women.The results suggest acceptable psychometric properties of the GOHAI for Brazilian women, with the same sociodemographics of the probabilistic sample composing this study.In this paper, we have not only given information about the metric quality of the GOHAI for the sample, but have also introduced the steps required for evaluating the quality of the data arising from the use of scales like the GOHAI.
The analysis of factorial validity and invariance in independent samples is a unique condition for determining the best model of data analysis and its stability in different samples.In this study, we Table 2. Goodness of fit indices of the confirmatory factor analysis (CFA), convergent validity (AVE), composite reliability (CR) of the models fitted to different samples/models (M0-M5), Pearson correlation between factors (r), squared correlations (r 2 ), and internal consistency (α) for the Geriatric/General Oral Health Assessment Index (GOHAI).Araraquara -SP / Brazil, 2011.M0: Three-factor model, M1: Adjusted three-factor model (excluding items 5 and 9 with correlation between errors 1 and 2), M2: M1 adjusted to the test sample, M3: M1 adjusted to the validation sample, M4: M1 adjusted sample of users of dental prosthetics, M5: M1 adjusted sample of nonuser dental prosthetics; ## D1: physical/functional, D2: psychosocial/psychological, D3: pain/discomfort noted that the evaluation of the perception of oral health among Brazilian women can be performed either considering the three-factor model (Table 2), preserving the content (tri-factorial) proposal of the items, 2 or using the one-factor model (Table 3).However, we observed (Figure 1) that the three factors have strong correlations with each other and that the convergent validity (Table 2) was below the desirable values, 19 indicating that the items that reflect these factors did not correlate more strongly with this factor than others.These estimates are strong indications that the unifactorial structure may be more parsimonious in representing the construct, despite adequate fit to the three-factor model.The comparison between the factorial structure obtained and the published research on GOHAI dimensionality has shown mixed results.In Turkey 12 and Sweden, 3 the two-factor model was that which presented the best fit to the data; in Saudi Arabia 6 and in India 11 the best fit was the unifactorial model; and in Mexico 13 the four-factor structure was proposed as offering the best fit to the data gathered.These mixed results also demonstrate a well-known fact: the validity of a scale is not characteristic of an instrument per se, but of the instrument as applied to the sample. 21Thus, the assessment of the psychometric properties of the data is the basic premise to be evaluated, prior to presenting any outcomes arising from the use of scales.However, we would like to point out that the assessment of these characteristics in psychometric instruments is essential to performing steps such as sensitivity analysis of psychometric items, as well as factorial, convergent, discriminant, concurrent, discriminant and predictive validity, which are usually neglected.These steps must be performed by all who intend to evaluate latent variables, since their evaluation is complex and requires the use of robust statistical modeling, and usually also large samples. 18espite the stability of the factor weights in the different samples, we observed no invariance in the correlations between factors, when applying the GOHAI in samples of users and nonusers of dental prosthetics.These findings suggest that the samples were associated to oral health perception factors differently.This observation is also supported by the variance explained by the model, which is higher for users (three-factor=74% and one-factor=71%) than for nonusers (three-factor=59% and unifacto-rial=57%) of dental prosthetics.This characteristic is consistent with the theoretical conception proposed by the GOHAI, designed to capture the perception of oral health in geriatric populations having more compromised oral health. 2 Despite the limitations of this study, namely that the sample was representative only of the female population, that it was a cross-sectional and correlational study and that no adjustment of the results was made in the oral clinical condition of the subjects, it should be noted that these limitations are common in validation studies of scales, and that the results attest to the reliability and validity of the Portuguese version of the GOHAI.

Conclusion
The GOHAI applied to the sample of women in this study presented good reliability, factor validity and stability in independent samples and between users and nonusers of dental prosthetics, when using both a three-factor and a one-factor structure.

Table 3 .
Quality of adjustment indices of the confirmatory factor analysis (CFA), convergent validity (AVE), composite reliability (CR) of the unifactorial models adjusted to different samples and internal consistency (a) of the Geriatric/General Oral Health Assessment Index (GOHAI).Araraquara -SP / Brazil, 2011.CFA: confirmatory factor analysis, AVE: average variance extracted, CR: composite reliability; # M1u: adjusted unifactorial model (excluding items 5 and 9 and with correlation between errors 1 and 2), M2u: M1u adjusted to the test sample, m3u: M1u adjusted to the validation sample, M4U: M1u adjusted to the sample of users of dental prosthetics, M5u: M1u adjusted to the sample of nonusers of dental prosthetics. *