Acessibilidade / Reportar erro

Construct validity of the Motor Development Scale (MDS)

Abstract

Objective

Construct validity for the Motor Development Scale (MDS) has not been established. The aim of this study was to examine whether the unidimensional model of MDS would be appropriate for children aged 4 to 6 years-old and provide construct validity for the items concerning this age group in Brazil.

Methods

A total of 938 children participated in the study (214 4-year-olds, 643 5-year-olds, and 81 6-year-olds). Confirmatory factor analysis (CFA) was used to evaluate construct validity of the MDS using a unidimensional model.

Results

The CFA for the unidimensional model showed excellent adequacy indices for age 4: χ2(2) = 0.581, p = 0.748, comparative fit index (CFI) = 1.000, Tucker-Lewis index (TLI) = 1.090, root mean square error of approximation (RMSEA) = 0.000 (90% confidence interval [90%CI] = 0.000 to 0.093, close fit [Cfit] = 0.841); age 5: χ2(2) = 2.669, p = 0.263, CFI = 0.993, TLI = 0.980, RMSEA = 0.023 (90%CI = 0.000 to 0.085, Cfit = 0.682), weighted root mean square residual (WRMR) = 0.407; and age 6: χ2(9) = 8.275, p = 0.506, CFI = 1.000, TLI = 1.010, RMSEA = 0.000 (90%CI = 0.000 to 0.118, Cfit = 0.653), WRMR = 0.495. Reliability was good: ω = 0.87 (95%CI = 0.81 to 0.92).

Conclusion

The proposed unidimensional solution for the MDS provides a concise, parsimonious and reliable way to assess motor development in children aged 4 to 6 years.

Motor skills; assessment; psychometrics; construct validity; children

Introduction

A variety of standardized assessment instruments have been used to examine children’s motor skill development because they can objectively identify children with motor delays. One of the most common assessments used by Brazilian professionals is the Motor Development Scale (MDS; in Portuguese Escala de Desenvolvimento Motor [EDM]).11. Rosa Neto F. Manual de avaliação motora - EDM. 2nd ed. Florianópolis: Governo do Estado de Santa Catarina; 2014. This Brazilian instrument assesses children motricity based on three important concepts, namely, coordination, proprioception and perception, divided into six domains: fine motricity, global motricity, balance, body schema, spatial organization, and temporal organization, with 10 items per domain. The instrument was translated into English and Spanish in 2018.

The MDS was developed to offer a multidimensional model (six domains). Even though content and criterion validity have been established for the whole instrument,11. Rosa Neto F. Manual de avaliação motora - EDM. 2nd ed. Florianópolis: Governo do Estado de Santa Catarina; 2014. it has not been evaluated for construct validity based on latent trait. There is concern that some assessments used for children may not measure the intended construct due to the nature of the skills. It is necessary to confirm that the MDS instrument can accurately measure motor skill development22. Pasquali L. Psychometrics. Rev Esc Enferm USP. 2009;43:992-9. through other kinds of validity, such as construct validity (validity based on internal structure) using structural equation modeling.

In order to make a succinct instrument, it is ideal to have the minimum number of dimensions and items that represent the most parsimonious solution corresponding to the underlying factor structure.33. Brown TA. Confirmatory factor analysis for applied research. New York: Guilford Publications; 2015. The aim of this study was to examine whether the unidimensional model of the MDS would fit for children aged 4 to 6-years-old and provide construct validity for the items concerning this age group in Brazil.

Methods

This research was approved by the ethics committee of research at Universidade Federal de São Paulo (UNIFESP). Parents provided informed consent and all children assented to participate in the study.

Participants

The study used previously collected data from a randomized, stratified, representative sample of 938 children of both sexes, collected at 29 public preschools from a metropolitan city of São Paulo (Embu).

The sample included 214 students aged 4 (mean age = 56.28 months, standard deviation [SD] = 1.98, 50% male); 643 aged 5 (mean age = 66.19 months of age, SD = 3.38, 53% male); and 81 aged 6 (mean age = 72.56 months, SD = 1.05, 54.3% male). Because we collected data from 29 schools, in the analysis we considered at least 10 participants per observed indicator variable (item of the test) as a rule-of-thumb for a lower bound adequate sample size,44. Nunnally JC, Bernstein IH, Berge JMT. Psychometric theory. New York: McGraw-Hill; 1967. totaling at least 80 children for each age.

Instrument and procedure

The Motor Development Scale is an assessment tool used to evaluate children with an ordinal scoring system. It has six dimensions (fine motricity, global motricity, balance, body schema, spatial organization and temporal organization). Each dimension has 10 unique items. The items are related to a particular age level. In our study, children were assessed at the level matching their chronological age. For example, if the participant was 5 years old, the task was initially selected at level 5. Children who successfully completed the task were scored 1; those who could not perform the task were scored 0. For the tasks requiring skills on left and right side of the body, a score of 0.5 was recorded for each side.

If the child could successfully perform the task at the level for his/her age, he/she was assessed at the next higher level; if the child could not perform the task, he/she was assessed at the previous lower level, until the child reached the limits of his/her skill level. That is, the test was stopped when the participant received a 0 score, either at the same level, above or below their chronological age. However, for this study, as we planned to test construct validity for the items by age in a unidimensional model, only the scores given for levels corresponding to the chronological ages of the participants were considered.

Data analysis

Confirmatory factor analysis (CFA) with clustering by school was used to evaluate: a) the goodness of fit of the measurement model; and b) the strength of the correlation between the items and overall motricity.33. Brown TA. Confirmatory factor analysis for applied research. New York: Guilford Publications; 2015. As such, the factor loading was a correlation between the observed categorical item and the latent measure of motricity. The higher this correlation, the lower the residual variance, which indicates a higher reliability index for each motricity indicator.

Several fit indices were used to evaluate the model solution, including chi-square (χ2), comparative fit indices (CFI), the Tucker-Lewis index (TLI), and root mean square error of approximation (RMSEA). The following cutoff criteria were used to determine good model fit: a non-statistically significant chi-square p-value (> 0.05); an RMSEA near or below 0.06; CFI and TLI near or greater than 0.95.55. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6:1-55. The weighted root mean square residual (WRMR) estimator was also used66. Muthén B, Du Toit SH, Spisic D. Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Psychometrika. 1997;75:1-45. because the observed indicators (i.e., motricity items) were ordinal (i.e., Likert scale). We adopted < 0.05 as statistically significant. All analyses were run using Mplus 8.0.77. Muthén LK, Muthén BO. Mplus user’s guide. 8th ed. Los Angeles: Muthén and Muthén; 2018.

To evaluate the reliability of the MDS, we used the omega total (ω).88. McDonald RP. Test theory: a unified treatment. Mahwah: Lawrence Erlbaum; 2013.,99. McDonald RP. The theoretical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. Br J Math Stat Psychol. 1970;23:1-21. Reliability measures are commonly reported as point estimates, and there is no clear cut-off point. Therefore, we followed the guidelines of Rodriguez et al.,1010. Rodriguez A, Reise SP, Haviland MG. Applying bifactor statistical indices in the evaluation of psychological measures. J Pers Assess. 2016;98:223-37. which state that ω > 0.8 indicates a sufficient relationship between the latent variable and item scores.

Results

Figure 1 shows a unidimensional model for MDS (motricity) items by age, with the factor loading and standard errors.

Figure 1
Unidimensional model for the Motor Development Scale by age: A) age 4; B) age 5; C) age 6. B = balance; BS = body schema; FM = fine motricity; GM = global motricity; spa = spatial organization; temp = temporal organization.

For age 4 (Figure 1A), the global motricity item was excluded because it had only one response category (all participants performed the task) and consequently did not present any variance. Temporal organization was also excluded from the model for this age because it made the model unidentifiable (bivariate empty cells = correlation of 1). Finally, the CFA with clustering by school for the unidimensional model with four items presented excellent indices of adequacy for age 4: χ2(2) = 0.581, p = 0.748, CFI = 1.000, TLI = 1.090, RMSEA = 0.000 (90% confidence interval [90%CI] = 0.000 to 0.093, close fit [Cfit] = 0.841), WRMR = 0.012. All the four items presented good reliability (factor loadings). The reliability of MDS item scores in the age 4 sample, computed as omega, was ω = 0.95 (95%CI = 0.81 to 0.99).

For age 5 (Figure 1B), body scheme was excluded from the model because it made the model unidentifiable (resulting in empty bivariate cells), and temporal organization was excluded because it presented a very low factor loading, suggesting that the item was not capturing the latent trait. After excluding those items, adequacy indices were excellent for the model with four items: χ2(2) = 2.669, p = 0.263, CFI = 0.993, TLI = 0.980, RMSEA = 0.023 (90%CI = 0.000 to 0.085, Cfit = 0.682), WRMR = 0.407. The reliability of MDS item scores in the age 5 sample was ω = 0.65 (95%CI = 0.58 to 0.72).

For age 6 (Figure 1C), the original model with six items presented excellent indices of adequacy: χ2(9) = 8.275, p = 0.506, CFI = 1.000, TLI = 1.010, RMSEA = 0.000 (90%CI = 0.000 to 0.118, Cfit = 0.653), WRMR = 0.495. The reliability of MDS item scores in the age 6 sample was ω = 0.87 (95%CI = 0.81 to 0.92).

Discussion

Using CFA, we evaluated the internal structure of MDS items for samples aged 4, 5 and 6 years. Specifically, the model for age 6 maintained the original six items, with good reliability. The models for ages 4 and 5 showed good reliability with four items. However, at age 4, global motricity was excluded because all participants had the same score, indicating that this item did not discriminate the participants’ performance. Also, temporal organization presented a correlation of 1 with the other items, indicating measurement redundancy (measuring the same aspect as other items), and therefore had to be excluded from the model. For age 5, body schema presented a correlation of 1 with the other items and needed to be excluded. In addition, temporal organization was excluded because for an item to remain in the model to capture the latent trait, the factor loading should be more than 0.3.1111. Raykov T, Marcoulides GA. A first course in structural equation modeling. London: Routledge; 2012.

This is important from a clinical perspective because it can shorten the testing time without compromising the identification of possible delays in motor performance. Thus, this study showed good internal consistency and reliability for the unidimensional model while taking into account the large number of schools sampled.

We evaluated children aged 4 to 6 from only one municipality in the metropolitan area of São Paulo. Therefore, these results do not necessarily apply to other age groups or to the whole Brazilian population. In addition, because no previous study has examined the construct validity of the MDS, our results could not be compared. Further tests of the psychometric properties of the MDS are needed, such as multimensional solution, bifactor model, measure invariance and others.

In summary, this is the first manuscript providing construct validity based on internal consistency for MDS items. The proposed unidimensional solution provides a concise, parsimonious, reliable way to assess motor development skills in children aged 4, 5 and 6 years, and could therefore be used in research and clinical settings.

Acknowledgments

This work was financed in part by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq; grant 466688/2014-8), Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES; Finance Code 001), and Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP; grant 2016/10120-1).

We thank Dr. Francisco Rosa Neto for contributing by answering our questions regarding the EDM.

References

  • 1
    Rosa Neto F. Manual de avaliação motora - EDM. 2nd ed. Florianópolis: Governo do Estado de Santa Catarina; 2014.
  • 2
    Pasquali L. Psychometrics. Rev Esc Enferm USP. 2009;43:992-9.
  • 3
    Brown TA. Confirmatory factor analysis for applied research. New York: Guilford Publications; 2015.
  • 4
    Nunnally JC, Bernstein IH, Berge JMT. Psychometric theory. New York: McGraw-Hill; 1967.
  • 5
    Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6:1-55.
  • 6
    Muthén B, Du Toit SH, Spisic D. Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Psychometrika. 1997;75:1-45.
  • 7
    Muthén LK, Muthén BO. Mplus user’s guide. 8th ed. Los Angeles: Muthén and Muthén; 2018.
  • 8
    McDonald RP. Test theory: a unified treatment. Mahwah: Lawrence Erlbaum; 2013.
  • 9
    McDonald RP. The theoretical foundations of principal factor analysis, canonical factor analysis, and alpha factor analysis. Br J Math Stat Psychol. 1970;23:1-21.
  • 10
    Rodriguez A, Reise SP, Haviland MG. Applying bifactor statistical indices in the evaluation of psychological measures. J Pers Assess. 2016;98:223-37.
  • 11
    Raykov T, Marcoulides GA. A first course in structural equation modeling. London: Routledge; 2012.

Publication Dates

  • Publication in this collection
    17 Apr 2020
  • Date of issue
    Jan-Mar 2020

History

  • Received
    18 Dec 2018
  • Accepted
    04 June 2019
Associação de Psiquiatria do Rio Grande do Sul Av. Ipiranga, 5311/202, 90610-001 Porto Alegre RS/ Brasil, Tel./Fax: (55 51) 3024 4846 - Porto Alegre - RS - Brazil
E-mail: trends@aprs.org.br