Construct validity and reliability of a checklist for volleyball serve analysis

Validade de construto e fidedignidade de uma lista de checagem para análise do saque do voleibol

Abstract

This study aims to investigate the construct validity and reliability of the checklist for qualitative analysis of the overhand serve in Volleyball. Fifty-five male subjects aged 13-17 years participated in the study. The overhand serve was analyzed using the checklist proposed by Meira Junior (2003), which analyzes the pattern of serve movement in four phases: (I) initial position, (II) ball lifting, (III) ball attacking, and (IV) finalization. Construct validity was analyzed using confirmatory factorial analysis and reliability through the Cronbach’s alpha coefficient. The construct validity was supported by confirmatory factor analysis with the RMSEA results (0.037 [confidence interval 90% = 0.020-0.040]), CFI (0.970) and TLI (0.950) indicating good fit of the model. In relation to reliability, Cronbach’s alpha coefficient was 0.661, being this value considered acceptable. Among the items on the checklist, ball lifting and attacking showed higher factor loadings, 0.69 and 0.99, respectively. In summary, the checklist for the qualitative analysis of the overhand serve of Meira Junior (2003) can be considered a valid and reliable instrument for use in research in the field of Sports Sciences.

Key words
Checklist; Motor skill; Volleyball

Resumo

O estudo tem como objetivo investigar a validade de construto e fidedignidade da lista de checagem para análise qualitativa do saque por cima do Voleibol. Participaram do estudo 55 sujeitos do sexo masculino com idades entre 13 e 17 anos. O saque por cima foi analisado utilizando a lista de checagem de Meira Junior (2003) que analisa o padrão de movimento do saque em quatro fases: posição inicial, levantamento da bola, ataque à bola e finalização. A validade de construto foi analisada utilizando análise fatorial confirmatória e a fidedignidade através do coeficiente alfa de Cronbach. A validade de construto foi suportada pela análise fatorial confirmatória com os resultados do RMSEA (0,037 [Intervalo de confiança 90% = 0,020-0,040]), CFI (0,97) e TLI (0,95) indicando um bom ajuste do modelo. Em relação à fidedignidade, o coeficiente alfa de Cronbach foi de 0,661, sendo este valor considerado aceitável. Dentre os itens da lista de checagem, o lançamento da bola e o ataque à bola apresentaram maiores cargas fatoriais, com 0,69 e 0,99, respectivamente. Em síntese, a lista de checagem para análise qualitativa do saque por cima de Meira Junior (2003) pode ser considerada um instrumento válido e confiável para utilização em pesquisas na área das Ciências do Esporte.

Palavras-chave
Habilidade motora; Lista de checagem; Voleibol

INTRODUCTION

The Volleyball serve is a predominantly closed sports skill, which allows testing basic motor learning issues, but with a task that has great control of variables and high ecological validity. Consequently, the findings of studies that use this task may also provide subsidies for professional intervention¹. Regarding Motor Behavior, researchers have used it to investigate the effect of factors that influence the acquisition of motor skills, such as contextual interference^2,3, performance knowledge^4,5 and demonstration⁶.

Performance on this ability can be measured through movement product or process measures. Product measures analyze the outcome of the motor action quantitatively, while process measures evaluate qualitative aspects related to the movement pattern⁷. In most of the experimental studies that have used the volleyball serve task, it has been more common to evaluate the product of the action, that is, the serve accuracy.

Among the instruments validated for the evaluation of the Volleyball serve accuracy found in international literature, we can mention the AAHPERD⁸ Serve Accuracy Test and the Volleyball Skills Test Battery⁹. Although they are fundamental to explain the change in behavior in a task that is oriented towards an external goal, these instruments do not evaluate changes in the execution pattern, that is, the process measures in the serve execution.

A process measure widely used by researchers is the checklist for qualitative analysis of Volleyball serve proposed by Meira Junior¹⁰, which has undergone the content validity process, but not construct. Construct validity needs to be tested so that an instrument is recommendable for research use, as well as the internal consistency of items (reliability), which make up the test^11,12. Pasquali¹¹ argues that construct validity is the most fundamental form of validity, and that construct is scientifically searchable when it is capable of behavioral representation. In turn, reliability refers to the degree of interrelation between components of the factor being tested¹².

Considering the applicability of the serve ability in experimental studies^2-6, as well as the importance of the development of valid instruments with items capable of measuring it adequately from a process perspective of movement, the checklist for analysis of the Volleyball serve pattern¹⁰ still lacks effort for other steps of its validation. The present study aims to verify the construct validity and reliability of the checklist proposed by Meira Júnior¹⁰ to analyze the pattern of movement of overhand serve in Volleyball.

METHODOLOGICAL PROCEDURES

Participants

Fifty-five right-handed male subjects, who did not participate in volleyball training aged 13-17, participated in the study. Participants were enrolled in public schools of Belo Horizonte-MG. The Free and Informed Consent Form was signed by legal representatives, consenting to the participation of adolescents in the study. In addition, all participants read and signed the consent form, which also contained information regarding the study objectives, procedures and risks involved in the research. The research project was submitted to the Research Ethics Committee of the Federal University of Minas Gerais (protocol No. 42988515.9.0000.5149) and approved by protocol No. 1.939.735.

Task and Instruments

The skill used in the present study was the overhand serve in Volleyball, conducted with participants positioned five meters away from the ‘A’ side of the court, with both feet resting on the ground and facing the target located on the side opposite of the court (Figure 1). The objective of the task was to hit the center of a target (located 4 meters from the net on the ‘B’ side of the court) with the overhand serve in Volleyball, based on the protocol of Ugrinowitsch and Manoel^13, used in later studies^5,6,14 and adapted for the present study. The TNT target (non-woven fabric) has 4 areas with diameters of 1, 2, 3 and 4 meters. Participants made three serve attempts over nine meters away from the center of the target, with intervals of 6 to 8 seconds between attempts.

Figure 1
Instrument for evaluating the serve performance.

An official volleyball court was used with the net set at the height of two meters. The service area was demarcated with tape. For the application of tests, the following materials were used: 1 digital camera with acquisition frequency of 240 frames per second, a tripod, 5 official Volleyball balls and memory card. The camera was laterally positioned at 45 degrees in relation to the server, close to the net and the lateral line on the right side of the court.

For the qualitative analysis of the serve movement pattern, a checklist was proposed by Meira Júnior¹⁰ . The checklist analyzes the preparation (starting position) and the serve performance, the last step being the ball throwing, ball attacking and finishing. In the present study, the intraclass correlation coefficients for inter-rater reliability (ICC_2,1) and intra-rater reliability (ICC_3,1) ranged from 0.814 to 0.998 in the different phases of the serve execution. The procedures for determining the intraclass correlation coefficients followed Weir’s recommendations¹⁵.

Procedures

After signing the Consent Form, participants were directed to the experiment site where they visualized the experimental environment. Explanation of the task objective (i.e., attempt to reach the center of the target with the ball using the tested skill) was given below. Prior to performing attempts, subjects were shown a video of the serve execution performed by a person with experience in the task. After this procedure, the participant was directed to the region from where the serve should be executed from “prepares” and “goes” commands for each attempt.

The analyses of videos were carried out in slow motion, after data collection, with the help of a computer and software for video playback. The pattern of serve movement was classified according to the normative criteria established by the author, in which score from 1 to 3 points is assigned to each of the serve stages (i.e., starting position, ball lifting and ball attacking).

Statistical analysis

Mean and standard deviation were used to describe the performance of participants in the different stages of serve execution. Construct validity was analyzed through confirmatory factor analysis. The existence of outliers was verified by the quadratic distance of Mahalanobis (D²), the normality of variables was evaluated by the uni and multivariate asymmetry (sk) and kurtosis coefficients (ku). The maximum likelihood method was used to estimate parameters. The adjustment indexes used to assess the suitability of the model included the Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), and Root Mean Squared Error of Approximation (RMSEA). The reliability of the instrument was examined using the Cronbach alpha index¹⁶. For analyses purposes, the Mplus (v. 7.0) and SPSS Statistics software (v. 20, SPSS, An IBM® Company, Chicago, IL) were used.

RESULTS

Table 1 presents the means and standard deviations for each of the four phases of the serve movement pattern. The Cronbach’s alpha coefficient to evaluate the reliability of the items was α = 0.661. Although this index is at the limit of adequate reliability due to the small number of items, this value is considered acceptable^12,17-19.

Table 1
Mean and standard deviation of the scores in each of the serve phases and Cronbach’s alpha value.

		Mean		Standard Deviation		Cronbach’s alpha
Initial position		1.96		0.40		0.661
Ball lifting		1.93		0.62
Ball attacking		2.10		0.40
Finalization		1.69		0.63

Analyses about data normality through the asymmetry and kurtosis index did not show violation distribution²⁰. The construct validity was supported by the confirmatory factor analysis with the RMSEA results (0.037 [confidence interval 90% = 0.020-0.040]), CFI (0.97), TLI (0.95), indicating good fit of the model. As for the factorial loads, among the checklist items, ball attacking and ball lifting presented higher standardized weights, 0.99 and 0.69, respectively. Modification indexes were not indicated in this analysis.

Figure 2
Confirmatory Factorial Analysis.

DISCUSSION

The present study aimed to investigate the construct validity and reliability of the checklist proposed by Meira Júnior¹⁰ to analyze the pattern of overhand serve movement in Volleyball. For this, a confirmatory factorial analysis with serve components (phases) was used to evaluate the construct validity and the Cronbach alpha coefficient to evaluate the reliability of the instrument. The results were promising, mainly in relation to the construct validity, with the confirmatory factorial analysis presenting indexes that point to good fit quality of the model²¹. Although literature is clear regarding the components that are part of the overhand serve, the values found are important because they allow us confirming the construct validity of the checklist. These results support the use of the instrument in research in the field of Motor Behavior^3,22,23 and in Sport Sciences in general.

Another implication of the confirmatory factor analysis is related to the total score of the serve. In the content validation process, a total score was established, which consists of a formula that adds the weighted scores assigned to each of the serve phases. Apparently, the different weights attributed to the phases (weight 1 for initial position, weight 3 for ball lifting, weight 4 for ball attacking and weight 1 for finalization) were established without a rigid analysis, that is, without a systematic verification regarding how much each component contributes to the performance in the skill. In fact, the present study demonstrates through the standardized weights of regressions that ball attacking (β = 0.99) followed by ball lifting (β = 0.69) are the items that most explain the model. On the other hand, the initial position (β = 0.43) presented greater weight than finalization (β = 0.23), which may lead us to question the weights assigned by the author of the instrument to obtain the total serve score. Considering these results, further studies with statistical support to improve the accuracy of the total score measurement should be performed.

Regarding the instrument reliability, although a minimum value of 0.7 is recommended for Cronbach’s alpha¹⁷, the low number of items²⁴ and a possible difference in the difficulty level of each item¹² justifies the adoption of a slightly lower coefficient (α = 0.661) as acceptable. As variability is influenced by the number of observations, it is expected that an instrument with higher number of items will have a higher α than another with lower number. Considering this argument, it can be assumed that the items of the instrument (serve phases) represent the same construct. According to Pasquali¹¹, reliability is fundamental to infer about the test precision, because if the items would covariate on a given occasion, they may present the same interaction at any occasion of test use.

CONCLUSION

In general, this study contributes to the fact that the checklist for analysis of the pattern of serve movement proposed by Meira Junior¹⁰ is a reliable instrument to be used in future research involving this skill. Since skills with greater number of degrees of freedom may have less experimental control, building instruments through a rigorous validation process is of great relevance. Considering that performance in a given skill depends on the relationship between movement product and process, the checklist for Volleyball serve associated with product measures such as serve accuracy may provide a more thorough assessment of performance changes in the process of motor learning. In addition, a study should be carried out to systematically establish the total serve score, based on the statistical analysis, considering that the skill components (serve phases) may have different levels of difficulty.

REFERENCES

• Ugrinowitsch H, Benda N. Aprendizagem motora : produção de conhecimentoe intervenção profissional. In: Corrêa UC, editor. Pesquisa em comportamentomotor: a intervenção profissional em perspectiva. São Paulo: EFP/EEFEUSP;2008. p. 240–59.
• ²
  Travlos AK. Specificity and variability of practice, and contextual interference in acquisition and transfer of an underhand volleyball serve. Percept Mot Skills 2010;110(1):298-312.
• ³
  Fialho JVAP, Benda RN, Ugrinowitsch H. The Contextual Interference Effect in a Serve Skill Acquisition with Experienced Players. J Hum Kinet 2006;50:65-78.
• ⁴
  Zubiaur M, Oña A, Delgado J. Learning Volleyball Serves: a Preliminary Study of the Effects of Knowledge of Performance and of Results. Percept Mot Skills 1999;89(1):223-32.
• ⁵
  Ugrinowitsch H, Fonseca FS, Carvalho MFSP, Profeta VLS, Benda RN. Efeitos de faixas de amplitude de CP na aprendizagem do saque tipo tênis do voleibol. Motriz: J Phys Ed 2011;17(1):82-92.
• ⁶
  Santos-Naves SP, Benda RN, Junqueira AHM, Alves GM, Veloso ALPM, Ugrinowitsch H. Efeito da demonstração distribuída na aprendizagem do saque do voleibol. Rev Bras Educ Fís Esporte 2014;28(4):629-39.
• ⁷
  Clark JE, Whitall J. What Is Motor Development? The Lessons of History. Quest 1989;41(4):183-202.
• ⁸
  American Association for Health, Physical Education, and Recreation. Skills test manual: volleyball for boys and girls. Washington: AAHPERD; 1969.
• ⁹
  Bartlett J, Smith L, Davis K, Peel J. Development of a valid Volleyball Skills Test Battery. J Phys Educ Recreat Danc 1991;62(2):18-21.
• ¹⁰
  Meira Jr CM. Validação de uma lista de checagem para análise qualitativa do saquedo voleibol. Motriz: J Phys Ed 2003;9(3):153-60.
• ¹¹
  Pasquali L. Psicometria. Rev Esc Enferm USP. 2009;43:992-9.
• ¹²
  Damásio B. Uso da análise fatorial exploratória em psicologia. Aval Psicol2012;11(2):213-28.
• ¹³
  Ugrinowitsch H, Manoel EJ. Interferência Contextual: Variação de programa e parâmetro na aquisição da habilidade motora saque do Voleibol. Rev Paul Educ Física 1999;13(2):197-216.
• ¹⁴
  Tertuliano IW, Souza Júnior OP, Silva Filho AS, Corrêa UC. Estrutura de práticae freqüência de “feedback ” extrínseco na aprendizagem de habilidades motoras. Rev Bras Educ Fís Esporte 2008;22(2):103-18.
• ¹⁵
  Weir JP. Quantifying test-retest reliability using the intraclass correlation coeficientand the SEM. J Strength Cond Res 2005;19(1):231-40.
• ¹⁶
  Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychol Bull 1955;52(4):281-302.
• ¹⁷
  Maroco J, Garcia-Marques T. Qual a fiabilidade do alfa de Cronbach? Questões antigas e soluções modernas? Lab Psicol 2006;4(1):65-90.
• ¹⁸
  Maroco J, Campos J, Bonafé F, Vinagre M da G, Pais-Ribeiro J. Adaptaçãotranscultural Brasil-Portugal da escala Brief-Cope para estudantes do ensinosuperior. Psicol Saúde Doenças 2014;15(2):300-13.
• ¹⁹
  Sim J, Wright C. Research in Health Care: Concepts, Designs and Methods. Cheltenham: Nelson Thornes Ltd; 2000.
• ²⁰
  Maroco J. Análise de equações estruturais: Fundamentos teóricos, software & aplicações. 2nd ed. Pêro Pinheiro: Report Number, Ltda; 2014.
• ²¹
  Campana ANNB, Tavares MCDCG, Silva D Da. Modelagem de Equações Estruturais: Apresentação de uma abordagem estatística multivariada para pesquisas em Educação Física. Motrici 2009;5(4):59-80.
• ²²
  Gonçalves WR, Ugrinowitsch H, Fonseca FDS, Benda RN. Efeitos do conhecimento de performance visual em uma frequência autocontrolada na aprendizagem de uma habilidade esportiva. Rev da Educ Física/UEM 2011;22(2):229-38.
• ²³
  Nhamussua D, Prista A, Basso L, Tani G. Interferência contextual e nível de habilidade na aprendizagem do serviço do voleibol. Rev Bras Educ Física Esporte 2012;26(4):731-40.
• ²⁴
  Cortina JM. What is coefficient alpha? An examination of theory and applications.J Appl Psychol 1993;78(1):98-104.

Publication Dates

• Publication in this collection
  Jan-Feb 2018

History

• Received
  24 May 2017
• Accepted
  30 Nov 2017