Content Validity of scenes of the Declarative Tactical Knowledge Test in Volleyball – DTKT:Vb

Validação de conteúdo das cenas do Teste de Conhecimento Tático Declarativo no Voleibol – TCTD:Vb

Abstract

Declarative Tactical Knowledge Tests are presented as important evaluation tools for the teaching-learning-training process regulation. This study aimed to establish the content validity of scenes of the Declarative Tactical Knowledge Test in Volleyball – DTKT:Vb. Five male coaches of the Brazilian Volleyball team who worked with male athletes participated as judges, being responsible for training formation categories up to 21 years, experts in the sport, with minimum ten years of experience. The judges evaluated 212 scenes containing extremity attack (n=55), central attack (n=33), setting (n=68) and block (n=60) situations and used a 1-5 point likert scale to assign a score to the scene according to requisites image clarity, practical relevance and item representativity. The Content Validity Coefficient (CVC) was used to determine the CVC for each scene and the instrument as a whole, with cutoff point of 0.80. The results demonstrated that “image clarity” (CVC=0.92), “practical relevance” (CVC=0.96) and “item representativity” criteria (CVC=0.96) showed satisfactory levels. After calculating CVC, the ecological validity of scenes was determined, which consists of the selection of scenes where there was convergence among decision made by judges and decision made by athletes. Thus, from 212 scenes initially prepared, 66 have been validated. Scenes validated using CVC enabled the evaluation of the Declarative Tactical Knowledge, assisting in the planning of teaching-learning-training processes of male volleyball athletes.

Key words
Cognition; Decision-making; Volleyball

Resumo

Os testes de conhecimento tático declarativo apresentam-se como procedimentos avaliativos importantes para a regulação do processo de ensino-aprendizagem-treinamento. Objetivou-se estabelecer a validade de conteúdo das cenas do teste de conhecimento tático declarativo no voleibol – TCTD:Vb. Participaram como juízes cinco treinadores masculinos da seleção brasileira de voleibol que trabalharam com atletas do sexo masculino, sendo responsáveis pelas categorias de formação até 21 anos, peritos na modalidade, com experiência mínima de dez anos na mesma. Os juízes avaliaram 212 cenas contendo situações de ataque de extremidade (n=55), ataque de central (n=33), levantamento (n=68) e bloqueio (n=60) e utilizaram uma escala tipo likert de 1 a 5 pontos para atribuir uma nota para cada cena nos quesitos clareza de imagem, pertinência prática e representatividade do item. Recorreu-se ao coeficiente de validade de conteúdo (CVC) para determinar o CVC para cada cena e do instrumento como um todo, sendo o ponto de corte 0,80. Os resultados demonstraram que os critérios “clareza de imagem” (CVC=0,92), “pertinência prática” (CVC=0,96) e “representatividade do item” (CVC=0,96) apresentaram níveis satisfatórios. Após o cálculo do CVC, determinou-se a validade ecológica das cenas, que consiste na seleção das cenas em que houve convergência entre a decisão apontada pelos juízes e decisão tomada pelo atleta. Desta forma, a partir das 212 cenas inicialmente elaboradas, 66 cenas foram validadas. As cenas validadas por meio do CVC possibilitam a avaliação do conhecimento tático declarativo auxiliando no planejamento dos processos de ensino-aprendizagem-treinamento de atletas de voleibol masculino.

Palavras-chave
Cognição; Tomada de decisão; Voleibol

INTRODUCTION

As a collective game (CG), volleyball demands cognitive processes of perception and decision-making (DM) from the player for the solution of problems through tactical-technical actions¹. In this sense, the knowledge of the game logics is crucial for directing cognitive processes such as attention, perception and DM^2,3.

In sports sciences, there is distinction between procedural tactical knowledge (PTK) and declarative tactical knowledge (DTK)^4,5. In volleyball, an eminently tactical-strategic sport, the question of player’s knowledge must be relevant when one intends to examine parameters of player and team’s performance⁶.

In this way, PTK assumes an important tactical dimension in volleyball, which consists of the athlete knowing “how to do”, that is, performing the motor gesture related to the player’s competence to perform the sports action, even if he is not able to express it or verbally describe it^4,5.

In turn, DTK is represented by the information present in the long-term memory, being accessed via working memory provided that it assists in the execution of what should be done; therefore, DTK is strongly related to the practitioner’s experience⁴.

DTK reflects the ability to express tactical decisions in a verbal or written way, why they were made and why they were integrated⁵. It refers to the player’s ability to identify and recognize “what to do” in the game context^4,5.

Gréhaigne, Godbout, and Bouthier⁷ suggest the use of tests that measure DTK. These DTK tests use game scenes and scenarios (i.e.: actual game recordings edited according to game situations or animations made in specific software to analyze the relevant signs of action) for the understanding and analysis of the tactical behavior in different sports. In this context, these tests are important evaluative procedures in the pedagogical process, which allow the collection of data of a certain variable, helping the teacher in choosing the most appropriate teaching models to address these contents and plan the necessary activities⁸.

In volleyball, instruments have been proposed for DTK analyses, among them the “DTK test – Setter“⁶ for setting situations, the “Questionnaire for DTK evaluation in volleyball”⁹ and the “Questionnaire for DTK evaluation in school volleyball”¹⁰ for general DTK evaluation, among others. However, none of them have been developed using real high-level game scenarios contemplating four tactical situations representing the DM of volleyball players.

Thus, the aim of the present study was to establish the content validity of volleyball game scenes to configure a declarative tactical knowledge test in the modality (DTKT:Vb) using the content validity coefficient (CVC) procedure¹¹. The challenge of the real evaluation of the construct emerges for use not only in the diagnostic area, but also as a pedagogical tool for directing the teaching-learning-training process (T-L-T) of the athlete’s tactical capacity.

METHODOLOGICAL PROCEDURES

Participants

Five male coaches (n = 05) of the Brazilian volleyball team (mean age 41.3 ± 8.5 years) who worked with male athletes voluntarily participated in the study, responsible for training formation categories up to 21 years, experts in the modality. As recommended by Balbinotti, Benetti, and Terra¹², the selection criteria adopted were (1) minimum of 10 years experience in sports initiation, specialization and high performance level, (2) currently playing the role of volleyball coach and 3) not having participated in any research stage.

This study adopted the definition that the expert is a very skillful individual, recognized in a specific field and with expertise level reached through the deliberate practice of at least ten years.

Ethical procedures

The present study was approved by the Ethics Research Committee of the Federal University of Minas Gerais (protocol No. 821.295) and all participants signed the Free and Informed Consent Form and were aware that they could quit participation at any time in the study without justifying their decision.

Validation procedures

a) Selection of scenes

In this study, all scenes were taken from official 2011-2012 Super league Volleyball matches and filmed from the top perspective, with approximate distance of seven to nine meters from the court, allowing the observer the full view of the playing court and depth perception in different situations, making the situations as close as possible to the environmental reality. Judges analyzed all game situations in real time and space sequence, and were asked to choose the best decision to be made by the athlete in defining the action in search for the score.

Initially, 212 scenes containing extremity attack - EA (n=55), central attack - CA (n=33), setting - SE (n=68) and block – BL (n=60) situations were selected to be analyzed by experts. From the 212 scenes, 82 obtained inter-rater agreement from the five experts, with Cohen’s Kappa values of 1.00 for image clarity, 0.83 for practical relevance, 0.81 for item representativity and 1.00 for DM. After 15 days in the first observations, intra-rater agreement was carried out among experts, and the Cohen Kappa Coefficient obtained in this procedure presented values of 1.00 for image clarity, 1.00 for practical relevance, 1.00 for item representativity and 0.96 for DM. The Kappa coefficient values showed perfect agreement, since they presented values between 0.81 and 1.00¹¹.

b) Ecological validity

After agreement obtained by experts, the criterion of ecological validity of scenes was adopted, where the choices of experts should be in accordance with the action performed by the athlete¹³. Thus, of the 82 validated scenes, 66 of them presented ecological validity, representing the choice of experts in agreement with the action performed by athletes in scenes that culminated in score.

After these analyses, as a criterion used by the authors, for the CVC evaluation, only the 66 scenes (9 of EA, 7 of CA, 25 of SE and 25 of BL) that met the inter- and intra-rater and ecological validity criteria were used to calculate CVC.

c) Content Validity

Within the theoretical procedures proposed by Pasquali¹³ for the elaboration of psychometric instruments, the semantic analysis of items and the analysis of judges aimed to verify the construct representativeness through items¹³. In this sense, the specialized literature recommends the use of CVC¹¹ in order to investigate the level of construct representativeness using the validation of concepts language clarity, practical relevance and theoretical relevance of items or criteria that make up the instrument to be validated, i.e., it is verified whether the instrument, in terms of content, accurately measures what it is proposed to measure¹⁴.

In the “image clarity”, the sharpness of the image of scenes was evaluated, taking into account the population of athletes and volleyball coaches who would later analyze them. Experts were asked: Does the image have good conditions for the tactical analysis of the situation? Are images clear? Is it possible to clearly see the ball, players, net, and side lines of the court in the scenes?

In the “practical pertinence” the scene relevance as a form of representation of situations for the DM in volleyball was evaluated. The following question was asked: Do you believe that these scenes represent appropriate situations to analyze the DM of a player in a volleyball game?

In the “item representativity” the level of demonstration of cognitive processes that one had interest in evaluating through the scenes was evaluated. Experts were asked: Is this image representative of the real game? Does it allow the DM analysis and the relevant signals that lead to it?

The literature evidences the use of CVC for the validation of the categorization protocol of teaching methodologies in sports¹⁵ and tennis test¹⁶. CVC was used in the process of development and validation of instruments for DTK evaluation in tennis¹⁶ and PTK in CSG according to Sports Guidance (SG)¹¹ and in Basketball¹⁷.

The mean Kappa was used for agreement among judges regarding image clarity, practical relevance and item representativity. This procedure was performed before the CVC calculation as recommended in literature¹¹. To verify the validity of scenes, the concordance among observers (CAO) was used¹⁶. Among observers, all judges pointed out the same response regarding the DM choice (100% inter-rater concordance). After 15 days of observations, judges re-evaluated the scenes and obtained 90% agreement in responses related to the DM choice (intra-rater concordance). The judges’ assessments were measured by a 5-point Likert scale (1 = Very low, 2 = Low, 3 = Medium, 4 = High, 5 = Very high). The agreement values should be above 80%, as could be observed. For all scenes and for all judges, making free commentaries was allowed.

Statistical analysis

The answers from experts were entered and analyzed in the Microsoft Office Excel 2010 software using the CVC formula. Data analysis was computed based on specific formulas for CVC calculation¹¹ and is presented as follows:

1. The average scores of each item (Mx) was first calculated based on scores assigned by experts. In the calculation, Σxi represents the sum of experts’ scores and J represents the number of experts who evaluated the item:

   $$M_{\mathit{x}}\mathit{}\mathit{=}\mathit{}\frac{{\displaystyle \underset{\mathit{i}\mathit{=}\mathit{1}}{\overset{\mathit{J}}{\mathit{\sum }\mathit{}{\mathit{x}}_{\mathit{i}}}}}}{\mathit{J}}$$

2. Using the average as basis, CVC was calculated for each item (CVCi), where Vmax represents the maximum value that the item could receive:

   $$CV{C}_i\mathit{}\mathit{=}\mathit{}\frac{M_x}{V_{máx}}$$

3. In order to discount possible biases of evaluators, the error (Pe_i) was calculated for each item:

   $$P{e}_i={\left(\frac{1}{J}\right)}^j$$

4. The final CVC of each item (CVCc) was also calculated:

   $$CV{C}_c=CV{C}_i-P{e}_i$$

5. In the calculation of the total CVC (CVC_i), for each of the characteristics (image clarity, practical relevance and item representativity), the following formula was used:

   $$CV{C}_t=Mcv{c}_i-Mp{e}_i$$

In the formula, the mean CVCs values of the test items are represented by Mcvc_i and the mean error of the test items by Mpe_i. The total CVC (CVC_t) refers to the value calculated for the instrument in general, that is, the average value calculated for all items referring to image clarity, practical relevance and item representativity.

The cutoff agreement adopted to represent satisfactory CVC levels was ≥0.80 for each criterion^11-13. This coefficient makes it possible to evaluate the degree to which the items represent the constructs through intentional actions¹¹.

RESULTS

Table 1 presents the CVC_t results for each criterion evaluated. The results show that all criteria, in their total values, obtained values equal to or greater than 0.92, demonstrating high content validity.

Table 1
CVCt results for each criterion evaluated.

Image clarity		Practical relevance		Item representativity
0.92		0.96		0.96

• * Values referring to the Total Content Validity Coefficient

“Image clarity”, which determines the image sharpness in relation to the objects present in the game situation, presented the lowest score in CVC_t (0.92), followed by “practical relevance”, which evaluates if scenes represent suitable situations for DM in a volleyball game (0.96) and by “item representativity”, that is, if scenes allow the DM and perception analysis (0.96).

DISCUSSION

The present study aimed to establish the content validity of scenes of the declarative tactical knowledge test in volleyball (DTKT:Vb) using the CVC procedure¹¹.

According to the evaluation of experts, CVC calculated for image clarity (CVC_t = 0.92), practical relevance (CVC_t = 0.96) and item representativity (CVC_t = 0.96) is satisfactory in relation to the proposal of DTK evaluation based on perception and DM in volleyball, since it is higher than the reference value¹¹.

Therefore, after the CVC calculation, the content validity was confirmed in a satisfactory number of scenes for the DTK evaluation in volleyball. The adequacy of procedures used in the present study to validate real game images was also explained. It is observed in specialized literature that the development of instruments for DTK evaluation in volleyball via images extracted from official games lacks objective procedures to establish content validity, which justifies the use of CVC as an important and indispensable procedure to adequately fulfill with all steps proposed in psychometry in the construction of psychological instruments referenced in the construct.

For the validation of a DTK test in tennis using game scenes, Aburachid and Greco¹⁹ reported CVC values of 0.89 for image clarity, 0.91 for practical relevance and 0.98 for item representativity, which corroborates CVC values reported in the present study and demonstrates evidence of content validity of scenes proposed to compose the DTKT:Vb.

A study conducted by Greco et al.¹¹ with the aim of investigating the evidence of content validity of tactical-technical actions of the Procedural Tactical Knowledge Test for Sports Guidance (PTKT:SG) used the scores assigned by a panel of judges considered experts in the basketball, handball and indoor soccer area in order to verify agreement in terms of language clarity, practical relevance and theoretical relevance through CVC calculation. Regarding basketball, the CVC reported values of 0.88 for language clarity, 0.92 for practical relevance and 0.98 for theoretical relevance. In the specific case of handball, the CVC values were 0.80 for language clarity, 0.94 for practical relevance and 0.95 for theoretical relevance. Finally, the CVC values calculated for indoor soccer were 0.82 for language clarity, 0.87 for practical relevance and 0.94 for theoretical relevance.

Preliminary studies on soccer and handball to obtain evidence of the content validity of PTKT:SG through CVC calculation reported values of 0.81 for language clarity and 0.80 for practical relevance in soccer, and values of 0.80 for language clarity and 0.94 for practical relevance in handball¹¹.

The PTKT:SG proposed in studies of Greco et al.²⁰ and Castro et al.²¹ is an instrument developed for PTK measurement in beginners in invasion collective games (for example: basketball, handball, soccer and indoor soccer). In this sense, the use of CVC favored the process to obtain evidence of the content validity of PTKT:SG in an objective and reliable way. That is, agreement was established among judges considered experts in the different invasion sports modalities during the procedure of evaluation of technical-tactical actions proposed by the test in the field of language clarity, practical relevance and theoretical relevance.

CVC has also been used in the preliminary validation procedure for the content of the Procedural Tactical Knowledge Test in Basketball (PTKT:Bb) proposed by Morales, Greco, and Andrade¹⁷. The CVC values reported (≥0.80) also favored obtaining satisfactory psychometric properties and the continuity of the process of construct and reliability validity.

The present study corroborates works that used CVC to validate scenes for tactical knowledge tests in various sports modalities such as tennis¹⁹, basketball^11,17, handball¹¹, indoor soccer¹¹ and soccer¹¹, presenting satisfactory values for CVC criteria image clarity, practical relevance and item representativity.

Valentini et al.²² calculated the CVC of a gross motor development test and obtained values of 0.94 for language clarity and 0.91 for practical relevance, which is in agreement with the present study, which presented satisfactory and very close values for the same items evaluated.

In order to make up at least a large part of the construct’s semantic extension, psychometry assumes that the validation process must be started with at least three items for later disposal, when they are not constituted based on a theory. When constituted based on a theory, it is not necessary to start with more than 10% in addition to the 20 items to be saved. In this sense, this study presented rigor since the choice and consolidation of the video scenes, which began with 212 images that were maintained for content validity processing, and finally, 66 scenes became items characterized as valid with regard to content, reaching values higher than those recommended in literature¹¹.

According to Costa et al.²³, minimum number of 6 and maximum of 13 items are sufficient to evaluate the construct regarding tactical knowledge. The results demonstrated that game scenes can be validated to build declarative knowledge tests. Studies based on psychometrics are mostly directed to the validation of questionnaires, personality tests and educational tests¹⁹.

Thus, the use of the theoretical framework suggested by Pasquali¹³ is appropriate for this type of test that uses motion images, which already has support in literature considering that several studies in the area of sports initiation ¹¹ and sports such as tennis¹⁶, soccer^11,15, handball¹¹ and basketball¹⁷ have used CVC to establish the content validity of instruments proposed for evaluation of both DTK and PTK.

CONCLUSION

The application of the previously validated DTKT:Vb scenes regarding content have enabled evaluating DTK in volleyball players and coaches of different experience levels, as well as in practitioners and non-practitioners of this sports modality. The purpose of applying this type of test is mainly related to pedagogical goals, that is, it is used as an available alternative for the regulation of the teaching-learning-training process over time.

Thus, it is an important pedagogical tool not only for the diagnosis of the DTK performance level, but also for directing and regulating the planning of teaching-learning-training processes of the tactical ability directed to the modality in its different expression levels of sports performance (school and high-performance sports, health, high-performance level, recreation and free time, among others).

Thus, the knowledge that the evaluator holds about the aspects he observes and perceives in different game situations is evaluated, and through the verbalization related to the relevant signals that one is able to understand, and from them, to elaborate information and produce answers. In this way, improvement and adequacy of the phases of tactical training (initial, positional and situational) of the sports modality, and adequacy, correction and development of the teaching-learning-training process are opportunely implemented.

The present study was limited to the content validity of official high-performance volleyball game scenes in the adult male category, which in a way restricts the use of these scenes with female volleyball players. Thus, further studies should develop and validate specific tests for female volleyball athletes with similar situational constraints, except for the specificities of each type and game characteristics.

In addition, the stratification of the level of difficulty of each scene presented by the test becomes interesting, allowing creating tests according to the level of athletes analyzed.

REFERENCES

• ¹
  Sáez-Gallego NM, Villa-Maldonado S, Hernández JA, Jordán ORC. Análisis del comportamiento visual y la toma de decisiones en el bloqueo en Voleibol. Cuad Psicol Dep 2013;13(2):31-44.
• ²
  Gil A, Moreno P, García-González L, Moreno A, Villar FD. Analysis of declarative and procedural knowledge in volleyball according to the level of practice and players’ age. Percep Motor Skill 2012;115(2):632-44.
• ³
  Gil A, Moreno MP, Moreno A, García-González L, Claver F, Del Villar F. Analysis of the relationship between the amount of training and cognitive expertise. A study of young volleyball players. J Streng Condition Res 2013;27(3):698-702.
• ⁴
  McPherson SL, Thomas J. Relation of knowledge and performance in boys’ tennis: age and expertise. J Experim Child Psychol 1989;48(2):190-211.
• ⁵
  McPherson SL, French KE. Changes in cognitive strategies and motor skill in tennis. J Sport Exerc Psychol 1991;13(1):26-41.
• ⁶
  Matias CJAS, Greco PJ. O conhecimento tático declarativo dos levantadores campeões de Voleibol. Motriz 2013;19(1):185-94.
• ⁷
  Gréhaigne JF, Godbout P. Tactical knowledge in team sports from a constructivist and cognitivist perspective. Quest 1995;47(4):490-505.
• ⁸
  Greco PJ, Memmert D, Morales JCP. The effect of deliberate play on tactical performance in basketball. Percep Motor Skills 2010;110(3):1-8.
• ⁹
  Moreno A, Moreno MP, García-Gonzalez L, Gil A, Del Villar F. Desarrollo y validación de um cuestinário para la evaluación del conocimiento declarativo en voleibol. Motri Eur J Hum Movem 2010; 25(2):189-95.
• ¹⁰
  Pritchard T, Hawkins A, Wiegand R, Metzler JN. Effects of two instructional approaches on skill development, knowledge, and game performance. Meas. Physical Educ Exerc Science 2008;12(4):219-236.
• ¹¹
  Greco PJ, Aburachid LMC, Silva SR, Morales JCP. Validação de conteúdo de ações tático-técnicas do Teste de Conhecimento Tático Processual – Orientação Esportiva. Motri 2014;10(1):38-48.
• ¹²
  Balbinotti MA, Benetti C, Terra PR. Translation and validation of the Graham-Harvey survey for the Brazilian context. Int J Manag Finance 2006;3(1):26-48.
• ¹³
  Pasquali L. Validade dos Testes Psicológicos: Será Possível Reencontrar o Caminho? Psicol Teoria Pesq 2007;23(Especial):99-107.
• ¹⁴
  Roberts P, Priest H, Traynor M. Reliability and validity in research. Nurs Stand 2006;20(44):41-5.
• ¹⁵
  Soares V, Santos H, Lima M, Aburachid L, Greco P. Validação de protocolo de categorização de metodologias de ensino nos esportes coletivos com base na Iniciação Esportiva Universal - uma escola da bola. Lecturas Educ Fís Dep 2010;15(144).
• ¹⁶
  Aburachid LMC, Greco PJ. Validação de conteúdo de cenas do teste de conhecimento tático no tênis. Estud Psicol 2011;28(2):261-7.
• ¹⁷
  Morales JCP, Greco PJ, Andrade RL. Validade de Conteúdo do Instrumento para Avaliação do Conhecimento Tático Processual no Basquetebol. Cuad Psicol Dep 2012;12(supl.1):31-6.
• ¹⁸
  Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159-74.
• ¹⁹
  Aburachid LMC, Greco PJ. Processos de validação de um teste de conhecimento tático declarativo no tênis. Rev Educ Física/UEM 2010;21(4):603-610.
• ²⁰
  Greco PJ, Morales JCP, Aburachid LMC, Silva SR. Evidência de validade do teste de conhecimento tático processual para orientação esportiva – TCTP:OE. Rev Bras Educ Fís Esp 2015;29(2):313-24.
• ²¹
  Castro HO, Morales JCP, Aburachid LMC, Greco PJ. Teste de Conhecimento Tático Processual 3x3 com os pés: alternativa para a orientação esportiva. Rev Bras Educ Fís Esp 2015;29(4):621-9.
• ²²
  Valentini NC, Barbosa ML, Cini GV, Pick RK, Spessato BC, Balbinotti MA. Teste de desenvolvimento motor grosso: validade e consistência interna para uma população gaúcha. Rev Bras Cineantropom Desempenho Hum 2008;10(4):399-404.
• ²³
  Costa JC, Garganta J, Fonseca A, Botelho M. Inteligência e conhecimento específico em jovens futebolistas de diferentes níveis competitivos. Rev Port Ciên Desp 2002;2(4):7-20.

Publication Dates

• Publication in this collection
  Nov-Dec 2016

History

• Received
  07 May 2015
• Accepted
  10 Jan 2017