Design and validation of an observational instrument for technical and tactical actions in beach volleyball

Desenho e validação de um instrumento de observação para ações tático-técnicas em voleibol de praia

Diseño y validación de un instrumento de observación para las acciones técnico-tácticas en voley-playa

Abstracts

Technical and tactical actions determine performance in beach volleyball. This research develops and tests an instrument to monitor and evaluate the manner of execution and efficacy of the actions in beach volleyball. The purpose of this paper was to design and validate an observational instrument to analyze technical and tactical actions in beach volleyball. The instrument collects information regarding: a) information about the match (context), b) information about game situations, c) information about technical situations (serve, reception, set, attack, block, and court defense) in relation to player execution, role, manner of execution, execution zone, and efficacy, and d) information about the result of the play (win-lose and way point is obtained). Instrument design and validation was done in seven stages: a) review of literature and consultation of experts; b) pilot observation and data analysis; c) expert review of instrument (qualitative and quantitative evaluation); d) observer training test; e) expert review of instrument (content validity); f) measurement of the ability of the instrument to discriminate the result of the set; and g) measurement of the ability of the instrument to differentiate between competition age groups. The results show that the instrument allows for obtaining objective and valid information about the players and team from offensive and defensive technical and tactical actions, as well as indirectly from physical actions. The instrument can be used, in its entirety or partially, for researching and coaching purposes.

performance; evaluation; team sport; match analysis

As ações tático-técnicas determinam o desempenho em voleibol de praia. Esta investigação desenvolveu e testou um instrumento para monitorar e avaliar a maneira de execução e eficácia das ações do jogo em voleibol de praia. O objetivo do presente estudo foi desenhar e validar um instrumento de observação para analisar as ações tático-técnicas em voleibol de praia. O instrumento coleta informações focando: a) informação sobre o jogo (contexto), b) informação sobre as situações do jogo, c) informação sobre as situações técnicas (serviço, recepção, distribuição, ataque, bloco, e defesa) em relação a execução do jogador, especialização, maneira de execução, zona de execução e eficácia, e d) informação sobre o resultado do jogo (ganhar-perder, e forma de obtenção dos pontos). O desenho e validação do instrumento foi realizado em sete fases: a) revisão da literatura e consulta dos experts; b) observação piloto e análise dos dados; c) revisão do instrumento pelo expert (avaliação qualitativa e quantitativa); d) treino-teste do observador; e) revisão do instrumento pelo expert (validade do conteúdo); f) medir a capacidade do instrumento para discriminar o resultado do set; e g) medir a capacidade do instrumento em relação a diferenças entre os grupos de idade na competição. Os resultados mostram que o instrumento permite a obtenção de informações objetivas e válidas sobre as ações tático-técnicas ofensivas e defensivas dos jogadores e das equipes, bem como (indiretamente) as ações físicas. O instrumento pode ser utilizado, totalmente ou parcialmente, para a investigação e treino.

desempenho; avaliação; desporto coletivo; análise do jogo

Las acciones técnico-tácticas determinan el rendimiento en voley-playa. Este trabajo desarrolla y prueba un instrumento para monitorizar y evaluar la forma de ejecución y eficacia de las acciones en voley-playa. El propósito de este trabajo fue diseñar y validad un instrumento observacional para analizar las acciones técnico-tácticas en voley-playa. Este instrumento registra información relativa: a) información sobre el partido (contexto), b) información sobre la situación de juego, c) información sobre la situación en la que se realiza las acciones técnicas (saque, recepción, colocación, ataque, bloqueo, y defensa en campo) en relación al jugador que ejecuta, su función de juego, la forma de ejecución, la zona de ejecución, y la eficacia, y d) información sobre el resultado de la jugada (ganar-perder y forma como su obtuvo el punto). El diseño y validación del instrumento se realizó en siete etapas: a) revisión de la literatura y papel de expertos; b) Estudio piloto de observación y análisis de datos; c) revisión del instrumento por expertos (evaluación cualitativa y cuantitativa); d) Estudio piloto de entrenamiento de observadores; revisión del instrumento por expertos (validez de contenido); f) valoración de la capacidad de discriminación de diferencias entre ganadores y perdedores; y g) valoración de la capacidad de discriminación de diferencias entre grupos de edad. Los resultados muestran que el instrumento permite obtener información validad y objetiva sobre los jugadores y los equipos en sus acciones técnico-tácticas, como indirectamente las acciones físicas. Este instrumento puede ser usado en su totalidad o parcialmente para investigación y entrenamiento.

rendimiento; evaluación; deporte de equipo; análisis de juego

Introduction

The goal of performance analysis is to improve the knowledge about the parameters that affect performance. In team sport, this information allows us to become more familiar with the dynamic system that is involved in team sport confrontations. Further, it provides reference values and criteria to the coaches that can be used to guide the training process and prepare teams for their sport's demands and their opponents. The use of technology to analyze sport has increased in the last few decades (Carling, Reilley, & Williams, 2009; Liebermann et al., 2010). Despite the use of technology, observation is still an important instrument for obtaining information. Normally, observation is carried out in the usual competition context, which requires non-standard instruments that are created specifically for this purpose. The creation of these observational instruments has to follow a strict protocol to ensure that the information obtained by them is reliable, objective, accurate, and valid (Elosua, 2003). However, this does not happen in all cases for some of the sports that use observation as the data collection instrument (Hughes, Cooper, & Nevill, 2002).

Beach volleyball is a fairly new net sport that follows this tendency. In a bibliography review of research studies, beach volleyball papers about match analysis do not include information about the way observation instruments were developed or validated. Only the information about observer training is included. The lack of an objective instrument can affect the data that are obtained regarding performance analysis in beach volleyball. The creation of a valid and reliable instrument is a necessary first step to obtain objective data from the sport. Therefore, the development of an observational instrument for researchers and coaches is necessary to properly study this sport (e.g. specific analysis of the players' technical behaviors in their natural context). This process involves establishing the way to code and record the data, the protocol for using the instrument, etc., in addition to the variables to be measured (Anguera, 2003; MacCall, 1984; Sykes, 1977).

The present paper develops a category system of observation to collect the actions carried out by players in competition or in training (McCall, 1984). The observational instrument collects information about players' and teams' actions in their natural context without interfering in their behavior. To prevent issues in instrument and observers' reliability, the instrument collects data only from the players' actions that can be observed (Anguera, 2003; Blanco, 1993; Kimberlin & Winterstein, 2008; McCall, 1984). The instrument allows us to obtain information about the technique and tactics used by players to resolve the game situations. It allows us to establish relationships between the manner of execution and temporal and spatial aspects of the execution with the effect of the actions on the game. These data allow us to set the player profile regarding technical-tactical actions and physical actions (Hughes & Bartlett, 2002), teams' game patterns, game dynamics, how tasks or environmental aspects affect game actions (Newell, 1986), etc. This applied instrument cannot describe the complexity of human motor performance (Glazier, 2010). However, it can provide information to researchers and academics to help them better understand team sports as well as to coaches and players to assist them in planning, monitoring, analyzing, and guiding the aspects that affect the game. Therefore, the purpose of this paper was to design and validate an observational instrument to analyze technical and tactical actions in beach volleyball.

Methods

The instrument design and validation was done in seven stages. The first and second stages had the objective of designing the observational instrument, a category system (Anguera, 2003). In the third to seventh stages, validity and reliability were established and calculated (Anguera, 2003; Trochim & Donnelly, 2007). The study was reviewed and exempted by the Ethical Committee of Catholic University of Saint Anthony (2007/05).

In the first stage, a draft of a list of player behaviors was created from the scientific literature and from expert consultation. A review of the databases of Web of Knowledge (WOK) of ISI (Thomson), Sport Discus, Google Scholar, Sponet, Scielo, and Dialnet was done. The key word was "beach volleyball". A review of the abstracts was done to select the papers related to match analysis. Twenty-seven articles about match analysis in beach volleyball were reviewed. From the variables studied in the research papers, as well as the analysis of the sport's characteristics and volleyball statistical systems, an initial list of variables was established. Indoor volleyball statistical systems were reviewed as the two sports have similar rules and characteristics. The list of variables included the definition of a behavior and its possible categories. The unit of analysis was the rally/complex phase. The data from the rally actions were collected concurrently, and the time when the rally started and ended was also registered. The variables were divided into four groups: a) information about the match, b) information about game situations, c) information about technical situations, and d) information about the result of the play. The group of variables related to the match and game situations provides information about the environmental constraints of the actions. The group of variables related to technical actions provides information regarding manner of execution (technique, temporal and spatial situations, and performance). The group of variables related to results of the game provides information regarding the outcome and the action's efficacy. The group of variables related to the manner of execution also provides information about the jumps and hits carried out by the players and the work and rest time of the rallies. A report about data from the observational instrument was also presented and reviewed by the experts. The first draft of the categorical system and its application was analyzed and reviewed by three experts and the researchers. The experts had the following characteristics: a) one was the Spanish men's national team coach, with more than 10 years of coaching experience in beach volleyball at the international level; and b) two coaches were from the national competition level with a minimum coaching experience of 10 years between volleyball and beach volleyball, were university professors, and had their doctorates in the area of physical activity and sport. The information found in the review was presented to the experts, and the process to develop the first draft was described. A consensual approach was utilized. All the experts had to agree in order to establish the variables.

In the second stage, a pilot observation test was done with the draft of the list of behaviors. Six sets of three men's matches from the 2007 World Tour were observed by one of the researchers from behind the court to ensure full view of all players' behaviors during the game (6-8 meters behind the court at an approximate height of three meters). The goal was to adapt and/or establish the criteria of the different variables in order to add them to the list of variables and category definitions. A draft of an observation manual for the instrument was developed at this stage. A descriptive analysis of the values obtained in the observation was likewise done. If the frequency of occurrence of the categories of variables was lower than 10%, the category of the variable was reviewed by the researchers and the expert group from the first stage in order to reorganize the categories. This occurred for the attack. The criterion of 10% was established by the expert group in the first stage. The coaches' perspectives were taken into account to set this criterion with the goal of ensuring pragmatic application of the data obtained from the instrument. An analysis of the data (frequency, percentages, Chi square test, and likelihood ratios) and a data report were produced by the researchers to ensure the applicability of the data (individual and collective reports for coaches). The analysis and the report were reviewed by the researchers and coaches. In this stage, the possibilities for automating or indirectly calculating part of the observation were also reviewed to reduce the amount of data to observe and record. After the pilot study, a second list of behaviors was made, including the modifications of and variations on the behaviors.

In the third stage, the operationalization, relevant content, and description of the instrument were reviewed by eight experts (coaches or former coaches with a minimum of 10 years of coaching experience in indoor or beach volleyball; four were university professors with doctorates) through a qualitative and quantitative evaluation of the instrument. Experts were asked about (Table 1): a) the degree of comprehension of the definitions of the behaviors from the observational instrument, b) the level of pertinence of behaviors, and c) the need to include other behaviors in the observational instrument. The level of pertinence was defined as whether the inclusion of the variable in the aspects to be observed was conceptually adequate. The degree of comprehension was defined as whether the variable and category were properly defined (clear criteria for carrying out the observation). The quantitative evaluation of comprehension and pertinence consisted of a scale from 0 to 10. Following a proposal by Bulger and Housner (2007), items with average values ≤ 7.0 were eliminated, items > 7.0 and < 8.0 were reviewed, and items that were ≥ 8.0 were accepted. After this feedback, a new list of behaviors was established.

Table 1.
Sample questionnaire sent to the experts.

Execution of the reception (a) Definition: Technique used to neutralize the serve done by the opponent. It is a categorical variable. Three categories were differentiated: 1) Bump, when the reception is executed using both of the forearms; b) overhead, when the ball is hit with the hands or forearms together and/or one over another; and c) other techniques, the rest of the techniques not included in the previous categories. Poorly defined 0 – 1 – 2 – 3 – 4 – 5 – 6 – 7 – 8 – 9 – 10 Very well defined Proposed definition, in case the previous one was not clear: (b) Pertinence: Does it seem pertinent to include the “execution of the reception” as a variable to be observed? Not Pertinent 0 – 1 – 2 – 3 – 4 – 5 – 6 – 7 – 8 – 9 – 10 Very pertinent (c) Inclusion: What other category would you add to the observation instrument for the initiation of the ball possession?

In the fourth stage, a second pilot study of observer training and an observation test were done. A group of four observers were trained in the use of the observational instrument during three sessions of two hours each (a 10-minute rest period was given after 55 minutes). The observers held a degree in sport science, had the highest volleyball coaching certification in Spain (level III), were former volleyball or beach volleyball players, and had experience as observers (e.g. research and sport scouting). The observation training was done following the criteria established by Anguera (2003) and Behar (1993). The training was directed by one of the researchers. An observation manual (on paper and video) was used in the training. After the sessions, four sets of two matches with posterior views of the court from the 2007 World Tour were used for training. The aspects for which there was no unanimous agreement were reviewed with the observers in an extra session. All comprehension problems regarding the variables, categories, and modifications of the category criteria were registered. After this stage, a new list of behaviors was established. Another match (two sets) with a posterior view of the court from the 2007 World Tour was observed to establish inter- and intra-agreement between the observers. Cohen's kappa was used to evaluate observer agreement. A researcher was used as the reference to establish the reliability. The first observation was done followed by the second observation a week later.

In the fifth stage, the review of the operationalization, relevant content, and description were repeated. Seven experts (coaches of higher Spanish divisions with a minimum of 10 years of coaching experience in volleyball or beach volleyball; four were university professors with their doctorates) did a qualitative and quantitative evaluation of the instrument. After this review, the list of behaviors was established. From the quantitative evaluation given by this group of experts, a descriptive analysis was done (mean, median, and mode for all variables). To calculate the content validity, Aiken's V was used (Merino & Livia, 2009; Penfield & Giacobbi, 2004).

In the sixth stage, the ability of the instrument to discriminate the result of the winning teams was measured (Trochim & Donnelly, 2007). Seventy sets from women's beach volleyball games from the 2008 Olympic Games were analyzed. A total of 7,111 rallies and 20,060 actions were analyzed. Only the first and second sets were taken into consideration, due to the special characteristics of the last set and different rules (Marcelino, Sampaio & Mesquita, 2012; Palao, Valades, Manzanares, & Ortega, 2014). The efficacy of the serve, reception, set, attack, block, and dig were analyzed. The matches were recorded in the same way as described in stage two. The observation was done by an observer that had a degree in sport science, had the highest volleyball coaching certification in Spain (level III), was a former beach volleyball player, and had experience as an observer (for research). The same protocol for observer training and data quality control that was described in previous stages was done. To calculate the intra-reliability, another trained observer was used. He had a doctorate in sport science and more than ten years of experience coaching volleyball and doing scouting for indoor and beach volleyball. The inter-observer and intra-observer agreement were higher than the levels found in stage four. A discriminant inferential analysis of the data was done to find those statistical variables that best differentiate winning and losing players (Ntoumanis, 2001). Structural Coefficients (SC) greater than or equal to .30 (Tabachnick & Fidell, 2001) were considered relevant for the interpretation of the linear vectors. All of the statistical analyses were done with a level of significance of p ≤ .05.

In the seventh stage, the ability of the instrument to differentiate between different age groups was measured (Trochim & Donnelly, 2007). Six sets of three matches from the men's under-19 World Championship, under-21 World Championship, and senior World Championship were studied (2010-2011 season). A total of 855, 944, and 978 rallies and 2,381, 2,669, and 2,821 actions, respectively, were analyzed from the men's under-19 World Championship, under-21 World Championship, and senior World Championship. The analyzed sets were from the semi-finals and finals of the Championship. Only the first and second sets were taken into consideration, due to the special characteristics of the last set as well as the different rules (Marcelino, et al., 2012; Palao, et al., 2014). The manner of execution of the different actions, their efficacy, play time, and efficacy of different game phases were analyzed. The observation was done by an observer that had a master's degree in sport science, had the highest volleyball coaching certification in Brazil, was a beach volleyball coach, and had experience as an observer (research). The same protocol for observer training and data quality control as described in the previous stage was done. The intra-reliability was calculated with the same trained observer as in stage six. The inter-observer and intra-observer agreement were higher than the levels found in stage four. An inferential analysis of the data was done using a Mann-Whitney U for the continuous variables and Chi-Square Test and likelihood ratios for the categorical variables to evaluate the ability to differentiate between competition age groups. The inferential analyses of the data were carried out with the SPSS 20.0 software. Statistical significance was set at p < .05.

In relation to the observational instrument structure, each row of data included the information from a complex or phase of the rally. The columns included the information about the observed variables. The variables were divided into four groupings: contextual variables, game situation, technical and tactical actions, and rally result. In the block of contextual variables, information about the match was collected. In the block of game variables, information about the game or set was collected. In the block of technical and tactical actions, information about the actions of a ball possession phase (complex) and the previous action was collected. For each action, the player that executed, his/her role, execution zone, technique and/or type, direction and/or destinations, and efficacy were recorded. Information collected about the previous action varied throughout the rally. In the first phase of the rally or side-out, the previous action is the serve. In the rest of the rally phases, the action before the offense is the block (first team action in defense). In the grouping about the rally result, information about the game or set was collected. The variables in this grouping were obtained indirectly from the previous grouping of variables after the observation.

Results

The list of actions from the observational instrument and their procedure after the first and second stages of the observational instrument design (literature review, first group of experts, and pilot studies for observation and data analysis) is shown in Table 2. The group of variables related to the match and game situations was set according to previous research and suggestions by the researchers or coaches. These variables collected information regarding the type of match and set, the score, team characteristics, game phase, game situation (player serving, and rotation) and rally duration. The variables studied in previous studies of indoor volleyball and beach volleyball were the starting point. The studies related to the nature of volleyball (Eom & Schutz, 1992a, 1992b; Palao, 2004), especially in relation to game phase or the order of the actions (e.g. serve-reception, reception-attack, etc.), were the basis for building the structure of the instrument. Therefore, the data from the actions were collected by rally phase or complex (group of three actions performed by a team and the previous action). The researchers reviewed the variables used in beach volleyball studies and they searched in the oldest available references related to the different categories (Burchuk & Burchuk, 1993; Coleman, Neville, & Gordon, 1969; FIVB, 2008; Homberg & Papageorgiu, 1994; Palao, 2004; Selinger & Ackermann-Blount, 1986). For all the variables related to technical actions, variables that collected information regarding player that executed, his/her role, technique used, manner of execution or situations of execution, zone of execution, zone of destination and action performance were included. To evaluate the effect of the different actions on the rally and the collective team efficacy, variables that collected data about the outcome and the actions' efficacies were also included in the instrument. The performance of the actions was evaluated by the effect of each action on the rally or the following actions. The terminal actions (serve, attack, and block) were measured using a 5-category scale (Coleman, Neville, & Gordon, 1969). The levels of the scale were: error, action allows the opponent all attack options, action limits opponent attack, action does not allow the opponent to attack, and point. The continuous actions (reception, set, and dig) were measured using a 4-category scale (Coleman, Neville, & Gordon, 1969). The levels of the scale were: error, action does not allow the opponent to attack, action limits the attack options, and action allows all attack options. The instrument does not include variables related to physical actions done by the player, but information about them can be obtained from the manner of execution and the work and rest time of the rallies.

Table 2.
Final behaviors of the first and second design stages of the observation instrument.

- Competition 1 - Gender 1 - Team A 2 - Team B 2 - Result of the match 1 - Set 1 - Result of the set 1 - Team A's points in the set 2 - Team B's points in the set 2 Game situation - Rotation of team A 2 - Rotation of team B 2 - Team in reception 1 - Start time of the rally 2 - End time of the rally 2 - Total play time of the rally 2 4 - Rally phase or complex 1 2 Technical-tactical actions (Serve / Block) A - Player that executes 1 - Player role 1 - Way of attacking toward the block 2 4 - Zone of execution 1 3 4 - Execution techniques 1 3 4 - Destination 12 - Efficacy 1

Technical-tactical actions (Reception / Defense) A - Player that executes 1 - Player role 1 - Zone of execution 1 2 4 - Execution techniques 1 2 - Efficacy 1 Technical-tactical actions (Set) - Player that executes 1 - Player role 1 - Zone of execution 1 - Depth 2 - Execution techniques 1 - Destination 1 - Efficacy 1 Technical-tactical actions (Attack) - Player that executes 1 - Player role 1 - Way of attacking 1 2 3 - Zone of execution 1 - Execution techniques 1 2 3 4 - Block intervention 2 3 4 - Destination 1 3 4 - Attack efficacy 1 Rally result - Rally phase or complex efficacy 2 - Way or actions through which the point is obtained 1 3 4 - Rally result 1

• 1 Behaviors suggested by the review of literature (beach volleyball and/or indoor volleyball); 2 Behaviors suggested or modified by experts and researchers; 3 Behaviors modified after the pilot test; and 4 Behaviors modified after the pilot data analysis. A In the analysis of side-out or rally phase, the actions collected include the serve and the reception, and in the rest of the game phase the actions collected are the block and the dig.

After the first two stages, the list was composed of 46 behaviors. Nine variables were about describing the context (five were suggested by the literature review and four were suggested by the experts and researchers). Seven variables were about observing the game situation (one was suggested by the literature review and six were suggested by the experts and researchers; one was modified after the pilot test). Twenty-seven variables were about observing the technical and tactical actions (22 were suggested by the literature review, four were suggested by the experts; eight were modified by the experts and researchers or after the pilot test). Three variables were about the rally result (two were suggested by the literature review, and one was suggested by the experts and researchers; one was modified after the pilot test).

In the third stage, after the evaluation by the second group of experts (n = 8), seven behaviors were modified. The experts' observations were related to the definitions of the categories of the variables. No variable was eliminated by the experts after the comprehension and pertinence evaluation (all variables had an average score > 7.0 out of 10). In the fourth stage (the second pilot test and observer training), twelve behaviors were modified or the criteria were more concretely specified to better differentiate the categories of the variables. All variables and categories had their definition and the criteria to establish the different categories. The observers had an inter-observer agreement coefficient > .82 and an intra-observer agreement coefficient > .99 for all the studied variables (Cohen's Kappa) (Table 3). In the fifth stage, the list of behaviors (categories, definitions, and criteria for differentiation) was evaluated by the third group of experts (n = 7). Aiken's V from the quantitative evaluation of the third group of experts are shown in table 4. Since all variables had an Aiken's V > .84, they were pertinent. The final instrument included 46 behaviors for observation (Table 5).

Table 3.
Observers' inter- & intra-agreement after training in the use of the observation instrument (fourth stage) and variables modified by experts (fifth stage).

Variables	Reliability		Variables		Reliability
	Intra	Inter			Intra	Inter
Contextual variables			Technical-tactical actions (Reception / Defense) A
Competition	1.00	1.00	Player that executes	1.00		0.99
Gender	1.00	1.00	Player role	1.00		1.00
Team A	1.00	1.00	Zone of execution * **	0.99		0.98
Team B	1.00	1.00	Execution techniques * **	0.99		1.00
Match result	1.00	1.00	Efficacy **	0.99		1.00
Set	1.00	1.00	Technical-tactical actions (Set)
Set result	1.00	1.00	Player that executes	0.99		0.99
Team A's points	1.00	1.00	Player role	0.99		1.00
Team B's points	1.00	1.00	Zone of execution	0.99		0.98
Game situation			Depth **	0.82		0.98
Team A's rotation	1.00	1.00	Execution techniques * **	0.99		1.00
Team B's rotation	1.00	1.00	Destination	0.99		0.98
Team in reception	1.00	1.00	Efficacy * **	0.99		1.00
Start time of the rally	0.92	1.00	Technical-tactical actions (Attack)
End time of the rally	0.90	1.00	Player that executes	1.00		1.00
Total play time of the rally	0.87	1.00	Player role	1.00		1.00
Technical-tactical actions (Serve / Block) A			Way of attacking * **	0.96		1.00
Player that executes	1.00	1.00	Zone of execution	0.85		1.00
Player role	1.00	1.00	Execution techniques * **	0.99		1.00
Way of attacking toward the block **	0.96	1.00	Block intervention **	0.99		1.00
Zone of execution	0.91	0.98	Destination	0.99		0.98
Execution techniques	0.94	1.00	Efficacy	0.99		0.99
Destination *  **	0.83	0.98	Rally result
Efficacy **	0.86	0.98	Rally phase efficacy	1.00		1.00
			Way point is obtained	1.00		1.00
			Rally result	1.00		1.00

• * Behaviors modified and included by expert judges during third stage; and * Behaviors modified by expert judges during fifth stage. A In the analysis of side-out or rally phase, the actions collected are the serve and the reception, and in the rest of the game phase the actions collected are the block and the dig.

Table 4.
Quantitative evaluation by eight expert judges about observation instrument (fifth stage).

Variables	AIKEN’S V		Variables		AIKEN’S V
	Definition	Pertinence			Definition	Pertinence
Contextual variables			Technical-tactical actions (Reception / Defense) A
Competition	1.00	1.00	Player that executes	1.00		1.00
Gender	1.00	1.00	Player role	1.00		1.00
Team A	1.00	1.00	Zone of execution	0.90		1.00
Team B	1.00	1.00	Execution techniques	0.92		1.00
Match result	1.00	1.00	Efficacy	0.95		1.00
Set	1.00	1.00	Technical-tactical actions (Set)
Set result	1.00	1.00	Player that executes	1.00		1.00
Team A's points	1.00	1.00	Player role	1.00		1.00
Team B's points	0.99	0.98	Zone of execution	0.95		0.95
Game situation			Depth	0.99		0.91
Team A's rotation	1.00	1.00	Execution techniques	1.00		1.00
Team B's rotation	1.00	1.00	Destination	0.95		0.95
Team in reception	1.00	1.00	Efficacy	0.95		1.00
Start time of the rally	1.00	1.00	Technical-tactical actions (Attack)
End time of the rally	1.00	1.00	Player that executes	1.00		1.00
Total duration of the rally	1.00	1.00	Player role	1.00		1.00
Technical-tactical actions (Serve / Block) A			Way of attacking	0.90		1.00
Player that executes	1.00	1.00	Zone of execution	0.95		0.95
Player role	1.00	1.00	Execution techniques	0.85		1.00
Way of attacking toward block	0.85	0.90	Block intervention	0.85		0.90
Zone of execution	0.98	1.00	Destination	0.90		1.00
Execution techniques	0.95	1.00	Efficacy	0.95		1.00
Destination	0.90	1.00	Rally result
Efficacy	0.95	1.00	Rally phase efficacy	0.98		0.90
			Way point is obtained	1.00		1.00
			Rally result	1.00		1.00

• A In the analysis of side-out or rally phase, the actions collected are the serve and the reception, and in the rest of the game phase the actions collected are the block and the dig.

Table 5.
Final behaviors and categories of the observation instrument

Contextual variables - Competition - Gender - Team A - Team B - Match result (win or lose) - Set - Set result (win or lose) - Team A's total points - Team B's total points Game situation - Rotation of team A (player serving for team A) 1 - Rotation of team B (player serving for team B) 1 - Team in reception (A or B) - Start time of the rally (minutes and seconds) - End time of the rally (minutes and seconds) - Total duration of the rally - Rally phase or complex (side-out or complex 1, complex 2, complex 3 or defense phase by team in side-out, and complex 4 or defense phase to the counter-attack of side-out team) 2 Technical-tactical actions (A. Serve / Block) A - Player that executes - Player role (blocker, digger, or universal) 3. - Way of attacking through the block (semi, second contact, penalty, and no attack (e.g. bump)) 3 4 * Only for evaluating block actions. - Zone of execution (six zones for serving (three lanes and two rows) and three zones for blocking (three lanes)) 1 B

Technical-tactical actions (A. Serve / Block) Cont. - Execution techniques (four types for serving (standing, power jump, floating jump, and other) and two for blocking (one player or two players) 3 4 - Destination (10 zones for serving (five lanes and two rows) and four options for blocking (no jump, block-out, court of the spiker, and court of the blocker)). B - Performance (5-categories scales: error, all attack options, limit attack, no attack options and point) 5 Technical-tactical actions (B. Reception / defense) A - Player that executes. Variable is calculated semi-automatically. - Player role (blocker, digger, or universal) 3. - Zone of execution (10 zones for reception (five lanes and two rows, near and far from the net) and six zones for defense (three lanes and two rows, near and far from the net)). B - Execution techniques (three types for serving (bump, overhead hit, and others) and five for digging (bump, hit, free-ball, acrobatic defense, and other)) 3 4 - Performance (4-categories scales: error, no attack, limit attack, and all options attack) 5. Technical-tactical actions (C. Set) - Player that executes - Player role (blocker, digger, or universal). - Zone of execution (three lanes) B - Depth (two distances: net to 4.5 m and more than 4.5 m) B - Execution techniques (jump set, overhead, bump, and others) - Destination (forward or behind). - Performance (4-categories scales: error, no attack, limit attack, and all options attack) 5

Technical-tactical actions (D. Attack) - Player that executes. Variable is calculated semi-automatically. - Player role (blocker, digger, or universal) 3. - Way of attacking (semi, second contact, penalty, and no attack (e.g. bump)) - Zone of execution (three lanes) B - Execution techniques (jump and no contact, hit, non-standard hit, and others) 3 4 6 - Block intervention (block jump and no contact, block-out, ball to court of the spiker, ball to the court of the blocker, no block jump) - Destination (seven zones (three lanes and two rows, plus ball out)). B - Performance (5-categories scales: error, all attack options, limit attack, no attack options and point) 5 Rally result (variables calculated automatically) - Rally phase or complex efficacy (scale of zero to four points in relation to the effect on the rally and the options it allows the opponent team) 5. - Way or actions through which the point is obtained (serve point, serve error, attack point, attack error, block point, block error, other errors). - Rally result (win or loss).

• Legend: 1 FIVB (2008); 2 Adapted from Palao (2004); 3 Adapted from Homberg & Papageorgiu (1994); 4 Adapted from Selinger & Ackermann-Blount (1986); 5 Coleman, Neville, & Gordon (1969); 6 Adapted from Burchuk & Burchuk (1993).A In the analysis of side-out or complex I, the actions collected are the serve and the reception, and in the rest of the game phases the actions collected are the block and the dig. B The zone of the execution of the serve, block, dig, set, and attack are establish dividing the net or court area in sizes equal parts. The zone of execution of the receptionist set regarding to the receptions positions and their movements. Zones were number from right deep side of the court of each team. The receptors were in zones 2 and 4. The lateral zones of zone 2 and 3 started when players have to do a step (displacement that involved move both foots). The closer zones stared when the players have to a step forward or have to nail to receive the ball.

In the sixth stage, the discriminate analysis showed that the instrument included variables that allow us to predict 79.3% of the game result (Table 6). The actions that predict winning or losing the set are: attack errors, attacks that limit the opponent's attack, digs that allow attacks, block points, blocks that limit the opponent's attack, serve points, and sets that allow an attack. In the seventh stage, significant differences were found in the 14 variables related to the way the actions were executed with regard to temporal, spatial, technique and efficacy between the different competition age groups analyzed (U-19, U-21, and World Championship). No significant differences were found for six variables (four related to the set actions and two related to the attack actions).

Table 6.
Standardized coefficients from the discriminant analysis done in the sixth stage (ability of the instrument to predict winning and losing women's teams in the 2008 Olympics Games).

Game statistics variable	Standardized coefficients	Game statistics variable	Standardized coefficients
Attack error	.450 *	Block error	.241
Attack that limits opponent attack	.406 *	Set that does not allow attack	.210
Dig that allows attack	.404 *	Reception that don't allow attack	.177
Block point	.404 *	Serve that does not limit opponent attack	.165
Block that limits opponent attack	.353 *	Set that allows all attack options	.162
Serve point	.322 *	Reception that allows all attack options	.162
Set that allows attack	.320 *	Serve error	-.151
Serve that limits opponent attack	.290	Attack point	.143
Reception that limits attack	.290	Serve does not allow opponent attack	.131
Dig that does not allow attack	.283	Dig error	.113
Reception error	.273	Block that does not limit opponent attack	.103
Block that does not let opponent attack	.255	Attack that allows all opponent attack options	.100
Set error	.253	Dig that allows all attack options	.085
Attack that does not allow opponent attack	.245

• Discriminant analysis (Wilks' Lambda: 0.644; Canonical Correlation: 0.597; Chi-square value: 59.643; Level of significance Chi-square test: p<.00; Classification ability: 79.3%). * SC discriminant value =|.30|

Table 7.
Differences found using the observation instrument between competitions of different age groups (seventh stage).

Variables		Age groups	Variables	Age groups
Contextual variables			Technical-tactical actions (Reception / Defense) A
Competition		-	Player that executes	-
Gender		-	Player role	-
Team A		-	Zone of execution 2	p<.001
Team B		-	Execution techniques 2	p<.017
Match result		-	Efficacy 2	p<.001
Set		-	Technical-tactical actions (Set)
Set result		-	Player that executes	-
Team A's points		-	Player role	-
Team B's points		-	Zone of execution 2	p<.117
Game situation			Depth 2	p<.342
Team A's rotation		-	Execution techniques 2	p<.401
Team B's rotation		-	Destination 2	p<.174
Team in reception		-	Efficacy 2	p<.001
Start time of the rally		-	Technical-tactical actions (Attack)
End time of the rally		-	Player that executes	-
Total duration of the rally 1		p<.001	Player role	-
Technical-tactical actions (Serve / Block) A			Way of attacking 2	p<.001
Player that executes		-	Zone of execution 2	p<.419
Player role		-	Execution techniques 2	p<.009
Way of attacking toward block 2		p<.003	Block intervention 2	p<.008
Zone of execution 2		p<.001	Destination 2	p<.271
Execution techniques 2		p<.001	Efficacy  2	p<.003
Destination 2		p<.001	Rally result
Efficacy 2		p<.001	Rally phase efficacy	-
			Way point is obtained 2	p<.001
			Rally result	-

• 1 Mann-Whitney U was used to establish the differences between age groups of competition. 2 Chi-Square Test and likelihood ratio were used to establish the differences between age groups of competition.

Discussion

This paper describes the stages done to design and validate an observational instrument to analyze technical and tactical actions in beach volleyball. Throughout the different stages carried out, a review of literature, expert review (coaches and researchers), pilot studies (observation, data analysis that includes both a descriptive analysis and preparing reports, and observer training), and statistical analyses were done to establish the validity of the observational instrument.

In the first stage, the review of the available literature was the starting point. Most of the beach volleyball studies that were reviewed used the same or similar categories used in indoor volleyball studies. Therefore, researchers reviewed the original studies or books that proposed those categories. The first draft of the instrument was reviewed by two research experts in match analysis and a coach. The collaboration in this stage of the Spanish men's National coach was critical to provide an applied perspective to the instrument, adapt the variables and categories to the reality of the game, establish possible applications of the data, and even to establish the proper vocabulary (common terminology between coaches and researchers). The help of the coach was valuable for establishing and defining the categories of the instrument. Another aspect that helped in the development of the instrument was the establishment of a sample data report for coaches that was created in this stage. After this experience, the researchers felt that the collaboration of researchers and coaches as well as keeping in mind the possible final use of the instrument are crucial in the design and validation of instruments to study sport performance. The report was used to provide scientific support to the men's Spanish national team for the 2008 Olympic Games (see Palao, 2012, for more information about the report sheet).

In the second stage, the first pilot study was used to test the data collection sheet (using a spreadsheet). The criteria used in the design of the data collection sheet was to make the instrument user-friendly for the observer (e.g. in the order of data collection, avoiding unnecessary cursor movements, etc.). Therefore, an automation of the data collection was done (calculating some variables indirectly from others, such as number of jumps, collective efficacy, etc.). Through the analysis of the data obtained from the first pilot study, the categories that had a low level of occurrence (which were regrouped with other categories) were not considered. No new variables or categories were included after the first pilot test. After this stage, a manual with the variables and categories of the instrument was developed.

In the third stage (review of the instrument by eight experts), the experts' opinions and contributions regarding the definition, categories, and criteria to differentiate between categories helped to develop the instrument and the manual for observer training. The qualitative evaluation done by experts helped specifically to more clearly define the variables and categories. The quantitative evaluation done by experts helped to establish the pertinence of the variables and categories (Bulger and Housner, 2007; Escurra, 1989; Padilla, Gomez, Hidalgo, & Muñiz, 2007; Zhu, Ennis, & Chen, 1998). For several variables, the researchers had different perspectives and opinions regarding level of specification of the variables (e.g. attack techniques). The researchers followed the criteria set by the first group of experts and the level of occurrence found in the pilot study done in stages two and four.

In the fourth stage, the observer training allowed us to improve and clarify the observation criteria. The development of a written and video manual for the instrument was decisive in this stage for improving observer agreement (training and final reliability). Also, a Frequently Asked Questions section was developed. The lack of agreement between observers resulted in several modifications of definitions and criteria for the categories in order to reach observer agreement. Final agreement was achieved in this phase by consensus. After the training, the reliability was calculated through test and re-test. The level of agreement between observers allowed us to establish that the measurement done by the observers after the training was reliable.

In the fifth stage (review of the instrument by seven experts), few changes to the instrument were made. From the quantitative review of the instrument done by the experts, higher values of content validity than the minimum proposed (Vo = .70) by Penfield and Giacobbi (2004) were found in the Aiken's V Test.

In the sixth stage, the discriminant analysis carried out shows the ability of the instrument to predict the result of the winning teams (79.3%). Human motor performance depends on a high number of aspects and perspectives (Glazier, 2010). However, the data found show the ability of the instrument to provide information about the way technical and tactical aspects influence the outcome in beach volleyball. The values found must not be considered a reference for futures studies because the characteristics of the sample may influence these results. In the present study, a sample of the 2008 Olympic Games was analyzed. However, the level of the teams affects the results. More studies are needed to better understand the stability, variability, and transitions between teams, players, etc. in team sport confrontation (Glazier, 2010; Lebed, 2006; MacGarry, Anderson, Wallace, Hughes, & Francis, 2002).

In the seventh stage, the results from the statistical analysis show the ability of the instrument to differentiate between different age groups of competition. In theory, players of higher age groups have more experience, years of training, and/or maturation. The instrument found differences between the ways the technical and tactical actions are carried out in men's beach volleyball according to different age groups. The instrument analyses players and teams in peak performance. The instrument does not qualitatively analyze the level of the motor movements done by the players (Knudson & Morrison, 2002). The goal of the instrument is to describe how technical and tactical actions are carried out and their efficacy.

The process and the data found demonstrate that the instrument may be suitable for measuring the technical and tactical actions executed (ball contacts) in beach volleyball. This instrument can be used by researchers in order to establish general game patterns and/or by coaches in order to establish specific game patterns. Regarding research, the instrument allows us to obtain a description of the technical and tactical actions executed by the players and teams to study sport evolution, effect of rules changes, references values, and much more. This instrument can be used by sport performance analysis or researchers from other research areas to study momentum, fatigue, etc. With regard to coaching, the instrument allows coaches to obtain information to monitor, analyze, and guide players and teams in practice and in competition.

The design and validation of an observation instrument for studying sport performance must take into consideration not only the variables, but also its use (observation) and the information that it can obtain (data analysis and its application). For that, the inclusion of coaches in the group of experts who work on the design and validation, and who test the instrument with regard to observation, data analysis, observer training, and usefulness is critical. Sports are constantly evolving (rule changes, players' and teams' evolution, sport materials, etc.). Therefore, the instrument described in this paper will need to be reviewed and adjusted in the future, if necessary.

Conclusion

This instrument allows for obtaining information about players and teams in technical and tactical actions as well as indirectly from physical actions (work and rest times, jumps, hits, etc.). The way data are collected allows us to analyze the influence of actions on each other (Eom & Schutz, 1992a, 1992b; Palao, 2004). However, the instrument does not register information about the players and the team without possession of the ball. The instrument only analyzes the team in possession of the ball and the actions of the player with the ball. This instrument lets us analyze the performance of beach volleyball teams from a technical and tactical perspective. Also, it allows us to evaluate the evolution of the sport, the differences between winning and losing teams, etc. The information that can be obtained from it can be used by coaches, players, and managers to make decisions with regard to team organization in practices and competitions.

Acknowlegements

This observational instrument was developed through the research grant "Technical, tactical, and physical monitoring in peak performance beach volleyball" (2008) supported by the Spanish High Council for Sport and Spain's Education and Science Ministry [Consejo Superior de Deportes y Ministerio de Educación y Ciencia] and through the research grant "Technical and tactical performance indicators and goal establishment in volleyball and beach volleyball" (2008-2010) supported by the Seneca Foundation (Región de Murcia, Spain).

The authors would like to acknowledge and thank professor María Teresa Anguera (University of Barcelona) for all the advice given throughout the years. This paper is based on the knowledge that she has helped us acquire.

The observational instrument, detailed instrument description, manual of the instrument, and training videos for the observer can be obtained at the following website https://sites.google.com/site/tebevol/ [Free download and use of all the material - Creative Commons Attribution-NonCommercial-ShareAlike].

References

• Anguera, M.T. (2003). Observational Methods (General). In R. Fernandez-Ballesteros (Eds.), Encyclopedia of Psychological Assessment, Vol. 2. (1st edition) (pp. 632-637). London: Sage.
• Behar, J. (1993). Sesgos del observador [Observer slant]. In M. T. Anguera (Eds.), Metodología observacional en la investigación psicológica - Volumen II [Observational methodology in psychology research - Volume II. (1st edition)] (pp. 27-76). Barcelona: Promociones y Publicaciones Universitarias.
• Blanco, A. (1993). Fiabilidad, precisión, validez y generalizabilidad de los diseños observacionales. In M. T. Anguera (Ed.), Metodología observacional en la investigación psicológica - Volumen II [Observational methodology in psychology research, Volume II. (1st edition)] (pp. 149-261). Barcelona: Promociones y Publicaciones Universitarias.
• Bulger, S.M., & Housner, L.D. (2007). Modified Delphi investigation of exercise science in physical education teacher education. Journal of Teaching in Physical Education, 26, 57-80.
• Burchuk, M., & Burchuk, M. (1993). Advanced attack skills. In Canadian Volleyball Associations (Eds.). Coaches Manual: Level 4 (pp 2-38). Montreal: CVA.
• Carling, C., Reilley T., & Williams, M. (2009). Performance assessment for field sports: physiological, psychological and match notational assessment in practice (1st edition). London: Routledge.
• Coleman, J.E., Neville, B., & Gordon, B. (1969). A statistical system for volleyball and its use in Chicago Women's Association, International Volleyball Review, 17: 72-73.
• Elosua, P. (2003). Sobre la validez de los test [About test validity]. Psicothema, 15, 315-321.
• Eom, H.J., & Schutz, R.W. (1992a). Statistical analyses of volleyball team performance. Research Quarterly for Exercise and Sport, 63, 11-18.
• Eom, H.J., & Schutz, R.W. (1992b). Transition play in team performance of volleyball: A log linear analysis. Research Quarterly for Exercise and Sport, 63, 261-269.
• Escurra, L. (1989). Cuantificación de la validez de contenido por criterio de jueces [Quantification of content validity through judge criteria]. Revista de Psicología, 6, 103-111.
• FIVB (2008). Official beach-volleyball rules 2008-2012. Lausanne: Fédération Internationale de Volleyball.
• Glazier, P.S. (2010). Game, set and match? Substantive issues and future directions in performance analysis. Sports Medicine, 40, 625-634.
• Homberg, S. & Papageorgiu, A. (1994). Handbook of beach volleyball. Aachen: Meyer & Meyer Sport.
• Hughes, M.D., & Bartlett, R.M. (2002). The use of performance indicators in performance analysis. Journal of sports sciences, 20, 739-754.
• Hughes, M.D., Cooper, S.M., & Nevill, A. (2002). Analysis procedures for non-parametric data from performance analysis. International Journal of Performance Analysis in Sport, 2, 6-20.
• Kimberlin, C.L., & Winterstein, A.G. (2008). Validity and reliability of measurement instruments used in research. American Journal of Health-System Pharmacy, 65, 2276-2284.
• Knudson, D.V., & Morrison, C.S. (2002). Qualitative analysis of human movement. Champaign, IL: Human Kinetics.
• Lebed, F. (2006). System approach to games and competitive playing. European Journal of Sport Science, 6, 33-42.
• Liebermann, D.G., Katz, L., Hughes, M.D., Bartlett, R.M., McClements, J., & Franks, I. M. (2002). Advances in the application of information technology to sport performance. Journal of Sports Sciences, 20, 755-769.
• Marcelino, R., Sampaio, J., & Mesquita, I. (2012). Attack and serve performances according to the match period and quality of opposition in elite volleyball matches. Journal of Strength and Conditioning Research, 26, 3385-3391.
• McCall, G. J. (1984). Systematic field observation. Annual Review of Sociology, 10, 263-282.
• McGarry, T., Anderson, D.I., Wallace, S.A., Hughes, M., & Francis, I.M. (2002). Sport competition as a dynamical self-organizing system. Journal of Sport Sciences, 20, 771-781.
• Merino, C., & Livia, J. (2009). Intervalos de confianza asimétricos para el índice de la validez de contenido: Un programa Visual Basic para la V de Aiken [Assymetric confidence interval for content validity index: a Visual Basic software for Aiken's V]. Anales de Psicología, 25, 169-171.
• Newell, K.M. (1986). Constraints on the development of coordination (pp. 341-360). In M.G. Wade, H.T.A. Whiting (Eds.), Motor development in children: Aspects of coordination and control. Boston, MA: Martinus Nijhoff.
• Ntoumanis, N. (2001). A step-by-step guide to SPSS for sport and exercise studies. London: Routledge.
• Padilla, J.L., Gómez, J., Hidalgo, M.D., & Muñiz, J. (2007). Esquema conceptual y procedimientos para analizar la validez de las consecuencias del uso de los test [Conceptual and procedures to analyze the validity of the consecuences of test use]. Psicothema, 19, 173-178.
• Palao, J.M. (2004). Incidencia de los complejos de juego y la posición del colocador sobre el rendimiento en competición [Effect of game phases and setter position on volleyball performance in competition]. Rendimiento deportivo, 9. Retrieved from URL: http://www.rendimientodeportivo.com.
  » http://www.rendimientodeportivo.com
• Palao, J.M. (2012). Apoyo científico al entrenamiento. Un caso práctico de diseño y aplicación de apoyo al alto rendimiento [Scientific support for sport training. A case study of the design and application of support to peak performance]. Apunts: Educación Física, 110, 52-60.
• Palao, J.M., Valades, D., Manzanares, P., & Ortega, E. (2014). Physical actions and work-rest time in men's beach volleyball. Motriz, 20, 257-261.
• Penfield, R.D., & Giacobbi, P.R. (2004). Applying a score confidence interval to Aiken's item content-relevance index. Measurement in Physical Education and Exercise Science, 8, 213-225.
• Selinger, A., & Ackermann-Blount, J. (1986). Arie Selinger's power volleyball. New York, NY: St. Martin Press.
• Sykes, R.E. (1977). Techniques of data collection and reduction in systematic field observation. Behavior Research Methods & Instrumentation, 9, 407-417.
• Tabachnick, B.G., & Fidell, L.S. (2001). Using multivariate statistics (3rd Edition). New York, NY: Harper Collins.
• Trochim, W., & Donnelly, J.P. (2007). The research methods knowledge base (3rd Edition). Mason, OH: Cengage Learning-Atomic Dog.
• Zhu, W., Ennis, C.D., & Chen, A. (1998). Many-faceted Rasch modelling expert judgment in test development. Measurement in Physical Education and Exercise Science, 2, 21-39.

• José Manuel Palao Andrés, PhD is affiliated with the Department of Physical Activity and Sport, Faculty of Sport Science at the University of Murcia, Spain (Universidad de Murcia, Murcia, Espanha). Former track and field and volleyball coach. Consultant and scientific support to athletes, teams and coaches.
• Policarpo Manzanares López has a master's degree and is affiliated with the Department of Physical Activity and Sport, Faculty of Physical Activity and Sport at the Catholics University of Saint Anthony, Murcia, Spain (Universidad Católica de San Antonio, Murcia, Espanha). College professor teaches volleyball and data analysis (2008-2012) and coaches the Spanish Superliga 2 volleyball.
• Enrique Ortega, PhD, is affiliated with the Department of Physical Activity and Sport, Faculty of Sport Science at the University of Murcia, Spain (Universidad de Murcia, Murcia, Espanha). Former coach of basketball teams.

Publication Dates

• Publication in this collection
  Apr-Jun 2015

History

• Received
  06 May 2014
• Accepted
  03 Mar 2015