Three-Dimensional Anxiety Scale for Sport: Development and Validity Evidence Based on Content

Escala Tridimensional de Ansiedade para o Esporte: Desenvolvimento e Evidências de Validade Baseadas no Conteúdo

Escala Tridimensional de Ansiedad para el Deporte: Desarrollo y Evidencias de Validez Basadas en Contenido

Abstract

This study aimed to develop and search for content validity evidences of an anxiety measure for athletes. An international literature review on the construct was carried out for the construction of 128 self-report items. The procedures of judge and semantic analysis were made after the elaboration of the instrument. The judges obtained 80.94% agreement and kappa indicated significant substantial agreement. By observing the kappa of each dimension, we conclude that the judges correctly identified the dimension of the items. However, the concordance in relation to the facets did not obtain the same indicators, a result that meets the studies of the three-dimensional model and may suggest the unsustainability of the facets. In the semantic analysis the participants indicated clarity regarding instructions and diagramming, with only three modified items. In general terms, the instrument presented favorable initial evidence of validity, and complementary studies are needed to gather evidence from other sources.

Keywords:
anxiety; sport psychology; psychometrics; test validity; scales

Resumo

O presente estudo teve por objetivo o desenvolvimento e a busca por evidências de validade baseadas no conteúdo de uma medida de ansiedade para atletas. Foi realizada uma revisão de literatura internacional sobre o construto para a construção de 128 itens de autorrelato. Após a elaboração do instrumento, foram realizados os procedimentos de análises de juízes e de semântica. Os juízes obtiveram 80,94% de concordância e o índice kappa demonstrou concordância substancial significativa. Ao observar o kappa de cada dimensão, concluiu-se que os avaliadores identificaram corretamente a dimensão dos itens. Entretanto, a concordância em relação às facetas não obteve os mesmos indicadores, resultado que vai ao encontro dos estudos do modelo tridimensional e podem sugerir a insustentabilidade das facetas. Na análise semântica, os participantes indicaram clareza em relação a instruções e diagramação, com apenas três itens modificados. Em termos gerais o instrumento apresentou evidências iniciais de validade favoráveis, sendo necessários estudos complementares para reunir evidências de outras fontes.

Palavras-chave:
ansiedade; psicologia do esporte; psicometria; validade do teste; escalas

Resumen

El presente estudio tuvo como objetivo desarrollar y buscar evidencias de validez de contenido de una medida de ansiedad para atletas. Se realizó una revisión de literatura internacional sobre el constructo para la elaboración de 128 ítems de autoinforme. Después de la elaboración del instrumento se realizaron los procedimientos de análisis de jueces y análisis semántico. Los jueces obtuvieron 80,94% de concordancia y el índice kappa indicó una concordancia sustancial significativa. Al observar el kappa de cada dimensión, se concluye que los evaluadores identificaron correctamente la dimensión de los ítems. Sin embargo, la concordancia con relación a las facetas no obtuvo los mismos indicadores, resultado que está vinculado con los estudios del modelo tridimensional y puede sugerir la insostenibilidad de las facetas. En el análisis semántico, los participantes indicaron claridad referente a las instrucciones y la diagramación, con solo tres ítems modificados. En términos generales el instrumento presentó evidencias iniciales favorables de validez, siendo necesarios estudios complementarios para reunir evidencias de otras fuentes.

Palabras clave:
ansiedad; psicología del deporte; psicometría; validez del test; escalas

Introduction

Anxiety in sport context is defined, according to the three-dimensional model of Cheng, Hardy and Markland (2009) as a functional evolutive strategy which emits warning signs and aims to prepare the individual to display the most appropriate response to adverse situations. However, the adaptive property of anxiety has been under-represented in theoretical models, since negative aspects are emphasized to the detriment of their potential to increase motivation, reach an optimal level of activation and contribute to attention processes (Carver & Scheier, 1986; Eysenck, 1992).

According to Coelho, Vasconcelos-Raposo and Mahl (2010) this is due to the fact that the concept of anxiety was proposed with the objective of diagnosing pathologies, which reinforced its negative connotation. It has been discussed in the literature for decades on how mistakenly anxiety has been treated in sports studies (Jones & Hanton, 2001; Jones & Swain, 1995). The central proposal about the study of anxiety is to characterize the emotional state that athletes experience before competitive situations, which is not always detrimental to sports performance (Martens, 1977). In this sense, the three-dimensional model seeks to understand the construct from three central dimensions: cognitive anxiety, physiological anxiety and perceived control.

The perceived control dimension aims to overcome the gap of other models, developed with clinical focus, promoting the investigation of the level of control that the athlete perceives to have about the situation. This dimension is defined as the athlete’s perception of his ability to achieve goals and achieve success even in the face of stressful situations (Teixeira, 2016). In addition, it refers to the athlete’s ability to perceive that he possesses the necessary qualities and to develop strategies to face adverse situations and achieve his goals, even in the face of anxiety symptoms.

Positive self-talk, visualization exercises, setting realistic goals, emotional and psychophysiological control strategies, among other strategies, helps the athlete to perceive himself in the control of the situation and to develop motivational and focus behaviors (Cheng & Hardy, 2016; Hanton & Connaughton, 2002; Hanton, O’brien, & Mellalieu, 2003). Thus, according to the feeling of control over the situation and their coping skills, the athlete can interpret their anxiety as positive or negative for achieving a good result (Cheng & Hardy, 2016).

The control dimension refers to the adaptive potential that covers the coping abilities involved in the anxiety dynamics in the context of the competitions. Therefore, it is considered possible to have both positive and negative consequences for sports performance, depending on the athlete’s perception of control over the situation (Cheng & Hardy, 2016). The proposition of a regulatory dimension promotes the insertion of coping skills in the anxiety model and is configured as an alternative to Jones’s directional scale (1995).

The cognitive dimension of the three-dimensional model points to the negative pole of the phenomenon and is divided into two facets, namely: worry and self-focus. The facet named worry is about a cognitive form of apprehension associated with unfavorable outcomes (Cheng et al., 2009). Its main characteristic is the anticipation of negative consequences that lead the athlete to feel apprehensive. This facet encompasses the fear of making mistakes, of failing, of situations of uncertainty, of the consequences of failure, of getting injured, and so on. All apprehensions involving sports practice situations are encompassed by this core of the cognitive dimension.

The facet named self-focus is defined as a state of self-evaluation with increased perceptions about the weaknesses that involves performance under pressure (Teixeira, 2016). The athlete focuses too heavily on how to perform the task, which triggers a critical and negative evaluative state with scrutiny over his motor actions. Consequently, one creates a fanciful thought that other people also pay attention to it and judge it. Athletes with a high self-focus tend to give excessive importance to the opinion of others. They tend to think that others do not find them capable and look for clues in the environment that meet their perception, thus keeping what they think about themselves (Cheng et al., 2009; Cheng & Hardy, 2016; Teixeira, 2016).

The physiological dimension, in turn, refers to the changes in the pattern of activation of the organism and that prepare it to respond to situational demands. It is also subdivided, according to the theoretical model, into autonomic hyperactivity and somatic tension. The first facet represents the physiological symptoms that are expressed through the involuntary musculature, glands and organs in response to a stressor. The physiological signs that are representative of this facet are the increase in heart and respiratory rate, feelings of cold sweat, shortness of breath, nausea, vomiting, intestinal derangement, “cold in the belly”, cold and wet hands, increased urinary frequency, and so on (Barlow & Durand, 2015; Cheng, Hardy & Markland, 2011).

On the other hand, physiological manifestations expressed through the voluntary musculature are comprised within the facet somatic tension. Muscle tension, fatigue, tremors, agitation and muscle spasms are its characteristic symptoms. For a symptom or physiological signal to be characterized as belonging to this facet it is not necessary for the individual to have control over it, but must necessarily be expressed through the voluntary musculature (Cheng et al., 2011).

The three-dimensional model was implemented in psychometric measurement through the Three Factor Anxiety Inventory (TFAI), elaborated by the authors of the theoretical model (Cheng et al., 2009). The instrument, with 25 items and a five-point scale varying from agree to disagree, was initially tested on samples from the UK and showed good psychometric properties, with satisfactory goodness-of-fit indexes (robust χ² [272] = 477.6; p < 0.001; RMSEA = .05; CFI = .97; SRMR = .07) and internal consistency (α between .75 and .86). Subsequently, the instrument underwent transcultural adaptation procedures and was tested in samples of Chinese athletes (Cheng et al., 2011). The Chinese version of the inventory with 21 items also produced satisfactory indicators of validity and reliability (robust χ² [186] = 303.4; p < 0.001; RMSEA = .05; CFI = .98; SRMR = .076; α between .80 and .87).

Regarding the measurement of anxiety in the Brazilian sport context, there is a lack of instruments available for professional use (SATEPSI, 2018). However, studies have focused on the process of creating, validating and/or improving instruments for Brazilian samples specific in the sporting context (Bartholomeu, Montiel, & Machado, 2013; Fernandes, Nunes, Vasconcelos-Raposo, & Fernandes, 2014; Fernandes, Nunes, Vasconcelos-Raposo, Fernandes, & Brustad, 2013; Fernandes, Vasconcelos-Raposo, & Fernandes, 2012). Most of the Brazilian research on anxiety measures in the sport context has been active in the production of knowledge about CSAI-2, an instrument based on the multidimensional theory of Martens, Burton, Vealey, Bump and Smith (1990).

The creation of psychological instruments is a complex process, since it represents the operationalization of the attributes of a latent trait in observable behaviors. According to Pasquali (2010), the construction of measures is based on a set of procedures that is subdivided into three major poles: the theoretical pole, the empirical pole and the analytical pole. The first pole describes the reasoning that underlies the measure. It begins with a scientific understanding of the phenomenon to be measured, that is, of the psychological theories through which the construct is formalized, to carry out its specification in behavioral categories and in the format of items (Reppold, Gurgel, & Hutz, 2014).

In this way, the theoretical pole bases the creation of the items from the meticulous review of the literature, passing through the decisions about the specifics of the psychological system that will be approached. It also focuses on the choice of central attributes for the understanding of the phenomenon, dimensional structure, constitutive and operational definitions, the format of the items and the properties of the scale used (Pacico, 2015). The theory has the supporting role for constructing the items and choosing which behaviors best represent the attributes of the construct within a continuum. In addition, evidence of validity seeks to verify if the content of the items adequately represents the attributes of the construct, and seek to evaluate its representativeness (Primi, Muniz, & Nunes, 2009).

Theories that systematize the construct, empirical studies, professional experience, expert evaluation and other psychological measures can act as sources of information for the elaboration of the instrument. During this process, the theoretical rules of measurement construction should act as guiding principles, as well as the criteria for writing items described in texts that are reference in the area, such as that of Pasquali (2010). Another crucial point for the proper functioning of the measure is the quantity of items developed to adequately represent the construct in its extension (Cohen, Swerdlik, & Sturman, 2014). The measure must be representative of the universe of behaviors that characterize the phenomenon, with items that express different magnitudes of the latent trait.

Resolution 007/2003 of the Federal Council of Psychology (CFP, 2003) proposes a definition of psychological evaluation that praises the understanding of psychological characteristics as resulting from the relationship between individual and context. Therefore, in addition to being representative, the measure must also be in line with the characteristics of the context for which it was elaborated. Considering this assumption, Silva, Foch, Guimarães and Enumo (2014) highlight the indiscriminate use of psychological instruments in the sports field.

Measures validated for use with clinical populations or in reference groups outside the sporting context have been used in psychological assessments of athletes (Garcia & Borsa, 2016; Silva et al., 2014). According to Rubio (2011), this fact can compromise the quality of the results and validity of the inferences proposed from these, given that the sport context presents specificities. The use of psychological instruments outside their specific context of application ends up reflecting the lack of qualification of the professionals in the practice of psychological evaluation (Borsa, 2016).

The use of specific instruments to assess anxiety in sports is essential. The phenomenon is closely related to context factors and peculiarities of sports practice (Jones, 1995). In addition, instruments that measure anxiety in the context of sport have a greater correlation with each other than with instruments for evaluation of general and clinical anxiety (Smith, Smoll, & Schutz, 1990; Teixeira, 2016). In this way, it is clear that the evaluation tools must be sensitive to the particularities of each application context. Therefore, the objective of this article was to develop and search for evidence of validity based on the content of an instrument for measuring anxiety in the sports context.

Method

Elaboration of the Instrument

An international literature review was carried out, covering theoretical models available, definitions, relations with other constructs and criteria, as well as evaluation instruments and their respective psychometric qualities. The elaboration of the state of the art represents a tool for the construction of representative items of the dimensions of the construct, according to the theoretical model and constitutive definitions. The theoretical framework used in the present study was based on the Three-Dimensional Model of Anxiety proposed by Cheng et al. (2009). This theoretical contribution was also used for writing the constitutive and operational definitions of dimensions and facets.

Subsequently the items were written, according to the criteria described by Pasquali (2010). It should be emphasized that, since the instrument will be indicated for athletes in general, regardless of their sporting modality, there was a care in the writing of the items not to mention specific terms of any sports modality. Thus, possible strangeness generated by the terms used is minimized, as well as the risks that athletes do not identify themselves with the content of the item.

The items were elaborated based on the main symptoms and characteristics described in the literature about anxiety, taking into account the specificities of the sport context and observing the criteria of construction. A wide range of physiological symptoms and signs have been attempted, ranging from changes in heart rate, exaggerated tension in the temporomandibular joint, to symptoms such as a feeling of suffocation. Other studies included in the literature review helped to complement the definitions, as well as in writing the items (e. g.Carver & Scheier, 1986; Eysenck, 1992; Hanton, Neil, & Mellalieu, 2011; Jones, 1995; Martens et al., 1990; Ohman, 2000; Smith, Smoll, Cumming, & Grossbard, 2006; Smith, Smoll, & Schutz, 1990; Spielberger, 1966; Teixeira, Nunes, & Cruz, 2016).

Items of other current instruments that obtained good indicators were also also used as models to generate the initial pool of items, as long as they were in agreement with the description of the dimensions of the three-dimensional model (CSAI-2; Fernandes et al., 2012; CSAI-2; Martens et al., 1990; SAS; Smith et al., 1990; SAS-2; Smith et al., 2006). Different versions of the same item were created, modifying the sequence of the words and the terms used, to verify which ones would work best for the target population. Fifteen reversed key items were also created to compose the initial item pool.

Initially, 128 items were built, in self-report format, to assess the dimensions of anxiety according to the three-dimensional anxiety model of Cheng et al. (2009). The amount of items required to properly evaluate the entire length of the construct is in line with the construction criteria described by Pasquali (2010). Approximately 25 items per dimension were elaborated to guarantee the adequacy of the representation, even after the exclusion stage of items with unfavorable psychometric properties.

Participants

Eight experts in anxiety and/or items construction, with minimum qualification of master, have participated as judges in content evaluation of the items. For the semantic analysis, ten athletes have participated. They were aged 18 years or more, with the 8th year of the elementary education required as minimum level of education, in order to allow the evaluation of the clarity of the information contained. The athletes who participated also met the following criteria: (a) participate in sports training and competitions, aiming at improving performance or results, (b) presenting formal registration in their sport federation, and (c) having competitive sports as their main activity. The semantic analysis sample was paired by gender and schooling. Two participants, one of each sex, were recruited for each level of schooling: complete elementary education, incomplete high school, complete high school, incomplete college and complete college.

Instruments

The instrument has three dimensions and five facets, according to the three-dimensional model of anxiety (Cheng et al., 2009). The first dimension is named Cognitive Anxiety and is subdivided into the Worry and Self-focus facets. The second dimension entitled Physiological Anxiety presents the facets Autonomic Hyperactivity and Somatic Tension. The third dimension is called Perceived Control and has no facets. The preliminary measure has 128 items in self-report format. A five-point Likert scale anchored in points 1 and 5 is used for response. The scale varies from 1 (it describes me very badly) to 5 (describes me very well) and expresses the degree of identification of the subject with the proposition.

Procedures

The judges received individually in their e-mail a spreadsheet with the items, as well as a brief description of the instrument, the intended audience, and a document with facet definitions. The judges were instructed to indicate the facet corresponding to each item, identify items with negative polarity, verify the suitability of the item to the target audience and its relevance to the evaluation of the construct, making observations if necessary. After the completion of the task, the judges returned the filled spreadsheet by e-mail.

The analysis of judges helps to obtain evidence related to the content, since it verifies the adequacy and coherence of the representation of behaviors in relation to the dimensions addressed by the instrument. This procedure allows the verification of possible divergences between judges as to the belonging of the items to dimensions and facets, which suggests problems that should be reviewed (Primi et al., 2009). In addition, items that have not been tied to any dimension may be irrelevant to construct measurement. Thus, the analysis of judges becomes essential for the process of the instrument elaboration.

Additionally, the general properties of the items were evaluated considering the construction rules, so as to ensure their intelligibility by the target population (Pasquali, 2010). From the results of the analysis of judges, a new version of the instrument was obtained, which was submitted to the semantic analysis. For the accomplishment of this procedure, the instrument was presented to a group of ten athletes. The semantic analysis was performed in a classroom, in the dependencies of the Federal University of Santa Catarina, in a quiet environment and only with the group of participants accompanied by the researcher. The application was in group and it was given through the traditional paper-and-pencil application.

First, the research and the procedure for responding to the instrument were presented with the proper instructions. Participants were asked to respond to the instrument and evaluate its clarity, wording and diagramming issues. At the end, time was made available for the recording of observations, doubts and suggestions pertinent to the evaluation of the instrument. The items were re-read for the participants so that they could express what they understood.

In this qualitative procedure, the instrument was answered by subjects of lower and higher schooling of the target population. This analysis was carried out to ensure that the contents of the instrument, from the instructions to the items, were understood by all participants, including the portion of lesser schooling, and that no disregard was made for the other group (Pasquali, 2010).

Data Analysis

To verify the inter-observer reliability of the judges, the kappa coefficient and the percentage of agreement between them were calculated (Souza, Alexandre, & Guirardello, 2017). The kappa was interpreted using the Landis and Koch (1977) criterion that classifies the magnitude of the indicator and considers kappa below 0,0 as poor; between .00 and .20 as slight; between .21 and .40 as fair; between .41 and .60 as moderate; between .61 and .80 as substantial and between .81 and 1.00 as almost perfect agreement.

The percentage of agreement should not be less than 80% according to the criteria recommended by Pasquali (2010), since items that reveal low agreement can share significant variance with other dimensions of the construct or be irrelevant, being characterized by an imprecise measure. Therefore, 80% agreement was the first cut line used for item selection. Secondly, items with a concordance between 70% and 80% were reassessed in order to verify their main problems and if they consisted of indispensable items for the evaluation of the construct, according to the underlying literature. For the semantic analysis, since it consists of a qualitative procedure of adjustment of the contents and diagramming, items and instructions considered confusing or inadequate were reformulated or eliminated.

Ethics

This research project was submitted to the Ethics Committee in Human Research of the Federal University of Santa Catarina, according to the norms of the National Health Council (Resolution 466/12 of the National Health Council) and approved under the 42882815.6.0000.0121 Certificate of Presentation for Ethical Appraisal.

The research is in line with the Directives and Norms Regulating Research Involving Human Beings (466/12). Individuals who agreed to participate were informed of their rights through the Free and Informed Consent Form signed by them. The participants were not harmed, had their welfare and dignity protected, and the protection of their rights were guaranteed.

Results

Judges Analysis

Initially, four items were identified that, according to the judges’ evaluation, were not relevant or did not seem to address the construct. Suggestions for altering the writing of some items were also examined, due to the quality of writing and observance of construction criteria, such as lengthy items or more than one idea. The suggestions were then considered and the necessary modifications were made.

The analysis by the percentage of agreement among judges indicated 80.94% of concordance among the eight judges for the 128 items. The kappa coefficient calculated from the categorization by dimensions, which can be verified in Table 1, indicated significant agreement among the evaluators (κ = .80, p < 0.001). The kappa coefficient calculated from the facet categorization also indicated substantial significant agreement (κ = .63, p < 0.001).

Table 1
Kappa for Facets e Dimensions

Facets	kappa (κ)	Dimensions	kappa (κ)
Worry	.55	Cognitive	.76
Self-focus	.49
Autonomic Hyperactivity	.74	Physiological	.90
Somatic Tension	.69
Perceived Control	.71	Perceived Control	.71
Total	.63	Total	.80

• Note. p < 0.001.

Autonomic hyperactivity was the facet with higher kappa (κ = .74, p < 0.001), followed by Perceived control (κ = .71, p < 0.001), Somatic tension (κ =.69, p < 0.001), Worry (κ = .55, p < 0.001) and Self-focus (κ = .49, p < 0.001). In relation to the dimensions, the highest kappa was found in the Physiological dimension (κ = .90, p < 0.001), indicating almost perfect agreement between the judges, followed by the Cognitive dimension (κ = .76, p < 0.001), and Perceived Control (κ = .71, p < 0.001).

Of the 128 items analyzed, 74 obtained 80% or more concordance between the judges, being this the first cut line used for the selection of items. In a second moment the judges’ considerations on the items with concordance between 70% and 80% were verified. Of these, 24 items whose theoretical rationale indicated relevance for the measurement of the construct were maintained. A set of 98 items were selected for the next stage: the semantic analysis.

Semantic Analysis

In general, the participants responded to the instrument without requiring interventions of the researcher to make any kind of explanation. They indicated clarity regarding instructions and layout. None of the athletes had any doubts about the content of the items and instructions However, questions about writing were raised in three of the 98 items. The first mentionated item was “I’m confident that I’ll be able to do my best” from the Perceived Control dimension. Participants suggested modifying the verb to denote how the participant feels, and then the item was rewritten as “I feel confident that I will be able to do my best”.

The second item, of the Self-focus facet, was “If I make a mistake, people will think I’m incapable.” One of the athletes pointed out that the item referred to the thinking of other people and not the athlete itself. After the suggestion, the item was rephrased: “I think if I make a mistake people will think I’m incapable.” Finally, the third item signaled by the participants was “People usually say that I don’t value my performance as it should”, also of the Self- focus facet. The argument used was similar to that of the previous item because it referred to the thinking of other people. The item was rewritten from the suggestion: “I tend to be very critical of my performance”.

Discussion

The objectives of this article were to construct an instrument for assessing anxiety in athletes and to seek evidence of validity based on content through judges and semantic analyses. According to the results, it was possible to identify satisfactory evidence. The preliminary instrument was shown to be in line with the theoretical model of Cheng et al. (2009), from which it is based, and mainly with the definitions constructed for each facet and dimension.

With regard to the judges’ analysis, few items were subject to suggestions regarding their writing, and underwent the necessary changes. The agreement index between the judges was considered satisfactory according to the assumptions of Pasquali (2010). The inter-observer reliability indicators among judges obtained for the total sample of items indicate that the evaluators understood the definitions and were able to categorize the statements effectively.

Analyzing the kappa coefficient for the dimensions, it was clear that they were satisfactory and indicated a substantial significant agreement (Landis & Koch, 1977). However, a closer look at the results of the facets is necessary. It is identified that there was less agreement, Self-focus and Worry, both with moderate agreement. The fact that these two facets have obtained the lowest agreement indicators within the model, both belonging to the same dimension, shows that its dividing line may be tenuous (Cheng et al., 2009; 2011). The fact that the Self-focus facet encompasses apprehensions about possible negative evaluations coming from representative people to the athlete represents an area of overlap with the definition proposed by the Worry facet. The same is true of a thorough analysis of your motor act, which can become a source of concern as it relates to the fear of failing and making mistakes.

By observing the kappa of each dimension, we conclude that the evaluators correctly identified the dimension to which the items belonged. However, due to the overlap between facets of the same factor and the probable shared variance between them, the concordance between judges was lower. These preliminary indicators are in line with the empirical results of the factorial studies by Cheng et al. (2009; 2011) and may suggest the unsustainability of the division between facets. The authors found better confirmatory indicators when the facets were fused, resulting in a simple three factor model.

Regarding to the two cognitive facets, the fact that both of them are negatively toned and relate to a self-evaluation state can be a reason to explain, in a way, its possible overlap (Cheng et al., 2009; 2011). A similar logic suggests the overlap of the somatic tension and autonomic hyperactivity facets. The kappa result for them were higher when compared to the facets of the cognitive dimension, although it still suggests that these two facets of physiological anxiety overlap, as reported in the literature (Cheng et al., 2009, 2011). lThe high correlation between the facets and the factorial findings of previous studies can be explained by the fact that both refer to physiological reactions in response to the perceived threat (Cheng et al., 2009).

In relation to the semantic analysis carried out with athletes, only three items underwent changes based on the participants’ suggestions. Of these, two belonged to the Self-focus facet, and one to the Perceived Control dimension. The writing changes in the self-focus items were consistent with their definition in the literature, being a state of self-evaluation, with greater attention to possible failures in performance and judgments (Cheng et al., 2009, 2011). The same can be said about the Perceived Control item. The change in this item highlights important characteristics of this dimension, such as the athlete’s perception of his ability to cope and achieve goals under stress (Cheng et al., 2009).

With regard to validity evidence based on content, the authors of the theoretical model and international measures (Cheng et al., 2009; 2011) indicate that they performed procedures to collect them, but no results are reported in their research articles, which does not allow the comparison of its results with those of the present research. Teixeira, Nunes and Cruz (2016) carried out a systematic review about the strategies of searching for evidence of validity used in several studies about instruments to evaluate anxiety in the sport context. There was a shortage of use and reporting of procedures performed to gather evidence based on content. Procedures performed, such as judges and semantic analyzes, were not reported in any study, even among studies of new measures.

Pacico (2015) brings the importance of the judges’ analysis as it is a theoretical evaluation of the behavioral representation of the construct and consists of verifying if the items are evaluating the dimensions and facets correctly. Equally important, the semantic analysis consists of discussing aspects of the instrument about the wording of the items, in order to verify that they are understandable and relevant to the members of the target population, allowing confusing items to be reformulated or excluded (Pasquali, 2010). It is important to emphasize that the difficulty in finding articles that have validity studies related to content for the aforementioned construct in its context is a limiting factor for the present research, since it makes it difficult to establish relationships with previously scientific literature. Evidence related to content is an important part of the validation process of a measure.

The specialized literature (International Test Commission, 2017; Reppold, Gurgel, & Hutz, 2014) points out that measures and their items, when created or adapted, may present different functions depending on the language, culture and target audience. In the process of creating the ETApE items’, aspects such as the education level of Brazilian athletes, different sports and competitive levels were taken into consideration, which resulted in items with nuances and specificities to better represent the theoretical model of anxiety adopted in Brazil.

The present study brings important contributions, as it performs procedures for searching evidence of validity based on the content, as well as present the empirical results of these stages of the instrument improvement. One of the objectives related to the construction of the items was to bring a more diversified sample of behaviors for the assessment of the construct, respecting the definitions of each of the dimensions. A scale with a variety of behaviors associated and relevant to measuring the construct helps to better understand the behavioral repertoire and symptoms associated with each level of the latent trait. The results point to the proper representation of the construct and the universe of behaviors that represent its attributes, that is, if the items appropriately describe the different nuances of the construct according to the theoretical model adopted (Cohen et al., 2014).

In relation to the main limitations of the study, the sample size is pointed out, both for the analysis of judges and for the semantic analysis. It arises, especially in relation to semantic analysis, the need to use samples from different regions of Brazil, given that there is great cultural variability that can influence the understanding of the instructions and the content of the items. After going through the theoretical stages of conceptualization and elaboration of the items, as well as gathering evidence of validity based on the content, it can be affirmed that the instrument presents favorable initial evidences of content-based validity. With the conclusion of the objectives proposed in this article, it is necessary that later studies submit the instrument to different psychometric procedures to gather evidence of validity of other natures.

References

• Barlow, D. H., & Durand, V. M. (2015). Psicopatologia: uma abordagem integrada (2ª ed.). São Paulo: Cengage Learning.
• Bartholomeu, D., Montiel, J. M., & Machado, A. A. (2013). Avaliação da Escala Likert dos itens do CSAI-2 em atletas. Interação em Psicologia, 17(1), 79-89. doi: 10.5380/psi.v17i1.26775
  » https://doi.org/10.5380/psi.v17i1.26775
• Borsa, J. C. (2016). Considerações sobre a formação e a prática em avaliação psicológica no Brasil. Temas em Psicologia, 24(1), 131-143. doi: 10.9788/TP2016.1-09
  » https://doi.org/10.9788/TP2016.1-09
• Carver, C. S., & Scheier, M. F. (1986). A control-process perspective on anxiety. Anxiety Research, 1(1), 17-22. doi: 10.1080/10615808808248217
  » https://doi.org/10.1080/10615808808248217
• Cheng, W. K., & Hardy, L. (2016). Three-dimensional model of performance anxiety: Testes of the adaptive potential of the regulatory dimension of anxiety. Psychology of Sport and Exercise, 22, 255-263. doi: 10.1016/j.psychsport.2015.07.006
  » https://doi.org/10.1016/j.psychsport.2015.07.006
• Cheng, W. K., Hardy, L., & Markland, D. (2009). Toward a three-dimensional conceptualization of performance anxiety: Rationale and initial measurement development. Psychology of Sport and Exercise, 10(2), 271-278. doi: 10.1016/j.psychsport.2008.08.001
  » https://doi.org/10.1016/j.psychsport.2008.08.001
• Cheng, W. K., Hardy, L., & Markland, D. (2011). Cross-cultural validation of a three-dimensional measurement model of performance anxiety in the context of Chinese sports. International Journal of Sport Psychology, 42(5), 417-435. Retrieved from <https://www.researchgate.net/profile/Wen-Nuan_Cheng/publication/282503272_Cross-cultural_validation_of_a_three-dimensional_measurement_model_of_performance_anxiety_in_the_Context_of_Chinese_Sports/links/5610e10808ae4833751a1f20/Cross-cultural-validation-of-a-three-dimensional-measurement-model-of-performance-anxiety-in-the-Context-of-Chinese-Sports.pdf>
  » https://www.researchgate.net/profile/Wen-Nuan_Cheng/publication/282503272_Cross-cultural_validation_of_a_three-dimensional_measurement_model_of_performance_anxiety_in_the_Context_of_Chinese_Sports/links/5610e10808ae4833751a1f20/Cross-cultural-validation-of-a-three-dimensional-measurement-model-of-performance-anxiety-in-the-Context-of-Chinese-Sports.pdf
• Coelho, E., Vasconcelos-Raposo, J., & Mahl, A. C. (2010). Confirmatory factorial analysis of the brazilian version of the Competitive State Anxiety Inventory-2 (CSAI-2). The Spanish Journal of Psychology, 13(1), 453-460. doi: 10.1017/S1138741600004005
  » https://doi.org/10.1017/S1138741600004005
• Cohen, R. J., Swerdlik, M. E., & Sturman, E. D. (2014). Testagem e Avaliação Psicológica Porto Alegre: AMGH.
• CFP - Conselho Federal de Psicologia. (2003). Resolução CFP nº 007/2003 Retrieved from <https://site.cfp.org.br/wp-content/uploads/2003/06/resolucao2003_7.pdf>
  » https://site.cfp.org.br/wp-content/uploads/2003/06/resolucao2003_7.pdf
• Eysenck, M. W. (1992). Anxiety: the cognitive perspective East Sussex: Erlbaum.
• Fernandes, M. G., Nunes, S. A. N., Vasconcelos-Raposo, J., & Fernandes, H. M. (2014). Efeitos da experiência nas dimensões de intensidade, direção e frequência da ansiedade e autoconfiança competitiva: um estudo em atletas de desportos individuais e coletivos. Motricidade, 10(2), 81-89. doi: 10.6063/motricidade.10(2).2930
  » https://doi.org/10.6063/motricidade.10(2).2930
• Fernandes, M. G., Nunes, S., Vasconcelos-Raposo, J., Fernandes, H. M., & Brustad, R. (2013). The CSAI-2: an examination of the instrument’s factorial validity and reliability of the intensity, direction and frequency dimensions with Brazilian athletes. Journal of Applied Sport Psychology, 25(4), 337-391. doi: 10.1080/10413200.2012.744780
  » https://doi.org/10.1080/10413200.2012.744780
• Fernandes, M. G., Vasconcelos-Raposo, J., & Fernandes, H. M. (2012). Propriedades psicométricas do CSAI-2 em atletas brasileiros. Psicologia: Reflexão e Crítica 25(4), 679-687. doi: 10.1590/S0102-79722012000400007
  » https://doi.org/10.1590/S0102-79722012000400007
• Garcia, R. P., & Borsa, J. C. (2016). A prática da avaliação psicológica em contextos esportivos. Temas em Psicologia, 24(4), 1549-1560. doi: 10.9788/TP2016.4-20
  » https://doi.org/10.9788/TP2016.4-20
• Hanton, S., & Connaughton, D. (2002). Perceived control of anxiety and its relationship to self-confidence and performance. Research Quarterly for Exercise and Sport, 73(1), 87-97. doi: 10.1080/02701367.2002.10608995
  » https://doi.org/10.1080/02701367.2002.10608995
• Hanton, S., Neil, R., & Mellalieu, S. D. (2011). Competitive anxiety theory and research. In T. Morris, & P. C. Terry, Sport and exercise psychology: the cutting edge. Morgantown, W.V: Fitness Information Technology.
• Hanton, S., O’Brien, M., & Mellalieu, S. D. (2003). Individual differences, perceived control and competitive trait anxiety. Journal of Sport Behavior, 26(1), 39-55. Retrieved from <https://www.researchgate.net/profile/Stephen_Mellalieu/publication/284771775_Individual_differences_perceived_control_and_competitive_trait_anxiety/links/5720752108aeaced788adb7b/Individual-differences-perceived-control-and-competitive-trait-anxiety.pdf>
  » https://www.researchgate.net/profile/Stephen_Mellalieu/publication/284771775_Individual_differences_perceived_control_and_competitive_trait_anxiety/links/5720752108aeaced788adb7b/Individual-differences-perceived-control-and-competitive-trait-anxiety.pdf
• International Test Commission. (2017). The ITC guidelines for translating and adapting tests (second edition). International Journal of Testing, 18(2), 101-134. doi: 10.1080/15305058.2017.1398166
  » https://doi.org/10.1080/15305058.2017.1398166
• Jones, G. (1995). More than Just a game: research developments and issues in competitive anxiety in sport. British Journal of Psychology, 86(4), 449-478. doi: 10.1111/j.2044-8295.1995.tb02565.x
  » https://doi.org/10.1111/j.2044-8295.1995.tb02565.x
• Jones, G., & Hanton, S. (2001). Pre-competitive feeling states and directional anxiety interpretations. Journal of Sports Sciences, 19(6), 385-395. doi: 10.1080/026404101300149348
  » https://doi.org/10.1080/026404101300149348
• Jones, G., & Swain, A. (1995). Predispositions to experience debilitative and facilitative anxiety in elite and nonelite performers. The Sport Psychologist, 9(2), 201-211. doi: 10.1123/tsp.9.2.201
  » https://doi.org/10.1123/tsp.9.2.201
• Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174. doi: 10.2307/2529310
  » https://doi.org/10.2307/2529310
• Martens, R. (1977). Sport competition anxiety test Champaign, Human Kinetics.
• Martens, R., Burton, D., Vealey, R. S., Bump, L. A., & Smith, D. E. (1990). Development and validation of the Competitive State Anxiety Inventory-2. In R. Martens, R. S. Vealey & D. Burton (Eds.). Competitive Anxiety in Sports Champaign: Human Kinetic Books.
• Ohman, A. (2000). Fear and anxiety: evolutionary, cognitive, and clinical perspectives. In M. Lewis, & J. M. & Haviland-Jones, Handbook of emotions (pp.573-593). New York: Guilford Press, 573-593.
• Pacico, J. C. (2015). Como é feito um teste? Produção de Itens. Em: C. S. Hutz, D. R. Bandeira & C. M. Trentini (Eds.). Psicometria (pp. 55-70). Porto Alegre: Artmed.
• Pasquali, L. (2010). Instrumentação psicológica: fundamentos e práticas. Porto Alegre: Artmed.
• Primi, R., Muniz, M., & Nunes, C. H. S. S. (2009). Definições Contemporâneas de Validade de Testes Psicológicos. Em: C. S. Hutz (Ed.). Avanços e polêmicas em avaliação psicológica (pp. 243-265). São Paulo: Casa do psicólogo.
• Reppold, C. T., Gurgel, L. G., & Hutz, C. S. (2014). O processo de construção de escalas psicométricas. Avaliação Psicológica, 13(2), 307-310. Retrieved from <http://pepsic.bvsalud.org/pdf/avp/v13n2/v13n2a18.pdf>
  » http://pepsic.bvsalud.org/pdf/avp/v13n2/v13n2a18.pdf
• Rubio, K. (2011). A avaliação em Psicologia do Esporte e a busca de indicadores de rendimento. Em: L. F. Angelo, L. F., & K. Rubio (Eds.). Instrumentos de avaliação em Psicologia do Esporte (pp. 13-26). São Paulo: Casa do psicólogo.
• SATEPSI - Sistema de Avaliação de Testes Psicológicos. (2018). Lista completa dos testes Retrieved from http://satepsi.cfp.org.br/listaTeste.cfm
  » http://satepsi.cfp.org.br/listaTeste.cfm
• Silva, A. M. B., Foch, G. F. L., Guimarães, C. A., & Enumo, S. R. F. (2014). Instrumentos aplicados em estudos brasileiros em psicologia do esporte. Estudos Interdisciplinares em Psicologia, 5(2), 77-95. doi: 10.5433/2236-6407.2014v5n2p77
  » https://doi.org/10.5433/2236-6407.2014v5n2p77
• Smith, R. E., Smoll, F. L., Cumming, S. P., & Grossbard, J. R. (2006). Measurement of multidimensional sport performance anxiety in children and adults: the sport anxiety scale-2. Journal of Sport & Exercise Psychology, 28, 479-501. doi: 10.1123/jsep.28.4.479
  » https://doi.org/10.1123/jsep.28.4.479
• Smith, R. E., Smoll, F. L., & Schutz, R. W. (1990). Measurement and correlates of sport-specific cognitive and somatic trait anxiety: the sport anxiety scale. Anxiety, Stress & Coping, 2(4), 263-280. doi: 10.1080/08917779008248733
  » https://doi.org/10.1080/08917779008248733
• Souza, A. C., Alexander, N. M. C., & Guirardello, E. B. (2017). Propriedades psicométricas na avaliação de instrumentos: avaliação da confiabilidade e da validade. Epidemiologia e Serviços de Saúde, 26(3), 649-659. doi: 10.5123/s1679-49742017000300022
  » https://doi.org/10.5123/s1679-49742017000300022
• Spielberger, C. (1966). Anxiety and behavior. New York: Academic Press.
• Teixeira, K. C. (2016). Construção e busca de evidências de validade e precisão de uma medida de ansiedade para atletas (Dissertação de Mestrado). Programa de Pós-Graduação em Psicologia, Centro de Filosofia e Ciências Humanas, Universidade Federal de Santa Catarina, Florianópolis, Brasil.
• Teixeira, K. C., Nunes, C. H. S. S., & Cruz, R. M. (2016). Estudos psicométricos de instrumentos de ansiedade no contexto esportivo: uma revisão sistemática. Avaliação Psicológica, 15(3), 362-370. doi: 10.15689/ap.2016.1503.09
  » https://doi.org/10.15689/ap.2016.1503.09

Publication Dates

• Publication in this collection
  04 Aug 2021
• Date of issue
  Apr-Jun 2021

History

• Received
  21 Mar 2019
• Reviewed
  26 Apr 2020
• Accepted
  30 May 2020