Examining the validity of the Athlete Engagement Questionnaire (AEQ) within a Portuguese sport setting

Sport psychology literature suggests that understanding engagement levels is pivotal to promote positive sporting experiences among athletes. The purpose of this study was to examine the psychometric properties of the Athlete Engagement Questionnaire among Portuguese sport athletes. Two distinct samples of Portuguese athletes from different competitive levels were collected, and the results of a confirmatory factor analysis demonstrated a good fit of the model to the data. A review of the psychometric properties indicated that all factors showed good composite reliability, convergent validity, and discriminant validity. In addition, a multi-groups analysis showed the invariance of the model in two independent samples providing evidence of cross validity. Implications of these results for scholars and coaches are discussed and guidelines for future studies are suggested.


Introduction
There is a growing body of literature focusing on engagement in different research settings (Hodge, Lonsdale, & Jackson, 2009;Martin, 2008;Maslach & Leiter, 1997).Engagement is a psychological construct that describes 'energy in action' in the form of a link between a person and an activity (Russel, Ainley, & Freydenberg, 2005).Previous studies suggest that en-gagement can be seen as a potential 'meta-construct,' gathering separate lines of research under a single conceptual model (e.g., Hodge et al., 2009).This 'meta-construct' is featured by having three components: behavioral (i.e., positive conduct, effort or participation), emotional (i.e., interest, belonging, and positive attitude about learning), and cognitive.(i.e., self-regulation, learning goals, investment in learning) (Appleton, Christenson, Kim, & Reschly, 2006;Fredricks, Blumenfeld, & Paris, 2004).
In sport settings, studying athlete engagement (AE) assumes a pivotal importance due to its relationship with the level of performance (Russell et al., 2005).The concept of AE is often described as a result of an enduring, relatively stable sport experience, which refers to generalized positive affects and cognitions about the individuals' involvement in the practice of a sport (Lonsdale, Hodge, & Jackson, 2007;Lonsdale, Hodge, & Raedeke, 2007).However, with few exceptions, little empirical research has been conducted in the sports field to understand the role of athletes' engagement in a sport environment.Of the few studies about this topic, Hodge et al. (2009) focused on an elite sport context and identified the antecedents (basic psychological needs) and consequences (dispositional flow) of athlete engagement.Notwithstanding, it is important to further explore the applicability of the engagement concept with athletes involved in different levels of competition.Prior studies highlighted that expert performance results from a long-term systematic engagement in a deliberate practice in a domain (Ericsson, Krampe, & Tesch-Römer, 1993;Martin, 2008).In this sense, examining the role of engagement of athletes in different competitive levels may prove to be vital to understand how they are driven to master skills, and continue to practice their sports (Liem & Martin, 2012;Martin, 2008) The majority of previous studies has focused on the more observable indicators of engagement levels (e.g.Furrer, Skinner, Marchand, & Kindermann, 2006;Zyngier, 2008).The measurement of engagement through athletes' behavior is a highly deductive method, and this procedure does provide precision in the definition of the engagement construct (Reschly & Christenson, 2006).In this sense, a focus on a more inductive approach (e.g.self-reported measures of athletes' cognitive engagement) may be important to better understand the multidimensional nature of engagement in sport environments (Appleton et al., 2006).Consequently, in order to improve previous theoretical conceptualizations, it is necessary to rely on empirical measurements of engagement according to the athletes' sense of experience and its meaning within a specific sport context (Appleton, et al., 2006;Connell, Halpern-Felsher, Clifford, Crichlow, & Usinger, 1995).In line with this concept, recent studies have increasingly focused on the development of measurement tools to assess athletes' perceived engagement with sports activities (Lonsdale, Hodge, & Jackson, 2007;Lonsdale, Hodge, & Raedeke, 2007).For example, Lonsdale et al. (2007) conducted an exploratory study using New Zealand elite athletes, and developed the Athlete Engagement Questionnaire (AEQ) consisting of four dimensions: confidence, dedication, vigor, and enthusiasm.In the subsequent study, Lonsdale, Hodge, and Raedeke (2007) examined the proposed factor structure using a larger sample of New Zealand and Canadian elite athletes.The AEQ revealed good psychometric properties proving to be a reliable tool to assess engagement in sport competitive environments, as well as to understand the relationship between burnout and engagement in athletes.Notwithstanding, the authors suggested that future research using the AEQ should be applied in different contexts in order to further validate the instrument, given that cultural differences may limit item interpretation and factor content.
A focus on reliable psychological instruments analyzing athletes' mental skills is a pre-requisite in the sport-related literature (Chartrand, Jowdy, & Danish, 1992;Salmela, Monfared, Mosayebi, & Durand-Bush, 2008).Thus, considering previous research (Lonsdale et al., 2007;Lonsdale, Hodge, & Raedeke, 2007), the current study uses the AEQ to examine athletes' engagement in a different sport environment.Specifically, the purpose of this study was to test the AEQ in a Portuguese athletes' sport setting.Therefore, we intend to further validate the AEQ instrument to the Portuguese language, given that every new application of the AEQ is a contribution to its validity consummation, and offers an important theoretical value (Baric & Horga, 2006).Additionally, it may aid sport psychologists and coaches in offering more positive sporting experiences (Hodge et al., 2009;Vecina, & Chacón, 2013), and skillfully guide athletes in a safer and more effective manner (Erikson, 2007).

Participants and data collection
The research sample consisted of two convenience groups and the participants were selected from different competitive levels (i.e.elite, national and regional levels), sports and regions of Portugal.All participants accepted voluntarily to participate in the study under the guarantee of anonymity of their responses.The total number of participants was 771 athletes, and all participants (and their parents when appropriate) were instructed about the purpose and procedures of the study before to fill an informed consent.Questionnaires were distributed in two separate moments.In each moment, a total of 500 questionnaires were distributed to the athletes before their training sessions.The questionnaires were self-administered and completion took approximately 12 minutes.After data screening, in a first moment, a total of 357 were deemed usable for data analyses.The mean age of the first sample was 20.2 years old (SD=6.28),and little more of two third were males (79.6%).Regarding the second moment, a total of 414 questionnaires were deemed usable for data analysis, with participants mean age of 17.2 years old (SD=4.63),and a great majority (i.e., about two thirds) were males (81.2.6%).It is important to note that in both data collections, samples sizes exceeded the minimum (n=200) recommended for structural equation modelling with maximum likelihood estimation (Hair, Black, Babin, & Anderson, 2009).

Measures
The original Athlete Engagement Questionnaire (AEQ) comprises sixteen items distributed in four dimensions, namely: confidence, dedication, enthusiasm, and vigor (see Table 1).
Respondents are asked to indicate how they felt in the past three months through a 5-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree).Sample items include, for example: 'I believe I am capable of accomplishing my goals in sport' (confidence); 'I am determined to achieve my goals in sport' (dedication); 'I feel excited about my sport' (enthusiasm); and 'I feel really alive when I participate in my sport' (vigor).The AEQ scale was translated to Portuguese and then back-translated to English to minimize discrepancies between the original and the translated version (Banville, Desrosiers, & Genet-Volet, 2000).Specifically, the AEQ instrument was first translated into Portuguese by two of the researchers in cooperation with an experienced Portuguese sport psychologist.Next, to test the equivalence of the items, back-translation into English was carried out by two natives of Portugal that are fluent in English.To verify the accuracy of the items, a bilingual expert was asked to assess differences in meaning between the original items and the back-translated items.The comparison of the two forms leaded to the conclusion that the two instruments were conceptually equivalent.

Data analysis
Data were analyzed using AMOS 21.0, and a confirmatory factor analysis (CFA) was performed to assess the psychometric properties of the AEQ instrument.Internal consistency of the constructs was assessed through composite reliability (Hair, et al., 2009).Convergent validity was evaluated through the average variance extracted (AVE), while discriminant validity was established when AVE for each construct exceeded the squared correlations between that construct and any other (Fornell & Larcker, 1981).The appropriateness of the model was tested using a variety of indices.Specifically, the measurement model was assessed with chi-square (χ²) statistical test, the ratio of χ² to its degrees of freedom, comparative -of-fit-index (CFI), goodness-of-fit index (GFI), parsimony comparative-of-fit-index (PCFI), parsimony goodness-of-fit index (PGFI), and root mean square error of approximation (RMSEA).The statistical significance was assumed at a .05level.Additionally, a multi-group analysis was conducted to compare the first sample with a second sample in order to assess cross validity.The model's invariance in both samples was tested by comparing the unconstrained model with constrained models (factor loadings fixed and variances/covariances fixed).Factorial invariance was accepted when the models did not differ significantly (p>0.05),according to the χ² statistic (Loehlin, 2003;Marôco, 2010).

Confidence
Represents a belief in one's ability to attain a high level of performance and achieve desired goals.

AEQ model
The results of the confirmatory analysis for each engagement factor are reported in Table 2.All estimated factor loadings exceed the cut-off point of .50(Hair et al., 2009), ranging from 0.67 to 0.88.Also, the Z-values ranged from 13.74 to 20.20.These results indicate that each item did load significantly on its construct.All composite reliability values exceeded the recommended minimum of 0.60 (Bagozzi & Kimmel, 1995) ranging from 0.85 to 0.88.Convergent validity was accepted for all constructs given the AVE values of each construct showed accepted levels and ranged from .58 to 0.66, with a mean of .62 (Fornell & Larcker, 1981).In addition, the results obtained for the measurement model demonstrated an acceptable fit to the data [χ²=319.38,p<0.01; χ²/df=3.26,PCFI=0.77,PGFI=0.66,CFI=0.95,GFI=0.91,RMSEA=0.074].Although the χ² was significant, this statistic result is sensitive to sample size, and the study was conducted with a large sample (n=771).However, both PCFI and PGFI were above the cut-off point of 0.60, while the CFI and GFI were higher than 0.90 (Hair et al. 2009).In addition, the RM-SEA value indicated an acceptable fit (Byrne, 2000).
Descriptive statistics for each constructs and its correlations are presented in Table 3. 'Enthusiasm' was the construct with the highest mean score (M=4.60,SD=0.58) while 'Confidence' had the lowest mean score (M=3.91,SD=0.81).Evidence of discriminant validity was accepted since none of the squared correlations exceeded the AVE values for each associated construct.

Discussion
The main goal of the current study was to extend previous research on athletes' engagement to the validation of the AEQ to the Portuguese sport competitive environment.Given the fact that every new application of a measurement instrument represents a contribution to improve the theoretical value of the research domain (Barić & Horga, 2006), this study extends the body of knowledge by confirming the validity of the AEQ instrument in a different research setting, as well as by improving the knowledge of how to aid sport psychologists and coaches to manage athletes' engagement.
The results of the CFA revealed that the four-factor structure proposed to assess athletes' engagement, including Confidence, Dedication, Vigor and Enthusiasm, showed a good fit to the data.In addition, all constructs showed good psychometric properties.These findings are consistent with previous research using the AEQ (Lonsdale et al., 2007, Lonsdale, Hodge, & Raedeke, 2007), and they confirm the importance of each of the four constructs in the understanding of the engagement levels of athletes.Also, there is another important finding to highlight about the AEQ that represents a contribution to the sports-related literature.Specifically, the model's invariance in two independent samples was supported, indicating cross validity.This is a pivotal step when evaluating psychometric properties of a measurement scale (Marôco, 2010), and this procedure has been reported when validating scales in recent studies conducted in the sports scenario (e.g., Alvarado, Sandín, Valdez-Medina, González-Arratia, & Rivera, 2012;Biscaia, Correia, Ross, Rosado, & Mâroco, 2013;Cabrita, Rosado, Leite, & Sousa, 2013;Theodorakis, Dimmock, Wann, & Barlas, 2010).population to better understand engagement among athletes.

Constructs
Additional samples with athletes from other countries could also be collected to compare the validity of the scale in different cultural contexts.Consistent with this view, Bollen (1989) refers that it is important to compare the fit of the model with the fit of prior research models, and the fit indices produced in the current study are comparable to the previously established fit indices (e.g.Lonsdale et al., 2007).Thus, the analysis of the AEQ in longitudinal studies using samples from different countries may constitute a step forward to aid scholars improving their understanding of the role of engagement in sports.
Second, the athletes' engagement construct was assessed with four dimensions, namely, confidence, dedication, vigor, enthusiasm.However, previous literature suggests that the existence of other important dimensions, such as enjoyment and preoccupation (Lonsdale et al., 2007a).As such, the insertion of these dimensions in futures empirical studies using the AEQ might be an important contribution to better understand the concept of athletes' engagement.
Third, while the current study focused on measuring the dimensions of engagement in sports settings, the antecedents and consequences of athletes' engagement were not systematically examined.According to Vallerand, Mageau, Elliot, Dumais, Demers, and Rousseau (2008), high levels of motivation are believed to be central at improving engagement.Also, Hodge et al. (2009) and Londsdale, Hodge, and Raedeke (2007) observed that burnout can be a consequence of low levels of engagement within a sport.In addition, the engagement levels have been suggested to be linked to the climate perceptions namely the quality of human relations (Busseri, Constain, Campbell, Rose-Krasnor, & Evans, 2011).In this sense, examining the relationship between engagement and athletic identity may also represent an important topic for future studies.Thusly, the simultaneous analysis of athletes' engagement, previous factors, and consequences could represent an important step in the pursuit of how sport participation and performance of athletes can be enhanced.
Engagement plays an important role for desirable outcomes that require emotional adjustments when the issue is to achieve high levels of performance by both elite athletes and beginners (Liem & Martin, 2012;Martin, 2008;Oliver, Hardy, & Mardland, 2010).Thus, findings from this study provide scholars and coaches with a tool to aid them in managing their athletes' levels of engagement.Previous studies suggest that expert performance in sport is the result of long-term engagement in a highly specialized form of training named deliberate practice (Maehr & Meyer, 1997;Wall & Coté, 2007).Furthermore, when an athlete's primary goal is to master his or her sport, it is more likely that his/her motivations will increase (Martin, 2007(Martin, , 2010)).Thus, from the scholars' point of view, the results of the current study may represent an important basis for the development of studies with a multidimensional framework including both motivations and engagement, in order to better understand an individual's energy and his/her drive to learn (Liem, & Martin, 2012;Martin, 2008).From the coaches' standpoint, this study provides a tool to assess engagement with practical implications in the improvement levels of confidence, dedication, enthusiasm, and vigor of athletes.Additionally, there is a research line suggesting that a large amount of sport-specific practice at a young age could lead to high levels of burnout among athletes in different stages of their sport careers (Hodge et al., 2009;Lonsdale et al., 2007, Lonsdale, Hodge, & Raedeke, 2007;Schaufeli, Salanova, González-Roma, & Bakker, 2002).Furthermore, studying AE could also be important for the understanding of how to avoid low levels of performance, enhanced pressure to win, and even dropout from sports (Liem & Martin, 2012).
According to the literature about AE, the study of engagement experiences among athletes at sub-elite levels may also be crucial to understand sport participation (Allender, Cowburn, & Foster, 2006;Ullrich-French & Smith, 2009).That is, studying the feelings of confidence in one's ability to perform at a high level (confidence), how much dedicated athletes are to sacrifice themselves to keep doing hard training (dedication), the physical, mental, and emotional energy of the athletes (vigor), and their high levels of enjoyment (enthusiasm) may contribute to extend the sport career of sub-elite athletes up to the elite level of participation (Coté, Horton, MacDonald, & Wilkes, 2009;Fraser-Thomas, Côté, & Deakin, 2008;Wall, & Côté, 2007).Given the importance of enhancing athletic performance and athletes' involvement since early ages, this measurement instrument examined in the current study would also be of pivotal interest to potentiate the transition from the sub-elite levels to elite levels of participation (Lonsdale et al., 2009;Salmela et al., 2008).Thus, the study of AE levels may contribute to draw a measurement of the athletes' risk of developing maladaptive emotions related to the sport setting (Hodge et al., 2009;Oliver et al., 2010;Trew, Kremer, Gallagher, Scully, & Ogle, 1997).

Limitations and future research
As with any study, this research has limitations that should be considered for future research.First, the study sample was not representative of the Portuguese athletes' population, and as such the results may lack generalizability.Thus, future studies should collect larger and stratified samples of the target

Table 1 .
Dimensions, description and corresponding items of the original version of the Athlete Engagement Questionnaire(Lonsdale,  Hodge, & Jackson , 2007, p.472).

Table 4 .
Results of the multi-group CFA models across the first and second application of the Portuguese Athletic Engagement Questionnaire.

Table 3 .
Mean (M), standard deviation (SD) and Squared Correlations between the first order constructs or among constructs.