VALIDATION OF THE LEARNING SITUATIONS SCALE FOR SPORTS COACHES - ESATE: DEVELOPMENT PROCESS AND VALIDITY EVIDENCE

VALIDAÇÃO DA ESCALA DE SITUAÇÕES DE APRENDIZAGEM PARA TREINADORES ESPORTIVOS - ESATE: PROCESSO DE ELABORAÇÃO E EVIDÊNCIAS DE VALIDADE

ABSTRACT

The objective of this study was to present the first pieces of evidence on the validity of the Learning Situations Scale for Sports Coaches [Escala de Situações de Aprendizagem para Treinadores Esportivos] (ESATE) - referring to content; model fit; and internal consistencies. Four evaluating judges participated in the content validation. In the other procedures, the ESATE was applied to 104 tennis coaches. Initially, the ESATE had 25 items divided into three dimensions: Mediated, Non-Mediated and Internal Situations. The items were evaluated in terms of clarity of language, practical relevance and theoretical dimensionality. Those items that did not reach desired levels were reformulated or reallocated into another dimension. All items presented satisfactory indices (CVC ≥ 0.80) for clarity of language and practical relevance, and “substantial”/”almost perfect” indices of agreement between judges (K ≥ 0.79) for theoretical dimensionality. The model that best fitted the available data was the second-order three-dimensional model with 18 items. It presented “reasonable”/”good” fit indices. The internal consistency of the scale was classified as “good” (α = 0.873), whereas the internal consistency of its dimensions was “weak”/”good” (0.676 ≤ α ≤ 0.801). Thus, it is possible to state that the final version of the ESATE showed satisfactory psychometric qualities for the assessment of learning situations when applied to tennis coaches.

Keywords:
Professional Development; Learning Situations; Coaches

RESUMO

O objetivo do estudo foi apresentar as primeiras evidências de validade da Escala de Situações de Aprendizagem para Treinadores Esportivos (ESATE) - relativas ao conteúdo; ao ajuste do modelo; e às consistências internas. Na validação de conteúdo participaram quatro juízes-avaliadores. Nos demais procedimentos, a ESATE foi aplicada a 104 treinadores de tênis. Inicialmente a ESATE possuía 25 itens divididos em três dimensões: Situações Mediadas, Não Mediadas e Internas. Os itens passaram por avaliação de clareza de linguagem, pertinência prática e dimensionalidade teórica. Aqueles itens que não atingiram níveis desejados passaram por uma reformulação, ou foram realocados em outra dimensão. Todos os itens apresentaram índices satisfatórios (CVC ≥ 0,80) para clareza de linguagem e pertinência prática, e índices “substanciais”/”quase perfeitos” de concordância entre juízes (K ≥ 0,79) para dimensionalidade teórica. O modelo que melhor se adequou aos dados disponíveis foi o tridimensional de segunda ordem com 18 itens. Apresentou índices de ajustamento “razoáveis”/”bons”. A consistência interna da escala foi classificada como “boa” (α = 0,873), e de suas dimensões foi “fraca”/”boa” (0,676 ≤ α ≤ 0,801). Assim, afirma-se que a versão final da ESATE demonstrou qualidades psicométricas satisfatórias para avaliação das situações de aprendizagem quando aplicada a treinadores de tênis.

Palavras-chave:
Desenvolvimento Profissional; Situações de Aprendizagem; Treinador

Introduction

The quality of a coach’s intervention is a determining factor for a positive or negative experience when it comes to children, youths and adults engaging in sports, both in participation and performance contexts¹. This recognition of the work done by coaches has triggered a global movement in search of the professionalization of this career², in which different institutions and agents have been working on the development/application of training programs that enable the professional development of coaches³.

This movement is followed by an increase in the interest from the scientific community in understanding the training and practice process of this professional⁴. A review conducted by Gilbert and Trudel⁵ provided the first insights on the theme at the international level and reported the prevalence of studies related to behavior, career development, and the thoughts/perceptions of sports coaches. Later, investigating only research concerning coach career development and referring to learning and professional development, Cushion et al.⁶ and He, Trudel and Culver⁷ found 36 studies that met the eligibility criterion (1993 to 2016), which were characterized, primarily, by the adoption of qualitative methodologies, with an emphasis on using interview as a data collection instrument.

In Brazil, research focusing on sports coaches is recent. A review study conducted by Galatti et al.⁸, referring to publications addressing the theme in the main national journals between 2000 and 2015, indicated that approximately 82% of the articles found were published as of 2009. Although there is a time mismatch in relation to the beginning of the works in the country, it is possible to observe that the national investigative agenda is now aligned with the international one, with a focus on studies aimed at analyzing the thoughts/perceptions and behaviors of coaches, as well as their career development, with prevalence of qualitative methodologies.

When it comes specifically to tennis, and in a broader search, presenting the “state of the art” of research on the modality in national journals, Cortela et al.⁹ found 102 articles published from 2000 to 2015, only three of which specifically addressed the perspective of coaches. These works fall into the category of career development, which includes studies related to learning and professional development.

If, on the one hand, the alignment with international research perspectives presents itself as an advance on the part of Brazilian researchers, on the other hand, it is clear that contributions in some fields and/or modalities, such as tennis, for instance, are still incipient, and that there is a lack of basic studies that can pave the ways to be walked on, whether in the sports or in the academic-scientific field. Viveiros et al.¹⁰ point out that the conduction of studies with descriptive design is a starting point for investigations, being considered an important phase for the development of research in Sports Science. In the view of these authors, this type of research founds and incites new questions that make it possible to carry out other studies that contribute to advances in the area.

Descriptive research can be developed by means of different approaches: qualitative, quantitative, or mixed¹¹. Particularly in national studies on coach career development, there is a prevalence of qualitative research, with interview as the main collection instrument. According to Galatti et al.⁸, this trend suggests that national researchers have been seeking to further explore the development of these professionals.

As main advantages, interviews have been proved to be more adaptable to the individual’s conditions and have a higher response rate. The fact that the interviewer is face to face with the interviewee allows the study questions to be reformulated, thus providing more pieces of information on the topic in subsequent questions¹². On the other hand, some factors limit the application of this instrument to large groups. The recognition of the interviewer’s identity by the interviewee and the time required to apply the interviews are some examples of these limitations¹³.

In a scenario where there is a need to obtain answers from large samples and in a wide geographic area, using questionnaires as an instrument for data collection presents itself as the most viable alternative for carrying out a research¹². Questionnaires can be divided into two categories: surveys and scales. Surveys consist of the elaboration of a series of questions related to a certain theme in order to explore and describe the participants’ perceptions and experiences, whereas scales are questionnaires composed of logically structured and combined questions that aim to establish an overall score for the items assessed¹⁴. According to these authors, the fundamental difference between surveys and scales lies in the psychometric properties, which must be proven and demonstrated on the scales.

In the present study, the discussions will target learning situations, an important element in a coach’s professional development process. Milistetd et al.¹⁵ shows that the concepts currently used in the international literature for this theme derive from general theories on learning, namely from the works of Jarvis^16,17 and Moon^18,19.

According to Milistetd et al.¹⁵, the operationalization of the concepts of mediated, non-mediated and informal situations, described by Moon¹⁹, relating them to the professional development of coaches, was performed by Werthner and Trudel²⁰. In mediated situations, the material and the learning context are not defined by the learner. In general, they are directed by an expert responsible for selecting the material and for the way in which it will be worked on. Conventional certification courses, seminars, workshops can be mentioned as examples of these situations. Non-mediated situations are understood as learning opportunities in which the learners themselves define the content and how the activities will be developed. Observing training, conversations with other coaches, group discussions, seeking solutions to real problems with the help of resources such as books, the internet, etc., are some examples of this type of learning situation. Finally, internal situations require that coaches reorganize their own knowledge, thus not producing new materials. Through deliberate reflections, based on previous experiences, coaches seek to reflect on a certain aspect, aiming at deepening learning^20,21.

To contribute with quantitative research that can lead to advances in the professional development of coaches, the present study intended to present the development process of the Learning Situations Scale for Sports Coaches (ESATE) and its first pieces of validity evidence - those referring to its content; the fit of the proposed model as to the available data; and the internal consistency of the scale as a whole, as well as its dimensions, for tennis coaches. The validation of this instrument may facilitate the access of researchers to coaches, allowing a larger number of professionals to be reached in a shorter period of time. The scale can also be used by people responsible for coach training programs so that they can recognize the specific preferences of certain groups of professionals, enabling the alignment of learning situations and strategies to be employed, according to the learner’s expectations.

Methods

The present study is divided into three phases: instrument content validation; confirmatory factor analysis; and internal consistency assessment.

Ethical Procedures

This study complied with the rules established by the National Health Council, being approved by the Ethics Committee of the Federal University of Rio Grande do Sul (CEP/UFRGS), under legal opinion No 1.890.257.

Instruments

The development of the ESATE was based on two classic works addressing the professional learning process of sports coaches, written by Werthner and Trudel²⁰, and by Trudel, Culver and Werthner²¹. These authors were responsible for the operationalization of the concepts described by Moon¹⁹ for the field. In their view, the term “learning situation” is associated with an individual’s own perception of learning, which can occur in a mediated, non-mediated, or internal manner²¹. In mediated situations, learning opportunities are not defined by the coach (learner), but controlled by other people - in general, a professional considered an expert in the field. Non-mediated situations are characterized by a more active role on the part of the coach, who defines the content and strategies that will be employed in learning. In their turn, internal situations are understood as opportunities in which the coach reorganizes previously acquired knowledge toward a deeper learning²⁰.

Based on this definition, the initial proposal for building the analytical matrix of the instrument presents a three-dimensional structure composed of the following professional learning situations: Mediated (eight items); Non-mediated (nine items); and Internal (eight items), totaling 25 items (ESATE-25). The items contained in each dimension were elaborated by the authors themselves, having as reference the conceptual framework previously described^20,21.

Figure 1
ESATE-25 Analytical Matrix

The ESATE is answered on a bidirectional Likert-type scale graded in five points. Starting from the guiding question “For your professional development, how important is it to participate in learning SITUATIONS such as those described below?”, the participants express their degree of agreement with a given statement, (1 = hardly important and 5 = extremely important). According to Rhind, Davis and Jowett¹⁴, Likert-type scales have been widely used to develop scales for research related to sports coaching.

In addition to this instrument, the coaches filled out a form containing these sociodemographic variables: age; sex; experiences as a coach and a player in the modality; initial training; and ongoing training.

Content Validation (CV) Procedure

The content validation (CV) process of the ESATE had as reference the proposals presented by Cassepp-Borges, Balbinotti and Teodoro²², and Hernández-Nieto²³. CV is a subjective evaluation done with the contribution of a group of evaluating judges. The purpose of this procedure is to verify whether the items present in the instrument are clear, relevant and aligned with the theoretical model employed. Through CV, it is also possible to identify, beforehand, possible flaws that could compromise the quality of the instrument²².

In this sense, each item in the instrument was evaluated by means of a specifically prepared spreadsheet, based on three criteria: Clarity of Language (CL); Practical Relevance (PR), and Theoretical Dimension (TD). For the first two criteria, the evaluation was carried out on a five-point Likert-type scale, in which 1 corresponds to “barely any”, and 5, to “very much” CL or PR. Based on the results found for each item, Hernández-Nieto²³ suggests calculating the Coefficient of Content Validity (CCV)²².

Sample

The study sample was composed of two groups of professionals: the evaluating judges, as experts responsible for evaluating the instrument from the CV perspective, filling out the specific spreadsheet; and the coaches, who effectively answered the ESATE, with the resulting data being used for the other analyses.

Evaluating Judges - CV

According to Cassepp-Borges, Balbinotti and Teodoro²², a group of evaluating judges plays a fundamental role in developing the instrument, by helping improve its quality in order to reduce its weaknesses and flaws. For CV analysis, a suggestion is to compose this group with three to five experts.

The present study counted on the collaboration of four evaluating judges, recognized by their notorious knowledge in the field of coach professional development. The experts in question were associated with different Brazilian public universities and responsible for a large portion of the national publications on the theme, besides having previous experience as coaches. With regard to initial training, they were all Physical Education and/or Sports graduates, with master’s degrees in the field. Three of these professionals held the title of PhD and were responsible for disciplines related to the research theme, in stricto sensu graduate programs. As suggested by Cassepp-Borges, Balbinotti and Teodoro²², the evaluating judges did not participate in any of the previous stages of the study.

Coaches

Concerning the coaches, the sample of the present study was exclusively made up of tennis professionals from clubs and gyms affiliated to the Tennis Federation of Paraná (FPT). The official website of the institution (http://www.fpt.com.br/Federacao/Clubes/Filiados) informs that, currently, there are 59 clubs or gyms duly registered. Through a personal contact made with the FPT on June 6, 2017, the latter disclosed that the estimated population of coaches working in these locations comprised 166 professionals. Thus, assuming a sampling error of five percentage points, a confidence level of 90%, the study sample was composed of 104 coaches.

As the main characteristics of this group, it is possible to observe a predominance of male coaches (95%), aged 35 years old or over (51.5%; x=35.5; SD=8.7), with more than 10 years in this career (58%; x=14.3; SD=9.4). The path taken during pre-professional socialization, a period that preceded the beginning of their career as coaches, is mostly associated with the role of ball catcher (46%), followed by contact with the Physical Education course (33%), and previous experience as players (21%). With respect to initial and ongoing training, there was a higher percentage of coaches with undergraduate degrees in Physical Education (56%) and who participated in training activities/courses in the last three years (92%). As for their technical level presented while playing the modality, half of the professionals reported being rated as second or third class by the FPT; whereas 37.5% claimed having scored in men’s or women’s professional rankings, or presenting a performance level compatible with that of the first class in the FPT, with the first being the highest level, and the eighth, the lowest one.

Data Collection Procedure

Evaluating Judges

Invitation for participation in the study was sent to the evaluating judges by email. After the professionals replied confirming their availability to analyze the instrument, the evaluation spreadsheet containing the scale items and the criteria to be observed (CL, PR and TD) was sent together with a guide containing the information necessary for it to be filled out, the deadline for return, and the Free and Informed Consent Form (FICF). Following the recommendations described in the consulted literature, the initial version submitted to the group of evaluating judges for CV had the 25 items of the scale randomized²².

Coaches

With the approval of the FPT, the coaches from affiliated clubs were contacted via email and invited to join the study. The invitation email contained the link for the survey to be taken online. The first page presented the FICF with all information relating to the study. To start filling out the instruments and participate in the survey, all coaches declared having read and agreed with the form. The estimated time to complete their participation was 25 minutes.

Data Analysis

Content Validity

Using the Excel software, version 2010, for Windows, the procedures referring to the CV of the instrument were carried out through the calculations proposed by Hernández-Nieto²³ and rectified by Balbinotti, Benetti and Terra²⁴ (Figure 2).

As suggested by Cassepp-Borges, Balbinotti and Teodoro²², indices greater than or equal to 0.80 (CCV_t≥0.80) for the CL and PR criteria were adopted as cutoff points. The items that did not reach this value were analyzed and reformulated, being submitted again to the group of evaluating judges in order to guarantee the quality of the instrument. For the TD criterion, the value suggested for Fleiss’ Kappa coefficient was greater than 0.7025.

Confirmatory Factor Analysis and Internal Consistency

For the Confirmatory Factor Analysis (CFA) of the initial suggested model of the instrument, the Mahalanobis squared distance was assessed first, in order to verify the normality of the cases and the presence of outliers, followed by an analysis of asymmetry (Skewness≤3) and kurtosis (Kurtosis≤10) to ascertain the normality of data distribution for each question. The results evidenced the presence of five coaches considered as outliers, who were removed for subsequent analyses to be performed, as suggested by Marôco²⁶.

Afterwards, with the aid of the IBM SPSS Amos 20.0 software, the tests and procedures referring to the CFA, proposed by Marôco²⁶, were applied through the Analysis of Structural Equations with the aim of evaluating the fit of the suggested model as to the available data. To establish the best fit for the final model of the instrument, Factor Loadings (FLs) and Modification Indexes (MIs) were assessed. The tests employed and their respective cutoff points were: Chi-square goodness of fit (X²), in which the lower the value, the better the quality of the fit; Chi-square degrees of freedom (X²/df), with values below 5 being considered acceptable; Goodness of Fit Index (GFI) and Comparative Fit Index (CFI) equal to or greater than 0.9; Penalty of the Comparative Fit Index (PCFI) and Penalty of the Goodness of Fit Index (PGFI) with values greater than 0.6; Root Mean Square Error of Approximation (RMSEA) with values below 0.08; and 90% Confidence Interval for the RMSEA (Prmsea). Goodness of fit was analyzed considering the values of the final model, obtained through the CFA, and those observed in the initial model, using the chi-square (X²) and the degrees of freedom (df) values.

Finally, the internal consistency of the instrument was evaluated by calculating Cronbach’s Alpha coefficient, which assesses the interrelationship between the items of one same dimension. The cutoff points adopted were those described by Hill and Hill²⁷, namely: α<0.6, unacceptable; 0.6<α<0.69, weak; 0.70<α<0.79, reasonable; 0.80<α<0.89, good; and α> 0.90, excellent.

Results

Content Validation

The overall result of the calculations for determining the CCV_t, executed for the CL criterion, satisfactorily attested to the clarity of the instrument, with 22 of the 25 items presenting CCV_c values greater than or equal to 0.90 (CCV_c≥0.90). In the first version sent to the evaluating judges, items 24 and 25 showed values below the cutoff point previously set for the study (Table 1). Based on the suggestions described by the experts in the “Note” field on the spreadsheet, these items were reformulated and sent back for evaluation. The modifications made improved the evaluators’ perception of the CL items (item 24 CCV_c≥0.80; and item 25 CCV_c≥0.95), making it possible to maintain them in the analytical matrix of the scale.

Table 1
CCV calculation for the CL and PR criteria, and Kappa coefficient for TD

Validation Criteria	CCVt	Reassessed and accepted CCVc
Clarity of Language (CL)	0.96	Item 24 (Internal Situation): 0.70 → 0.80 Item 25 (Non-Mediated Situation): 0.75 → 0.95
Practical Relevance (PR)	0.96	______
Theoretical Dimension (TD)	0.79	______

• Source: Prepared by the authors

For the PR criterion, the 25 items present in the initial version of the instrument were rated as satisfactory, with CCV_c ranging from 0.80 to 1, denoting the relevance of the items elaborated²².

Because it is a nominal variable, the TD was assessed by calculating the Kappa coefficient. The results were: (K=0.86) for the Mediated dimension; (K=0.87) for the Non-Mediated dimension; (K=0.80) for the Internal dimension; and (K=0.79) for the overall dimension assessment. Based on the results observed in the experts’ evaluation, item three (Q3), initially formulated for the Internal dimension, was reallocated, being considered for subsequent analyses as belonging to the Mediated dimension.

Confirmatory Factor Analysis and Internal Consistency

The CFA was performed having as starting point the three theoretical dimensions postulated by the literature, validated through the CV bias by the experts and attested to the 25 items of the scale (ESATE-25). A detailed analysis of the initial first-order model of the analytical matrix (1^st Round) showed that only the X²/df results and the PCFI index reached satisfactory scores²⁶ (Table 2).

Based on these results, the fitting process of the CFA model began in order to make it better fit the items present in the analytical matrix of the instrument. In the 2^nd Round, item Q6 was excluded from the analysis (Table 2) because the FC (0.17) presented by the question was unsatisfactory²⁸.

After this question was removed, six fitting rounds were performed until the model presented values deemed acceptable. The exclusions and correlations established had as criterion the indications described in the MIs and the recommendations proposed by Marôco²⁶. Thus, in the 3^rd Round, item Q25 was removed for presenting causal trajectories with items Q24 (MI: 22.849), Q18 (MI: 13.461) and Q14 (MI: 11.018), as well as presenting correlation between the errors of items Q24 (MI: 22.154) and Q18 (MI: 13.991). In the 4^th Round, item Q21 was removed for presenting a causal trajectory with item Q24 (MI: 14.278). In the 5^th Round, item Q14 was removed for presenting a causal trajectory with item Q17 (MI: 11.868), as well as presenting correlation with the error of item Q19 (MI: 20.019). In the 6^th Round, item Q17 was removed for presenting causal trajectories with items Q24 (MI: 12.704) and Q16 (MI: 11.638), as well as presenting correlation with the errors of items Q24 (MI: 11.723) and Q16 (MI: 16.916). In the 7^th Round, item Q18 was removed for presenting causal trajectories with item Q1 (MI: 10.362), in addition to assisting in the model, as the correlation between the errors of items Q15 and Q18 was eliminated, referring to the 4^th Round of the model fitting process. Finally, in the 8^th Round, item Q12 was removed from the analysis for presenting a causal trajectory (MI: 13.773) and correlation with the error (MI: 13.598) of item Q15 (Table 2).

It should be noted that items Q20 and Q22, from the Mediated and Non-Mediated dimensions, respectively, showed moderate correlation (r=0.42) (correlation inserted in the 3^rd Round), while items Q7 and Q19, from the Non-Mediated dimension, evidenced negative moderate correlation (r=-0.57) (correlation inserted in the 5^th Round) (Table 2).

Table 2
Fitting values for first-order Confirmatory Factor Analysis

Round	Actions Taken	X2 <best	X2/df <5	CFI >9	PCFI >6	GFI >9*	PGFI >6	RMSEA ≤0.08	P(rmsea) >0.05
1st	Initial Model	585.349	2.152	0.675	0.612	0.657	0.550	0.108	<0.001
2nd	Q6 removed	530.894	2.132	0.698	0.630	0.673	0.559	0.107	<0.001
3rd	Q25 removed - Q20-Q22 correlation	447.363	1.979	0.744	0.665	0.713	0.584	0.100	<0.001
4th	Q21 removed - Q15-Q18 correlation	388.327	1.904	0.771	0.681	0.744	0.600	0.096	<0.001
5th	Q14 removed - Q7-Q19 correlation	317.341	1.734	0.820	0.715	0.775	0.614	0.087	<0.001
6th	Q17 removed	260.301	1.587	0.860	0.743	0.806	0.629	0.077	0.009
7th	Q18 removed - Removed from Q18-Q15 correlation	233.179	1.586	0.863	0.742	0.816	0.632	0.077	0.012
8th	Q12 removed	188.733	1.452	0.900	0.765	0.836	0.635	0.068	0.091

• Note: X2 (Chi-square goodness of fit); X²/df (Chi-square degrees of freedom); CFI (Comparative Fit Index); PCFI (Penalty of the Comparative Fit Index); GFI (Goodness of Fit Index); PGFI (Penalty of the Goodness of Fit Index); RMSEA (Root Mean Square Error of Approximation); 90% Confidence Interval for the RMSEA (Prmsea)
• Source: Prepared by the authors

The final first-order model reached satisfactory indices after the elimination of seven questions: three referring to the Mediated dimension (Q14, Q17 and Q21); and two, to the Non-Mediated (Q12 and Q25) and the Internal (Q6 and Q18) dimensions, respectively. It is noteworthy that only the GFI showed a value below that considered “good”. However, the score obtained is not classified as inadequate fit²⁶.

As described in Figure 3, the final second-order results observed for the Learning Situations assessment model showed fitting values identical to those found in the first-order final analysis. Thus, it is evident that the final version of the fitted model is composed of 18 items (ESATE-18), namely: six referring to Mediated situations; seven to Non-Mediated situations; and five to Internal situations.

Figure 3
Final second-order model: Confirmatory Factor Analysis of the Learning Situations construct

It is important to stress that statistically significant differences were found between the results observed in the X² test and their respective degrees of freedom, comparing the initial and final fitting models (df: 272-130=142; fitted X²: 585.349-188.733=396.616; p <0.001). Such evidence showed the best fit of the final model compared to the initial one.

Finally, the following coefficients of internal consistency were found in the construct of the instrument: α of 0.676 for the Mediated dimension; α of 0.801 for the Non-Mediated dimension; and α of 0.783 for the Internal dimension. The α of the internal consistency of the overall assessment of the instrument was 0.873.

Discussion

Identifying a coach’s preference for a certain learning situation is an important point in the process of acquiring knowledge and skills. By promoting situations that meet the learner’s expectations, learning possibilities are broadened, since motivation is one of the triggers that boost this process¹⁹. The recognition of this importance has legitimized the study of learning situations as a field of academic investigation⁶.

The analysis of the CL criterion is performed to verify if the items of a certain instrument has a language that suits the characteristics of the population to be evaluated. According to the group of evaluating judges, the items contained in the ESATE were clear and understandable to the population of sports coaches for which it was designed. The CCV_c and CCV_t found satisfactorily complied with the recommendations described by Cassepp-Borges, Balbinotti and Teodoro²², which allowed maintaining all initially elaborated items in the scale.

In line with the findings for the CL, the PR of the items was rated as satisfactory by the group of experts. This criterion seeks to identify whether the items are actually important to assess the concept of interest to the point of remaining in the final structure of the instrument²².

Having as reference the cutoff points described by Landis and Koch²⁵, the values found for the Kappa coefficient proved to be adequate. For the Mediated and Non-Mediated dimensions, the agreement observed between the experts and the TD were classified as “almost perfect”, whereas for the Internal dimension and in the overall dimension assessment, the results were considered “substantial”.

The preference for using CFA instead of an exploratory factor analysis derived from the existence, in the literature, of a conceptual body that supported the three-dimensional structure for the assessment of the learning situations construct^20,21. In addition to this factor, the dimensions and questions of the instrument, created from a theoretical model, were evaluated by four experts in the field, who evidenced that the instrument had acceptable CL, PR and TD.

It is worth noting that due care was taken to present the results referring to the CFA in accordance with the recommendations of Kline²⁹ and Brown³⁰. They suggest the presentation of at least one absolute fit index (χ², χ²/df, GFI and PGFI), which allows assessing the similarity between the observed variance-covariance matrix and the estimated matrix; a parsimonious correction index (RMSEA and Prmsea), which is similar to the previous index, but performs a statistical correction that allows fixing possible initial model misfits; and a comparative fit index (CFI and PCFI), which allows assessing the fit of the hypothetical model through the bias of the null model, that is, covariances equal to zero.

With the exception of the result observed for the GFI, the other procedures employed in the CFA presented scores ranging from reasonable to good. However, it is worth highlighting that the GFI was not unacceptable²⁶. The hypothesis test showed significant differences between the values for X² and degrees of freedom presented by the initial and final models. These results mean that the final model (simplified, ESATE-18) fits better the existing correlational structure between the items than the initial model (original, ESATE-25)²⁶.

The questions removed during the analysis of the model had two characteristics. Specifically in the case of item Q6, the low factor loading²⁶ determined the exclusion, suggesting that the question categorized in the Internal dimension was not linked to the proposed initial model. For the other items excluded, the MIs were responsible for the elimination of the questions.

In the model fitting process, the questions related to the Mediated dimension, which expressed a passive role on the part of the coaches in the learning process, were suppressed. Despite the lower importance assigned by experienced coaches²¹, mediated situations can effectively contribute to the learning process, reducing the time necessary for knowledge acquisition and helping less experienced coaches to access quality materials that suit the moment they find themselves in their professional development, and referring to the skills required in the intervention^21,31.

Some items addressing discussions on topics of interest on the internet and access to support materials, such as books and articles as sources of knowledge, were suppressed from the Non-Mediated dimension. Despite the current scenario, in which online interactions by means of applications and social media have a strong presence in society, the coaches report daily discussions with peers as the main source of learning^32,33. Although they mention reading books as a source of knowledge acquisition, the same cannot be observed for scientific articles, which are little representative in this professional community^32,33.

The question removed based on the suggestion of the MI of the Internal dimension (Q18) referred to the role of critical reflection on one’s professional practice as a source of learning for the coach. Egerland, Nascimento and Both³⁴, when studying the professional competence of sports coaches in different modalities, evidenced that the lowest levels of mastery and importance attributed to the professional skills dimension were associated with self-reflection. The literature points out the difficulties that coaches have in deliberately engaging in reflective practices as a means to reorganize their cognitive structure and learning through reflection²¹. The absence of strategies and opportunities for involvement with this practice during the training process is the main element that contributes to this scenario²¹. According to Trudel, Gilbert and Rodrigue³¹, as professionals make progress in their continuum of professional development, they tend to give greater importance to internal situations, associated with reflection. However, this scenario lacks an in-depth analysis for tennis coaches, since the results reported by Cortela et al.³⁵ showed that these opportunities are less valued by the group of more experienced professionals and suggest the need for training programs to support reflective practices.

According to Marôco²⁶, the correlations found between items may point to other intervening factors not directly considered by the model, factors which, in general, derive from errors in the interpretation of the items. The moderate and negative correlation suggested between questions Q7 “Debates with coaches in which you, or your group, can define topics for discussion” and Q19 “Learning situations that can count on the participation of more experienced coaches, who will be able to debate and help you solve problems of interest to you, experienced in your work routine” in the Non-Mediated dimension, is an example of the multifactor character and complexity of a coach’s professional learning process. According to the literature, competitiveness causes exchanges of experiences and debates to occur less frequently in certain groups of coaches. In this sense, professional identity and age appear as factors to be considered in this relation^7,31.

In the specific case of the correlation between questions Q20 “Seminars in which content is passed on by a specialist in the field” and Q22 “Participate in conversations with other coaches on topics of interest”, belonging to the Mediated and the Non-Mediated dimensions, respectively, seem to be associated with the interpretation given by the coaches to the seminars. Despite being characterized as a typically mediated learning situation, in which content and learning environment, in general, are defined by a specialist who is responsible for controlling the entire process, seminars are held in a non-formal context²¹. In this sense, He, Trudel and Culver⁷ emphasize that seminars are seen by some coaches as a space to meet and exchange information, especially with more experienced professionals, who seem to be more open to making these exchanges happen.

Finally, the overall assessment of the internal consistency of the instrument was classified as “good”. Isolating the dimensions, the resulting classifications were considered as “weak”, but acceptable, for Mediated, “good” for Non-Mediated, and “reasonable” for Internal²⁷.

Conclusions

Despite the limitation for more robust analyses, as a consequence of the size of the population of coaches eligible to participate in the study, it is concluded that the fitted model of the ESATE, with 18 items, showed satisfactory psychometric qualities for the assessment of learning situations through biases concerning content validity (ESATE-25), the goodness of fit presented by the final second-order model, and internal consistency (ESATE-18), for application to tennis coaches from Paraná.

It is worth noting that the dimensions contained in the Learning Situations construct derive from a theoretical framework widely used in the international literature, facilitating dialogues with works carried out in other countries. The generic character of the items that compose the scale is an alert to the possibility of testing it and subjecting it to different validation procedures, for samples of sports coaches in general. This scenario may contribute to raising the number of quantitative investigations in the field, promoting access to a larger number of professionals and accelerating the mapping of information relating to the national context.

References

• ¹
  Lefebvre JS, Evans MB, Turnnidge J, Gainforth HL, Côté J. Describing and classifying coach development programmes: A synthesis of empirical research and applied practice. Int J Sports Sci Coach 2016;11(6):887-899. Doi: 10.1177/1747954116676116.
  » https://doi.org/10.1177/1747954116676116
• ²
  Duffy P, Hartley H, Bales J, Crespo M, Dick F, Vardhan D, Nordmann L. et al. Sport coaching as a 'profession': challenges and future directions. Int J Sports Sci Coach 2011;5(2):93-124.
• ³
  International Council for Coaching Excellence [Internet]. ICCE's Standards for Higher Education Bachelor Coaching Degree Programmes - Consultation Draft; 2016 [acesso em 05 mar 2018]. Disponível em: https://www.icce.ws/_assets/files/icds-draft-4-final-november-23.pdf
  » https://www.icce.ws/_assets/files/icds-draft-4-final-november-23.pdf
• ⁴
  Trudel P, Gilbert W, Werthner P. Coach Education Effectivenes. In: Lyle J, Cushion CJ, editors. Sports Coaching: Professionalization and Practice. London: Elsevier; 2010, p. 135-152.
• ⁵
  Gilbert WD, Trudel P. Analysis of coaching science published from 1970-2001. Res Q Exerc Sport 2004;75(4):388-399. Doi: 10.1080/02701367.2004.10609172.
  » https://doi.org/10.1080/02701367.2004.10609172
• ⁶
  Cushion C, Nelson L, Armour K, Lyle J, Jones R, Sandford R, et al. Coach learning and development: a review of literature. London: Leeds Sports Coach UK; 2010.
• ⁷
  He C, Trudel P, Culver D. Actual and ideal sources of coaching knowledge of elite Chinese coaches. Int J Sports Sci Coach 2018;13(4):496-507. Doi: 10.1177/1747954117753727.
  » https://doi.org/10.1177/1747954117753727
• ⁸
  Galatti LR, Bettega OB, Brasil VZ, Sobrinho AEPS, Bertram R, Tozetto AVB, et al. Coaching in Brazil sport coaching as a profession in Brazil: An analysis of the coaching literature in Brazil from 2000-2015. ISCJ 2016;3:316-331. Doi:10.1123/iscj.2014-0103.
  » https://doi.org/10.1123/iscj.2014-0103
• ⁹
  Cortela CC, Gonçalves GHT, Klering RT, Balbinotti CAA. O "estado da arte" das publicações sobre tênis em periódicos nacionais. Coleç Pesqui Educ Fís 2016;15(2):143-151.
• ¹⁰
  Viveiros L, Moreira A, Bishop D, Aoki MS. Ciência do esporte no Brasil: Reflexões sobre o desenvolvimento das pesquisas, o cenário atual e as perspectivas futuras. RBEFE 2015;29(1):163-75. Doi: 10.1590/1807-55092015000100163.
  » https://doi.org/10.1590/1807-55092015000100163
• ¹¹
  Gaya A. Ciências do movimento humano: Introdução à metodologia da pesquisa. Porto Alegre: Artmed; 2008.
• ¹²
  Thomas JR, Nelson JK, Silverman SJ. Métodos de pesquisa em atividade física. 6. ed. Porto Alegre, Artmed; 2012.
• ¹³
  Marconi MA, Lakatos EM. Fundamentos da metodologia científica. 7. ed. São Paulo: Atlas; 2010.
• ¹⁴
  Rhind D, Davis L, Jowett S. Questionnaires. In: Nelson L, Groom R, Potrac P, editores. Research methods in sports coaching. Abingdon: Routledge; 2014, p.111-122.
• ¹⁵
  Milistetd M, Duarte T, Ramos V, Mesquista IMR, Nascimento JV. Aprendizagem profissional de treinadores esportivos: desafios da formação inicial universitária em educação física. Pensar Prát 2015;18(4):982-994. Doi: 10.5216/rpp.v18i4.34988.
  » https://doi.org/10.5216/rpp.v18i4.34988
• ¹⁶
  Jarvis P. Towards a comprehensive theory of learning. London: Routledge; 2006.
• ¹⁷
  Jarvis P. Learning to be a person in society. London: Routledge; 2009.
• ¹⁸
  Moon JA. Short courses and workshops: improving the impact of learning, training and professional development. London: Kogan Page; 2001.
• ¹⁹
  Moon J. A handbook of reflective and experiential learning: theory and practice. London: Routledge Falmer; 2004.
• ²⁰
  Werthner P, Trudel P. A new theoretical perspective for understanding how coaches learn to coach. Sport Psychol 2006; 20:198-212. Doi: 10.1123/tsp.20.2.198.
  » https://doi.org/10.1123/tsp.20.2.198
• ²¹
  Trudel P, Culver D, Werthner P. Looking at coach development from the coach-learner's perspective: considerations for coach development administrators. In: Potrac P, Gilbert W, Denison J, editores. Routledge handbook of sports coaching. Abingdon: Routledge; 2013, p. 375-387.
• ²²
  Cassepp-Borges V, Balbinotti MAA, Teodoro MLM. Tradução e validação de Conteúdo: Uma proposta para a adaptação de instrumentos. In: Pasquali L, editor. Instrumentação psicológica: Fundamentos e práticas. Porto Alegre: Artmed; 2010, p. 506-520.
• ²³
  Hernández-Nieto R. Contributions to statistical analysis. Mérida, España: Los Andes University Press; 2002.
• ²⁴
  Balbinotti MAA, Benetti C, Terra PRS. Translation and validation of the Graham-Harvey survey for the Brazilian context. IJMF 2007;3(1):26-48. Doi: 10.1108/17439130710721644.
  » https://doi.org/10.1108/17439130710721644
• ²⁵
  Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159-174.
• ²⁶
  Marôco J. Análise de equações estruturais - fundamentos teóricos, software e aplicações. 2.ed. Pêro Pinheiro: ReportNumber; 2014.
• ²⁷
  Hill M, Hill A. Investigação por Questionário. Lisboa: Edições Sílabo; 2000.
• ²⁸
  Hair K, Black W, Babin B, Anderson R, Tatham R. Multivariate data analysis. New Jersey: Pearson Education; 2005.
• ²⁹
  Kline RB. Principles and practice of structural equation modeling. 4.ed. Nova Iorque: The Guilford Press; 2015.
• ³⁰
  Brown TA. Confirmatory factorial analysis for applied research. 2.ed. New York: The Guilford Press; 2015.
• ³¹
  Trudel P, Gilbert W, Rodrigue F. The journey from competent to innovator: Using Appreciative inquiry to enhance high performance coaching. AI Practitioner 2016;18(2):40-46. Doi: 10.12781/978-1-907549-27-4-5.
  » https://doi.org/10.12781/978-1-907549-27-4-5
• ³²
  Stoszkowski J, Collins D. Sources, topics and use of knowledge by coaches. J. Sports Sci 2016; 34(9):794-802. doi: 10.1080/02640414.2015.107¬79.
  » https://doi.org/10.1080/02640414.2015.107¬79
• ³³
  Gonzalez-Rivera M, Campos-Izquierdo A, Villalba A, Hall, N. Sources of knowledge used by Spanish coaches: A study according to competition level, gender and professional experience. Int J Sports Sci Coach 2017; 12(2):162-174: Doi: 10.1177/1747954117694733.
  » https://doi.org/10.1177/1747954117694733
• ³⁴
  Egerland EM, Nascimento JV, Both J. Nível de associação entre importância atribuída e competência percebida de treinadores esportivos. Pensar Prát 2009;12(3):1-13. Doi: 10.5216/rpp.v12i3.7621.
  » https://doi.org/10.5216/rpp.v12i3.7621
• ³⁵
  Cortela CC, Milistetd M, Both J, Fuentes JP, Balbinotti CAA. Desenvolvimento profissional de treinadores de tênis: situações e contextos de aprendizagem. Retos 2020;38:700-707.

Publication Dates

• Publication in this collection
  13 Nov 2020
• Date of issue
  2020

History

• Received
  23 Jan 2019
• Reviewed
  04 June 2020
• Accepted
  20 June 2020