Design , validation , and reliability of survey to measure female athlete triad knowledge among coaches

The purpose of this study was to design and to test the validity and reliability of an instrument to evaluate coaches’ knowledge about the female athlete triad syndrome and their confidence in this knowledge. The instrument collects information regarding: knowledge of the syndrome, components, prevention and intervention; confidence of the coaches in their answers; and coach’s characteristics (gender, degree held, years of experience in coaching females, continuing education participation specific to the syndrome and its components, and sport coached). The process of designing the questionnaire and testing the validity and reliability of it was done in four phases: a) design and development of the instrument, b) content validity, c) instrument reliability, and d) concurrent validity. The results show that the instrument is suitable for measuring coaches ́ female athlete triad knowledge. The instrument can contribute to assessing the coaches ́ knowledge level in relation to this topic.


Introduction
The "female athlete triad" syndrome involves three components: low energy availability, sometimes associated with disordered eating, menstrual dysfunction, and low bone mineral density (Barrack, Ackerman, & Gibbs, 2013;Hobart, 2000;Otis, Drinkwater, Johnson, Loucks, & Wilmore, 1997;Smith, 1996).This syndrome often starts when an athlete consciously or unconsciously decreases her caloric intake or increases her energy expenditure (Sundgot-Borgen, 1994).As a result of low energy availability, menstrual function is altered (Barrack, Ackerman, & Gibbs, 2013;Otis et al., 1997;Rust, 2002).The decreased estrogen level leads to lower bone mineral density, osteopenia or osteoporosis.The female athlete triad syndrome can have serious long-term consequences (e.g.pathogenic behaviors, full-fledged eating disorders, hormonal issues, and loss of bone density) for which treatment can be difficult (Thompson & Gabriel, 2004).The ideal treatment plan is multdisciplinary in nature.Thus, prevention is widely regarded as the key to combating the syndrome (Beals, Brey, & Gonyou, 1999;Joy, et al., 1997a;Rust, 2002).Prevention is where coaches can play an important role.Coaches create the training environment and they are a key piece in preventive measures or screening (Eagle, Lohman, & Jarman, 2013;Joy et al., 1997a;Sherman, Thompson, Dehass, & Wilfert, 2005).
The female athlete triad syndrome is a relatively new issue.It is not clear what coaches know about the syndrome (low energy availability, menstrual dysfunction, and low bone mineral density), its effects, risk factors, or their roles as coaches in the prevention of this syndrome.Along these lines, some national governing bodies have considered mandatory education for coaches (governing bodies not cited; Joy et al., 1997b).Previous research has been focused on coaches´ knowledge regarding nutrition, parts of the triad, or their attitudes or actions (Froiland, Koszewski, Hingst, & Kopecky, 2004;Jacobson, Sobonya, & Ransone, 2001;Joy et al., 1997a;Juzwiak & Ancona-Lopez, 2004;Pantano, 2006;Sossin et al., 1997;Troy, Hoch, & Stavrakos, 2006).These studies shows coaches are aware of the menstrual problems of their athletes and identified more symptomatic athletes (Sherman, et al., 2005) but also that they recommend dangerous weight control practices (Juzwiak & Ancona-Lopez, 2004).Further, some coaches do not have enough knowledge about the syndrome (Troy, et al., 2006;Pantano, 2006).Thus, it is not clear whether coaches are prepared to provide information and to set proper and safe training environments for their athletes and teams.
Information regarding what coaches know about the syndrome is critical for establishing the need for specific training on the issue and the characteristics of the training program.Most instruments used to measure coaches´ knowledge involve asking coaches about the perceptions of their knowledge, including few questions about the syndrome, or they focus on one of the components (low energy availability, menstrual dysfunction, or low bone mineral density) or aspects (effects, signs, or recognition) of the syndrome.Carrying out educational programs and monitoring coaches´ knowledge needs require an instrument to measure this knowledge at different levels, such as the syndrome, its components (characteristics and effects), how to prevent it, and how to intervene.This instrument will allow us to establish whether coaches have the knowledge and training to assist in preventing and referring this problem or whether their specific continuing education program is working.To best estimate the coaches´ receptiveness toward receiving training, the effect of the program, or the measurement of the application of the coaches' knowledge, an instrument should also evaluate the confidence that coaches have in their knowledge (Turk, Prentice, Chappell, & Shields Jr,. 1999;Vaughan, King, & Cottrell, 2004).Without confidence in one´s knowledge, the step from knowledge to application is more difficult or even dangerous, as without accurate knowledge they can create risky situations (Wozney, Venkatesh, & Abrami, 2006).This instrument can provide information about the factors influencing coaches´ knowledge, such as sociodemographic characteristics, experiences, education, etc.Therefore, the purpose of this study was to design, validate, and test the reliability of an instrument to assess the knowledge that coaches have about the female athlete triad syndrome and the confidence they have in this knowledge.

Sample
The sample for the content validity was nine experts.All experts held a PhD, MD, or similar degree and had at least 5 years of experience carrying out research related to the female athlete triad syndrome or one of its components or treating female athletes.The experts were from the following fields: medicine/family practice, nutrition, physical therapy, gynecology, eating disorder recovery, triad research in athletes, coaching female athletes, and sport psychology.
The sample for the reliability test was 12 coaches.The characteristics of this group of coaches were: an average age of 39.4±11.2years, an average coaching experience at the college level of 11.3±7.7 years, 41.6% were female, and all coaches had a university degree.
The sample for the concurrent validity were 45 college coaches from an endurance sport that emphasized weight control (cross-country) and 63 coaches from a sport that theoretically does not emphasize weight control (basketball) in the 2006-2007 season.All coaches were from the athletics department of an American Division I university.The characteristics of this group of cross country coaches were: an average age of 40.8±13.1 years, an average coaching experience at the college level of 12.4±9.2years, 37.8% were female, and all coaches had a university degree.The characteristics of this group of basketball coaches were: an average age of 36.6±9.2 years, an average coaching experience at the college level of 10.3±8.3 years, 58.8% were female, and all coaches had a university degree.The study was approved by the University of Texas-Pan American Institutional Review Board (IRB#500; 2007/04/24).

Procedures
There were four phases in the process of designing the questionnaire and testing the validity and reliability of it: a) design and development of the instrument, b) testing the content validity, c) testing the instrument reliability, and d) testing the concurrent validity (Trochim & Donnelly, 2007).
The first phase, the design and development of the instrument, involved the use of specific literature about the female athlete triad syndrome and knowledge instruments related to the topic.Reviews in the following databases were done: Web of Knowledge (WOK) of ISI (Thomson), Medline, SPORTDiscus, Google Scholar, Sponet, Scielo, and Dialnet.The key word searches included: "female athlete triad," "eating disorders," "amenorrhea," "osteoporosis," "knowledge," and "coaches" (as well as equivalents in Portuguese and Spanish for the Scielo and Dialnet databases, respectively).Abstracts were reviewed to select the papers related to the female athlete triad syndrome.The criterion followed to select the article was that it studied co-aches´ or athletes´ knowledge regarding the female athlete triad syndrome or its components.Several articles and surveys were selected from the literature regarding nutrition, the components of the syndrome, or coaches´ attitudes or actions regarding the components of the syndrome (Beals, et al., 1999;Bradney, 2002;Hobart, 2000;Juzwiak & Ancona-Lopez, 2004;Joy et al., 1997a;Joy et al., 1997b;Lassiter, 2002;Martínez-Pecino, Mulas-Sánchez, Fernández-Palacín, & Bayón-Suárez, 1997;Sossin et al., 1997;Turk, 1995).After reviewing the bibliography, the parts and topics of the instrument were established (Table 1).Several instruments (Bradney, 2002;Lassiter, 2002;Martínez-Pecino, et al., 1997;Turk, 1995) and literature about creating an original instrument (Hague, Hague, & Morgan, 2004;Thomas, 2004) were used as guides to design the instrument presented in this article.
From the review of the literature and the surveys that were used in empirical research that were found in the literature, the outline of the survey and the first draft of questions were established (Table 1).Questions were designed to ask about these aspects and about the coaches' characteristics.The aspects were grouped into: a) knowledge of the syndrome, components, prevention and intervention; b) coaches´ confidence in their responses (coaches´ perception of the adequacy of their answers); and c) coaches´ characteristics (gender, degree held, years of experience in coaching females, continuing education participation specific to the syndrome and its components, and sport coached).Finally, questions about the sample's characteristics (ethnicity, number of coaches in their program, experience, level of competition, sport, and formal training) were also included.For the questions related to sociodemographics of the coaches, multiple choice items were used, except for questions related to degree held, number of coaches in program, and number of years coaching females, where the possibility to type in the answer was given.For the questions related to knowledge, multiple choice, and true/false were used, depending on the objective.A 4-point Likert scale was provided to determine level of confidence: 1) not at all confident; 2) somewhat confident; 3) confident; and 4) completely sure.
The evaluation of the level of confidence was included to measure the confidence the coach had in his or her knowledge (Turk, 1995), as a low level of confidence has been related to a lack of application of one´s knowledge (Bandura & Wood, 1989).Knowledge and confidence level were taken into account to calculate the overall knowledge score.A correct answer with the highest confidence resulted in the highest weighted score for that question, while an incorrect answer with the highest confidence resulted in the lowest weighted score (Table 2).In fact, this score was lower than answering "don't know", as it is potentially dangerous to have inaccurate information but to believe that it is correct.The overall score was between -35.00 and 35.00.
In the second phase, nine experts in fields related to at least one of the components of the syndrome provided input regarding the instrument.The experts evaluated the qualitative (open questions) and quantitative questions (scale from 1 to 10) from the survey regarding: degree of definition and understanding of the survey's questions; degree of adequacy of the survey's questions, the scale used, and the need to include more questions in the survey.The collective suggestions from the experts were considered and the appropriate changes were made.Effects of the syndrome and/or components 14, 20, 23, 24, 30 1 3 6 7 Population and incidence of the syndrome and/or components 13, 25 1 3 7 Risk factors and detection of the syndrome 16, 26 2 3 4 7 10 Causes of the syndrome and/or components 27, 33 1 3 4 7 9 10 Signs and symptoms of the syndrome and/or components 15, 29, 31 1 3 4 7 9 10 Prevention of the syndrome and/or components 28, 32, 34-38 1 5 7 10 Intervention 39-42 5 9 Coaches´ formation and continuing education 43-50 8 9 10 In the third phase, the reliability of the instrument was calculated.The questionnaire was sent to selected college coaches of an NCAA Division II program by e-mail.Twelve coaches completed the test-retest process.A final section allowing for comments took into consideration their understanding of the questionnaire, the time taken to complete the survey, and questions or concerns they had with the instrument.One week after completing the instrument, the questionnaire was sent back to the same sample of coaches for test-retest procedures for determining instrument reliability.
In the fourth phase, we measured the ability of the instrument to differentiate between coaches of different types of sport (Trochim & Donnelly, 2007).The type of sport was established using the classification by Rust (2002).Forty-five coaches from an endurance sport that emphasized weight control (cross-country) and 63 coaches from a sport that theoretically does not emphasize weight control (basketball) were analyzed (2006-2007 season).The survey was sent to the coaches by e-mail.
The coaches´ e-mails were obtained from their websites.Fifty percent of NCAA Division I intercollegiate universities were selected randomly from the NCAA website.The response rate was 15% for cross country and 11.9% for basketball.

Statistical analysis
To test the content validity of the instrument, a descriptive analysis of coaches´ responses (mean, median, and mode) was also done.Following Bulger and Housner (2007), questions with values lower than 7.0 were eliminated, questions with values between 7.1 and 8.0 were modified, and questions with values greater than 8.0 were accepted or accepted with modifications.With the values from the quantitative evaluation done by the experts, the Aiken´s V was calculated (Penfield & Giacobbi, 2004).
The reliability of each item of the instrument was calculated using the Kappa Index for each of the questions (categorical variables).To test the ability of the instrument to differentiate between coaches of different types of sport (concurrent validity), an inferential analysis of the data (one-factor ANOVA) was done to establish whether there were differences between coaches of both types of sport using the SPSS 15.0 software with a level of statistical significance set at p < .05.

Results
The draft of the survey had 48 questions after the first phase.The survey comprised three parts: a) 10 questions related to the coaches' characteristics, b) 30 questions related to knowledge of the syndrome, and c) 8 questions related to coaches´ education and continuing education.The knowledge questions were subdivided into seven parts: a) definition of the syndrome and/or components, b) effects of the syndrome and/or components, c) population and incidence of the syndrome and/or components, d) risk factors and detection of the syndrome, e) causes of the syndrome and/or components, f) signs and symptoms of the syndrome and/or components, g) prevention of the syndrome and/or components, and h) intervention.
After reviewing the experts´ evaluation of the draft of the survey, one question related to knowledge was eliminated, three questions were added, two questions were merged, and 12 questions were modified.The experts´ observations were related to the terms used, the need to clarify the terminology and questions, and expressing the questions and the scale in a positive way.All questions of the draft of the survey at this phase had an average score > 7.0.Aiken´s V from the quantitative evaluation of the experts was calculated.The Aiken´s V was pertinent (> .84 for the lowest value).
From the score of the test-retest process that was carried out with 12 coaches of an NCAA Division II program, the total reliability of the questionnaire was calculated (the smallest of these calculations).An intra-class correlation coefficient of .68 was found.Regarding the ability of the instrument to differentiate theoretically different groups of coaches (Table 3), significant differences were found in the knowledge score of the coaches of an endurance sport that emphasizes weight and coaches of a sport that theoretically does not emphasize weight.Further, significant differences were found in the education received about this topic, this interest in this topic, and the coaches´ involvement (p < .000).The final version of the survey is included (Appendix).more information about it, be more aware of it, and have more experience with it (Eagle, et al., 2013;Joy et al., 1997a;Sherman, et al., 2005).The results show coaches of a sport with a higher incidence of the female athlete triad syndrome had higher knowledge scores as well as more confidence in their responses.However, these findings need to be carefully considered due to the response rate (11-15%).The response rate was 15% lower than similar research (Pantano, 2006).These may demonstrate bias in the data, as only coaches interested in the topic may have responded.Still, findings would show differences between coaches interested in the topic in both sports.The process that was followed and the data found show that this instrument is suitable for measuring coaches´ female athlete triad syndrome knowledge in college coaches.This instrument can be used by researchers, athletic directors, and/ or coaches to evaluate the need for specific education as well as the effect of educational training done by the coaches.This instrument lets us monitor the knowledge of this topic, detect the need for education by coaches, and assess the effect of continuing education as well as the evolution of this knowledge over time.Performance sport involves the risk of developing unhealthy behaviors.The role of coaches must be proactive to avoid these problems.Knowledge of and involvement in prevention are key aspects in reducing health problems in sport.The instrument that was developed and validated in this paper can contribute to knowing how the coaches´ knowledge level is related to this topic.
Limitations and future research The instrument design and the processes of validation and reliability that were carried out are only the first steps in the process of developing an instrument due to these steps being carried out in a specific sample.Testing it in other populations, obtaining reference values, and translating it to other languages is needed.Coaches have an important role in the health of their athletes (Eagle, et al., 2013;Joy et al., 1997a;Sherman, et al., 2005).They must be aware of possible health issues.The present instrument can be used to obtain information regarding coaches´ knowledge, to study the effect of specific educational training, or to study the relationship between knowledge and application (information provided, training environment, or stress perceived by athletes).

Discussion
The results show that the instrument is suitable for measuring coaches´ female athlete triad syndrome knowledge.Coaches have an important role in creating safe training environments and providing preventive measures or screening to reduce the incidence of the female athlete triad syndrome (Eagle, et al., 2013;Joy et al., 1997a;Sherman, et al., 2005).Given that this syndrome is a relatively new issue for coaches; the instrument can provide information about the coaches´ knowledge about this topic.The instrument allows us to collect information regarding the coaches' characteristics, their education and continuing education, and about their specific knowledge about the female athlete triad syndrome (components, effects, risks, signs, causes, prevention, and intervention).
The instrument allows us to obtain information about co-aches´ knowledge regarding a basic perspective (definition, effects, or risks), regarding an applied perspective (how to detect environments that can foster the syndrome, signs and symptoms, prevention, and/or steps to take if a case is detected), and coa-ches´ knowledge confidence.The knowledge about the coaches´ confidence provides us information about their receptiveness as well as their knowledge perception.All this information can help us understand coaches´ need for knowledge.The instrument can also be used to measure the effect of educational programs or to obtain information about the knowledge of coaches from different sports, levels, or characteristics.This information provides data that could be useful for planning and implementing educational training for the specific needs of these populations.
Specifically, regarding the validity and reliability process, the procedures that were followed combined the experts´ evaluation with testing the instrument with a group of coaches.The coaches' evaluation involved qualitative and quantitative assessments of the instrument, thus allowing us to combine the advantages of both types of assessment.Their evaluations helped to develop the instrument by contributing to establishing the adequacy, level of understanding, and terminology (Bulger & Housner, 2007;Escurra, 1989;Padilla, Gómez, Hidalgo & Muñiz, 2007;Zhu et al., 1998).The experts positively evaluated the inclusion of the confidence in the questions to assess the coaches´ perception about their knowledge.
The reliability of the instrument (test-retest procedure) was "substantial" and pertinent (Landis & Koch, 1977).The value of the intra-class correlation coefficient that was found may be affected by coaches either doubting their responses after completing the survey or reviewing information about the topic.The reliability was tested with an online survey, as those were the conditions under which the survey was going to be administered in the future.However, this procedure may involve a lack of control over the conditions in which the coaches completed the survey.On the other hand, it was done in a more natural situation.
The findings regarding the concurrent validity show the ability of the instrument to differentiate between groups that, in theory, are different.Theoretically, coaches and athletes from sports with a higher incidence of the female athlete triad syndrome or some of its components should have received Prevention of the triad involves emphasizing to athletes the amount of macro-and micronutrients that they need to compete as opposed to the foods they should avoid.
True / False / Don't know Confidence 1 2 3 4 An efficient and easy way for medical staff to screen for the triad involves using menstrual health history and nutritional history questionnaires.
True / False / Don't know Confidence 1 2 3 4 The coaching staff member or the athletic trainer with the best relationship with the athlete is the person who should intervene when a case of the triad is suspected.
True / False / Don't know Confidence 1 2 3 4 For health reasons, an athlete with a confirmed case of the female athlete triad must stop competing immediately.True / False / Don't know Confidence 1 2 3 4 When confronted about disordered eating or triad concerns, typically the athlete will be relieved and desire help.True / False / Don't know Confidence 1 2 3 4 The intervention team should minimally involve a medical physician, dietician, and a psychiatrist or psychologist.True / False / Don't know Confidence 1 2 3 4 Finally, please complete the following questions about your program, choosing one answer unless otherwise noted.
The athletic department currently has a policy in place that specifies what to do when we suspect that an athlete suffers from one or more aspects of the female athlete triad Yes / No / Don't know The athletic department currently has a policy in place that specifies what to do with a confirmed case of an athlete with one or more aspects of the female athlete triad.
Yes / No / Don't know The athletic department currently has a policy in place that specifies what to do when we suspect that an athlete suffers from one or more aspects of the female athlete triad Yes / No / Don't know The athletic department currently has a policy in place that specifies what to do with a confirmed case of an athlete with one or more aspects of the female athlete triad.
Yes / No / Don't know The athletic department provides educational programming for the athletes about the female athlete triad.

Table 1 .
Structure of the survey (parts and topics of the different questions of the survey) and references used to develop the survey.

Table 2 .
Calculation of points for survey.

Table 3 .
Knowledge score and average confidence based on type of sport coached.
Degree Major area of study: _________ Year: ________ □ Doctoral Degree Major area of study: _________ Year: ________ Sport(s) currently coaching at this level.Indicate the sport by checking the box then type the number of years* coaching this sport (if appropriate, choose more than one): *Please enter years in a numerical format.Questionnaire to measure coaches' female athlete triad knowledgePlease answer the following demographic and experiential questions.Unless otherwise noted, please choose one answer per question.Because of the impact of running on bone density, bone loss does not occur in many light-weight athletes.If weight is a concern for the health of an athlete or for avoiding injury, giving the athlete a range of acceptable weights or body fat compositions will help her to feel less pressure related to weight control.Once an athlete has a confirmed case of one of the three components of the triad, screening for the other two components should follow.It is an invasion of the athlete's privacy for the coaching staff to ask if her menstrual cycles are regular.
Yes / No / Don't know As a coach, I personally provide information for the athletes about the female athlete triad.Yes / No / Don't know Although I do not personally speak to the athletes about it, I do provide educational programming about the female athlete triad for them.Yes / No / Don't know I have received training on the subject of the Female Athlete Triad as a collective entity.Yes / No / Don't know I have received training on at least one of the components of the Triad.Indicate the type of continuing education you take part in at least once a year.(checkall that apply in general and specific to the triad)