Validation of the Brazilian Portuguese version of Evidence-based Practice Knowledge Assessment in Nursing (EKAN) Validação da versão Brasileira da Evidence-based Practice Knowledge Assessment in Nursing (EKAN) Validación de versión brasileña de la Evidence-based Practice Knowledge Assessment in Nursing (EKAN)

1 Loma Linda University School of Nursing, Loma Linda, California, United States of America. 2 Kettering College Division of Nursing, Kettering, Ohio, United States of America. 3 Universidade de São Paulo, Escola de Artes, Ciências e Humanidades, Curso de Obstetrícia, São Paulo, SP, Brazil. 4 University of São Paulo, School of Nursing, The Brazilian Centre for Evidence-informed Healthcare: a Joanna Briggs Centre of Excellence, São Paulo, SP, Brazil. 5 Centro Universitário Adventista de São Paulo, São Paulo, SP, Brazil. 6 Indiana University School of Nursing, Bloomington, Indiana, United States of America. ABSTRACT Objective: The Evidence-based Practice Knowledge Assessment in Nursing (EKAN) is an objective measure of evidence-based practice (EBP) knowledge. The aims of the project were to translate the EKAN instrument into Brazilian Portuguese, test its psychometric properties, and document a baseline assessment of students’ EBP knowledge. Method: This study used methodological design. The EKAN-Brazilian Portuguese was administered to nursing students from two baccalaureate programs. Data were analyzed using Rasch procedures. Results: 123 students participated (87% ranged 19-25 years old; 60% were juniors; 38% perceived having no exposure to EBP). Validity indices showed the mean difficulty index = -2.48 to 2.07, the Infit (WMS) mean was.985 (SD = 2.13), and the Outfit (UMS) mean was.975 (SD = 0.17). Reliability indices showed sufficient item separation index (4.47); moderate person separation index (.47); high item reliability (.95); and low person reliability (.18). The mean EKAN sum score (max = 20) was 7.6 (SD = 2.13). Conclusion: This study on the EKAN-Brazilian Portuguese version provided sufficient evidence for validity and reliability; the person reliabilities showed the need for improvement in incorporating EBP concepts and content in nursing programs.


INTRODUCTION
Evidence-based practice (EBP) is a significant global movement and is pertinent to all health professions. Research shows the relationship between best practice and patient outcomes (1)(2) . These results demonstrate the need to teach EBP in the academic setting. Since EBP is an international movement, a global instrument that objectively measures EBP knowledge is important to combat variability in assessment practices and diminish the use of subjective measures as self-ratings of knowledge correlate poorly with actual knowledge (3)(4)(5) .
The EKAN instrument, originally developed in English as an objective measure of EBP knowledge in nurses and has been tested across levels of education in academic and practice settings (5)(6) . The EKAN was selected because it is an objective instrument with evidence of strong validity and reliability when tested in student nurse populations from different levels of nursing degree programs (6) . It is a 20-item multiple choice examination that provides three answer options and requires a single-correct response. The EKAN focuses on measuring not only knowledge but also application of EBP principles and was developed from the Essentials for Baccalaureate Education for Professional Nursing Practice (7) , and the Quality and Safety Education for Nurses Competencies (8) . Items evaluate students' knowledge of steps of the EBP process, interpretation of statistical results, and phases in a global quality improvement cycle.
The EKAN has already been validated in Spanish (4) , English (6) , and French (9) . Validation in Japanese and Korean languages has been completed but not yet published. The team chose to develop and test the EKAN in the Portuguese language since, according to the World Health Organization, Portuguese is the sixth most natively spoken language in the world (10) . Portuguese is the official or co-official language in Angola, Brazil, Cape Verde, Guinea-Bissau, Mozambique, Portugal, Timor-Leste, Macau, São Tomé and Príncipe (11)(12) .
Establishing instrument validity in multiple languages will facilitate a standardized, global approach to documenting the state of nursing's knowledge of EBP. Therefore, the objectives of this research were to translate the EKAN instrument into Brazilian Portuguese, and test the psychometric performance using Rasch model (13) . An additional aim was to document a baseline assessment of baccalaureate nursing student's knowledge of EBP at two universities in Brazil.

Design of stuDy
This methodological study is part of a global research project that has translated and tested the EKAN in several languages.

scenario
The data collection took place in the classroom of a private university (Institution A) and a public university (Institution B), both located in São Paulo, Brazil. The private university is a faith-based institution that enrolls 100 baccalaureate nursing students annually. Students complete 4,440 hours (theory and clinical) over ten semesters. The public university is maintained by the State of São Paulo and enrolls 80 full-time students each year. Students complete 4,170 hours (clinical and theory) over eight semesters. In both institutions, EBP concepts are taught, but not formally in any one course, nor threaded systematically throughout the whole curriculum. Both campuses offer internet access for searching research.

sample Definition
Junior and senior nursing students from two baccalaureate programs were recruited and responses from the two groups were combined for psychometric analysis. The recommended sample size with Rasch models is between 100 responses but less than 200 responses per test item; when the size falls in this range, the model provides stable results (14) . To protect student identity, neither the demographic nor the EKAN forms asked for any personal or identifying information. However, a consent form was required by both study sites. This study followed all the requirements of the Resolution nº 466 of the Conselho Nacional de Saúde (CNS) of the Brazilian Health Ministry (15) .
In Question 2, the back translation used the trans-literated "distributed randomly" like in the Portuguese version "distribuídos aleatoriamente" rather than the original English term "random assignment." The term used in the Portuguese version was verified as contextually correct. In Question 5, the back translated version used "What property are you evaluating" which is correct, while the Portuguese version used "se referindo" or "What property are you referring to." The word was changed to "está avaliando" which more closely aligns with the original version. In Question 9, the back translation used "allocate," as the Portuguese version also used the term "alocar." Yet the original English term used was "allow," which has a different connotation. This term was changed to "disponibilizar" which is contextually similar to the original meaning. In Question 13, the back translation used the term "affect the decision" rather than "influence the decision" as is in the original. The Portuguese version was checked and it correctly used "influenciarão," and was retained as is. Finally, in Question 16, the back translation used "self-assessment" which is different contextually than the original version's "self-report." When checked, the Portuguese version used "self-report," matching the original, and was not changed. In summary, two of the five questions needed word refinement to correlate more closely to the original.
With those changes, the EKAN-Brazilian Portuguese was finalized in June 2017. The Brazilian co-investigators also translated the Demographic Questionnaire and SIS from English to Brazilian Portuguese, but those forms did not require back translation.
Demographic questionnaire. The seven-item IRB approved Demographic Questionnaire asked subjects a series of questions about characteristics such as age, gender, and the completion of specific courses (EBP, statistics, research). It also asked subjects to rate the extent of their agreement with the following statement, "I am sure I can deliver evidence-based care" on a 5-point, Likert-type scale (1= Strongly Disagree, 5= Strongly Agree). This last question enabled data to assess correlations between student's knowledge of EBP and self-rated confidence in providing evidence-based care. The demographic form did not require back translation.
Study Information Sheet (SIS). The SIS provided information regarding the study, stressed that participation was voluntary, the student could leave the room before finishing and actions would not negatively affect the student's course grade or status within the program. The researchers posted an advertisement using the IRB approved "Study Information Sheet" (SIS) in the school of nursing buildings where students had classes. The co-investigators also provided verbal announcements about the study a few months prior to, the week of, and the morning of data collection. In addition, at Institution B, students received an email invitation along with the SIS. The SIS form did not require back translation.

Data collection
Data collection occurred during scheduled, proctored sessions with the principal investigator and onsite co-investigators. Co-investigators communicated with the students using a standardized protocol to ensure information and distribution of study materials was similar in each session at both study sites. The protocol reiterated details of the study purposes, voluntary status of participation, and the multiple-choice format. Subjects were further instructed to not put any identifying information on the forms, and to not talk or access any electronic resources, books, or notes during the session. After the instructions, interested students signed the informed consent and received the study instruments. To reduce the threat of crosstalk among subjects, multiple sessions were scheduled in rapid sequence on the same day at each site.
Upon finishing each data collection session, the principal investigator collected the forms, placed them in an envelope, and transported the forms to a locked office for data entry at a later time. MS Excel was used for demographic management and Winsteps v. 4.5.0 software was used for Rasch Analyses (17) . Two researchers performed data entry and data cleaning together to ensure reliability in these two processes. During inspection of the EKAN data, two subjects did not complete the last four questions; this was the only overlap of unanswered questions. One additional subject had seven non-responses but there was no overlap in any of the unanswered questions with the other two. The non-overlap indicated randomness to the non-responses which is important, as randomness of missing data provides unbiased item responses in the Rasch model (18) . Therefore, the records from the three individuals were retained, making the total sample size N = 123 for all analyses.

Data analysis
The team used Rasch model for dichotomous data to analyze data because it enables researchers to assess student's ability and the instrument's item difficulty and discrimination (19) . Rasch models provide "information-weighted fit statistic (Infit)" and "outlier-sensitive fit statistic (Outfit"). While common Infit and Outfit statistics are considered local fit testing measures for the Rasch model, it is also recommended to perform global fit testing including determination Type I α error since the model is sensitive to sample size in relation to the number of items in the instrument (20) . The x 2 /df of Anderson's Conditional Likelihood Ratio is recommended since this fit test is the most invariant to changes in sample size. Based on the instrument's length of 20 items, a maximum cutoff of 1.32 would minimize the possibility of a Type 1 error using 68% CI. The mean x 2 /df for the Anderson's Conditional Likelihood Ratio goodness of fit for the EKAN-Brazilian Portuguese is 1.22, supporting appropriate use of the Rasch model analytics.

Demographics
A total of 123 students volunteered to participate. Most students at both study sites were between the ages of 19 to 25 years and female. All students self-identified as in the 5 th , 7 th , or 9 th semester. No student took more than 30 minutes to complete the forms. Sample characteristics are summarized in Table 1. Of the sample, 32% of students from Institution A already completed a statistics course versus 100% of students from Institution B. Since some of the questions on the EKAN involve interpretation of statistical results, this could have affected scores in one institution over the other. There was great variation in students' reported exposure to EBP content. At Institution A, 60% perceived they had already taken an entire EBP course though co-investigators at the site reported there was no content in the curriculum yet. At Institution B, almost 70% perceived they had not taken any EBP coursework and co-investigators confirmed that students were not systematically exposed to this content. When asked to rate their confidence in providing evidence-based care, 91% of students from Institution A strongly agreed or agreed they could provide evidence-based care compared to 48% of students from Institution B.

psychometric performance
Since students from both institutions had similar cultural background, age and gender distribution, in the same type of nursing program, and both programs provided limited exposure to EBP, the team pooled data from the two institutions and treated them as one group for psychometric testing. A description of the 20 EKAN items and corresponding fit statistics are presented in Table 2. All items fell within recommended infit WMS (Weighted Mean Square) limits while all, but one item of the outfit UMS (Unweighted Mean Square) parameters fell within the recommended boundaries. The one item that did not was.79, missing the recommended range by.01. When standardized, all items met recommended infit WMS and outfit UMS parameters. The team analyzed response patterns for each item in the continue… To test validity properties, WMS and UMS statistics were used. Reliabilities were tested, using the item and person separations and reliabilities. Results are seen in Table 3.
Validity indices. In reviewing parameters for dimensionality, EKAN-Brazilian Portuguese showed all 20 items with infit WMS ranging from 0.92 to 1.10 with a mean of 0.983 and median of 0.97. The outfit UMS ranged from 0.93 to 1.10 with one student outside the range at 0.79. The mean and median of the outfit UMS without the outlier was 0.98 and 0.98; with the outlier included, the outfit UMS was 0.97 and 0.97 for both mean and median. The included outlier shifted the mean slightly lower. The scale quality statistics with an adjusted variance of 0.98 for the items (SD = 0.99), centers on 1.0, confirming the newly translated EKAN instrument was unidimensional in measuring EBP knowledge. The fit statistics are within parameters by Linacre (17) of 0.5-1.5 as well as other researchers who use more conservative parameters of 0.7-1.3. These results show support for the items to fit the model, assessing different aspects of EBP knowledge as a unidimensional trait.
The mean difficulty index (θ = -2.48 to 2.07) met benchmark parameters set by Bond and Fox (21) . The Infit WMS mean and standard deviation was 0.985 (SD = 2.13) with a range of 0.69 to 1.43. The Outlier-sensitive fits (outfit UMS) should also center on 1.0; those that do not, show either predictability (values less than about 1) or randomness/guessing of answers (values more than about 1). For this study, outfit UMS mean was 0.975 (SD = 0.17) (range of 0.057-2.11) and fell outside the benchmark of 0.6 to 1.4. This indicates some students were too predictable in their responses (e.g., A, A, A, A for each item) while other students did not have the knowledge and simply guessed at the answers.
Reliability Indices. Reliability, tested via item and person separations and item and person reliabilities, can also be seen in Table 3. The item separation index of 4.47 showed items could separate knowledge into categories; meanwhile the person separation index was 0.47 indicating the instrument could not distinguish between high and low performers.
Item reliability (IR) was.95 and met standard benchmarking of >.80, indicating items will perform similarly again; IR meeting benchmark also indicate the sample size was sufficient (22) . Person reliability (PR) was very low at 0.18, indicating there was very little variance in the sample's knowledge (homogeneity) and has a narrow range in responses (16,23) . Therefore, the person reliability parameter could not be used to inform certain aspects of total reliability but did clearly indicate a need for the addition of EBP content in the curriculum. The homogeneity in scores was logical, given that EBP has not been taught consistently covered in either of the curricula. performance of items relateD to stuDents' ability A Rasch Wright map (Figure 1) that of the 20 items on the EKAN, 13 items were at or above the mean difficulty (harder to answer); 10 of the 13 items fell within one standard deviation (S), being moderately difficulty to answer, and three items were very difficult to answer. Seven items fell below the mean, indicating there was a higher probability that students with average EBP knowledge were able to answer these items correctly. The distribution of students' ability displays a normal parametric curve. Half the students (n = 60, 49%) were above the mean (M) and half were below the mean (n = 63, 51%). The mean (M) item difficulty is half a logit higher than mean (M) student ability.

ebp knowleDge
Sum scores and ranges are seen in Table 4. Site A had lower scores (Note: one-third of students from site A had completed a statistics course); site B had higher scores (100% from site B had completed a statistics course). While minimum scores varied among the sites, the maximum score of 13 points was the same for both institutions.

DISCUSSION
The sample size provided sufficient responses for each item, and therefore delivered valid psychometric results for the EKAN-Brazilian Portuguese version. Results (item separation index and item reliability) showed strong item performance in the newly translated instrument. Person reliability showed limited consistency in student responses from one item to the next and person separation index reflected the restricted trait range of students' EBP knowledge (16,23) . These results, while expected because EBP has not been systematically integrated into the curriculum at either study site, will be helpful to gauge the impact of teaching enhancements in courses and across curriculum. While the EKAN is intended for single administration, it has been used as a pre/post measure to evaluate the effectiveness of an EBPrelated course (24) , which may be particularly helpful as programs begin to develop class content, teaching strategies, and assignments. The EKAN can also be used across levels of a program (25) to provide feedback on whether the curriculum Note: *Results from Spurlock & Wonder (6) ; **Validity benchmarks recommended by Bond & Fox (21) ; ***Reliability benchmarks recommended by Tennant & Conaghan (23)  is supporting progressive knowledge development versus stagnation or decline. Results from the current study can be used as a baseline measure and, when repeated, can help gauge the impact of targeted interventions (e.g., educational exposures, faculty development) on students' EBP knowledge development.
The one outlier score that fell outside the acceptable range in the Outfit UMS could have happened for various reasons. Since the Outfit UMS measures excessive predictability in choosing the same response (below the range) or excessive guessing of answers (above the range), one explanation is the student answered several questions with the same option without reading or taking care to answer the question based on knowledge or "best guess". This is the most plausible answer since the other students' scores fell within the range, and they all had the same level of exposure to the content.
Testing of dimensionality showed support for the items included in the EKAN and good fit with the Rasch model-the items measured the same dimension of EBP knowledge. These results are consistent with the original EKAN (6) . The inability of the instrument to distinguish between high and low performers (as evidenced by the low person separation index) can be explained by the variance; achieving a higher person separation index occurs when variance in their knowledge ability is wider. In general, students' knowledge ability was more narrow and lower since coverage of content was lower. An increase in ability variance would occur as knowledge increases; and this would occur when EBP content is increased in the curriculum.
The Wright map showed students' difficulty to correctly answer more difficult EBP questions. When more EBP content is added to the curriculum, a closer alignment between mean item difficulty and student ability will occur. The EKAN will enable ongoing program evaluation to gauge the impact of course and curriculum enhancements.
The newly adapted instrument showed a mean EKAN sum score lower or similar when compared to other validation studies using the EKAN (4,6,9) . The lack of correspondence between objective (EKAN) and subjective (confidence rating on ability to provide evidence-based care) measures is a phenomenon that has been shown in other disciplines (1) , in nursing students (4,(6)(7) , and in practicing nurses (5) . The low knowledge scores yet high confidence is consistent in all the EKAN validation studies. In the study to validate the EKAN-Spanish version, although 84% of 139 nursing students agreed or strongly agreed they were "confident in providing EBP care," while the EKAN-Spanish mean score was only 6.5 (SD = 2.03) out of 20 (4) . In the original EKAN validation study in the US, 80% of 200 nursing students felt "confident in providing EBP care," yet the mean score in this population was 10.4 (SD = 2.31) (6) . In the EKAN-French validation study, 81% of 105 Haitian nursing students agreed or strongly agreed they were "confident in delivering evidence-based care" despite the mean score of the EKAN-French score was 5.9 (SD = 1.93) out of a maximum score of 20 (9) .
These studies, as well as the current study, show student confidence in delivering EBP care is poorly correlated with EBP knowledge.
Results from the current study also showed incongruence or student's confusion between their concept of research content (in this case primary research content), and EBP content. Each element of these results should encourage institutions to acknowledge the need for a more systematic approach to introducing EBP concepts in courses across the curriculum.

limitations
One of the limitations of cross-sectional research is that results provide a snapshot or an assessment of a population at one point in time. Hence, the design makes it difficult to establish a strong association between EBP knowledge and exposure to EBP content. Additionally, the timing of the snapshot does not guarantee a representative sample, especially when sampling is non-random as was the case with this study, making a potential source of bias. The great variation in percentages of students who had completed a statistics course between both institutions also shows the impact of timing. This bias may have negatively contributed to the almost two-point difference in mean EKAN sum scores between sites, since some items required statistical interpretation.
generalizability Positive psychometric outcomes were obtained due to four factors, showing evidence of generalizability. Sample size requirements for the model were met, the instrument was tested in two different academic institutions, and minimal data were missing. Further, the randomness of the missing data showed no serious nonresponse issues. All of these factors support representativeness and the ability to use the translated EKAN instrument in the target population of junior-and senior-level, Brazilian nursing students. Testing the EKAN-Brazilian Portuguese with students in other degree levels should also be conducted. As EBP knowledge and abilities are a growing expectation, this instrument will enable fruitful educational research in Brazil.

CONCLUSION
This study provides baseline results on baccalaureate nursing students' EBP knowledge in Brazil and contributes to a larger body of work aimed at promoting the global use of evidence-based care. Validity results for the newly translated EKAN-Brazilian Portuguese contributes performance data for an instrument that was originally developed in the US and then translated and tested in Colombia, Dominican Republic, Japan, Korea, and Haiti. Ongoing, rigorous evaluation will be needed as Brazil's faculty develop and test curricular content and teaching strategies for EBP. The EKAN enables a consistent approach for evaluation, allowing faculty to collaborate globally on nursing education and EBP.