Non-technical skills assessment scale in nursing: construction, development and validation 1

ABSTRACT The introduction of non-technical skills during nursing education is crucial to prepare nurses for the clinical context and increase patient safety. We found no instrument developed for this purpose. Objectives: to construct, develop and validate a non-technical skills assessment scale in nursing. Method: methodological research. Based on the literature review and experience of researchers on non-technical skills in healthcare and the knowledge of the principles of crisis resource management, a list of 63 items with a five-point Likert scale was constructed. The scale was applied to 177 nursing undergraduate students. Descriptive statistics, correlations, internal consistency analysis and exploratory factor analysis were performed to evaluate the psychometric properties of the scale. Results: scale items presented similar values for mean and median. The maximum and the minimum values presented a good distribution amongst all response options. Most items presented a significant and positive relationship. Cronbach alpha presented a good value (0.94), and most correlations were significant and positive. Exploratory factor analysis using the Kaiser-Meyer-Olkin test showed a value of 0.849, and the Bartlett’s test showed adequate sphericity values (χ2=6483.998; p=0.000). One-factor model explained 26% of the total variance. Conclusion: non-technical skills training and its measurement could be included in undergraduate or postgraduate courses in healthcare professions, or even be used to ascertain needs and improvements in healthcare contexts.

of NTS in the activities of nurses in general. Such an instrument is necessary to benchmark good NTS and to guide a formative feedback to the future practice of nursing students, and that is what we aim to discuss in this paper: to present the development and validation studies of a scale built based on theories and previous studies of NTS, specifically adapted for nursing undergraduate students, as it can be used to assess NTS in order to enable a greater understanding of these skills and enhance the performance of nursing undergraduate students in their future practice and patient safety (22) .
In this sense, since there was no specific instrument for the context of nursing education, we carried out a panel discussion to adapt the CRM principles to the context of nursing practice, according to the language and the specific activities performed in nursing.   The NTS-NAS was constructed and developed in Portuguese, however, in this paper we will translate the necessary parts into English.

Method
To select the sample, the following inclusion criteria were considered: there should be 2 nd , 3 rd or 4 th grade nursing students, because clinical experience and knowledge were required to answer the scale; and exclusion criteria: 1 st grade nursing students (these undergraduate students have no clinical experience and knowledge yet to answer the scale).
The study version of the scale resulted in a list of 63 items, with a five-point Likert scale: "totally disagree", "partially disagree", "neither agree nor disagree", "partially agree", and "totally agree", and the option "non-applicable". It is subdivided into 14  In order to analyze the psychometric properties of the NTS-NAS, SPSS (version 23.0) was used.
The following statistical analyses were performed: descriptive statistics (for sensitivity); correlations; internal consistency (Cronbach's alpha), and exploratory factor analysis.

Results
The scale was applied to a random sample of 177 nursing undergraduate students from the School of  (Table 3). and "Call for help early" being the most consistent ones. These results suggest that the dimensions "Cross (double) check", "Distribute the workload", "Use cognitive aids", and "Have a good teamwork" do not have a solid internal consistency and, hence, may not be assessing what they are supposed to assess. In addition, the dimensions "Use all available information", "Prevent and manage fixation errors", and "Set priorities dynamically" could not be assessed since they have only one item each. Considering the items in particular, the exclusion of four items could potentially benefit the internal consistency of the respective dimension.
The corrected item-total correlation coefficients were also analyzed, which correspond to the correlation of each item with the total score of the respective dimension by excluding the item itself. Therefore, a low coefficient (bellow 0.30) suggests that the item does not measure the same construct measured by the other items included (23) . Overall, these correlations corroborate the results of internal consistency, since the dimension "Have a good teamwork" is the one that presents the lowest correlation coefficients, which means that probably some items are not fulfilling     The total model explained 26% of the total variance.
In general, the factor loadings were between 0.37 and 0.73, which suggests that the items are influenced by the underlying factor and, therefore, belong to this unidimensional model. In addition, items presented commonality values between 0.24 and 0.53.  To conclude, the NTS-NAS was built based on the 14 CRM principles and it was expected that 14 dimensions would be found, however, a unidimensional structure emerged for this questionnaire, which seems to be valid.
In this sense, the final version of NTS-NAS resulted in a list of 63 items, with one dimension, NTS, and with a five-point Likert scale: "totally disagree", "partially disagree", "neither agree nor disagree", "partially agree", and "totally agree", and a "non-applicable" option.

Conclusion
This research was conducted in order to construct, develop and validate an instrument capable of measuring and representing NTS in nursing practice.
This instrument seems to be appropriate to adequately assess NTS in nursing clinical contexts, however, more studies are needed to further validate the unidimensional model NTS-NAS, with a more representative sample of students/professionals from different healthcare settings. On the one hand, it is suggested that this instrument can be used in training settings, both in curricular internships and in specific workshops/ intervention programs focused not only on technical habilities, but also on NTS. These types of intervention and respective assessment may significantly improve the performance, confidence, and self-efficacy of nursing students, and be an added value, as they can help them to better adjust to the complex clinical context, improve their clinical performance and ultimately, contribute to the safety and well-being of patients. On the other hand, NTS training and its measurement by using the NTS-NAS could also be included in postgraduate courses in healthcare professions and even be used to ascertain needs and improvements in healthcare contexts, such as in hospitals and private practices.