Design of a nursing objective structured clinical examination of a first-year clinical practice program

1 University of Cádiz, Faculty of Nursing and Physiotherapy, Cádiz, Cádiz, Spain. 2 University of Cádiz, Faculty of Nursing, Algeciras, Cádiz, Spain. 3 University of Cádiz, School of Nursing “Salus Infirmorum”, Cadiz, Cádiz, Spain. ABSTRACT Objective: The aim of the present study was to design a content-valid nursing objective structured clinical examination attending a first-year clinical nursing practice program. Method: The examination was designed following a procedure based on the consensus of experts which was comprised of three phases: selection of the activities in which students should be competent according to the learning outcomes of the course, clinical case design, and integration of the clinical cases designed into the stations of the test. Results: Of the 44 surveys submitted for the design of the stations, 37 were answered, of which 31 respondents met the inclusion criteria of the panel of experts. The activities on which the experts reached the highest degrees of consensus were: basic physical assessment and monitoring of vital signs, assessment of hygiene and skin status, ability to develop care plans, management of safety principles in administration of medication and administration of oral medication. Based on the selected activities, the experts developed 20 clinical cases, from which a four-station nursing objective structured clinical examination was designed. Conclusion: The structured methodology based on the design of experts enabled the design of a content-valid objective structured clinical examination appropriate for the evaluation of the learning outcomes achieved by the students attending a clinical practice program.


INTRODUCTION
In the last decade, Nursing degrees have undergone fundamental changes in the adaptation of their subjects to competency-based education and training. Practical clinical training has acquired particular relevance in this degree, and, as a consequence, the evaluation of the student's clinical competence presents a great challenge for teaching staff. In Spain, public healthcare resources are available for use in practical teaching of clinical aspects of the Nursing degree. Joint responsibility and partnership is established between universities and healthcare institutions for the training of Health Sciences students (1) .
The evaluation of the level of clinical competence acquired by the health sciences student is a complex process that must fulfill various requirements to guarantee its quality (2) . As a result, there are several evaluation strategies used.
Specifically, the Objective Structured Clinical Examination (OSCE) is a test based on the principles of objectivity and standardization used to assess student performance through a standardized scoring system (3) . This test consists of a test comprised of a set of stations in which different clinical situations are simulated which students must solve (4) . During the OSCE, students rotate sequentially through each of the stations where they have to perform one or more activities. Each one of the test stations contains different assessment elements, such as standardized patients, simulation dummies, videos, and/or clinical documentation, which allow students to sufficiently demonstrate their clinical competence, knowledge, skills, and attitudes (5) . At each station, there is an expert observer who scores the students' performance using an objective, structured, and predetermined assessment list. After a specified period of time, a signal alerts students to move on to the next station. All of the students follow the circuit in sequence, which may consist of a varying number of stations.
The OSCE is increasingly taken into account in nursing education (6)(7)(8) . The implementation of the OSCE in the context of nursing has required some modifications to the original format designed for medical students (9) . There are logical arguments that support these variations, such as the comprehensive approach to the patient. The cases tested at the stations demand a specific number of nursing knowledge and skills (10) . Therefore, the case designed should include a detailed patient profile that would allow students to demonstrate their skills to make an assessment, the prioritization of problems and clinical judgment, the elaboration of a care plan, its implementation, and the evaluation of the care provided.
The contents of a high-quality OSCE must be in line with the learning level of students (11) . This is one of the first premises that should guide the design of OSCEs and their subsequent analyses.
The importance of providing a detailed description of the method used in the design of OSCE-type assessment tools to endow them with content validity has been emphasize (12) . Among the first steps to establish this validity is the establishment of specific objectives and activities to be performed by the student at the stations.
Based on these considerations, the objective of this study was to design a content-valid nursing OSCE based on the learning outcomes of a clinical nursing practice program of the first year of the Nursing degree.

Design of stuDy
The stations were designed following a structured design methodology based on expert consensus (13) . The design took into account the principles and recommendations of expert-developed guidelines for the assessment of clinical competence using OSCE in both Medicine and Nursing (9)(10) .
The methodology was developed in three phases. The first phase consisted of the selection of priority activities to be included in the nursing OSCE. In the second phase, clinical cases were created based on the selected activities. Finally, in the third phase, the stations of the nursing OSCE were agreed upon, based on the integration of similar or complementary clinical cases.

First phase: panel oF experts and selection oF activities
In order to select the activities, the Delphi method was used. This is a technique whose objective is to obtain a reliable consensus between the opinions of a panel of experts, and which has been used successfully to solve problems and reach a consensus on the research criteria in teaching methodology and evaluation (14) .
A self-administered questionnaire was created for the selection of the panel of experts and for the identification of the priority activities in which students were to be competent. This questionnaire was transferred to a web platform of online surveys in order to facilitate the completion and collection of data.
The questionnaire was divided into two parts. The first part was aimed at identifying the profile of potential experts in order to justify their selection. The second part included all of the activities to be selected by the panel of experts in which students had to be competent.
The professionals who could make valid contributions based on their knowledge, healthcare practice, and academic experience were considered to be experts. The following inclusion criteria were considered: more than five years of experience in practical clinical teaching in Nursing; having participated in at least two officially acknowledged teaching innovation projects at the university level. As for the number of experts required, there is no consensus in the literature, although some authors point out that they should not exceed 30 because synthesizing all of the information could then become a problem (15) .
The second part of the questionnaire included 40 activities extracted from the program of the Practicum I course (16) . These activities were grouped into four categories according to the learning outcomes of the course: "patient assessment", "care planning", "collaboration in the performance of diagnostic/therapeutic interventions", and "management and administration of medication". Activities derived from basic patient care, such as patient safety, respect for their autonomy and confidentiality, resource organization and management, as well as communication skills, were included as key skills in the stations of the nursing OSCE and were therefore excluded from the study.
Experts were asked to mark the specific activities of the various blocks which they considered to be of high priority. Only activities reaching a consensus level higher than 75% were considered for the design of the test (17) .
Forty-four candidates with experience in practical clinical teaching were contacted via e-mail with a link to the questionnaire. The candidates had 2 weeks to complete the questionnaire.

second phase: elaboration oF clinical cases
The second phase of the design consisted of e-mailing the panel of experts the list of priority activities, based upon which they would develop clinical cases that would form the core of the nursing OSCE stations.
Experts were asked to describe real clinical cases that are prevalent in the context of nursing, whose resolution would involve demonstrating one or more competencies, and which would reflect the nature of nursing practice (10) . The resolution of the cases was to be based on the available scientific evidence (18) . In addition, experts were asked to prepare an assessment list, including the key points that would allow them to conclude whether students had reached the required level of competence.
In the instructions provided to the experts for the design of the clinical cases, the information to be included in each clinical case was specified, i.e. the objectives set, the learning outcomes to be assessed and their weighting in the final grade, activities to be carried out by the students, necessary materials, information for students, information for the assessor, evaluation criteria, and information for the standardized patient, if the type of station so required. The literature emphasizes the importance of collecting this information in writing, as well as the importance of its periodic updating to improve its validity (10)(11) . third phase: consensus on the stations In order to select the different nursing OSCE stations, the consensus development conference technique was used (19) . The participants of this phase were the professors responsible for the course, who jointly analyzed the various clinical cases provided by the experts. At the meeting, the proposed stations were homogenized by synthesizing or supplementing them as deemed necessary by integrating those which were sufficiently similar and adapting them to the available resources. The most representative stations were chosen, as well as those including most of the abilities to have been acquired in the first year of practical clinical training. The stations to be included in the nursing OSCE were decided upon as a result of the expert consensus conference. The designed stations were detailed in writing.
The overall design of the test was represented in a table of contents, also known as a blueprint. Numerous authors in the literature stress the need to develop a blueprint to support content validity (20)(21) . Thus, this table included the stations, the assessment tools used, and the weighting of each learning outcome in the final score.

Data analysis anD treatment
Descriptive statistics were used to summarize the variables. Ordinal variables were expressed as frequencies and percentages, and numerical variables as means and standard deviations. The data were coded and analyzed with the statistical programs SPSS 23.0.

ethical aspects
The present study was reviewed and approved by the Innovation and Teaching Improvement Commission of the participating university in 2015. Participants were duly informed before starting the study. Participation in the study was anonymous and voluntary.

RESULTS
Of the 44 surveys submitted for the design of the stations, 37 were answered, of which 31 respondents met the inclusion criteria of the panel of experts. 52% of the experts reported that university teaching was their main area of work, whereas 48% reported healthcare. The panel consisted 13 (41.9%) male and 18 (58.1%) female, with a mean age of 54.23 (SD 7.182) years.
Presented below are the results obtained on the content that, in the opinion of the group of experts, should be considered as priority in the nursing OSCE of students attending a clinical nursing practice program of the first year of the Nursing degree. Figure 1 shows the patient assessment activities ranked according to the percentage of experts who consider them as priority.
Regarding the "care planning" category, 80.6% of the respondents considered it necessary to include, in the nursing OSCE, activities to assess the students' abilities to design a care plan in a clinical situation. Figure 2 shows the activities related to the students' collaboration in the performance of diagnostic/therapeutic interventions, ordered according to the priority level assigned by the group of experts.
With respect to the category "management and administration of medication", 90.5% of the experts considered it essential to evaluate the student's ability to follow the principles of medication administration. Figure 3 shows the order of priority of the routes of medication administration to be considered in the student's evaluation.    Based on the activities with the highest percentage of experts who consider them as priority, the panel of experts created 20 clinical cases which, using the consensus development conference technique, gave rise to a four-station nursing OSCE of 10 minutes each (Chart 1).
Using the blueprint, the different stations were linked to the learning outcomes or abilities evaluated by them by indicating their corresponding weighting in the scores attained by each student at each station, as shown in Chart 2. The final grade of the test would correspond to the mean score across all stations.

DISCUSSION
This paper presents the implementation of a structured methodology, which was based on the consensus of experts, for the design of OSCE. It takes as reference the learning outcomes of a practical clinical program, as well as the activities listed in the Portfolio of the program.
Currently, the implementation of OSCEs has become widespread, employed as a good method to assess clinical competencies (22)(23) . However, studies on the methodology used in the design of OSCE stations based on learning outcomes are more limited despite its importance.
In order to address the design of the OSCE-type tests, it was crucial to have the participation of a work group consisting of professors from both the academic and the healthcare fields who were also involved in nursing teaching, as recommended by some authors (24) . The collaboration between both fields was necessary to enhance the content validity of the test, so that the test would be adjusted to the type of training provided to students in the health institutions where they carry out their clinical practice. In the literature, we find several authors who agree on the convenience of designing this type of test based on expert consensus methodologies to achieve adequate content validity (25)(26) .
In terms of content, the majority of the experts in the study considered it a priority to evaluate the student's ability to assess and monitor vital signs, assess hygiene and skin status, prepare care plans, and monitor the safety principles in (mainly oral) medication administration. In the literature consulted, it was possible to verify that the main topics addressed in the OSCE stations presented clinical situations in which the student must interview the patient, carry out a physical examination, interpret laboratory data, perform a differential diagnosis, use clinical reasoning, or suggest a treatment, depending on the different systems of the body and following a biomedical model (5) .
Nursing practice, based on a holistic view of the individual and focused on the diagnosis and treatment of human responses to vital processes or health issues, either real or potential, requires the adaptation of the test model and its content. In this respect, the experiences carried out in nursing education tend to integrate different stations in a single, longer one (9) . This also avoids one of the major drawbacks encountered in the adaptation of the OSCE test format to the nursing practice, which is the fragmentation of patient care in different unrelated stations instead of providing a holistic view. Medical studies have also taken into account continuity of care, have modified the OSCE format, and concluded that this modification provides another approach for assessing clinical performance, which balances the benefits of traditional OSCEs with continuity of care (27)(28) .
In addition, in the present study, the number of nursing OSCE stations depended on the availability of human and material resources. All stations were documented following the recommendations published. These recommendations were also followed regarding the realism of scenarios, the appropriate preparation of actors, the training of assessors, the elaboration of assessment tools with their respective rubrics, and also to ensure case specificity during the implementation of the test (29) . Documenting the nursing OSCE was a fundamental aspect for the standardization of the stations, as well as the creation of a bank of stations that would facilitate their revision and use in subsequent evaluations.
To avoid the contamination of the students' performance, several similar versions of the stations were available and thus required different responses from the students. For instance, in the hygiene and skin care station, two video sequences were recorded. In one sequence, the patient had risk for pressure ulcers, whereas in the other the patient did not. Both sequences were alternated during the examination.
The blueprint allowed checking the weighting of the different learning outcomes in the final evaluation. Additionally, the blueprint showed that the stations with a standardized patient facilitate the exploration of a greater number of learning outcomes in an integrated way, among them the communication skills of the student, in consonance with other studies published (30) .
The use of a structured methodology based on expert consensus is the first step in the design of valid nursing OSCE stations for clinical competence assessment. However, psychometric studies are needed to prove the reliability and validity of these stations. Moreover, it is also necessary to continue seeking a consensus on these stations and create a bank of valid stations. These stations may derive from existing ones, could be alternated within the same assessment system to avoid bias, and could have different difficulty levels to assess undergraduate and postgraduate students, as well as health professionals.

CONCLUSION
The structured methodology based on expert design enabled the design of a content-valid nursing OSCE to evaluate the learning outcomes achieved by students attending a clinical nursing practice program of the first year of the Nursing degree. It has been found that stations with a standardized patient made it possible to integrate and evaluate a greater number of clinical competencies, both specific and key skills, offering a holistic view of the patient and reducing the number of stations needed in the test.