Nursing students’ learning from involvement in research projects: an integrative literature review

Rev Bras Enferm. 2022;75(1): e20210053 https://doi.org/10.1590/0034-7167-2021-0053 7 of ABSTRACT Objective: To identify the learning outcomes and skills obtained of undergraduate nursing students involved in research projects. Methods: This was an integrative literature review, based on a research protocol in the CINAHL Complete databases; Cochrane Central Register of Controlled Trials; Cochrane Database of Systematic Reviews; Cochrane Methodology Register; MedicLatina; MEDLINE, Scopus and JBI, including primary and secondary studies, published between 2015 and 2020. Results: A total of five heterogeneous articles were included, which were categorized using Kirkpatrick’s (adapted) model. Seventeen learning outcomes acquired through participation in research projects were identified, from the learning of new knowledge and skills to the development of new attitudes and behaviors. Final considerations: The involvement of nursing students in research projects is important to their professional development. Future investment in research on this topic can help cement the potential of this type of student involvement. Descriptors: Evidence-Based Practice; Learning; Students; Nursing; Research.


INTRODUCTION
There has been growing interest in recent years in the experience of involving nursing students in research and development projects (1)(2) . Some studies have shown that the participation of students in research teams can bring benefits to their learning process, to project development, and to higher education institutions (3) .
The main guiding policies for health care and undergraduate and graduate nursing education, both in Brazil and internationally, challenge higher education institutions (HEIs) to involve students in research, as this promotes learning about the process of developing scientific knowledge in nursing (4)(5) . In particular, there are recommendations for HEIs to create and develop opportunities for collaborative research, involving students (6) .
Some studies show that the active and supervised participation of students promotes greater awareness of the object of study, simultaneously stimulating critical thinking, promoting the construction of a practice that is aware of contextual influences, and that allows for greater familiarity with the research process (7)(8) . These results are echoed in the schools of thought that defend that it is essential to instill among students passion and enthusiasm for research and its relevance to daily life. It is important to ensure that the knowledge obtained from these experiences is translated into evidence-based practice in clinical contexts, because when professionals are inserted in a network of interprofessional cooperation, nursing care becomes increasingly complex (9) . Therefore, these recommendations promote the involvement of nursing students as co-researchers, while also encouraging them to incorporate evidence into their practice and being more proactive in seeking graduate-level education (8) .
The practical application of knowledge has the great potential to improve quality of care (9) and, consequently, reduce the time gap between the production of knowledge and its implementation. Furthermore, the integration of students in projects led by HEIs in all phases of the research process -from the creation of research questions to the publication of results -seems to increase levels of learning motivation (10) , develop interprofessional work skills (8) and contribute to the progress of research programs of HEIs (8,11) .
Although several benefits have been observed as a consequence of nursing student participation in research projects led by HEIs, no studies have been identified that systematize the learning results that result from these experiences.

OBJECTIVE
To identify the learning outcomes and skills obtained by nursing students involved in research projects.

Ethical aspects
An integrative literature review (ILR) is a type of secondary study that does not require permission from an ethics committee as it does not directly involve human beings.
When developing this study, a pre-defined protocol was used based on the recommendations of some authors (12)(13)(14) to ensure the rigor of the studies' different methodological procedures and ensure their validity (14) . The formulation of the research question abided by the principles of clarity, accuracy, pertinence, and objectivity.
The process of selecting, extracting, and analyzing the sample was carried out respecting the work and results obtained by the other researchers. The framework of this article is based on the best academic and scientific practices.

Type of study
The integrative literature review (ILR) is a research method that generates new knowledge on a given topic of study, presenting a review, critique, and summary of the most representative literature (12) . Thus, it is the most comprehensive methodological approach, because it includes experimental and non-experimental studies (13) to understand the chosen phenomenon. Moreover, the ILR allows for the inclusion of relevant literature to support decision-making, through the synthesis of what is already known about the studied phenomenon, helping to identify gaps that need to be clarified with new research (14) .
The choice for an ILR was based on the need for a thorough understanding of the chosen phenomenon. It is a current, pertinent, and underexplored topic that allows for recommendations in the field of nursing research and education, impacting how nursing students value and apply scientific evidence to their practice.

Methodological procedures
The pre-defined protocol consisted of six steps: 1) identifying the topic and defining the research question, 2) defining the eligibility criteria for inclusion and exclusion of studies, 3) defining the information to be extracted, 4) evaluating the included studies, 5) interpreting the results and 6) presenting the review/synthesis of knowledge (14) .
The following research question was formulated: "What are the learning outcomes and skills developed by undergraduate nursing students when they are involved in research projects?". It was created using the PICo mnemonic (population, phenomenon of interest and context), which also guided the definition of eligibility criteria (Chart 1).

Data collection and organization
Based on the research question and the definition of the inclusion criteria, the study was carried out between December 2019 and February 2020. The descriptors used in Portuguese and English and in associations (AND and OR) were: (students OR undergraduate students) and (research OR research project OR Technology and Innovation OR Activities OR R&D activities or evidence-based practice) and (learning or learning outcomes) and nurs*. The analysis of the literature and the experience of the authors contributed to the selection of descriptors.
The search was conducted in the following databases: CINAHL®Complete; Cochrane Controlled Trials Register; Cochrane Database of Systematic Reviews; Cochrane Methodology Register; MedicLatina; MEDLINE, Scopus, and JBI. The timeframe was restricted to the years 2015 to 2020, given the importance of describing current knowledge, an assumption associated with the potential of IRLs to produce changes in clinical practice as a consequence of research findings (14) . 7 of Nursing students' learning from involvement in research projects: an integrative literature review Loura DS, Bernardes RA, Baixinho CL, Henriques HR, Félix IB, Guerreiro MP. Figure 1 illustrates the process of selecting the studies until the final sample was obtained.
outcomes (16) . The model was later revised and adapted by other authors; the present study started out with the version that appeared in the article by Steinert and collaborators (16) , which was adapted to the present review. It includes four levels, represented by a K, followed by a number indicating the level: K1 (reaction); K2 (learning, knowledge, and skills, which help change attitudes and behavior); K3 (behavior change); and K4 (change in the institution/organization associated with the educational program) (16) .

RESULTS
The results obtained in this integrative review were extracted from the articles included in the sample after they were read in full. A total of five articles were included, which were heterogeneous in terms of their objectives, methodology, results, and conclusions. The five publications were classified as presenting different levels of evidence and were developed using non-similar research designs.
The articles were published between 2015 and 2020, with a wide geographical distribution -two were from Norway, one from Russia, one from the USA, and one from the Netherlands. The main results of the articles are shown in Chart 2.
In relation to Kirkpatrick's Model (16) , of a total of 17 learning outcomes, most were at the K2 level, corresponding to the learning of new knowledge and skills, as well as new skills that promote attitude change (n=10). Next, there were changes related to behavior (K3; n=5) and, finally, those related to the response regarding the importance of the phenomenon (K1; n=1) and the results of change in the institution/organization related to the educational program (K4; n=1). The main findings by category are described below.

Learning new knowledge and skills to develop new attitudes
The articles showed that the most common outcome of undergraduate nursing students' participation in research projects was learning new knowledge and developing competencies and skills relevant to the profession and society. The knowledge and skills acquired at the research level (18) , especially with regard to data analysis and interpretation and project implementation (8,17) , are integrated into the clinical dimension, which enhances general and specific technical-scientific knowledge (8) , and interdisciplinary (8) and relational (17) cooperation skills. The potential sample consisted of 107 articles. Two researchers read and analyzed the titles and abstracts, which immediately led to the elimination of 89 studies (4 duplicates and 85 that did not meet the eligibility criteria). The reading and analysis of the abstracts whittled the selection down to 18 and the analysis of the full text to 5. Thirteen articles were eliminated after reading the full text because they did not answer the research question.
Most of the screened studies were eliminated by evaluating evidence-based research or research training programs, which did not involve students in research projects. The review identified one systematic review about the involvement of students in research, but the article did not identify the specific gains for nursing students, so therefore it was not included in the sample (15) .

Data analysis
A table was constructed to record the data collected based on the sample of articles, including the following data: title of the article; year; author(s), type of article; objective(s); method; level of evidence, and main outcomes/conclusions. Then a synthesized version of this table was produced, according to the guidelines of the journal, as presented in Table 2.
The nature of the studies and their outcomes enabled analysis and synthesis of their content. Categories defined a priori based on Kirkpatrick's (K) model were used to evaluate learning The emergence of new attitudes towards research was also highlighted. Participation in research projects molds future professionals who have the potential to constructively question clinical practice and are able to conceive it within the framework of a given organizational culture (7) . This change in attitude was also revealed in the acquisition of an initiated degree of research development, which can lead to the development of scientific work in the short, medium, or long term (7) .

Behavior change
Behavioral change emerges from the ability to interconnect the contexts of clinical practice and knowledge production. First, the involvement of students in research projects enhances the optimization of time management (17) , and increases awareness and attention in clinical practice (7) , along with self-confidence (17) and self-efficacy of participating students (19) . Additionally, it seemed to encourage them to apply the knowledge they had acquired and continued to acquire into their practice, translating such knowledge and promoting evidence-based practices (19) .

Response regarding the importance of the phenomenon
The recognition of the importance of student involvement in research projects appeared to be cross-sectional to the entire population mentioned in the articles. However, it was only mentioned explicitly in one article that explored the role of supervisors in fostering increased motivation in students to participate in research (18) . Chart 2 -Articles included in the bibliographic sample, Lisbon, 2020

Titles
Year Countries Design / number of participants and level of evidence (LE) (13) Interventions Outcomes Student nurses' learning outcomes through participation in a clínical nursing research project: A qualitative study (17) 2020 Norway Cross-sectional qualitative study N = 12 LE = 4 To examine the learning experiences of nursing students after participating in a clinical research project.
The authors suggested that students develop communication and relationship skills with patients (K2), as well as translation and application of theoretical knowledge to practice (K3), within the scope of clinical evaluation and observation. The optimization of time management (K3) and the increase of self-confidence (K3) were also mentioned, as well as the improvement in the interpretation of research articles (K2).

Role of Students and Supervisors' Interaction in Research Projects:
Expectations and Evaluations (18) 2015 Russia Mixed design: quantitative (cross-sectional observational) and qualitative N=40 LE = 4 To identify the students' expectations of participating in research, and to compare the subjective supervisors' estimates of students' research abilities with students' research abilities measured by objective indicators.
Analysis of student expectations suggested that supervisor support may be one of the most important factors to motivate students from all study cycles to participate in research projects (K1). There were gains in theoretical knowledge and research skills (K2), and the presence of the supervisor was indicated to be a contributing factor to the students' development in all phases of research (K4).

Involving Undergraduate Nursing Students in Participatory Health
Research: Implications from the Netherlands (19) 2018 Netherlands Retrospective observational qualiquantitative study N=70 LE = 4 To describe how the inclusion of final year undergraduate nursing students as co-researchers in participatory health studies was processed; to illustrate the value of these activities to the development of students' role as co-researchers.
Students gained knowledge about nursing research methods and competencies in the area (K2). The self-efficacy of students in applying these skills to their practice was apparently increased as a result of involvement (K3). Participation is presented as a way to help the student transfer theoretical knowledge to practice, applying it in each context and gaining specific knowledge of each area. (K3) Nursing students' longitudinal learning outcomes after participation in a research project in a hospital (7) 2019 Norway Cross-sectional qualitative study N = 52 LE = 4 To understand the longterm outcomes after active participation in research during the second year of a nursing program and its influence on learning outcomes.

Change in institutions/organizations associated with the educational program
The sample did not present a great number of outcomes that could allow for the extrapolation of the changes that took place in the institutions that hosted the research projects. However, there seemed to be several opportunities for this to happen. An example was the importance assigned to the supervisor as a strategy to monitor and guide students (18) , a figure who, contributes to the development of all phases of the research project and, as such, can cause a change in the perception of HEIs regarding the necessity of their role.

DISCUSSION
The articles included in this literature review were mostly observational, with different methods and techniques, and considering their limitations, they were able to answer the research question.
The involvement of students in research projects has been a theme prominently addressed at a global level. There are several scientific perspectives about its importance for those involved, but also for individual and collective health. The results reflect the importance of encouraging and motivating students to participate in research projects, in addition to the need to generate awareness among HEIs to promote these practices (7)(8)(17)(18)(19) .
The aspects described above are corroborated by evidence that suggests that HEIs need to invest in such programs, because most undergraduate students who complete research curricular units are not in fact competent in research methodology and research processes. In addition, in real-world learning, students often face complex situations that involve reflection and wellgrounded clinical decision-making. Therefore, research skills is one of the topics that should be included in the curricula as soon as possible (20) , so that students are able to address and solve complex health-disease situations.
There is consensus that research is important to nursing, not only at the level of education (7,21) , but also professionally. Three articles included in the sample highlighted the importance of translating theoretical knowledge into practice and how student involvement in research activities fosters this potential (7)(8)19) . Other researchers have reinforced not only the academic importance of research, but also its relevance to clinical practice and, consequently, to the interconnection of care with the latest scientific evidence, noting that greater exploration of research activities can more effectively enable students to acquire certain core competencies for clinical care (22) .
Evidence-based practice is a key product of adopting strategies to engage students in research projects (7,19) . The future and current success of the correct use of evidence seems to be related to the need to bring students closer to the contexts in which it is created (23) . This study corroborates the opinion of authors who argue that, although the topic of transferring knowledge and using it in clinical settings is not a recent topic in the academic and public debate, it is an emerging practice (24) .
The sample in this literature review suggests that the participation of students in research helps develop their professional identity and, as such, acquire new knowledge and skills. This perspective is supported by the literature, because the exercise of reflecting on and questioning issues, combined with performing tasks in various real-life contexts -activities which occur in research projects -promote the complex integration of core knowledge to the profession, such as professional judgment, skills, values, and attitudes (25) .
Studies have also underscored the importance of supervisors as a guiding actor in the teaching-learning process. However, this should be explored in future research, because in the present study, the analysis of the evidence did not make it clear which models and strategies should be used. In fact, there is evidence that the dynamics between supervisor and supervised lacks planning that allows for the coordination of the project objectives, the supervisor's skills, and the students' monitoring needs (26)(27)(28) , which would prevent the potential loss of interest in research and its alienation from clinical practice. Therefore, knowledge about the best processes to support and monitor students is important to promote more effective learning (27) and enable a more adequate use of knowledge with health gains.
In the long term, investing in the training of nurses with research skills seems to contribute not only to the improvement of the quality of health services but also stimulates innovation industries in the health sector, improving the health level of the population and contributing to national competitiveness and economic growth (28) . Also in the educational sphere, there is a paradigmatic shift, with a pronounced call for the transition from research-informed education, in which students are passive consumers of knowledge, to research-based education, in which students are effectively involved in studies, thus actively assimilating knowledge (29) .
Research is an essential competence for health professionals and implies that they are aware of the latest developments in the health field, are able to critically evaluate the scientific literature, and can use scientific knowledge in clinical decision-making (29)(30)(31) . Therefore, participating in research during the undergraduate programs is an emerging focus to help students develop the skills and competencies necessary for health professionals today (15,19) .

Study limitations
The observational design of the studies included in the sample weakens the demonstration of the importance of the involvement of undergraduate students in research projects because causability cannot be attributed to the intervention in question.
In addition to this factor, the use of internal and/or non-validated measuring instruments by the authors of the articles included in this ILRto assess the extent of student involvement and its impact also represents a limitation. This may have compromised the accuracy of the outcomes, generating greater uncertainty about the translation of the study phenomenon and hindering the dissemination of the instruments used for similar purposes.
Although two studies were qualitative and three were qualiquantitative, which allows for a more in-depth investigation of reality (32) , the small number of articles included in the sample and their level of evidence (level 4), makes it difficult to systematize the evidence about the phenomenon. Although the literature states that the "synthesis of knowledge of the studies included in the review 7 of Nursing students' learning from involvement in research projects: an integrative literature review Loura  reduces the uncertainty about the recommendations, practices, and [allows] (...) an accurate generalization about the phenomenon based on the available information (...)" (14) , more research is needed in the area with more solid designs to better clarify the implications involving nursing students in research projects.

Contributions to the field of nursing
The results of this integrative literature review suggest that the participation of undergraduate nursing students in research projects contributes positively to their learning outcomes and the development of attitudes and scientific skills. Students can be involved in all phases of the research process, with benefits to their learning, motivation, and the development of their own research projects. This pedagogical practice enables learning, reflection and integration of knowledge, development of communication skills, scientific writing, time management, critical judgment, while increasing satisfaction with the learning process.
If these results are corroborated, there are numerous gains to be gained for nursing. It would allow for the training of more qualified professionals, who are aware of the importance of evidence-based practice and health research as a driver of great advances that make a difference in people's lives. The key to a competent health workforce lies in the quality of the training of future professionals.
This area should receive more investment and be further explored, with studies designed to demonstrate causality between the benefits addressed here and the involvement of undergraduate students in research projects. The construction of validated measurement instruments can add value to these same projects and maximize the potential of this topic and its operationalization.

FINAL CONSIDERATIONS
Based on the available evidence, this study presented an overview of the learning outcomes and skills developed by nursing students involved in research projects, promoting greater awareness about the topic. Additionally, it listed benefits that seem inherent to this participation and suggested implications for future professional exercise.
Participation in research projects seems to influence student development in different domains. This includes the acquisition of knowledge and skills, defining new attitudes that promote research culture, and behavior change, which translates knowledge into the practice of health care.
The results point to the importance of creating opportunities from now on to promote the participation of students in research activities during the degree in nursing. Aligned with the main guiding policies, curricular restructuring is a strategy that can bring students and research closer together, fostering its potential benefits. Higher education institutions have a double role: to promote the restructuring of their curricula and "topdown" organizational changes, and promoting changes led by researchers, who within the scope of their activities create the structure, resources, and incentives for students to explore the challenges inherent to research projects.

FUNDING AND ACKNOWLEDGMENT
This work was developed as part of students' engagement in the Train4Health project. The project has received funding from the Erasmus+ Programme of the European Union under the grant agreement no. 2019-1-PT01-KA203-061389. The Funder had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript. The European Commission's support for the production of this publication does not constitute an endorsement of the contents, which reflect the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.
We would also like to thank Nursing School of Lisbon for all the support in the production and publication of this article.