Evaluation of an admission program for the nursing team

ABSTRACT

Objective:

To evaluate the efficacy of training programs administered to the nursing team during the integration period.

Methods:

This evaluation and experimentation research project was developed with 52 nursing professionals. Data collection included the evaluation of reactions, which measured professionals' feelings about the instructor and the training content, as well as the evaluation of learning, which measured the training efficacy through technical multiple-choice questions concerning the content given in two moments: before the beginning and in the end of the integration period.

Results:

The means of the evaluation of reactions of the practical and laboratory training were considered to be excellent, and pre- and post-training grades were 5.9 and 7.2 points (p < 0.0001) respectively.

Conclusion:

Trainings ministered during the integration period were effective and developed the skills needed to improve patients' health care quality and safety.

Keywords:
Nursing, Team; Educational Measurement; In-service Training

RESUMO

Objetivo:

Avaliar a eficácia dos treinamentos ministrados à equipe de enfermagem, no período de integração.

Métodos:

Pesquisa de avaliação somativa e experimentação desenvolvida com 52 profissionais de enfermagem. Para a coleta de dados, foram consideradas a avaliação de reação, que mensurou o sentimento do profissional em relação ao instrutor e ao conteúdo do treinamento, e a avaliação da aprendizagem, que mensurou a eficácia do treinamento, por meio de questões técnicas de múltipla escolha sobre o conteúdo ministrado em dois momentos: antes do início e no término do período de integração.

Resultados:

As médias da avaliação de reação do treinamento prático e de laboratório foram consideradas excelentes, e as notas do pré e pós-teste foram, respectivamente, 5,9 e 7,2 pontos (p < 0,0001).

Conclusão:

Os treinamentos ministrados durante o período de integração foram eficazes e desenvolveram as competências necessárias para a melhoria da qualidade da assistência e segurança do paciente.

Palavras-chave:
Equipe de Enfermagem; Avaliação Educacional; Capacitação em Serviço

RESUMEN

Objetivo:

Evaluar la eficacia de los entrenamientos ministrados al equipo de enfermería durante el período de integración.

Métodos:

Investigación de evaluación somática y experimentación desarrollada con 52 profesionales de enfermería. Para la recolección de datos fueron consideradas la evaluación de reacción, que medió el sentimiento del profesional acerca del instructor y al ejercicio, y la evaluación del aprendizaje, la cual mensuró la eficacia de la capacitación por medio de cuestiones técnicas de selección múltiple sobre el contenido ministrado en dos momentos: antes del inicio de integración y al final del periodo.

Resultados:

Las métricas de evaluación de reacción del ejercicio práctico y del laboratorio fueron consideradas excelentes, y las notas antes y después del teste fueron, respectivamente, 5,9 y 7,2 puntos (p < 0,0001).

Conclusión:

Los ejercicios realizados durante el período de integración fueron eficaces y desarrollan las competencias necesarias para una mejoría de la cualidad de asistencia y seguridad del paciente.

Palabras clave:
Grupo de Enfermería; Evaluación Educacional; Capacitación em Servicio

INTRODUCTION

Organizations have faced economic challenges as new employees need to adapt to their workplaces and, due to the requirements of production, their training period is compromised and the image of such organizations is consequently at risk¹.

To minimize risks, some institutions have managed to maintain their admission training programs, aiming to facilitate the adaptation of new employees to their norms, routines, procedures and protocols, thus standardizing health care¹.

Aiming to continually improve the performance of health professionals, who frequently encounter situations that require immediate decision-making, the importance of continuing education and in-service education to guarantee quality health care to patients stands out^2,3.

With regard to education, four areas should be emphasized: introduction or admission to work, training, updating and development. In any of these areas, educational projects must be aimed at meeting the needs of participants, institutions and patients².

For this reason, the process of educational project evaluation is essential for those responsible to intervene, aiming to make it more effective, although this is an aspect still poorly developed in in-service education proposals⁴.

Educational program evaluations aim to identify whether the objectives of training programs have been met. A successful evaluation depends on the analysis of needs for training, instructors' skills, and the institutions' preparation, which includes the involvement of managers and trainers⁵.

The first ideas about educational program evaluation were published in 1959⁵ and dealt with the evaluation of the efficacy of training programs on four levels: reaction (what those instructed think and feel about this training); learning (increase in knowledge or ability to change and reproduce new behavior), behavior (improvement in the ability to use and apply knowledge), and results (practical effects on the environment resulting from the performance of those instructed)⁶. The author recommends the application of all these measures for a complete and meaningful evaluation of learning in organizations⁵.

In Brazil, studies on the evaluation of educational programs and actions performed by health professionals are not common yet and, for this reason, the investigation of such theme is extremely important⁷, as improvement and qualification are key conditions for professional development, which is directly related to patients' health care quality and safety.

In view of what has been described, the present study aimed to assess the efficacy of nursing team training programs during the integration period.

METHODS

A study on summative evaluation and experimentation⁸ was performed in a not-for-profit private institution in the countryside of the state of São Paulo, in Brazil. This institution has 238 beds and provides health care to patients covered by the Sistema Único de Saúde (SUS - Unified Health System) for urgency, emergency, orthopedic, and medical and surgical clinic services.

The total number of individuals admitted in the nursing area in the past ten months (n = 200) was considered to calculate the sample size, including a sampling error of 10% and a confidence interval of 95%, totaling a minimum sample size of 66 professionals. The sample was obtained by convenience and the following inclusion criteria were used: newly-hired nurses and nursing technicians and assistants who participated in the integration program.

This program aims to facilitate the adaptation of newly-hired professionals to their new work environment, familiarizing them with the institution's norms, routines, procedures and protocols.

Data collection was performed between May and August 2014 with the nurse/continuing education coordinator during the integration period of newly-hired professionals. The following were used to collect data: a file to characterize the sample, an instrument to evaluate learning, and an instrument to assess the reactions to training. The analysis was based on two aspects: the evaluation of reaction, aimed at assessing the feelings of those instructed towards their instructor and the content taught during the training program; and the evaluation of learning, aimed at assessing whether this program contributed to increasing their knowledge. The evaluation of knowledge and results was not included as this was a cross-sectional study and thus did not allow these professionals to be followed during the period of their professional performance.

A characterization file was developed aiming to assess participants' personal and professional profile, comprised of questions about their personal (age, sex and marital status) and professional characteristics (qualification, position, more than one employment relationship in the nursing area, length of experience in nursing, participation in refreshing courses in the previous year and work sector).

To assess reaction, the instrument developed by the institution was used, including options of response for each item presented in a Likert scale that varied from one to four points: 1 is bad, 2 is fair, 3 is good, and 4 is great, i.e. the higher the score, the more positive the feeling of those instructed towards their training.

With regard to learning, an evaluation file was used, including 15 multiple-choice questions about the technical content taught during training programs (venous puncture, airway aspiration, surgical wound dressing, medications, giving baths, performing electrocardiograms, vital signs, expiry date of medical devices, nursing notes/prescriptions, gas therapy, catheter fixation and dressing, oral hygiene and precautions) with the nurse/continuing education coordinator of the institution and based on protocols and the content taught by them during the integration period of newly-hired professionals.

Tests were applied immediately before the beginning and after the last training program held during the integration period to verify whether there was a variation in knowledge that could be attributed to this training. Pre- and post-tests were identical, so that there was no difference in their level of difficulty and so that it would be possible to compare the performance of those being trained. The mean grade attributed to each participant varied from zero to ten points and the higher scores represented better performances. It should be emphasized that only one model of test was taken by all participants from different nursing professional categories, as there is not a specific training program for each.

Integration took place every week, from Mondays through Fridays, except for the last week of every month. Professionals stayed with those responsible for several hospital sectors from 7:30am to 5:00pm.

On the first day (Friday), newly-hired professionals from all areas participated in the institution's integration program, which included those admitted to all areas of this institution, when they received guidance on the electronic timecard, transportation allowance, human resources department norms, cafeteria opening hours, psychology services, Information Technology (IT), Núcleo de Educação Permanente (NEP - Continuing Education Center), nursing management, quality, safety, press advisory, Serviço de Controle de Infecção Hospitalar (SCIH - Hospital Infection Control Service), Serviço Especializado em Engenharia de Segurança e Medicina do Trabalho (SESMT - Labor Medicine and Safety Engineering Service), removal of Personal Protective Equipment (PPE) from the warehouse and introduction to work sectors (except for the nursing team, whose sector was introduced on the last day of the admission training program).

On the second day (Tuesday), professionals began their jobs, except for the nursing team, who continued their specific theoretical-practical training program throughout the week. Nursing professionals received guidance and attended lectures with a more specific content for the area of nursing, including the SCIH, SESMT, pharmacy, psychology, Equipe Multiprofissional de Terapia Nutricional (EMTN - Multi-professional Team of Nutritional Therapy) and physiotherapy.

On the third day (Wednesday), the following themes were approached in the laboratory and classroom: gas therapy, airway aspiration, orotracheal tube/tracheostomy, peripheral catheter fixation, venous puncture, expiry date of medical devices, electrocardiogram, hospital electronic system training, wound prevention and treatment committee, and ostomies.

On the fourth day (Thursday), nursing professionals received practical training in different hospital sectors, where routines were approached, such as measuring vital signs, bed/aspersion bath, simple wound dressing, peripheral venous puncture, transportation to in-hospital exams, operational system, patient safety (identification of patients and expiry date of medical devices). In the afternoons, they would watch a motivational video and presentations given by those responsible for the clinical engineering, clinical nutrition, hotel management and speech therapy sectors.

On the fifth and last day (Friday), newly-hired professionals received practical training in different sectors again to become acquainted with the hospital's routine. Apart from what was approached on the previous day, the nursing process and notes in the hospital's electronic system were also emphasized. In the afternoon, they attended lectures given by those responsible for the Comissão Intra-Hospitalar de Doação de Órgãos e Tecidos para Transplante (CIHDOTT - Intra-Hospital Committee of Organ and Tissue Donation for Transplants), nephrology and electronic medical records. Finally, professionals received instructions about the nursing records and post-death care and were shown around all hospital sectors.

Data were typed into the Excel-Windows/XP^® software and treated statistically by a specialized professional. Descriptive analysis was made, frequency tables and percentage of categorical variables were prepared and the position of continuous variables (mean and standard deviation) was measured. The Mann-Whitney test was used to analyze the domains of nominal variables with two categories, while the Kruskal-Wallis test was used for variables with three or more categories. The comparison of means before and after the training program was assessed with Student's t-test.

This research project was submitted to the Research Ethics Committee of the Jundiaí Medical School and, upon approval, data collection was conducted (official opinion 706.501/2014).

RESULTS

The study sample was comprised of 52 newly-hired professionals with a mean age of 30.1 years (standard deviation - SD ± 6.2) and length of experience in this profession of 2.0 years (SD ± 3.8). The majority were females (90.4%) who did not have an employment relationship (80.8%). The remaining information about sample characterization is described in Table 1.

Table 1
Description of sample characteristics. City of Jundiaí, SP, Brazil, 2014

Characteristics	n	%
Marital status
Single	26	50.0
Married	19	36.5
Divorced	7	13.5
Qualification
Nursing assistant	27	51.9
Nursing technician	12	23.1
Nurse	5	9.6
Nursing specialist	8	15.4
Job performed
Nursing assistant	39	75.0
Nurse	13	25.0
Work sector where one performs
Medical clinic	22	42.3
Emergency unit	13	25.0
Surgical clinic	8	15.4
Intensive Care Unit	7	13.5
Surgical center	2	3.8

With regard to the evaluation of reaction, the means attributed to participants during laboratory and practical training programs are described in Table 2.

Table 2
Description of means of grades from the evaluation of reaction to training programs. City of Jundiaí, SP, Brazil, 2014

Variable	n	Mean	Standard deviation
Practical training program	52	3.8*	0.4
Laboratory training program	52	3.8*	0.4

• ^*
  Paired Student’s t-test

The mean of theoretical test grades obtained by newly-hired professionals was 5.9 points (SD ± 1.4) in the pre-test and 7.2 points (SD ± 1.5) in the post-test (p < 0.0001).

Table 3 shows the results obtained when comparing the means obtained in the theoretical evaluation in the pre- and post-tests among different professional categories.

Table 3
Means of pre- and post-test grades according to professional categories. City of Jundiaí, SP, Brazil, 2014

	Nursing assistant	Nursing technician	Nurse	p-value
Pre-test mean	5.4	6.3	7.0	0.04*
Post-test mean	6.5	7.4	8.5	0.00*

• ^*
  Kruskal-Wallis test.

Another significant finding was the positive correlation between the grades obtained by participants and the length of experience in this profession, i.e. the longer the experience, the better the performance in the tests (p = 0.01).

DISCUSSION

The minimum number of participants was not reached, as the number of professionals hired during data collection was not similar throughout the months included for sample calculation, thus resulting in an increase in 2% in sample error.

The personal and professional characteristics of study participants are very similar to the Brazilian nursing profile, which is mostly comprised of women, despite the increasing number of male nurses⁹.

Although some studies have emphasized the fact that nursing is not well paid in Brazil¹⁰, the results of the present study corroborate those that have already been found by other authors, regarding professionals having only one employment relationship¹¹.

The search for professional qualification can also be observed as some have a higher level of education than that required by their function. The Nursing Professional Qualification Project, which consisted in a governmental initiative to promote the improvement of health care quality through the reduction in the number of less qualified personnel, offering free nursing technician and assistant courses¹², could have contributed to this situation.

Additionally, the majority of all the nurses hired had a postgraduate degree and this finding can be justified by the growing number of undergraduate nursing courses in Brazil, which increases the availability of such professionals in the job market, so that nurses seek professional improvement to stand out in selective processes, aiming to be included¹³.

The evaluation of reaction is considered to be the first effect of training, although this can be compromised by several aspects, such as fear of being judged or punished for assessing the workplace negatively¹⁴. Considering the fact that participants of this study were evaluated during the integration period, i.e. when they had just been admitted to the institution and, for this reason, were in the probationary period. Thus, their positive evaluation could reflect one of two contexts: either participants actually found the instructor's teaching skills and the content taught to be excellent or this was a reflection of their feeling of insecurity, taking into consideration the fact that even participants who did not have a good performance assessed their instructors and content taught very positively.

When learning was evaluated, the training program was found to have positively influenced the post-test grades obtained by participants, which was also observed in another study, emphasizing the understanding that training programs held in the workplace represent an improvement in tasks routinely performed by professionals and should direct knowledge in a positive way so that all individuals can develop the abilities and behavior required to improve the results of their institution⁷.

Functions are divided according to levels of complexity and professional qualification, i.e. nursing assistants are responsible for repetitive activities of an average complexity that include simple supporting services under the supervision of a nurse, while nurses (higher education) are responsible for more complex activities, some of which are private, and for all activities developed by nursing technicians and assistants, as nurses are expected to have more knowledge.

This result could be observed in the present study, as nurses showed a better performance both in the pre- and post-tests, compared to the other professional categories with less qualification¹⁵. Considering what has been described here and the fact that tests had the same level of complexity for all categories, perhaps the performance of nurses should have been better.

Another interesting finding was the better performance of professionals with more experience, thus showing that the presence of previous knowledge can facilitate the acquisition of new knowledge.

FINAL CONSIDERATIONS

Training programs for newly-hired nursing professionals during the integration period were effective. In-service training improves knowledge and the techniques usually performed by these professionals, helping them to develop the abilities required for their work.

Institutions must develop strategies to assess behavior and the results as a way to better identify the training needs, including tools to support the development of qualification programs that meet the institution's objectives and improve patients' health care quality and safety.

Different and more in-depth contents and more rigorous evaluations, including more representative samples of the population, should be applied to nurses, given the fact that they have a higher level of qualification and perform functions that require more responsibility in their workplace.

REFERÊNCIAS

• ¹
  Bucchi SM, Mira VL, Otrenti E, Ciampone MH. Enfermeiro instrutor no processo de treinamento admissional do enfermeiro em unidade de terapia intensiva. Acta Paul. Enferm. 2011;24(3):381-7.
• ²
  Bellan MC, Araujo II, Araujo S. Capacitação teórica do enfermeiro para o atendimento da parada cardiorrespiratória. Rev. Bras. Enferm. 2010;63(6):1019-27.
• ³
  Mira VL. Avaliação de programas de treinamento e desenvolvimento da equipe de enfermagem de dois hospitais do município de São Paulo [tese]. São Paulo: Escola de Enfermagem, Universidade de São Paulo; 2010.
• ⁴
  Peduzzi M, Guerra DA, Braga CP, Lucena FS, Silva JA. Atividades educativas de trabalhadores na atenção primária: concepções de educação permanente e de educação continuada em saúde presentes no cotidiano de Unidades Básicas de Saúde em São Paulo. Interface: Comunicação, Saúde, Educação 2009;13(30):121-34.
• ⁵
  Kirkipatrick DL. Evaluation training programs: the four levels. San Francisco: Berrett-Koehler; 1994.
• ⁶
  Tasca JE, Ensslin L, Ensslin SR. A construção de um referencial teórico sobre a avaliação de desempenho de programas de capacitação. Ensaio: Aval. Pol. Públ. Educ. 2013;21(79):203-38.
• ⁷
  Mira VL, Peduzzi M, Melleiro MM, Tronchin DM, Prado MF, Santos PT, et al. Análise do processo de avaliação da aprendizagem de ações educativas de profissionais de enfermagem. Rev. Esc. Enferm. USP. 2011;45(esp):1574-81.
• ⁸
  Lobiondo-Wood G, Haber J. Pesquisa em enfermagem: métodos, avaliação crítica e utilização. Rio de Janeiro: Ed. Guanabara Koogan; 2001.
• ⁹
  Fundação Oswaldo Cruz (FIOCRUZ). Pesquisa inédita traça perfil da enfermagem no Brasil [Internet]. Rio de Janeiro: FIOCRUZ; 2015 [citado 2015 out 9]. Disponível em: http://portal.fiocruz.br/pt-br/content/pesquisa-inedita-traca-perfil-da-enfermagem-no-brasil
  » http://portal.fiocruz.br/pt-br/content/pesquisa-inedita-traca-perfil-da-enfermagem-no-brasil
• ¹⁰
  Grazziano ES, Bianchi ER. Impacto del estrés ocupacional y burnout em enfermeros. Enferm. Glob. [periódico na Internet]. 2010 [citado 2015 out 9];18:2-5. Disponível em: http://scielo.isciii.es/pdf/eg/n18/revision1.pdf
  » http://scielo.isciii.es/pdf/eg/n18/revision1.pdf
• ¹¹
  Rissardo MP, Gasparino RC. Exaustão emocional em enfermeiros de um hospital público. Esc. Anna Nery. 2013;17(1):128-32.
• ¹²
  Bógus CM, Bersusa AA, Martins CL, Escuder MM. Conhecendo egressos do curso técnico de enfermagem do PROFAE. Rev. Esc. Enferm. USP. 2011;45(4):945-52.
• ¹³
  Carrijo CI, Bezerra AL, Munari DB, Medeiros M. A empregabilidade de egressos de um curso de graduação em enfermagem. Rev. Enferm. UERJ. 2007;15(3):356-63.
• ¹⁴
  Andrade RJ, Silva NF. ROI - uma análise comparativa entre os métodos de avaliação sobre o retorno do investimento em treinamento [Internet]. VII Coimbra Administração - Congresso Brasileiro de Administração [citado 2015 out 9]. Disponível em: http://www.convibra.org/upload/paper/adm/adm_1357.pdf
  » http://www.convibra.org/upload/paper/adm/adm_1357.pdf
• ¹⁵
  Conselho Regional de Enfermagem (COREN). Principais legislações para o exercício da enfermagem. 2.ed. São Paulo: Coren; 2013.

Publication Dates

• Publication in this collection
  2016

History

• Received
  28 Oct 2015
• Accepted
  28 Feb 2016