Validation of the Practice Environment Scale among nursing technicians and aides

Abstract

Objective

Evaluate the reliability and validity of the Brazilian version of the Practice Environment Scale among nursing technicians and aides.

Methods

This is a methodological cross-sectional study with 91 randomly selected nursing professionals. The following variables were evaluated: nursing practice environment (using the Brazilian version of the Practice Environment Scale), emotional exhaustion (using the Maslach Burnout Inventory subscale), job satisfaction, and safety climate (using two subscales of the Safety Attitudes Questionnaire – Short Form), perception of care quality, and intention to leave employment. To assess the reliability and validity of subscales of the Practice Environment Scale, Cronbach’s alpha coefficient and Spearman’s correlation coefficient were calculated, respectively, between the subscales of the Practice Environment Scale and the other study variables.

Results

Cronbach’s alpha coefficient ranged from 0.70 to 0.88 among the subscales of the Practice Environment Scale, and all subscales presented a significant correlation with the variables of emotional exhaustion, job satisfaction, safety climate, perception of care quality, and intention to leave employment.

Conclusion

The instrument showed evidence of satisfactory reliability and validity in the assessment of the nursing practice environment. A validated instrument that measures the practice environment of a larger number of nursing professionals can help administrators implement and evaluate strategies to improve the results of patients, nursing professionals, and institutions.

Health facility environment; Licensed practical nurses; Nursing sssistants; Validation studies; Reproducibility of results

Resumo

Objetivo

Avaliar a confiabilidade e a validade da versão brasileira da Practice Environment Scale entre técnicos e auxiliares de enfermagem.

Métodos

Estudo metodológico, transversal, realizado com 91 profissionais selecionados de maneira aleatória. As seguintes variáveis foram avaliadas: ambiente da prática profissional da enfermagem (por meio da versão brasileira da Practice Environment Scale ), exaustão emocional (por uma subescala do Inventário de Burnout de Maslach), satisfação profissional e clima de segurança (por meio de duas subescalas do Safety Attitudes Questionnaire Short – Form ), percepção da qualidade do cuidado e intenção de deixar o emprego. Para avaliar a confiabilidade das subescalas da Practice Environment Scale e a validade, foram calculados, respectivamente, o coeficiente alfa de Cronbach e o coeficiente de correlação de Spearman entre as subescalas da Practice Environment Scale e as demais variáveis em estudo.

Resultados

O coeficiente alfa de Cronbach variou de 0,70 a 0,88 entre as subescalas da Practice Environment Scale , sendo que todas obtiveram correlação significante com as variáveis exaustão emocional, satisfação profissional, clima de segurança, percepção da qualidade do cuidado e intenção de deixar o emprego.

Conclusão

O instrumento demonstrou evidências de confiabilidade e validade satisfatórias para avaliar o ambiente da prática desses profissionais. A disponibilização de um instrumento validado que mensura o ambiente da prática de um maior contingente de profissionais da enfermagem pode auxiliar os gestores a implementarem e avaliarem estratégias que influenciem a melhoria dos resultados com pacientes, profissionais e instituições.

Ambiente de instituições de saúde; Técnicos de enfermagem; Auxiliares de enfermagem; Estudos de validação; Reprodutibilidade dos testes

Resumen

Objetivo

Evaluar la confiabilidad y la validez de la versión brasileña de la Practice Environment Scale entre técnicos y auxiliares de enfermería.

Métodos

Estudio metodológico, transversal, realizado con 91 profesionales seleccionados de manera aleatoria. Se evaluaron las siguientes variables: ambiente de la práctica profesional de enfermería (mediante la versión brasileña de la Practice Environment Scale ), agotamiento emocional (por una subescala del Inventario Burnout de Maslach), satisfacción profesional y clima de seguridad (mediante dos subescalas del Safety Attitudes Questionnaire Short – Form ), percepción de la calidad del cuidado e intención de dejar el trabajo. Para evaluar la confiabilidad de las subescalas de la Practice Environment Scale y la validez, se calcularon, respectivamente, el coeficiente alfa de Cronbach y el coeficiente de correlación de Spearman entre las subescalas de la Practice Environment Scale y las demás variables en estudio.

Resultados

El coeficiente alfa de Cronbach tuvo una variación de 0,70 a 0,88 entre las subescalas de la Practice Environment Scale , y todas obtuvieron correlación significativa con las variables agotamiento emocional, satisfacción profesional, clima de seguridad, percepción de la calidad del cuidado e intención de dejar el trabajo.

Conclusión

El instrumento demostró evidencias de confiabilidad y validez satisfactorias para evaluar el ambiente de la práctica de estos profesionales. Poner a disposición un instrumento validado que mide el ambiente de la práctica de un mayor contingente de profesionales de la enfermería puede ayudar a los gestores a implementar y evaluar estrategias que influyan en la mejora de los resultados con pacientes, profesionales e instituciones.

Ambiente de instituciones de salud; Enfermeros no diplomados; Asistentes de enfermería; Estudios de validación; Reproducibilidad de los resultados

Introduction

Nursing professionals constitute the largest group of workers in a hospital. As they have an important role and influence on the results of patients, it is extremely important to provide this team with an environment where they can develop their practice with quality.( ¹ )

The nursing practice environment can be assessed by the presence or absence of characteristics that favor the development of their activities. These favorable characteristics include: nurse participation in hospital affairs; staffing and resource adequacy; nursing foundations for quality of care; collegial nurse-physician relations; and nurse manager ability, leadership, and support of nurses.( ² )

Organizations that are able to maintain these characteristics above have better results, not only regarding patients, but also with nursing professionals and institutions.( ³ - ⁶ )

Regarding patients, there is evidence of a better perception of the nursing team regarding care safety, lower mortality rates and fewer complications.( ⁷ - ¹⁰ )In relation to nursing professionals, lower levels of emotional exhaustion( ¹¹ , ¹² )and greater satisfaction( ⁷ , ¹¹ )are observed. For institutions, an influence on reducing staff turnover is reported.( ⁷ , ¹³ )

In the context described above, instruments were developed to evaluate the presence of characteristics that contribute to the development of nursing activities, including the Practice Environment Scale (PES), which was developed from the first studies with magnet hospitals. PES has been used as a measurement of nursing care performance, allowing the classification of the nursing environment of institutions as: mixed, favorable or unfavorable settings.( ² )It has been used in different cultures and has demonstrated satisfactory measurement properties.( ¹⁴ - ¹⁶ )

In Brazil, it was adapted and validated with a sample of nurses.( ¹⁶ )Considering that 80% of Brazilian nursing professionals are nursing technicians and aides,( ¹⁷ )the following question guided this study: “Is the PES also a reliable and valid tool to measure the presence of characteristics that favor the professional practice of nursing technicians and aides?”

To answer this question, this study was conducted to evaluate the reliability and validity of the Brazilian version of the PES among nursing technicians and aides.

Methods

This is a methodological cross-sectional study conducted in a public hospital with 260 beds in the state of São Paulo, which provides tertiary care through the Unified Health System (SUS) – the public health system in Brazil – and performs teaching and research activities. The sample size was based on the study objective, which was to validate the instrument by assessing the validity of relationships with external variables, i.e., with related constructs.( ¹⁸ )For this reason, the following conditions were assumed: 80% power, 5% significance level, 0.30 correlation coefficient( ¹⁹ ), and 0.00 correlation coefficient as the null hypothesis, resulting in a minimum sample of 84 participants.

Inclusion criteria were nursing technicians and aides from all sectors of the hospital (emergency room, operating room, pediatrics, surgical and medical units, and adult, pediatric and neonatal intensive care units) directly involved in patient care, with experience in the unit of three months or more. Professionals who did not answer any item or who answered “not applicable” to more than 50% of the subscales of the Safety Attitudes Questionnaire (SAQ) – Short Form 2006 were excluded.

Data were collected using the Brazilian version of the PES, the emotional exhaustion subscale of the Maslach Burnout Inventory (MBI), SAQ subscales of job satisfaction and safety climate, and two questions – one to evaluate the perception of nursing professionals regarding the quality of patient care and one to evaluate their intention to leave employment the following year.

The original PES version consists of 31 items; however, after a confirmatory factor analysis, the Brazilian version was developed with 24 items in five subscales: “nurse participation in hospital affairs” (items 5, 13, 17, 19, 22); “staffing and resource adequacy” (items 1, 7, 8, 10); “nursing foundations for quality of care” (items 4, 14, 15, 18, 21, 23, 24); “nurse manager ability, leadership, and support of nurses” (items 3, 6, 9, 11, 16), and “collegial nurse-physician relations” (items 2, 12, 20). A Likert measurement scale of 1-4 points was used, with higher scores representing the greater presence of attributes favorable to professional nursing practice. Scores of subscales should be obtained with the average scores of participant answers.( ¹⁶ )

The MBI subscale of emotional exhaustion evaluates how often professionals experience situations of work-related physical and mental exhaustion. This subscale has nine items and it is the most evident manifestation and the main element of the burnout syndrome. A Likert scale of 1-5 points was used, with higher scores expressing negative feelings of professionals regarding employment. Scores of subscales should be obtained with the average scores of participant answers.( ²⁰ )

The answers to the subscale of job satisfaction, which consists of five items that represent the professional’s positive view of the workplace, and to the SAQ subscale of safety climate, which has seven items that represent the institution’s commitment to patient safety from the professional’s perspective, were obtained through a Likert scale of 1-5 points. Scores should be added up and divided by the number of questions answered, excluding “not applicable” answers. Scores greater than 75 indicate satisfied professionals who consider the environment safe for the patient.( ²¹ )

Besides these subscales, two questions based on the literature were used to evaluate the perception of quality of patient care and the professional’s intention to leave employment the following year.( ⁶ , ⁸ )Both questions were evaluated using a scale of 0-10 points, where higher scores represented better perception of quality and greater intention to leave employment.

After the randomization of nursing professionals, data were collected from November 2017 to February 2018 by one of the researchers. Nursing technicians and aides were contacted at their workplaces and those who agreed to participate received an envelope containing the assessment instruments which they should fill out. These instruments were filled out by participants and returned to the researcher in the same work shift.

Data were inserted in spreadsheets of Microsoft Excel 2010 for Windows® and analyzed using Statistical Analysis System (SAS) for Windows®, version 9.1.3. Descriptive statistics were used to calculate absolute and relative frequencies for categorical variables, and position and dispersion measurements were calculated for continuous variables.

Cronbach’s alpha coefficient was calculated to evaluate internal consistency, where values equal to or greater than 0.70 were considered satisfactory.( ²² )Evidence of validity based on relationships between related constructs was assessed using Spearman’s correlation coefficient between the subscales of the Brazilian version of the PES and the other study variables to test the following hypotheses: the higher the score in the subscales of the PES, the higher the score of job satisfaction, safety climate and perception of care quality, and the lower the score of emotional exhaustion and intention to leave employment. The level of significance adopted for all tests was 5% (p<0.05).

The project was approved by the university’s research ethics committee (process nº 2.302.308) and complied with all national ethical standards and procedures for research with human beings. Participants were informed about the confidentiality of the information obtained and all participants signed an informed consent form.

Results

The sample consisted of 91 nursing technicians and aides, mean age of 40.8 years (sd=9.8), experience in the role of 10.5 years (sd=7.5), and time in the nursing sector of 6.2 years (sd=4.8). Most participants were female (95.6%), of completed high school (70.3%), and were nursing aides (50.6%). In addition, 48.2% were married and 27.5% worked in intensive care units, 27.5% in inpatient units, 26.4% in the operating room, and 18.7% in the emergency room. Table 1 shows the values from the evaluation of instrument reliability and validity.

Table 1
Reliability and correlation of subscales of the Practice Environment Scale with the variables of emotional exhaustion, job satisfaction, safety climate, perception of care quality, and intention to leave employment.

Practice Environment Scale	Reliability	Emotional exhaustion	Job satisfaction	Safety climate	Perception of quality	Intention to leave employment
Nurse participation in hospital affairs	0.87*	-0.40†	0.48†	0.67†	0.25‡	-0.31‡
Nursing foundations for quality of care	0.88*	-0.43†	0.52†	0.57†	0.34‡	-0.31‡
Nurse manager ability, leadership, and support of nurses	0.86*	-0.41†	0.53†	0.69†	0.28‡	-0.37‡
Staffing and resource adequacy	0.78*	-0.48†	0.53†	0.45†	0.39‡	-0.24‡
Collegial nurse-physician relations	0.70*	-0.44†	0.43†	0.58†	0.22‡	-0.22‡

• *Value obtained by Cronbach’s alpha coefficient; †p<0.0001 and ‡p<0.05 obtained by Spearman’s correlation coefficient.

Discussion

Several instruments are available for the scientific community. However, many of them have not been properly validated, so researchers must carefully evaluate the psychometric properties of questionnaires before selecting them for a study, especially in terms of reliability and validity of measurements.( ²³ )

The reliability assessment of an instrument shows its ability to reproduce consistent results in time and space. By choosing internal consistency to assess reliability, researchers analyze whether the subscales of an instrument measure the same characteristic and whether the answers to the items are consistent.( ²³ )

When comparing Cronbach’s alpha values obtained by other authors in the validation of PES in different cultures, this study showed similar values (0.70-0.88) to the original version (0.71-0.84), Chinese (0.65-0.87), Portuguese (0.71-0.89), Korean (0.80-0.84), Spanish (0.71-0.84), Australian (070-0.89), and Japanese (0.78-0.86).( ² , ²⁴ - ²⁹ )

These differences reflect variations in reliability depending on the researchers, the population, the questionnaire application, and study type and purpose; therefore, setting a minimum acceptable value is very important. All coefficients obtained in the reliability evaluation of the subscales of the PES in this study were equal to or greater than 0.70.( ²² , ²³ )

When assessing validity, researchers want to check whether the instrument measures what it intends to measure, highlighting the importance of assessing both reliability and validity, as instruments showing high reliability values may not be valid.( ²² )When correlating the subscales of the PES with the variables that indirectly evaluated patient results, data agree with those found in the literature showing favorable environments for the development of nursing activities contribute to a better perception of care quality and safety climate.( ¹⁶ , ³⁰ - ³² )

Regarding the variables to estimate the influence of the environment on the results from nursing professionals, the findings of this study are again consistent with those found in the literature, demonstrating that favorable environments contribute to higher job satisfaction and lower levels of emotional exhaustion and burnout.( ¹⁶ , ³¹ )

In addition, regarding the influence of the environment on the results of institutions, data obtained in this study agree with those from the national and international literature showing favorable environments directly influence the intention of these professionals to leave their jobs.( ¹³ , ³¹ )Many authors use the intention of professionals to leave employment as an indicator that measures the impact of the environment on the institutions, as staff turnover generates costs that can be up to three times the average salary of a nursing professional.( ³² )

When using instrument validity based on the relations between related constructs to validate the Brazilian version in this new population, all hypotheses were confirmed, that is, in environments that favor the development of professional activities, nursing technicians and aides have positive perceptions regarding the quality of patient care and safety climate, feel more satisfied and less emotionally exhausted, and have less intention to leave their jobs.

A validated instrument that measures the practice environment of a larger number of nursing professionals can help administrators implement and evaluate strategies to improve the environment where nursing professionals develop their activities, contributing to favorable results of patients, nursing professionals, and institutions.

Conclusion

The Brazilian version of the PES is a reliable and valid tool to measure characteristics that favor the professional practice of nursing technicians and aides.

Referências

• ¹
  Boaretto F, Haddad MCFL, Rossaneis MA, Gvozd R, Pissinati PSC. The work environment of nurses who perform care activities in a university hospital. Cogitare Enferm. 2016;21(2):1-10.
• ²
  Lake ET. Development of the practice environment scale of the Nursing Work Index. Res Nurs Health. 2002;25(3):176–88.
• ³
  Aiken LH, Patrician PA. Measuring organizational traits of hospitals: the Revised Nursing Work Index. Nurs Res. 2000;49(3):146–53.
• ⁴
  Stimpfel AW, Sloane DM, McHugh MD, Aiken LH. Hospitals known for nursing excellence associated with better hospital experience for patients. Health Serv Res. 2016;51(3):1120–34.
• ⁵
  Aiken LH, Clarke SP, Sloane DM, Lake ET, Cheney T. Effects of hospital care environment on patient mortality and nurse outcomes. J Nurs Adm. 2008;38(5):223–9.
• ⁶
  Aiken LH, Sloane DM, Clarke S, Poghosyan L, Cho E, You L, et al. Importance of work environments on hospital outcomes in nine countries. Int J Qual Health Care. 2011;23(4):357–64.
• ⁷
  Dorigan GH, Guirardello EB. Nursing practice environment, satisfaction and safety climate: the nurses’ perception. Acta Paul Enferm. 2017;30(1):129–35.
• ⁸
  Kutney-Lee A, Stimpfel AW, Sloane DM, Cimiotti JP, Quinn LW, Aiken LH. Changes in patient and nurse outcomes associated with magnet hospital recognition. Med Care. 2015;53(6):550–7.
• ⁹
  Cho E, Sloane DM, Kim EY, Kim S, Choi M, Yoo IY, et al. Effects of nurse staffing, work environments, and education on patient mortality: an observational study. Int J Nurs Stud. 2015;52(2):535–42.
• ¹⁰
  McHugh MD, Rochman MF, Sloane DM, Berg RA, Mancini ME, Nadkarni VM, et al.; American Heart Association’s Get With The Guidelines-Resuscitation Investigators. Better nurse staffing and nurse work environments associated with increased survival of inhospital cardiac arrest patients. Med Care. 2016;54(1):74–80.
• ¹¹
  Gasparino RC, Guirardello EB. Professional practice environment and burnout among nurses. Rev Rene. 2015;16(1):90–6.
• ¹²
  Vandenbroeck S, Van Gerven E, De Witte H, Vanhaecht K, Godderis L. Burnout in Belgian physicians and nurses. Occup Med (Lond). 2017;67(7):546–54.
• ¹³
  Nelson-Brantley HV, Park SH, Bergquist-Beringer S. Characteristics of the nursing practice environment associated with lower unit-level RN turnover. J Nurs Adm. 2018;48(1):31–7.
• ¹⁴
  Lake ET. The nursing practice environment: measurement and evidence. Med Care Res Rev. 2007;64(2 Suppl):104S–22S.
• ¹⁵
  Jesus EH, Roque SM, Amaral AF. Estudo RN4Cast em Portugal: ambientes de prática de enfermagem. Rev Invest Enferm. 2015:26–44.
• ¹⁶
  Gasparino RC, Guirardello EB. Validation of the Practice Environment Scale to the Brazilian culture. J Nurs Manag. 2017;25(5):375–83.
• ¹⁷
  Fundação Oswaldo Cruz (Fiocruz). Pesquisa inédita traça perfil da enfermagem no Brasil [Internet]. Rio de Janeiro: Fiocruz; 2015 [citado 2018 Out 22]. Disponível em: https://portal.fiocruz.br/noticia/pesquisa-inedita-traca-perfil-da-enfermagem-no-brasil
  » https://portal.fiocruz.br/noticia/pesquisa-inedita-traca-perfil-da-enfermagem-no-brasil
• ¹⁸
  Santos AA, Anache AA, Amaral AE, Werlang BS, Reppold CT, Nunes CH, et al. Avaliação psicológica: diretrizes na regulamentação da profissão. Brasília (DF): Conselho Federal de Psicologia; 2010.
• ¹⁹
  Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. USA: LEA; 1988. p. 79–80.
• ²⁰
  Tamayo MR. Relação entre a síndrome do burnout e os valores organizacionais no pessoal de enfermagem de dois hospitais públicos [dissertação]. Brasília (DF): Universidade de Brasília; 1997.
• ²¹
  de Carvalho RE, Cassiani SH. Cross-cultural adaptation of the Safety Attitudes Questionnaire - Short Form 2006 for Brazil. Rev Lat Am Enfermagem. 2012;20(3):575–82.
• ²²
  Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use. 5th ed. Oxford, UK: Oxford University Press; 2015.
• ²³
  Souza AC, Alexandre NM, Guirardello EB. Psychometric properties in instruments evaluation of reliability and validity. Epidemiol Serv Saude. 2017;26(3):649–59.
• ²⁴
  Chiang HY, Lin SY. Psychometric testing of the Chinese version of nursing practice environment scale. J Clin Nurs. 2009 ;18(6):919–29.
• ²⁵
  Ferreira MR, Martins JJ. Study of adaptation and validation of the Practice environment scale of the nursing work index for the Portuguese reality. Rev Esc Enferm USP. 2014;48(4):690–7.
• ²⁶
  Cho E, Choi M, Kim EY, Yoo IY, Lee NJ. [Construct validity and reliability of the Korean version of the practice environment scale of nursing work index for Korean nurses]. J Korean Acad Nurs. 2011;41(3):325–32.
• ²⁷
  Alzate LC, Bayer GL, Squires A. Validation of a Spanish version of the practice environment scale of the nursing work index in the Colombian context. Hisp Health Care Int. 2014;12(1):34–42.
• ²⁸
  Parker D, Tuckett A, Eley R, Hegney D. Construct validity and reliability of the Practice Environment Scale of the Nursing Work Index for Queensland nurses. Int J Nurs Pract. 2010;16(4):352–8.
• ²⁹
  Ogata Y, Sasaki M, Yumoto Y, Yonekura Y, Nagano M, Kanda K. Reliability and validity of the practice environment scale of the nursing work index for Japanese hospital nurses. Nurs Open. 2018;5(3):362–9.
• ³⁰
  Aiken LH, Sloane DM, Ball J, Bruyneel L, Rafferty AM, Griffiths P. Patient satisfaction with hospital care and nurses in England: an observational study. BMJ Open. 2018;8(1):e019189.
• ³¹
  Nogueira LS, Sousa RM, Guedes ES, Santos MA, Turrini RN, Cruz DA. Burnout and nursing work environment in public health institutions. Rev Bras Enferm. 2018;71(2):336–42.
• ³²
  Wan Q, Li Z, Zhou W, Shang S. Effects of work environment and job characteristics on the turnover intention of experienced nurses: the mediating role of work engagement. J Adv Nurs. 2018;74(6):1332–41.

Publication Dates

• Publication in this collection
  10 June 2020
• Date of issue
  2020

History

• Received
  19 Aug 2019
• Accepted
  21 Oct 2019