Acessibilidade / Reportar erro

Validation of the Nursing Activities Score in Portuguese intensive care units

ABSTRACT

Objective:

to describe the process of adaptation and validation of the Nursing Activities Score to the Portuguese context.

Method:

this was a pilot study of adaptation and validation of the Nursing Activities Score with a sample consisting of 67 patients hospitalized in the intensive care units of three Portuguese hospitals. The construct validity was assessed through factor analysis procedures and the internal consistency of the items was measured through the Cronbach's alpha coefficient.

Results:

a mean workload value of 63.04% (SD = 14.25; Median = 61.30) was obtained. Psychometric data revealed a Cronbach's alpha of 0.71 in the total scale, indicating an acceptable accuracy. Confirmatory factor analysis suggested an appropriate adjustment between the model and the data (χ2(199) = 214.5, p = 0.214; CFI = 0.95; RMSA = 0.035).

Conclusion:

in the present study, the Portuguese version of the Nursing Activities Score was found to be a valid instrument, enabling a safe assessment of the workload of nurses.

Descriptors:
Workload; Intensive Care Units; Nursing; Management; Nursing Activities Score

RESUMO

Objetivo:

descrever o processo de adaptação e validação do Nursing Activities Score para o contexto português.

Método:

trata-se de um estudo-piloto de adaptação e validação do Nursing Activities Score, com amostra de 67 doentes internados em unidades de cuidados intensivos de três hospitais portugueses. A validade de constructo avaliou-se mediante procedimentos de análise fatorial e a consistência interna dos itens através do coeficiente Alpha de Cronbach.

Resultados:

obteve-se um valor médio da carga de trabalho de 63,04% (DP = 14,25; Mediana = 61,30). Os dados psicométricos revelaram um Alpha de Cronbach de 0,71, na escala total, indicando uma fidelidade aceitável. A análise fatorial confirmatória sugeriu um ajustamento adequado entre o modelo e os dados (χ2(199) = 214,5, p = 0,214; CFI = 0,95; RMSA = 0,035).

Conclusão:

neste estudo, a versão portuguesa do Nursing Activities Score revelou-se um instrumento válido, permitindo avaliar a carga de trabalho dos enfermeiros com segurança.

Descritores:
Carga de Trabalho; Unidades de Cuidados Intensivos; Enfermagem; Gestão; Nursing Activities Score

RESUMEN

Objetivo:

describir el proceso de adaptación y validación del Nursing Activities Score al contexto portugués.

Método:

estudio piloto de adaptación y validación del Nursing Activities Score, con muestra de 67 pacientes internados en unidades de cuidados intensivos de tres hospitales portugueses. La validez del constructo se evaluó mediante análisis factorial y por consistencia interna de los ítems evaluados a través del coeficiente Alpha de Cronbach.

Resultados:

se obtuvo un valor medio de carga de trabajo de 63,04% (SD=14,25; Mediana=61,30). Los datos psicométricos expresaron un Alpha de Cronbach de 0,71 en la escala total, indicando fidelidad aceptable. El análisis factorial confirmatorio sugirió un ajuste adecuado entre el modelo y los datos (χ2(199)=214,5; p=0,214; CFI=0,95; RMSA=0,035).

Conclusión:

en este estudio, la versión portuguesa del Nursing Activities Score demostró ser un instrumento válido, permitiendo evaluar la carga de trabajo de los enfermeros con precisión.

Descriptores:
Carga de Trabajo; Unidades de Cuidados Intensivos; Enfermería; Gestión; Nursing Activities Score

INTRODUCTION

In recent years, in Portugal, there has been an increasing concern regarding the development of efficient management models and formulas to calculate the number of nursing professionals(11 Portugal. Ordem dos Enfermeiros. Norma para o cálculo de dotações seguras dos cuidados de enfermagem. Lisboa: Ordem dos Enfermeiros [Internet]. 2014[cited 2014 Dec 14]; Available from: http://www.ordemenfermeiros.pt/comunicacao/paginas/norma-para-o-calculo-de-dotacoes-seguras-dos-cuidados-de-enfermagem-publicada-em-diario-da-republica.aspx
http://www.ordemenfermeiros.pt/comunicac...
) required in each unit in order to enable acceptable levels of safety and quality in health and an effective management of costs(22 Padilha K, Barbosa R, Oliveira E, Andolhe R, Ducci A, Secoli S. Patient safety in Intensive Care Units: development of a research project. Rev Esc Enferm USP [Internet]. 2015[cited 2014 Sep 14];49(Esp):154-60. Available from: http://www.scielo.br/pdf/reeusp/v49nspe/en_1980-220X-reeusp-49-spe-0157.pdf
http://www.scielo.br/pdf/reeusp/v49nspe/...
). In this area, it is pertinent to identify objective indicators that enable the measurement of the real needs in terms of nursing professionals in the various health units and organizations. Furthermore, considering the specific and highly specialized nature of the provision of nursing care, particularly in multipurpose intensive care units (UCIPs), a reliable and sustainable theoretical and practical framework is required to characterize the interventions resulting from the needs presented by critical patients. It should be noted that in these units the requirement of intensive care with quality and safety is extremely expensive(33 Telles SC, Castilho V. Staff cost in direct nursing care at an intensive care unit. Rev Latino-Am Enfermagem [Internet]. 2007[cited 2014 Sep 14];15(5):1005-9. Available from: http://www.scielo.br/pdf/rlae/v15n5/v15n5a18.pdf
http://www.scielo.br/pdf/rlae/v15n5/v15n...
-44 Panunto MR, Guirardello EB. Carga de trabalho de enfermagem em Unidade de Terapia Intensiva de um hospital de ensino. Acta Paul Enferm [Internet]. 2012[cited 2014 Sep 14];25(1):96-101. Available from: http://www.scielo.br/pdf/reeusp/v49n6/pt_0080-6234-reeusp-49-06-0960.pdf
http://www.scielo.br/pdf/reeusp/v49n6/pt...
).

In this context, this study highlights the Nursing Activities Score (NAS) as a tool that enables the assessment of the workload of nurses based on the time spent on nursing activities inherent to the provided care and independent of the level of severity of the disease presented by the critical patient(55 Miranda D, Nap R, Rijk A, Schaufelli W, Lapichino G. Nursing Activities Score. Crit Care Med [Internet]. 2003[cited 2014 Sep 14];31(2):374-382. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12576939
http://www.ncbi.nlm.nih.gov/pubmed/12576...
). The total scoring translated a percentage that may range between a minimum fixed value of 1.4% and maximum of 177%. The authors established a cut-off point, and total scores above 100% represent indicators of the need for at least two nursing professionals to provide health care to a given patient/day(55 Miranda D, Nap R, Rijk A, Schaufelli W, Lapichino G. Nursing Activities Score. Crit Care Med [Internet]. 2003[cited 2014 Sep 14];31(2):374-382. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12576939
http://www.ncbi.nlm.nih.gov/pubmed/12576...
-66 Queijo A, Padilha K. Nursing Activities Score (NAS): Cross-cultural adaptation and validation to Portuguese language. Rev Esc Enferm USP [Internet]. 2009[cited 2014 Sep 14];43(Spe): 1001-8. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/a04v43ns
http://www.scielo.br/pdf/reeusp/v43nspe/...
).

The NAS was cross-culturally adapted to the Portuguese language spoken in Brazil(66 Queijo A, Padilha K. Nursing Activities Score (NAS): Cross-cultural adaptation and validation to Portuguese language. Rev Esc Enferm USP [Internet]. 2009[cited 2014 Sep 14];43(Spe): 1001-8. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/a04v43ns
http://www.scielo.br/pdf/reeusp/v43nspe/...
), and it was used to assess the workload of nurses in intensive care units (ICUs)* * In the Brazilian context, UCI refers to Intensive Care Units - ICUs (Unidade de Terapia Intensiva – UTI – in Portuguese). . In these studies, the reported mean scores in terms of working hours of nurses ranged between 51.5% and 66.5% in general adult ICUs(77 Nogueira L, Santos M, Mataloun S, Moock, M. Nursing Activities Score: comparação com o índice APACHE II e a mortalidade em pacientes admitidos em unidade de terapia intensiva. Rev Bras Ter Intensiva [Internet]. 2007[cited 2014 Sep 14];19(3):327-30. Available from: http://www.scielo.br/pdf/rbti/v19n3/v19n3a10.pdf
http://www.scielo.br/pdf/rbti/v19n3/v19n...
); between 66.7% and 73.7% in cardiology ICUs(66 Queijo A, Padilha K. Nursing Activities Score (NAS): Cross-cultural adaptation and validation to Portuguese language. Rev Esc Enferm USP [Internet]. 2009[cited 2014 Sep 14];43(Spe): 1001-8. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/a04v43ns
http://www.scielo.br/pdf/reeusp/v43nspe/...
,88 Ducci A, Zanei S, Whitaker I. Nursing workload to verify nurse/patient ratio in a cardiology ICU. Rev Esc Enferm USP [Internet]. 2008[cited 2014 Sep 14];42(4):673-8. Available from: http://www.scielo.br/pdf/reeusp/v42n4/en_v42n4a08.pdf
http://www.scielo.br/pdf/reeusp/v42n4/en...
); and around 65.2% in neurology ICUs(99 Queijo A. Estudo comparativo da carga de trabalho em unidades de terapia intensiva geral e especializadas, segundo o Nursing Activities Score (NAS) [Tese]. São Paulo: Universidade de São Paulo; 2008.). Other NAS application initiatives have been conducted in units with patients requiring a high dependency of care who are still in the units of other specialties, such as gastroenterology, nephrology, trauma, and others(1010 Lima MK, Tsukamoto R, Fugulin FM. Aplicação do Nursing Activities Score em pacientes de alta dependência de enfermagem. Texto Contexto Enferm [Internet]. 2008[cited 2014 Sep 14];17(4):638-46. Available from: http://www.scielo.br/pdf/tce/v17n4/03.pdf
http://www.scielo.br/pdf/tce/v17n4/03.pd...

11 Panunto M, Guirardello E. Nursing workload at a gastroenterology unit. Rev Latino-Am Enfermagem[Internet]. 2009[cited 2014 Sep 14];17(6):1009-14. Available from: http://www.scielo.br/pdf/rlae/v19n5/11.pdf
http://www.scielo.br/pdf/rlae/v19n5/11.p...

12 Trepichio PB, Guirardello EB, Duran ECM, Brito AP. Patient profile and nursing workload at the nephrology unit. Rev Gaúcha Enferm [Internet]. 2013 [cited 2016 Aug 19];34(2):133-9. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1983-14472013000200017&lng=en
http://www.scielo.br/scielo.php?script=s...
-1313 Goulart L, Aoki R, Vegian C, Guirardello E. Carga de trabalho de enfermagem em uma unidade de terapia intensiva de trauma. Rev Eletrônica Enf [Internet]. 2014[cited 2015 Sep 14];16(2):346-51. Available from: https://www.fen.ufg.br/fen_revista/v16/n2/pdf/v16n2a10.pdf
https://www.fen.ufg.br/fen_revista/v16/n...
). Results of Brazilian studies conducted in ICUs are similar to the NAS average found in other countries such as Spain(1414 Carmona-Monge F, Jara-Pérez A, Quirós-Herranz C, Rollán-Rodríguez G, Cerrillo-González I, García-Gómez S, et al. Assessment of nursing workload in three groups of patients in a spanish ICU using the nursing activities score scale. Rev Esc Enferm USP [Internet]. 2013[cited 2014 Sep 14];47(2):335-40. Available from: http://www.scielo.br/pdf/reeusp/v47n2/en_09.pdf
http://www.scielo.br/pdf/reeusp/v47n2/en...
) and Italy(1515 Lucchini A, De Felippis C, Elli S, Schifano L, Rolla F, Pegoraro F, et al. Nursing Activities Score (NAS): 5 years of experience in the intensive care units of an Italian University hospital. Intensive Crit Care Nurs [Internet]. 2014[cited 2014 Sep 14];30(3):152-8. Available from: http://www.intensivecriticalcarenursing.com/article/S0964-3397(13)00113-4/abstract
http://www.intensivecriticalcarenursing....
).

In terms of the practical pertinence of the instrument, some authors report that their studies enabled the adequacy of the number of nurses with a consequent reduction of costs in a private hospital; evidence of a deficit of nursing professionals in a public teaching hospital; and the identification of the particularities of patients hospitalized in public and private ICUs in relation to the required care needs(1616 Padilha KG, Sousa RMC, Garcia PC, Bento ST, Finardi EA, Hatarashi RHK. Nursing workload and staff allocation in an intensive care unit: a pilot study according to Nursing Activities Score (NAS). Intensive Crit Care Nurs [Internet]. 2010[cited 2014 Sep 14];26(2):108-13. Available from: http://www.ee.usp.br/Pos/proesa/public/doc/Padilha%20KG,%20Sousa%20RMC%20de,%20Garcia%20PC,%20Bento%20SCT,%20Finardi%20EM,%20Hatarashi%20RHK..PDF
http://www.ee.usp.br/Pos/proesa/public/d...

17 Inoue K, Matsuda L. Sizing the nursing staff in an intensive care unit for adults. Acta Paul Enferm [Internet]. 2010[cited 2014 Sep 14];23(3):379-84. Available from: http://www.scielo.br/pdf/ape/v23n3/en_v23n3a11.pdf
http://www.scielo.br/pdf/ape/v23n3/en_v2...
-1818 Nogueira L, Koike K, Sardinha D, Padilha K, Sousa R. Nursing workload in public and private intensive care units. Rev Bras Ter Intensiva [Internet]. 2013[cited 2014 Sep 14];25(3):225-32. Available from: http://www.scielo.br/pdf/rbti/v25n3/en_0103-507x-rbti-25-03-0225.pdf
http://www.scielo.br/pdf/rbti/v25n3/en_0...
). In this sense the NAS constitutes a reliable tool from the investigative perspective and has been shown to be extremely useful in the clinical practice context, especially in relation to an effective management. It may even support managers for an appropriate sizing of the nursing staff.

Considering the scored aspects on the importance of applying the NAS to properly measure the workload of nurses in intensive care units, the present study aimed at describing the processes of adaptation and validation of the Nursing Activity Score to the Portuguese context.

METHOD

Ethical aspects

The study was initiated upon formal consent by the original authors of the instrument and the respective linguistic and cross-cultural adaptation to the Portuguese context. It is important to mention that the process of making a version of an international instrument and its application equivalent in a given country consists in a requiring its use in the investigation. Therefore in the present study the translation of the original version into the Portuguese language with subsequent back-translation of the items was performed by nursing experts in the area. This procedure enabled the translation to be appropriate to the Portuguese reality. The next step consisted in the development of a new translation based on the feedback to a spoken pretest and subsequent back-translation to the original language by qualified bilingual experts. Finally, a comparison of the translated version with the back-translation was performed in order to identify inconsistencies and implement the required adjustments, resulting in the definitive instrument.

The data collection procedure involved approval of the research project by the Ethics Subcommittee for Life and Health Sciences of the University of Minho, the administration boards of the hospitals involved, and the respective ethics committees. Subsequently, meetings were conducted with the leading nurses of the respective ICUs in order to clarify the objectives of study, expose the planned methodology to collect data, and define the best strategy for its implementation.

All participating nurses were referred to the investigation team by the leading nurse. Subsequently, formation meetings were developed in order to implement a homogeneous, objective, clarified data collection procedure and obtain voluntary participation and informed written consent. Furthermore, anonymity and confidentiality of data were preserved through the use of specific codes. After discharge, all instruments were delivered to the investigation team in a sealed envelope.

Design, site, and period of study

The NAS adaptation and validation pilot study was conducted in the PICUs of three hospital institutions with different management models located in the northern region of Portugal. The data collection period was 30 consecutive days, encompassing the period between November 2013 and February 2014, according to the availability of each hospital.

Population or sample: inclusion and exclusion criteria

Overall the ICU staff consisted of a total of 87 nurses. The assessment of workload of Portuguese nurses included all adult patients that were hospitalized in these ICUs for a period of 24 hours or more, resulting in a total sample of 67 patients. However, only data referring to the first day of workload assessment after 24 hours of hospitalization in the ICU were considered for the NAS adaptation and validation study, similarly to other studies(55 Miranda D, Nap R, Rijk A, Schaufelli W, Lapichino G. Nursing Activities Score. Crit Care Med [Internet]. 2003[cited 2014 Sep 14];31(2):374-382. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12576939
http://www.ncbi.nlm.nih.gov/pubmed/12576...
,99 Queijo A. Estudo comparativo da carga de trabalho em unidades de terapia intensiva geral e especializadas, segundo o Nursing Activities Score (NAS) [Tese]. São Paulo: Universidade de São Paulo; 2008.).

Study protocol

Data collection protocol consisted of the following assessment tools: (1) a clinical and sociodemographic questionnaire specifically developed to assess the typology of critical patients; and (2) a Portuguese version of the NAS to measure the workload of nurses. Each patient hospitalized in the ICU was subject to an assessment at the end of each work shift by the nurse in charge through the NAS and during the period of hospitalization in the ICU. After the 24 hours corresponding to the beginning of the morning shift (9:00 A.M.), the reference nurse conducted a validation of the records, not only for the NAS but also for patient characterization data.

Statistics and data analysis

The collected data were entered and processed in the software SPSS version 23.0 for Windows. Then an exploratory analysis of the data was performed to assess the fulfillment of the necessary presuppositions to the use of parametric statistics. This analysis consisted in a study of the sensitivity of the items of the instrument as an essential step to the determination of the psychometric characteristics of that instrument. Cronbach's Alpha coefficient was calculated to assess the internal consistency of the items. The validity of the construct was assessed through analysis of the main components and confirmatory factor analysis through the use of the software AMOS, version 23(1919 Marôco J. Análise de equações estruturais: fundamentos teóricos, software & aplicações. Pêro Pinheiro: Report Number; 2010.).

RESULTS

The total sample (N = 67) consisted of patients hospitalized in ICUs of three hospitals in the northern region of Portugal with distinct management models (e.g., public-private partnership [n = 21; 31.3%]; company [n = 29; 43.3%]; and private [n =17; 25.4%]). The mean age of the participants was 66.86 years (SD = 66), consisting of 40 (59.7%) male and 27 (40.3%) female individuals. It should be noted that only 16.4% (n = 11) of the participants had a paid professional activity; the others had inactive employment status at the moment (e.g., retired, housewife, student).

In relation to the admission diagnoses of patients in the ICUs, respiratory pathology was the most recurrent (n = 23; 34.3%), followed by complications in the postoperative period (n = 12; 17.9%), shock (n =10; 14.9%), cardiovascular/cerebrovascular pathologies (n = 7; 10.4%), and trauma (n = 6; 9%). In addition, it was found that 44 (65.7%) patients presented other physical pathologies, and 16 (23.9%) were diagnosed with mental pathologies. In parallel, it was verified that the most recurrent sites of origin consisted of the emergency service (n = 24; 35.8%) and external hospital centers (n = 17; 25.4%). The majority of the target participants of the study (n = 64; 94%) remained in the hospitalization unit for more than 24 hours, with a mean period of time corresponding to 24.59 days (SD = 115.43; Median = 6), ranging between a minimum value of 1 and a maximum of 745 days.

As previously mentioned, only data referring to the first day of assessment after 24 hours of hospitalization in the ICU were considered for the adaptation of the NAS instrument, corresponding in this study to 64 patients and 64 observations (i.e., days of assessment of workload). Therefore, in the considered group (n = 64) an overall workload mean value corresponding to 63.04% was observed (SD = 14.25; Median = 61.30), ranging between a minimum value of 38.00% and maximum of 115.00%.

In the present study the psychometric properties of the NAS were assessed in terms of the reliability and validity of the construct; however, such analysis started with the study of sensitivity of the items. An analysis of the frequency of interventions that make up the instrument was conducted for the sensitivity study. Such frequencies were found to be variable according to the domains that make up the NAS. Although none of the items presented a frequency of 100%, some presented relatively high frequencies, such as: (1) item 2, pertinent to the presence of "biochemical and microbiological laboratory investigations" (n = 63; 98.4%); (2) item 3, referring to the need for "drug therapy, excluding vasoactive drugs" (n = 63; 98.4%); (3) item 5, covering the implementation of "care with drains, except gastric probe" (n = 54; 84.4%); (4) item 6b, pertinent to the implementation of "mobilization and positioning more than 3 times in 24 hours or with 2 nurses" (n = 58; 90.6%); (5) item 8a, considering the implementation of "administrative and routine management tasks with time below 2 hours in a shift" (n = 61; 95.3%); (6) item 9, related to the presence of "respiratory support" (n = 61; 95.3%); and (7) item 17, covering the presence of "quantitative measurement of diuresis" (n = 62; 96.9%).

In contrast, significantly lower frequencies were found for the following items: (1) item 15, related to the presence of "cardiopulmonary resuscitation in the last 24 hours" (n = 2; 3.1%); (2) item 16, related to the need for "dialysis techniques" (n = 1; 1.6%); (3) item 18, related to the need for "intracranial pressure monitoring" (n = 4; 6.3%); and (4) item 20, related to the administration of "total parenteral nutrition" (n = 7; 10.9%). Finally, item 14, related to the need for "left atrium monitoring," had a frequency of zero. For this reason, item 14 was removed from the analysis of the study of the psychometric properties of the instrument in this population. Table 1 shows the results of descriptive statistics of the NAS.

Table 1
Descriptive statistics of the Nursing Activities Score in the first 24 hours (N = 64)

In relation to the reliability analysis, a Cronbach's alpha (α) internal consistency coefficient of 0.705 for the total scale was found. This analysis enabled the identification of items 2, 3, 9, 13, 15, 20, 21, and 23 as critical, that is, items presenting an inappropriate correlation with the total scale (r item/total < 0.20). However, as no significant alteration was verified in the internal consistency value in the total scale when the item was eliminated, and due to the theoretical and practical relevance of the items in the assessment of the workload construct in ICUs, none of these items were removed from the instrument, similarly to other validation studies. In parallel, it was found that the correlations of the remaining items with the total scale ranged between 0.22 and 0.50, suggesting a significant contribution to the total scale.

In relation to the respective scales making up the instrument, low and inadequate internal consistency values were found for the dimensions Basic Activities (α = 0.52), Cardiovascular support (α = 0.05), Renal Support (α = 0.04), and Specific Interventions (α = 0.24). In contrast, higher and acceptable internal consistency coefficients were observed for the dimensions Ventilatory Support (α = 0.60) and Metabolic Support (α = 0.76).

In the validity study of the construct, the items of the instrument were submitted to an analysis of the main components without predefinition of the number of factors, with orthogonal varimax and eigenvalue rotation (≥ 1). The analysis revealed values of commonalities higher than 0.40. Moreover, the analysis of the main components showed that the items of this instrument were organized by seven factors, explaining 63.8% of the total variance. In this case, although a resemblance was verified, there is not an exact reproduction of the original model of the seven factors.

Confirmatory factor analysis was complementarily conducted in order to test the factorial structure proposed in the original model of the instrument. This analysis produced the following adjustment indicators: χ2(199) = 214.5; p = 0.214; χ2/df = 1.078; CFI = 0.95; p CFI = 0.816; RMSA = 0.035; p (RMSA ≤ 0.05) = 0.750. These results suggest a good overall adjustment between the model and the data. The probability of the statistics χ2 being higher than 0.05 and the RMSA value being lower than 0.05 indicates the existence of a good measure of the proximity of adjustment between the model and the data. Finally, the value of the Comparative Fit Index (CFI), reaching 0.95, also suggests a good adjustment of the model. In general terms, the data of the AFC propose that the model may be deemed appropriate.

DISCUSSION

The sociodemographic characteristics of the participants included in the present study refer to a majority of elderly, male patients hospitalized due to respiratory pathologies or postoperative complications, coming from emergency services or external hospital centers, with a mean length of stay in the ICU of 24.59 days. The authors of a Brazilian study conducted in a general adult ICU in the city of São Paulo describe that the participants presented a mean age of 70.4 years, were predominantly male (66.7%), with a mean length of stay of 17 days, coming from the surgical center and emergency service (only 3% had been transferred from other institutions). Regarding the reason for hospitalization in that ICU, 60.6% of the patients had been admitted due to clinical pathologies, and 39.4% due to surgical pathologies(2020 Conishi R, Gaidzinski R. Nursing Activities Score (NAS) como instrumento para medir carga de trabalho de enfermagem em UTI adulto. Rev Esc Enferm USP [Internet]. 2007[cited 2014 Sep 14];41(3):346-54. Available from: http://www.scielo.br/pdf/reeusp/v41n3/02.pdf
http://www.scielo.br/pdf/reeusp/v41n3/02...
). Interestingly, results related to the same characteristics were also found in another study conducted in a university hospital in Madrid, where the sample consisted predominantly of male patients (72.5%) with a mean age of 62.5 years, and a mean length of stay of 3.6 days. Reasons for hospitalization in that ICU included coronary heart disease (58.1%) and acute respiratory failure (34.1%)(1414 Carmona-Monge F, Jara-Pérez A, Quirós-Herranz C, Rollán-Rodríguez G, Cerrillo-González I, García-Gómez S, et al. Assessment of nursing workload in three groups of patients in a spanish ICU using the nursing activities score scale. Rev Esc Enferm USP [Internet]. 2013[cited 2014 Sep 14];47(2):335-40. Available from: http://www.scielo.br/pdf/reeusp/v47n2/en_09.pdf
http://www.scielo.br/pdf/reeusp/v47n2/en...
). Consequently, the present study evidences that despite the existence of data that are similar to other investigations in relation to gender, mean age, reasons for hospitalization, and origin, the patients presented a mean length of stay in the ICU higher than those mentioned in the literature.

The mean value of the workload found in this study was 63.04% (SD = 14.25; Median = 61.30); some similarities were observed in comparison with the results of international investigations. This result is close to the results described by the NAS translation and validation study in Brazil, in which the author introduces a mean score of 61.7% (SD= 8.42)(99 Queijo A. Estudo comparativo da carga de trabalho em unidades de terapia intensiva geral e especializadas, segundo o Nursing Activities Score (NAS) [Tese]. São Paulo: Universidade de São Paulo; 2008.). In another investigation also conducted in the Brazilian context, the authors found a mean NAS score of 66.5% (SD = 9.1%; Median = 70.7)(2121 Gonçalves L, Padilha K. Fatores associados à carga de trabalho de enfermagem em Unidade de Terapia Intensiva. Rev Esc Enferm USP [Internet]. 2007[cited 2014 Sep 14];41(4):645-52. Available from: http://www.scielo.br/pdf/reeusp/v41n4/14.pdf
http://www.scielo.br/pdf/reeusp/v41n4/14...
). In studies conducted in hospital institutions in Spain and Italy, European researchers described NAS mean values of 65.9% (SD= 6.6) and 65.97% (SD = 2.53), respectively(1414 Carmona-Monge F, Jara-Pérez A, Quirós-Herranz C, Rollán-Rodríguez G, Cerrillo-González I, García-Gómez S, et al. Assessment of nursing workload in three groups of patients in a spanish ICU using the nursing activities score scale. Rev Esc Enferm USP [Internet]. 2013[cited 2014 Sep 14];47(2):335-40. Available from: http://www.scielo.br/pdf/reeusp/v47n2/en_09.pdf
http://www.scielo.br/pdf/reeusp/v47n2/en...
-1515 Lucchini A, De Felippis C, Elli S, Schifano L, Rolla F, Pegoraro F, et al. Nursing Activities Score (NAS): 5 years of experience in the intensive care units of an Italian University hospital. Intensive Crit Care Nurs [Internet]. 2014[cited 2014 Sep 14];30(3):152-8. Available from: http://www.intensivecriticalcarenursing.com/article/S0964-3397(13)00113-4/abstract
http://www.intensivecriticalcarenursing....
). By presenting values that are so close to those referred to in the investigation, the present study translates a similarity of realities, suggesting that the Portuguese version of the instrument fits the Portuguese intensive care contexts.

Regarding other mean NAS scores found in the literature, another study conducted in ICUs in Norway presented a higher mean NAS value (≅ 96.2%). The authors relate this fact to the time spent with family members of critical patients (as in this reality there is no time limit for visits) and also with the time spent with intra-hospital transportation, as this is performed by the nursing staff of the ICUs(2222 Stafseth S, Solms D, Bredal I. The characterisation of workloads and nursing staff allocation in intensive care units: a descriptive study using the Nursing Activities Score for the first time in Norway. Intensive Crit Care Nurs [Internet]. 2011[cited 2014 Sep 14];27(5):290-4. Available from: http://www.intensivecriticalcarenursing.com/article/S0964-3397(11)00074-7/abstract
http://www.intensivecriticalcarenursing....
). From an opposite perspective, other investigations reported lower NAS mean values in comparison with the present study, such as the original study of NAS with value of 56% (SD = 17.5%; Median = 54)(55 Miranda D, Nap R, Rijk A, Schaufelli W, Lapichino G. Nursing Activities Score. Crit Care Med [Internet]. 2003[cited 2014 Sep 14];31(2):374-382. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12576939
http://www.ncbi.nlm.nih.gov/pubmed/12576...
); a study conducted in a Spanish university hospital that presented mean NAS of 50.40% (SD = 15.29; Median = 54.80)(2323 Adell AB, Campos RA, Rey MC, Bellmunt JQ, Rochera ES, Muñoz JS, et al. Nursing Activity Score (NAS). Nuestra experiencia con un sistema de cómputo de cargas de enfermería basado en tempos. Enferm Intensiva [Internet]. 2005[cited 2014 Sep 14];16(4):164-73. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16324544
http://www.ncbi.nlm.nih.gov/pubmed/16324...
); and a study conducted in a state hospital of Rio de Janeiro, in Brazil, that found a mean score of 51.50% (SD = 12.1)(77 Nogueira L, Santos M, Mataloun S, Moock, M. Nursing Activities Score: comparação com o índice APACHE II e a mortalidade em pacientes admitidos em unidade de terapia intensiva. Rev Bras Ter Intensiva [Internet]. 2007[cited 2014 Sep 14];19(3):327-30. Available from: http://www.scielo.br/pdf/rbti/v19n3/v19n3a10.pdf
http://www.scielo.br/pdf/rbti/v19n3/v19n...
).

From the perspective of human resources management, the mean workload value obtained in the assessed Portuguese ICUs shows that a hospitalized patient demanded 63.04% of the working time of a nurse during a shift. In the study conducted to build the original version of the NAS, the authors report that the sum of the values found for each assessed item is related to the nursing working time provided per patient during the 24 hours. As an example, a patient with a NAS score of 100% indicates the need for full dedication of a nurse during the full shift(55 Miranda D, Nap R, Rijk A, Schaufelli W, Lapichino G. Nursing Activities Score. Crit Care Med [Internet]. 2003[cited 2014 Sep 14];31(2):374-382. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12576939
http://www.ncbi.nlm.nih.gov/pubmed/12576...
).

Regarding the analysis of frequency of the interventions that make up the instrument, similar results to a previously mentioned Brazilian study were found. A high frequency (95.5%) was also described by the author in item 17 in relation to the performance of quantitative measurement of diuresis. Similarly to the present study, low frequencies of cardiopulmonary resuscitation interventions, hemofiltration techniques, performance of intracranial pressure monitoring, and use of total parenteral nutrition were verified. Finally, in item 14, related to the need for monitoring the left atrium, the Brazilian study described a value of 2.5%, differing from the frequency of zero found in the present study(99 Queijo A. Estudo comparativo da carga de trabalho em unidades de terapia intensiva geral e especializadas, segundo o Nursing Activities Score (NAS) [Tese]. São Paulo: Universidade de São Paulo; 2008.). Thus, it is possible to assume that these interventions are underused in the selected and studied ICUs, consisting of a type of care that requires minor participation by the nurse.

In relation to the analysis of reliability, the value of the Cronbach's alpha coefficient is higher than the value reported in the translation and validation study in Brazil(66 Queijo A, Padilha K. Nursing Activities Score (NAS): Cross-cultural adaptation and validation to Portuguese language. Rev Esc Enferm USP [Internet]. 2009[cited 2014 Sep 14];43(Spe): 1001-8. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/a04v43ns
http://www.scielo.br/pdf/reeusp/v43nspe/...
), and was set as an indicator of a reasonable internal consistency of the instrument(66 Queijo A, Padilha K. Nursing Activities Score (NAS): Cross-cultural adaptation and validation to Portuguese language. Rev Esc Enferm USP [Internet]. 2009[cited 2014 Sep 14];43(Spe): 1001-8. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/a04v43ns
http://www.scielo.br/pdf/reeusp/v43nspe/...
). The low and inappropriate internal consistency values for the dimensions Basic Activities, Cardiovascular Support, Renal Support, and Specific Interventions are consistent with values reported in another validation study(66 Queijo A, Padilha K. Nursing Activities Score (NAS): Cross-cultural adaptation and validation to Portuguese language. Rev Esc Enferm USP [Internet]. 2009[cited 2014 Sep 14];43(Spe): 1001-8. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/a04v43ns
http://www.scielo.br/pdf/reeusp/v43nspe/...
). In contrast, higher and more acceptable internal consistency coefficients were observed for the dimensions Ventilatory Support and Metabolic Support, exceeding the results found in the referred study. It should be noted that, as in the Brazilian NAS validation study(66 Queijo A, Padilha K. Nursing Activities Score (NAS): Cross-cultural adaptation and validation to Portuguese language. Rev Esc Enferm USP [Internet]. 2009[cited 2014 Sep 14];43(Spe): 1001-8. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/a04v43ns
http://www.scielo.br/pdf/reeusp/v43nspe/...
), it was decided not to eliminate any of the items of the instrument due to the theoretical and practical relevance of the items in the assessment of the construct of workload in ICUs.

According to a research study published in the United States in 2005(2525 Costello AB, Osborne J. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Eval [Internet]. 2005[cited 2014 Sep 14];10(7):1-9. Available from: http://pareonline.net/pdf/v10n7.pdf
http://pareonline.net/pdf/v10n7.pdf...
), the observed commonalities values show a significant contribution by the items for the dimensions of the instrument in the present study. Furthermore, although not precisely reproducing the original model(55 Miranda D, Nap R, Rijk A, Schaufelli W, Lapichino G. Nursing Activities Score. Crit Care Med [Internet]. 2003[cited 2014 Sep 14];31(2):374-382. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12576939
http://www.ncbi.nlm.nih.gov/pubmed/12576...
), the analysis of main components showed that the items of this instrument are organized by the seven factors. Considering the similarity to the original model and the possibility to establish comparisons with cross-cultural studies, it was decided to maintain the organization proposed by the authors, as it happened in other studies(66 Queijo A, Padilha K. Nursing Activities Score (NAS): Cross-cultural adaptation and validation to Portuguese language. Rev Esc Enferm USP [Internet]. 2009[cited 2014 Sep 14];43(Spe): 1001-8. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/a04v43ns
http://www.scielo.br/pdf/reeusp/v43nspe/...
,1515 Lucchini A, De Felippis C, Elli S, Schifano L, Rolla F, Pegoraro F, et al. Nursing Activities Score (NAS): 5 years of experience in the intensive care units of an Italian University hospital. Intensive Crit Care Nurs [Internet]. 2014[cited 2014 Sep 14];30(3):152-8. Available from: http://www.intensivecriticalcarenursing.com/article/S0964-3397(13)00113-4/abstract
http://www.intensivecriticalcarenursing....
). Confirmatory factor analysis data revealed a good measure of adjustment proximity between the model and the data, suggesting that the model may be deemed appropriate.

One last point to be considered is that the present study assessed the workload of nurses in ICUs considering only nursing activities conducted according to the needs of the hospitalized patients. That is, it disregarded other factors that may affect the workload of nurses and human resources planning.

CONCLUSION

ICUs represent a scenario of innovation, development of technology, and work processes; they are recognized as contexts that promote quality in health, particularly in differentiated nursing care. The present pilot study was conducted in ICUs with different management models, and it aimed for the adaptation and validation of the NAS instrument to the Portuguese context.

Overall it was verified that the workload of nurses in the studied units is in accordance with studies conducted at the international level, strengthening the homogeneity in relation to the comparison of results. However, the study also enables the identification of a high and representative overload of work in Portuguese nurses.

Some items assessed through the NAS presented low frequencies because they were virtually absent from the nursing interventions within the period of the study. However, it is possible to consider that the items with higher frequencies depict the needs of care presented by patients and the respective nursing activities in Portuguese ICUs. In this sense, the NAS constitutes an important instrument of assessment of the workload of nurses in ICUs; its psychometric results shows consistency and robustness, according to the results of the analyzed original studies. Further in-depth studies are suggested to assess the workload in ICUs in other Portuguese hospital realities with different management models in order to achieve results that support safe staffing and the maintenance of quality nursing care.

It should also be noted that the definitive Portuguese version of the instrument-Nursing Activities Score, Braga, Portugal, 2013-shall be made available by the authors upon request.

  • *
    In the Brazilian context, UCI refers to Intensive Care Units - ICUs (Unidade de Terapia Intensiva – UTI – in Portuguese).

ACKNOWLEDGMENTS

Thank you to the administration boards of the hospitals that supported the implementation of this study. Thank you to the participant nurses; without their dedicated, honest work this investigation would not be possible. Thank you to the population of the sample that agreed to participate in the study. Finally, thank you to other researchers that conducted the same NAS validation processes in other countries for their help and experience. A special thanks to Professor Miranda and Professor Katia Padilha.

REFERÊNCIAS

  • 1
    Portugal. Ordem dos Enfermeiros. Norma para o cálculo de dotações seguras dos cuidados de enfermagem. Lisboa: Ordem dos Enfermeiros [Internet]. 2014[cited 2014 Dec 14]; Available from: http://www.ordemenfermeiros.pt/comunicacao/paginas/norma-para-o-calculo-de-dotacoes-seguras-dos-cuidados-de-enfermagem-publicada-em-diario-da-republica.aspx
    » http://www.ordemenfermeiros.pt/comunicacao/paginas/norma-para-o-calculo-de-dotacoes-seguras-dos-cuidados-de-enfermagem-publicada-em-diario-da-republica.aspx
  • 2
    Padilha K, Barbosa R, Oliveira E, Andolhe R, Ducci A, Secoli S. Patient safety in Intensive Care Units: development of a research project. Rev Esc Enferm USP [Internet]. 2015[cited 2014 Sep 14];49(Esp):154-60. Available from: http://www.scielo.br/pdf/reeusp/v49nspe/en_1980-220X-reeusp-49-spe-0157.pdf
    » http://www.scielo.br/pdf/reeusp/v49nspe/en_1980-220X-reeusp-49-spe-0157.pdf
  • 3
    Telles SC, Castilho V. Staff cost in direct nursing care at an intensive care unit. Rev Latino-Am Enfermagem [Internet]. 2007[cited 2014 Sep 14];15(5):1005-9. Available from: http://www.scielo.br/pdf/rlae/v15n5/v15n5a18.pdf
    » http://www.scielo.br/pdf/rlae/v15n5/v15n5a18.pdf
  • 4
    Panunto MR, Guirardello EB. Carga de trabalho de enfermagem em Unidade de Terapia Intensiva de um hospital de ensino. Acta Paul Enferm [Internet]. 2012[cited 2014 Sep 14];25(1):96-101. Available from: http://www.scielo.br/pdf/reeusp/v49n6/pt_0080-6234-reeusp-49-06-0960.pdf
    » http://www.scielo.br/pdf/reeusp/v49n6/pt_0080-6234-reeusp-49-06-0960.pdf
  • 5
    Miranda D, Nap R, Rijk A, Schaufelli W, Lapichino G. Nursing Activities Score. Crit Care Med [Internet]. 2003[cited 2014 Sep 14];31(2):374-382. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12576939
    » http://www.ncbi.nlm.nih.gov/pubmed/12576939
  • 6
    Queijo A, Padilha K. Nursing Activities Score (NAS): Cross-cultural adaptation and validation to Portuguese language. Rev Esc Enferm USP [Internet]. 2009[cited 2014 Sep 14];43(Spe): 1001-8. Available from: http://www.scielo.br/pdf/reeusp/v43nspe/a04v43ns
    » http://www.scielo.br/pdf/reeusp/v43nspe/a04v43ns
  • 7
    Nogueira L, Santos M, Mataloun S, Moock, M. Nursing Activities Score: comparação com o índice APACHE II e a mortalidade em pacientes admitidos em unidade de terapia intensiva. Rev Bras Ter Intensiva [Internet]. 2007[cited 2014 Sep 14];19(3):327-30. Available from: http://www.scielo.br/pdf/rbti/v19n3/v19n3a10.pdf
    » http://www.scielo.br/pdf/rbti/v19n3/v19n3a10.pdf
  • 8
    Ducci A, Zanei S, Whitaker I. Nursing workload to verify nurse/patient ratio in a cardiology ICU. Rev Esc Enferm USP [Internet]. 2008[cited 2014 Sep 14];42(4):673-8. Available from: http://www.scielo.br/pdf/reeusp/v42n4/en_v42n4a08.pdf
    » http://www.scielo.br/pdf/reeusp/v42n4/en_v42n4a08.pdf
  • 9
    Queijo A. Estudo comparativo da carga de trabalho em unidades de terapia intensiva geral e especializadas, segundo o Nursing Activities Score (NAS) [Tese]. São Paulo: Universidade de São Paulo; 2008.
  • 10
    Lima MK, Tsukamoto R, Fugulin FM. Aplicação do Nursing Activities Score em pacientes de alta dependência de enfermagem. Texto Contexto Enferm [Internet]. 2008[cited 2014 Sep 14];17(4):638-46. Available from: http://www.scielo.br/pdf/tce/v17n4/03.pdf
    » http://www.scielo.br/pdf/tce/v17n4/03.pdf
  • 11
    Panunto M, Guirardello E. Nursing workload at a gastroenterology unit. Rev Latino-Am Enfermagem[Internet]. 2009[cited 2014 Sep 14];17(6):1009-14. Available from: http://www.scielo.br/pdf/rlae/v19n5/11.pdf
    » http://www.scielo.br/pdf/rlae/v19n5/11.pdf
  • 12
    Trepichio PB, Guirardello EB, Duran ECM, Brito AP. Patient profile and nursing workload at the nephrology unit. Rev Gaúcha Enferm [Internet]. 2013 [cited 2016 Aug 19];34(2):133-9. Available from: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1983-14472013000200017&lng=en
    » http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1983-14472013000200017&lng=en
  • 13
    Goulart L, Aoki R, Vegian C, Guirardello E. Carga de trabalho de enfermagem em uma unidade de terapia intensiva de trauma. Rev Eletrônica Enf [Internet]. 2014[cited 2015 Sep 14];16(2):346-51. Available from: https://www.fen.ufg.br/fen_revista/v16/n2/pdf/v16n2a10.pdf
    » https://www.fen.ufg.br/fen_revista/v16/n2/pdf/v16n2a10.pdf
  • 14
    Carmona-Monge F, Jara-Pérez A, Quirós-Herranz C, Rollán-Rodríguez G, Cerrillo-González I, García-Gómez S, et al. Assessment of nursing workload in three groups of patients in a spanish ICU using the nursing activities score scale. Rev Esc Enferm USP [Internet]. 2013[cited 2014 Sep 14];47(2):335-40. Available from: http://www.scielo.br/pdf/reeusp/v47n2/en_09.pdf
    » http://www.scielo.br/pdf/reeusp/v47n2/en_09.pdf
  • 15
    Lucchini A, De Felippis C, Elli S, Schifano L, Rolla F, Pegoraro F, et al. Nursing Activities Score (NAS): 5 years of experience in the intensive care units of an Italian University hospital. Intensive Crit Care Nurs [Internet]. 2014[cited 2014 Sep 14];30(3):152-8. Available from: http://www.intensivecriticalcarenursing.com/article/S0964-3397(13)00113-4/abstract
    » http://www.intensivecriticalcarenursing.com/article/S0964-3397(13)00113-4/abstract
  • 16
    Padilha KG, Sousa RMC, Garcia PC, Bento ST, Finardi EA, Hatarashi RHK. Nursing workload and staff allocation in an intensive care unit: a pilot study according to Nursing Activities Score (NAS). Intensive Crit Care Nurs [Internet]. 2010[cited 2014 Sep 14];26(2):108-13. Available from: http://www.ee.usp.br/Pos/proesa/public/doc/Padilha%20KG,%20Sousa%20RMC%20de,%20Garcia%20PC,%20Bento%20SCT,%20Finardi%20EM,%20Hatarashi%20RHK..PDF
    » http://www.ee.usp.br/Pos/proesa/public/doc/Padilha%20KG,%20Sousa%20RMC%20de,%20Garcia%20PC,%20Bento%20SCT,%20Finardi%20EM,%20Hatarashi%20RHK..PDF
  • 17
    Inoue K, Matsuda L. Sizing the nursing staff in an intensive care unit for adults. Acta Paul Enferm [Internet]. 2010[cited 2014 Sep 14];23(3):379-84. Available from: http://www.scielo.br/pdf/ape/v23n3/en_v23n3a11.pdf
    » http://www.scielo.br/pdf/ape/v23n3/en_v23n3a11.pdf
  • 18
    Nogueira L, Koike K, Sardinha D, Padilha K, Sousa R. Nursing workload in public and private intensive care units. Rev Bras Ter Intensiva [Internet]. 2013[cited 2014 Sep 14];25(3):225-32. Available from: http://www.scielo.br/pdf/rbti/v25n3/en_0103-507x-rbti-25-03-0225.pdf
    » http://www.scielo.br/pdf/rbti/v25n3/en_0103-507x-rbti-25-03-0225.pdf
  • 19
    Marôco J. Análise de equações estruturais: fundamentos teóricos, software & aplicações. Pêro Pinheiro: Report Number; 2010.
  • 20
    Conishi R, Gaidzinski R. Nursing Activities Score (NAS) como instrumento para medir carga de trabalho de enfermagem em UTI adulto. Rev Esc Enferm USP [Internet]. 2007[cited 2014 Sep 14];41(3):346-54. Available from: http://www.scielo.br/pdf/reeusp/v41n3/02.pdf
    » http://www.scielo.br/pdf/reeusp/v41n3/02.pdf
  • 21
    Gonçalves L, Padilha K. Fatores associados à carga de trabalho de enfermagem em Unidade de Terapia Intensiva. Rev Esc Enferm USP [Internet]. 2007[cited 2014 Sep 14];41(4):645-52. Available from: http://www.scielo.br/pdf/reeusp/v41n4/14.pdf
    » http://www.scielo.br/pdf/reeusp/v41n4/14.pdf
  • 22
    Stafseth S, Solms D, Bredal I. The characterisation of workloads and nursing staff allocation in intensive care units: a descriptive study using the Nursing Activities Score for the first time in Norway. Intensive Crit Care Nurs [Internet]. 2011[cited 2014 Sep 14];27(5):290-4. Available from: http://www.intensivecriticalcarenursing.com/article/S0964-3397(11)00074-7/abstract
    » http://www.intensivecriticalcarenursing.com/article/S0964-3397(11)00074-7/abstract
  • 23
    Adell AB, Campos RA, Rey MC, Bellmunt JQ, Rochera ES, Muñoz JS, et al. Nursing Activity Score (NAS). Nuestra experiencia con un sistema de cómputo de cargas de enfermería basado en tempos. Enferm Intensiva [Internet]. 2005[cited 2014 Sep 14];16(4):164-73. Available from: http://www.ncbi.nlm.nih.gov/pubmed/16324544
    » http://www.ncbi.nlm.nih.gov/pubmed/16324544
  • 24
    Marôco J. Análise estatística com o SPSS Statistics. 6 ed. Pêro Pinheiro: Report Number; 2014.
  • 25
    Costello AB, Osborne J. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Eval [Internet]. 2005[cited 2014 Sep 14];10(7):1-9. Available from: http://pareonline.net/pdf/v10n7.pdf
    » http://pareonline.net/pdf/v10n7.pdf

Publication Dates

  • Publication in this collection
    Sep-Oct 2016

History

  • Received
    23 May 2016
  • Accepted
    15 June 2016
Associação Brasileira de Enfermagem SGA Norte Quadra 603 Conj. "B" - Av. L2 Norte 70830-102 Brasília, DF, Brasil, Tel.: (55 61) 3226-0653, Fax: (55 61) 3225-4473 - Brasília - DF - Brazil
E-mail: reben@abennacional.org.br