Agreement among evaluators in the Nursing Activities Score application

Abstract

Objective

To assess the agreement among evaluators in the Nursing Activities Score (NAS) application in an adult Intensive Care Unit (ICU).

Methods

This is a methodological study, carried out in an ICU of a public and university hospital in southern Brazil. The researchers were trained to use the NAS, and after that, a researcher considered a reference standard (RR), two research assistants (RA1 and RA2) and nursing assistants (NUR) applied the instrument independently, considering the same patients. Agreement was tested using the intraclass correlation coefficient (ICC) for the final value of NAS, and kappa coefficient, for analysis of the NAS 23 items.

Results

In the final mean of NAS, there was a strong agreement among RR and RA1 (ICC=0.92; 95%CI: 0.89-0.95) and substantial agreement among RR and RA2 (ICC=0.78; 95%CI: 0.64-0.87) and RR and NUR (ICC=0.75; 95%CI: 0.62-0.84). Agreement regarding NAS items was considered perfect or almost perfect in 14 of the 23 items scored by the pairs of evaluators formed by RR with RA1 and RA2, and in two of the 23 items scored by the pair formed by RR with NUR. Agreement was considered poor or relative, with at least one pair of raters, in seven of the 23 items.

Conclusion

Despite the good agreement both in the general mean and in most of the NAS items, divergences were observed, especially in the items of greater subjectivity of the instrument.

Workload; Personnel downsizing; Intensive care units; Critical care; Nursing care

Resumo

Objetivo

Avaliar a concordância entre avaliadores na aplicação do instrumento Nursing Activities Score (NAS) em um Centro de Terapia Intensiva (CTI) adulto.

Métodos

Trata-se de um estudo metodológico, realizado em um CTI de um hospital público e universitário do sul do Brasil. Os pesquisadores foram capacitados para a utilização do NAS, e, após, uma enfermeira considerada padrão de referência (PR), dois assistentes de pesquisa (AP1 e AP2) e os enfermeiros assistenciais (ENF) aplicaram o instrumento de modo independente, considerando os mesmos pacientes. Testou-se a concordância por meio do coeficiente de correlação intraclasse (CCI) para o valor final do NAS e do coeficiente kappa para a análise dos 23 itens que compõem o NAS.

Resultados

Na média final do NAS, obteve-se forte concordância entre a PR e o AP1 (CCI=0,92; IC95%: 0,89-0,95) e concordância substancial entre a PR e o AP2 (CCI=0,78; IC95%: 0,64-0,87) e a PR e os ENF (CCI=0,75; IC95%: 0,62-0,84). A concordância dos itens que compõe o NAS foi considerada como perfeita ou quase perfeita em 14 dos 23 itens pontuados pelas duplas de avaliadores formadas pela PR com o AP1 e AP2, e em dois dos 23 itens pontuados pela dupla formada pela PR com os ENF. A concordância foi considerada pobre ou relativa, com ao menos uma dupla de avaliadores, em sete dos 23 itens.

Conclusão

Apesar da boa concordância tanto na média geral quanto na maior parte dos itens do NAS, observou-se divergências especialmente nos itens de maior subjetividade do instrumento.

Carga de trabalho; Dimensionamento de pessoal; Unidades de terapia intensiva; Cuidados críticos; Cuidados de enfermagem

Resumen

Objetivo

Evaluar la concordancia entre evaluadores en la aplicación del instrumento Nursing Activities Score (NAS) en una Unidad de Cuidados Intensivos (UCI) adulta.

Métodos

Se trata de un estudio metodológico, realizado en una UCI de un hospital público universitario del sur de Brasil. Los investigadores fueron capacitados para la utilización del NAS. Después, una enfermera considerada modelo de referencia (PR), dos asistentes de investigación (AP1 y AP2) y los enfermeros asistenciales (ENF) aplicaron el instrumento de modo independiente, considerando los mismos pacientes. Se probó la concordancia por medio del coeficiente de correlación intraclase (CCI) del valor final del NAS y del coeficiente kappa para el análisis de los 23 ítems que componen el NAS.

Resultados

En el promedio final de NAS, se obtuvo una fuerte concordancia entre la PR y el AP1 (CCI=0,92; IC95 %: 0,89-0,95) y concordancia substancial entre la PR y el AP2 (CCI=0,78; IC95 %: 0,64-0,87) y la PR y los ENF (CCI=0,75; IC95 %: 0,62-0,84). La concordancia de los ítems que componen el NAS fue considerada perfecta o casi perfecta en 14 de los 23 ítems marcados por los pares de evaluadores formados por la PR con el AP1 y el AP2, y en dos de los 23 ítems marcados por el par formado por la PR con los ENF. La concordancia fue considerada pobre o relativa, con al menos un par de evaluadores, en siete de los tres ítems.

Conclusión

A pesar de la buena concordancia, tanto en el promedio general como en la mayor parte de los ítems del NAS, se observaron divergencias, especialmente en los ítems de más subjetividad del instrumento.

Carga de trabajo; Reducción de personal; Unidades de cuidados intensivos; Cuidados críticos; Cuidados de enfermería

Introduction

The Nursing Activities Score (NAS) is an instrument for measuring the nursing workload in intensive care. It was proposed by Miranda et al(¹) in 2003 and developed from a restructuring of the Therapeutic Intervention Scoring System 28 (TISS-28).(²)Its final score results from the sum of the scores obtained through assessment of 23 items and represents what proportion of the time of a nursing professional a patient required for their direct care in the last 24 hours. Each NAS point corresponds to 14.4 minutes and the maximum sum of possible points to be reached is 176.8%. A score of 100% represents that a patient required 100% of a nursing professional’s time in their care in the last 24 hours; therefore, in cases where the final NAS score is greater than 100%, it means that two professionals are needed for adequate assistance.(¹)

Brazil was one of the 15 countries included in the multicenter study that originated the NAS, and was responsible for 5% of the total sample used to derive the instrument.(¹)The NAS was translated into Portuguese and validated by Queijo(³) in 2002. Later, in 2009, its cross-cultural adaptation for use in Brazil was published.(⁴)

In compliance with Brazilian legislation,(⁵,⁶)Intensive Care Units (ICUs) must adopt a Patient Classification System (PCS) to assess hospitalized patients and, thus, estimate the number of nursing staff. The chosen PCS must be reliable and have been published in specialized scientific literature. Currently, the NAS is the PCS used in several Brazilian ICUs.

Despite being widely used, the NAS seems to have little impact on the management routines of health services, which can be explained by several factors. The first one refers to the characteristics of nursing practice in Brazil, which differs from the country of origin of the NAS. In its genesis, the NAS estimates the need for care from a “nursing professional”, not allowing to differentiate the specific demand of nurses and nursing technicians. In Brazil, where the two categories of nursing professionals (nurses and nursing technicians) still provide care, this is a major constraint, because it makes it impossible to estimate the time required by each of the categories and, consequently, to plan a work schedule based exclusively on NAS data. The second point refers to the NAS’ retrospective nature, which estimates, based on a patient’s status in the 24 hours prior to its calculation, the time dedicated to care already performed, limiting its use in a care area as dynamic as ICUs. In this way, the estimate may not correspond to the care required by patients in the different 24-hour work shifts, or provide a more accurate prediction for subsequent work shifts.(⁷)

Finally, it is noteworthy that, despite the NAS being a validated instrument for use in Brazil, there is a need to test scores in each environment in which it is planned to use them, since local factors can affect their accuracy.(⁸)The few studies that were responsible for assessing the agreement among raters in NAS application presented several subitems of the instrument with low agreement.(⁹,¹⁰) Although the NAS is the instrument adopted since 2009 to measure workload in the Intensive Care Unit (ICU) where the present study was developed, agreement among evaluators in this scenario has never been tested. Therefore, the objective of this research was to determine the agreement among evaluators in the NAS application.

Methods

This is a methodological study, carried out in October and November 2019 in a general ICU, except trauma, of a reference hospital in southern Brazil. For the convenience sample, patients older than 18 years were selected, regardless of the reason for hospitalization. Sample demographic and clinical data were obtained from electronic medical records.

Each NAS assessment was performed in duplicate by a nurse, researcher on the subject and active in intensive care for 10 years, with residency in the critical adult area (assessment adopted as a reference standard, called reference researcher (RR)) and by one of two undergraduate nursing students, called research assistant 1 (RA1) and research assistant 2 (RA2). Also, NAS assessment performed by assistants (NUR) was considered.

The NAS was applied retrospectively, considering the previous 24 hours. NUR followed the already established routine, which recommends filling out the NAS once a day, with registration in computerized medical records. RR and RA applied the NAS considering the same period as the NUR. All evaluators applied the NAS independently and blinded to the records of the others. As observed in another study(¹¹)that assessed agreement, initially, 50 assessments were foreseen in duplicate, a number that was exceeded with all pairs of evaluators.

Before data collection, nursing students received training in order to standardize: a) how to approach patients and obtain consent for the study; b) filling in the data collection instrument for sample characterization; and c) understanding of NAS. This last item took place in a theoretical and practical way, with in-depth detailing of each NAS item. Also, RA received specific training for insertion in the field, contemplating good practices in research, clinical and patient safety in research. NUR receive training on the routine use of NAS when they are admitted to work in the unit. Both nurses and nursing students followed the information contained in a manual available for consultation at the study site, whose content summarizes the recommendations for NAS scoring available in literature.(⁴,¹²,¹³)

Also, the percentages of agreement among evaluators were assessed considering each of the NAS categories. Although there is no standardized consensus classification in the literature for the workload (light, moderate or heavy) from the NAS, for the purposes of this study, based on the reasoning of the number of patients that can be cared for by a professional, it was decided to use three categories, based on the classification previously adopted in another study,(¹⁴) according to the NAS range: (1) NAS ≤50%: light workload (one professional for every two patients); (2) NAS between 50.1-99.9%: moderate/high workload (one professional for each patient); and (3) NAS ≥100%: very high (two professionals for each patient).

The analyzes were performed using the Statistical Package for the Social Sciences (SPSS) version 20.0. Values of p<0.05 were considered statistically significant. Global agreement among raters was assessed using the intraclass correlation coefficient (ICC). Agreement among each of the instrument items and subitems was established using the Kappa coefficient, considering: Kappa: 0 to 0.19 = poor agreement; 0.20-0.39 = relative agreement; 0.40-0.59 = moderate agreement; 0.60-0.79 = substantial agreement; 0.80-0.99; = almost perfect agreement; and 1.00 = perfect agreement.(¹⁵)The project was approved by the institution’s Research Ethics Committee, with CAAE (Certificado de Apresentação para Apreciação Ética - Certificate of Presentation for Ethical Consideration) 16288619.0.0000.5327.

Results

We included 56 patients, predominantly men (57.1%), whose mean age was 58.3±17.3 years, admitted for neurological causes (25%), sepsis (21.4%) and respiratory causes (17.9%), with a mean Simplified Acute Physiology Score 3 (SAPS 3) of 64.2±15.1. The most prevalent previous diseases were hypertension (55.4%) and cancer (30.4%). The median ICU hospital stay was 5 (P25:2.2 - P75: 13.7) days, and 21.4% died (Table 1).

Table 1
Sample characteristics

Variables	Patients n(%)
Male	32(57.1)
Age (years) (Mean±SD)	58.3±17.3
SAPS 3 (n= 50) (Mean±SD)	64.2±15.1
Hospitalization type
Clinical	50(89.3)
Surgical	6(10.7)
Reason for ICU admission
Neurological	14(25)
Sepsis	12(21.4)
Respiratory	10(17.9)
Cardiological	8(14.3)
Post-operatory	6(10.7)
Others	6 (10.7)
Previous diseases
Hypertension	31(55.4)
Cancer	17(30.4)
DM	15(26.8)
CKD	11(19.6)
Stroke	8(14.3)
HF	5(8.9)
COPD	4(7.1)
CAD	3(5.4)
Median ICU hospitalization days (P25 - P75)	5(2.2 – 13.7)
Death in the ICU	12(21.4)

• SAPS3 - Simplified Acute Physiology Score 3; ICU - Intensive Care Unit; DM - Diabetes Mellitus; CKD - chronic kidney disease; HF - heart failure; COPD - chronic obstructive pulmonary disease; CAD - coronary artery disease.

In the 56 patients, 250 assessments were performed in duplicate, with 101 assessments by the RR with RA 1, 61 assessments by the RR with RA 2 and 88 by the RR with NUR. Considering the final NAS score, there was strong agreement among the RR and RA1 and substantial agreement among RR and RA2 and RR and NUR (Table 2).

Table 2
Mean values and standard deviations of nursing activities scores assessed by the reference researcher by the second observer and agreement among each pair of evaluators

Researchers	Number of assessments	NAS value %	ICC (95%CI)
Reference researcher	101	79.5±20.7	0.92 (0.89 – 0.95)
Research assistant 1		79.8±23.9
Reference researcher	61	81.3±18.8	0.78 (0.64 – 0.87)
Research assistant 2		78.6±20.5
Reference researcher	88	79.6±21.2	0.75 (0.62 – 0.84)
Nursing assistants		79.7±20.4

• NAS – Nursing Activities Score; ICC – Intraclass Correlation Coefficient; CI - confidence interval

Figure 1 shows the proportion of agreement among raters according to workload category (NAS ≤50%: light workload; NAS between 50.1-99.9%: moderate/high workload; NAS ≥100%: very high workload, in shades of gray). While assessments among RR and RA1 were fully in agreement in the light workload categories (NAS < 50.0%), it was in agreement only in 50% of assessments carried out with RA2 and with NUR. The opposite was identified with regard to the NAS category of moderate/high workload (NAS between 50.1-99.9%), where lower agreements were established with RA1. In the category of very high workload (NAS > 100%), there was little agreement in the classification with RA2, increasing (66.7%) with NUR and reaching close to 90% with RA1.

Figure 1
Proportion of assessments in which there was a coincidence among evaluators in the NAS classification subdivided into three categories in the three pairs of evaluators

RR – reference researcher; RA1 – research assistant 1; RA 2 – research assistant 2; NUR – nursing assistants; NAS - Nursing Activities Score.

Agreement among evaluators in the NAS 23 subitems was tested by obtaining the Kappa coefficient, presented in Table 3. Agreement was considered perfect in items such as “Drug therapy, except vasoactive drugs” and “Intracranial pressure monitoring” among all pairs of evaluators, and in items such as “Left atrium monitoring, pulmonary artery catheter” and “Cardiorespiratory resuscitation in the last 24 hours” in the assessments performed by RR and RA. Also, almost perfect agreement was observed in items such as: “Care with drains”, “Care with artificial airways” in the assessments carried out by RR and RA. On the other hand, agreement was considered poor or relative, with at least one pair of evaluators, in the following items: “Monitoring and control”, “Hygiene procedures”, “Mobilization and positioning” and “Support and care to patients and their families” and “Management and administrative tasks”, “Treatment of metabolic acidosis/alkalosis” and “Specific interventions in the ICU”.

Table 3
Agreement among the reference researcher, research assistants and nursing assistants in the Nursing Activities Score 23 subitems

NAS subitems	RA 1 n= 101 k (95%CI)	RA 2 n= 61 k (95%CI)	NUR n= 88 k (95%CI)
1. Monitoring and control	0.5(0.36-0.63)	0.47(0.29-0.64)	0.32(0.16-0.47)
2. Laboratory investigations: biochemical and microbiological	0.87(0.73-1)	1	0.67(0.44-0.89)
3. Drug therapy, except vasoactive drugs	1	1	1
4. Hygiene procedures	0.38(0.21-0.54)	0.18(0.04-0.41)	0.08(0.03-0.19)
5. Care with drains: all (except gastric tube)	0.82(0.68-0.95)	0.85(0.71-0.99)	0.46(0.21-0.7)
6. Mobilization and positioning	0.22(0.08-0.35)	0.43(0.22-0.63)	0.05(0.02-.019)
7. Support and care to patients and their families	0.48(0.05-0.9)	0.31(0.16-0.79)	0.11(0.03 -0.19)
8. Management and administrative tasks	0.41(0.26-0.55)	0.34(0.11-0.56)	0.27(0.08-0.45)
Ventilatory support 9. Respiratory support 10. Care with artificial airways 11. Treatment of pulmonary function	0.94(0.87-1) 1 0.91(0.83-0.99)	1 1 0.93(0.84-1)	0.63(0.44-0.81) 0.79(0.66-0.91) 0.62(0.45-0.78)
Cardiovascular support 12. Vasoactive drug therapy 13. Intravenous restitution of major fluid loss 14. Left atrium monitoring, pulmonary artery catheter 15. Cardiopulmonary resuscitation in the last 24 hours	0.96(0.9-1) 0.52(0.16-0.88) 1 1	1 0.57(0.23-0.9) 1 1	0.72(0.58-0.87) 0.05(0.01-0.22) 0.64(0.31-0.96) *
Renal support 16. Hemofiltration techniques, dialysis techniques 17 Quantitative measure of diuresis	1 1	1 1	0.78(0.62-0.93) 0.42(0.08-0.76)
Neurological support 18. Intracranial pressure monitoring	1	1	1
Metabolic support 19. Treatment of metabolic acidosis/alkalosis 20. Total parenteral nutrition 21. Enteral nutrition	0.70(0.51-0.89) 1 0.98(0.95 – 1)	0.76(0.53-0.98) 1 1	0.25(0.01-0.51) 0.82(0.51-1) 0.61(0.44-0.78)
Specific interventions 22. Specific interventions in the ICU 23. Specific interventions out of the ICU	0.72(0.58-0.85) 0.91(0.79-1)	0.67(0.47-0.83) 0.93(0.77-1)	0.36(0.16-0.55) 0.48(0.22-0.74)

• NAS – Nursing Activities Score; RA1 – research assistant 1; RA2 – research assistant 2; NUR – nursing assistants; ICU – Intensive Care Unit; k - Kappa coefficient; 95%CI - 95% confidence interval; * Kappa not significant.

It is noteworthy that in the assessments by RR and RA1 and RA2, 14 of the 23 items had almost perfect or perfect agreement, while by RR and NUR, only 2 items had perfect or almost perfect agreement.

Discussion

In the present study, moderate or substantial agreement was found among evaluators for the final score of the NAS instrument. When the final value was categorized (workloads: light; moderate/high and very high), it was observed that the assessments did not coincide especially in the extreme categories of workload, light and very high. Agreement in each of the NAS 23 items was also presented, showing that although there was perfect or almost perfect agreement in 14 of the 23 items scored by pairs of evaluators formed by RR with RA, agreement was poor or relative in seven items, between at least one pair of evaluators.

The characteristics of the sample studied by us are similar to those described in other studies, such as the one that presented the epidemiological profile of patients hospitalized in an ICU in Florianópolis. Although 52.5% of patients were surgical, different from our sample that had a majority of clinical patients, the study showed that 61.6% of hospitalized patients were male, aged 40 to 69 years, and mortality rate of 20.4%, (¹⁶)similar characteristics to our sample. Similarly, a study was conducted in an adult ICU of a large university hospital in the countryside of Rio Grande do Sul, in which the patients treated were predominantly male (58%), with a mean age of 64.8±5.65 years and a mortality rate of 50%, higher than ours, which was explained by the authors due to the high incidence of septic shock and multiple organ failure. (¹⁷)

The Federal Nursing Council (COFEN) (⁵)Resolution determines that the minimum reference for the nursing staff should consider for patients in intensive care 18 hours of nursing per patient in the 24 hours. The mean NAS found in our study was 79.5±20.7%, which corresponds to 19.08 hours of nursing care, similar to that determined by COFEN.

The mean NAS described in our study can be compared to that identified in other national and international studies. In a clinical and surgical ICU of a university hospital in Londrina, a mean of 74.4±8.8% was found for the NAS applied in 437 patients, similar to our study. Research,(¹⁸)developed in an adult general ICU of a private hospital in the city of São Paulo (Brazil), included 33 patients and found that the mean 24-hour NAS was 69.6±18.2%, lower than that found in our study, which can be explained by the difference in profile of patients seen in public versus private services. In the international literature, it is possible to identify the NAS application in countries such as Italy, where a study(¹⁹)presented a five-year historical series of NAS use in three ICUs (general, neurosurgical and cardiopulmonary) of a university hospital. A total of 5,856 patients were included, with a mean NAS of 66.0±2.5%.

Regarding agreement, we observed that although the final NAS value agreement was moderate or substantial with the three pairs of evaluators, this was not observed when the NAS value was categorized. Also, when the instrument items were assessment, seven items showed poor or relative agreement. In a study conducted in a general ICU in Norway, the NAS was applied to 101 patients by three evaluators: a NUR, an intensive care physician and a manager nurse. It was observed that the mean total NAS score scored by the physician was significantly lower than the mean scored by a NUR and a manager nurse (83.7% ± 21.18; 88.4% ± 16.2 and 88, 7% ± 24.5, respectively, p<0.05). As in our study, there was poor or relative agreement on items such as: “Monitoring and control”, “Hygiene procedures”, “Mobilization and positioning”, “Support and care to patients and their families” and “Management and administrative tasks”. These items are divided into three subitems, which may explain the weaker agreement, in addition to their subjective assessment.

Since the beginning of NAS use in Brazil, the subjective character of these items is described as a challenge. In 2007, Gonçalves et al(²⁰)published a proposal for the instrument application, motivated especially by the difficulty observed in its application, due to the subjectivity in these same items (1, 4, 6, 7 and 8). Over the years, this subjective character has also motivated the creation of other manuals(¹²,¹³)for using the NAS. Even so, it is possible that, in different centers or according to the evaluator, the instrument is scored considering different guidelines. As suggested by Miot,(¹⁵)agreements may vary from one scenario to another, among evaluators, between instruments, which justifies assessing the NAS reproducibility, even though its use is established in the context of Brazilian ICUs.

A study developed in Spain(⁹) showed the agreement among evaluators in the NAS application from the assessment of three NUR. It was not considered a reference standard and the results for the agreement of the 23 items were presented through ICC. It was observed that nine items presented perfect agreement, 12 items presented almost perfect agreement and only two items presented relative agreement. It is noteworthy that, as a method of collection, the researchers used, in addition to the information generated from the assistance, a collection instrument at the bedside, filled in by the care team during the work shift and, also, a stopwatch to assess procedure times. When compared to this study, in general, the agreement found by us was weaker. However, in our study, agreement was assessed using the same method as the instrument is applied in daily practice, without additional instruments or stopwatch. It should be noted that the instrument presents several hourly measures that end up being estimated in 24 hours, given the impossibility of measuring precise time in intensive care nursing interventions, characterizing the punctual estimate, in the context of care.

Our study presented the agreement ratio among evaluators according to the workload category. We did not find another study that described a similar result. We observed that the moderate workload category had a higher proportion of agreement in general. This can be explained by the fact that this category is broad and concentrates the largest number of assessments, while the extreme categories had a smaller number of classified patients. Results different from ours were obtained by Perroca et al,(¹¹)when they identified less agreement among evaluators in the extreme categories of workload when using another type of scale (other than the NAS) in ICU patients. On the other hand, our overall results (general agreement) are close to those obtained by the researchers who carried out the cross-cultural adaptation of NAS.(⁴)

Although it was not the object of this study, it can be considered that some factors may contribute to facilitate, or hinder, the agreement among evaluators. In addition to training at the time of admission of NUR, it seems necessary to think about strategies for periodic reinforcement in the training of these professionals regarding the use of measurement instruments used in health during clinical practice. The routine application of instruments such as NAS, whose results or applicability are little or not discussed with nurses, could arouse disinterest in their use. Including the NAS as an indicator for managerial decisions could contribute to better engagement of teams in filling out the instrument, with better accuracy in filling it out. Also, adjustments to the available manuals could reduce subjectivity in items with lower agreement.

Our study has some limitations: as it is a small sample, in a single center, our results may not reflect the reality of other locations. Furthermore, data collection was performed by three profiles of evaluators: research nurse, RA and NUR. Considering that professional experience can influence decision-making for NAS application, RA, still in training, could inaccurately assess the instrument items. To minimize this effect, these RA were trained until they had adequate understanding about each item that composes the instrument. Also, the profile of nurses who assessed patients belonging to this sample and, therefore, their experience with the NAS, was not detailed. Even so, this study presents unpublished data that can collaborate both locally and with the generalization of the findings to other institutions. It is the first Brazilian study to identify in which NAS subitems there is less agreement among evaluators, indicating which aspects of this score deserve attention in the training of nurses and intensive care nurses.

Conclusion

In general, agreement among nurses and, also, trained nursing students, in NAS application is good. However, of the 23 items that make up the score, some deserve attention with regard to divergences, especially items that have more than one subitem as an option for scoring. As in our institution, these isolated disagreements may be happening in other institutions, affecting the accuracy of nursing workload estimates.

Acknowledgments

To the Research Incentive Fund (FIPE) of the Hospital de Clínicas de Porto Alegre and to the Brazilian National Council for Scientific and Technological Development (CNPq - Conselho Nacional de Desenvolvimento Científico e Tecnológico), for granting a scientific initiation scholarship to Julya Cyrino Veras.

Referências

• ¹
  Miranda DR, Nap R, de Rijk A, Schaufeli W, Iapichino G; TISS Working Group. Therapeutic Intervention Scoring System. Nursing activities score. Crit Care Med. 2003;31(2):374-82.
• ²
  Miranda DR, de Rijk A, Schaufeli W. Simplified Therapeutic Intervention Scoring System: the TISS-28 items--results from a multicenter study. Crit Care Med. 1996;24(1):64-73.
• ³
  Queijo AF. Tradução para o português e validação de um instrumento de medida de carga de trabalho de enfermagem em Unidade de Terapia Intensiva: Nursing Activities Score (N.A.S.) [dissetação]. São Paulo: Universidade de São Paulo; 2002.
• ⁴
  Queijo AF, Padilha KG. Nursing Activities Score (NAS): adaptação transcultural e validação para a língua portuguesa. Rev Esc Enferm USP. 2009;43(Spe):1018–25.
• ⁵
  Conselho Federal de Enfermagem. Resolução COFEN 543/2017 Atualiza e estabelece parâmetros para o Dimensionamento do Quadro de Profissionais de Enfermagem nos serviços/locais em que são realizadas atividades de enfermagem. Brasília (DF): Conselho Federal de Enfermagem; 2019.
• ⁶
  Brasil. Ministério da Saúde. Resolução-RDC N^o 7, de 24 de fevereiro de 2010. Dispõe sobre os requisitos mínimos para funcionamento de Unidades de Terapia Intensiva e dá outras providências. Brasília (DF): Ministério da Saúde; 2010.
• ⁷
  Ducci AJ, Padilha KG. Nursing activities score: estudo comparativo da aplicação retrospectiva e prospectiva em unidade de terapia intensiva. Acta Paul Enferm. 2008;21(4):581–7.
• ⁸
  Oliveira G, Camargo F, Gonçalves E, Duarte C, Guimarães C. Revisão sistemática da acurácia dos testes diagnósticos: uma revisão narrativa. Rev Col Bras Cir. 2010;37(2):153–6. Review.
• ⁹
  Valls-Matarín J, Salamero-Amorós M, Roldán-Gil C, Quintana-Riera S. Grado de concordancia interevaluador de la escala «Nursing Activities Score» en cuidados intensivos. Enferm Clin. 2015;25(4):204–8.
• ¹⁰
  Stuedahl M, Vold S, Klepstad P, Stafseth SK. Interrater reliability of nursing activities score among intensive care unit health professionals. Rev Esc Enferm USP. 2015;49(Spe):117–22.
• ¹¹
  Perroca MG, Gaidzinski RR. Avaliando a confiabilidade interavaliadores de um instrumento para classificação de pacientes - coeficiente Kappa. Rev Esc Enferm USP. 2003;37(1):72–80.
• ¹²
  Padilha KG, Stafseth S, Solms D, Hoogendoom M, Monge FJ, Gomaa OH, et al. Nursing activities score: an updated guideline for its application in the intensive care unit. Rev Esc Enferm USP. 2015;49(Spe):131–7.
• ¹³
  Vieira DF, Padilha KG, Nogueira LD. Manual do Nursing Activities Score-NAS. São Paulo: Associação de Medicina Intensiva Brasileira; 2014. p. 1–6 [citado 2022 Fev 20]. Disponível em: https://docplayer.com.br/10481933-Manual-do-nursing-activities-score-nas.html
  » https://docplayer.com.br/10481933-Manual-do-nursing-activities-score-nas.html
• ¹⁴
  Batassini E, Silveira JT, Cardoso PC, Castro DE, Hochegger T, Vieira DF. Nursing Activities Score: qual periodicidade ideal para avaliação da carga de trabalho? Acta Paul Enferm. 2019;32(2):162–8.
• ¹⁵
  Miot HA. Agreement analysis in clinical and experimental trials [Editorial]. J Vasc Bras. 2016;15(2):89–92.
• ¹⁶
  Rodriguez AH, Bub MB, Perão OF, Zandonadi G, Rodriguez MJ. Epidemiological characteristics and causes of deaths in hospitalized patients under intensive care. Rev Bras Enferm. 2016;69(2):229–34.
• ¹⁷
  Favarin SS, Camponogara S. Perfil dos pacientes internados na Unidade de Terapia Intensiva Adulto de um hospital universitário. Rev Enferm UFSM. 2012;2(2):320–9.
• ¹⁸
  Conishi RM, Gaidzinski RR. Evaluation of the nursing activities score (nas) as a nursing workload measurement tool in an adult ICU. Rev Esc Enferm USP. 2007;41(3):346–54.
• ¹⁹
  Lucchini A, De Felippis C, Elli S, Schifano L, Rolla F, Pegoraro F, et al. Nursing Activities Score (NAS): 5 years of experience in the intensive care units of an Italian University hospital. Intensive Crit Care Nurs. 2014;30(3):152-8,
• ²⁰
  Gonçalves LA, Padilha KG, Cardoso Sousa RM. Nursing activities score (NAS): a proposal for practical application in intensive care units. Intensive Crit Care Nurs. 2007;23(6):355-61.

Edited by

Associate Editor (Peer review process): Alexandre Pazetto Balsanelli (https://orcid.org/0000-0003-3757-1061) Escola Paulista de Enfermagem, Universidade Federal de São Paulo, SP, Brazil

Publication Dates

• Publication in this collection
  09 Dec 2022
• Date of issue
  2022

History

• Received
  7 Nov 2021
• Accepted
  11 Apr 2022