Delirium assessment in postoperative patients: validation of the portuguese version of the Nursing Delirium Screening Scale in critical care

Abstracts

EXPERIÊNCIA E OBJETIVOS: O objetivo desse estudo foi validar a versão portuguesa da escala Nursing Delirium Screening Scale (Nu-DESC) para uso em anbientes de terapia intensiva. MÉTODOS: Simultânea e independentemente, avaliamos para ocorrência de delírio todos os pacientes pós-operatórios internados em uma Unidade de Terapia Intensiva Cirúrgica (UTIC) ao longo de um período de um mês, utilizando as versões portuguesas de Nu-DESC e da escala Intensive Care Delirium Screening Checklist (ICDSC) dentro de 24 horas a contar da internação, tanto pelo médico da equipe de pesquisa como pelo enfermeiro diretamente encarregado do paciente. Determinamos a acurácia diagnóstica de Nu-DESC utilizando análises de sensibilidade, especificidade e da curva ROC. Avaliamos a confiabilidade entre enfermeiros/médico da equipe de pesquisa para Nu-DESC pelo coeficiente de correlação intraclasse (CCI). Avaliamos concordância e confiabilidade entre Nu-DESC e ICDSC por percentuais globais e específicos de concordância, e por estatística kappa. RESULTADOS: Com base na escala ICDSC, diagnosticamos delírio em 12 dos 78 pacientes. Para o escore Nu-DESC total, consideramos como alta a confiabilidade entre enfermeiros/médico da equipe. Quanto ao diagnóstico, consideramos perfeita a concordância entre enfermeiros e médico da equipe. O percentual de concordância global entre Nu-DESC e ICDSC para o diagnóstico de delírio foi 0,88, e kappa variou de 0,79 a 0,93. A sensibilidade da escala Nu-DESC foi 100%, e a especificidade, 86%. CONCLUSÕES: A versão portuguesa da escala Nu-DESC parece ser instrumento de avaliação e monitoramento acurado e confiável para o diagnóstico de delírio em ambientes de terapia intensiva.

Delírio; Terapia pós-operatória; Terapia intensiva

EXPERIENCIA Y OBJETIVOS: El objetivo de este estudio fue validar la versión portuguesa de la escala Nursing Delirium Screening Scale (Nu-DESC) para uso en ambientes de cuidados intensivos. MÉTODOS: Simultánea e independientemente, evaluamos el aparecimiento de delirio en todos los pacientes postoperatorios ingresados en una Unidad de Cuidados Intensivos Quirúgica (UCIQ) a lo largo de un período de un mes, utilizando las versiones portuguesas de Nu-DESC y de la escala Intensive Care Delirium Screening Checklist (ICDSC) dentro de 24 horas a partir del momento del ingreso, tanto por el médico del equipo de investigación, como por el enfermero directamente encargado del paciente. Determinamos la exactitud diagnóstica de Nu-DESC utilizando análisis de sensibilidad, especificidad y de la curva ROC. Evaluamos la confiabilidad entre enfermeros/médico del equipo de investigación para Nu-DESC por el coeficiente de correlación intraclase (CCI). Evaluamos la concordancia y la confiabilidad entre Nu-DESC y ICDSC por porcentajes globales y específicos de concordancia, y por estadística kappa. RESULTADOS: Basándonos en la escala ICDSC, diagnosticamos delirio en 12 de los 78 pacientes. Para la puntuación Nu-DESC total, consideramos como alta la confiabilidad entre enfermeros/médico del equipo. En cuanto al diagnóstico, consideramos perfecta la concordancia entre enfermeros y médico del equipo. El porcentaje de concordancia global entre Nu-DESC y ICDSC para el diagnóstico de delirio fue de 0,88, y kappa varió de 0,79 a 0,93. La sensibilidad de la escala Nu-DESC fue de un 100%, y el de especificidad de un 86%. CONCLUSIONES: La versión portuguesa de la escala Nu-DESC parece ser un instrumento de evaluación y monitoreo exacto y confiable para el diagnóstico del delirio en ambientes de cuidados intensivos.

Delirio; Terapia postoperatoria; Cuidados intensivos

SCIENTIFIC ARTICLE

Delirium assessment in postoperative patients: validation of the portuguese version of the nursing delirium screening scale in critical care^*

Fernando Abelha^I,II; Dalila Veiga^II; Maria Norton^II; Cristina Santos^III; Jean-David Gaudreau^IV,V,VI

^IAnesthesiology and Perioperative Care Unit, Surgical Department, School of Medicine, University of Porto, Porto, Portugal

^IIDepartment of Anesthesiology, Centro Hospitalar de São João, Porto, Portugal

^IIIClinical and Health Services Research, School of Medicine, University of Porto, Porto, Portugal

^IVDepartment of Psychiatry, L'Hôtel-Dieu de Québec, Québec, QC, Canada

^VCentre Hospitalier Universitaire de Québec, Québec, QC, Canada

^VISchools of Pharmacy and Medicine and Department of Psychiatry, School of Medicine, Laval University, Québec, QC, Canada

Corresponding author

ABSTRACT

BACKGROUND AND OBJECTIVES: The aim of this study was to validate the Portuguese version of the Nursing Delirium Screening Scale (Nu-DESC) for use in critical care settings.

METHODS: We simultaneously and independently evaluated all postoperative patients admitted to a surgical Intensive Care Unit (SICU) over a 1-month period for delirium, using the Portuguese versions of both the Nu-DESC and the Intensive Care Delirium Screening Checklist (ICDSC) within 24 hours of admission by both the research staff physician and one bedside nurse. We determined the diagnostic accuracy of the Nu-DESC using sensitivity, specificity and ROC curve analyses. We assessed reliability between nurses and the research staff physician for Nu-DESC by intraclass correlation coefficient (ICC). We assessed agreement and reliability between Nu-DESC and ICDSC by overall and specific proportions of agreement and by kappa statistics.

RESULTS: Based on the ICDSC, we diagnosed delirium in 12 of the 78 patients. Reliability between nurses and the staff physician for total Nu-DESC score was high. Agreement between nurses and staff physician in the delirium diagnosis was perfect. The proportion of overall agreement between Nu-DESC and ICDSC in the delirium diagnosis was 0.88 and the kappa ranged from 0.79 to 0.93. Nu-DESC Sensitivity was 100 and specificity was 86%.

CONCLUSIONS: The Portuguese version of the Nu-DESC appears to be an accurate and reliable assessment and monitoring instrument for delirium in critical care settings.

Keywords: Delirium; Postoperative care; Critical care

Introduction

Postoperative delirium is associated with higher mortality, prolonged duration of intensive care unit (ICU) stay, more frequent medical complications and higher health care costs.^1-6 Many risk factors have been described for the development of delirium,^7-9 many of them being constantly present in these populations (e.g. high analgesic doses, heavy sedation). Delirium is often the presenting sign of a more serious underlying condition, whether physical abnormality or drug toxicity. The reported incidence of delirium in the ICU ranges widely from 16% to 89%, with numerous studies reporting incidence values close to 75%.^1,4,10,11

Detection of delirium in critical care patients can be particularly difficult, which could partly explain the wide variations in measured incidence, as multiple factors can hamper making a clear clinical diagnosis e.g. lack of verbal communication for intubated patients, fast-paced and technologically advanced environment. Specific instruments have been developed to take these particularities into account such as the Confusion Assessment Method for the ICU (CAM-ICU).¹² These are widely used despite being more burdensome to rate than round-the-clock observational instruments that can be rated quickly and show promise for use in critical care settings such as the Nursing Delirium Screening Scale (Nu-DESC).^13-15 Since early recognition and treatment of delirium is of crucial importance in reducing its duration and severity,^16-18 current practice guidelines recommend that ICU patients be routinely assessed for delirium using a validated instrument.¹⁹ This would help to better identify the causes of delirium and prepare strategies for prevention of delirium in critical care settings.

Operationalized criteria derived from the Diagnostic and Statistical Manual of Mental Disorders IV (DSM-IV)²⁰ are often used as the gold standard for diagnosing delirium although validated instruments such as the Confusion Assessment Method (CAM) and the Intensive Care Delirium Screening Checklist (ICDSC) are also widely used in delirium studies. The ICDSC was validated using operationalized DSM-IV criteria as the gold standard.¹⁰ Gaudreau et al.¹³ developed the Nursing Delirium Screening Scale (Nu-DESC) to assess delirium on a continuous 24-hour basis. The Nu-DESC can be fully integrated into routine clinical care and can be completed in less than 1 minute. It has already been used for delirium assessment in the recovery room and in critical care settings.^21,22 This instrument has shown very good discriminative power, with high sensitivity and specificity and high levels of agreement with DSM-IV classification. Recently, Luetz et al.¹⁴ validated the Nu-DESC for the ICU and found that it presented high sensitivity (82%-83%) and specificity (81%-83%).

This study aimed to prospectively validate the Portuguese version of the Nu-DESC for use in critical care settings.

Methods

Subjects and setting

This prospective study was carried out in the post anaesthesia care unit (PACU) of the Hospital São João, Porto, Portugal, an 1100-bed community teaching hospital in Porto, Portugal. Within the PACU, there is a surgical intensive care unit (SICU) with 5 beds, where critically ill patients are admitted, closely monitored, and treated. We included in the study all consecutive Portuguese-speaking adult patients admitted to the SICU after non-cardiac and non-intracranial surgery between 1 May 2011 and 31 May 2011 (31 days) who were expected to remain in the hospital postoperatively for more than 48 hours.

We excluded patients who did not provide or were incapable of providing informed written consent before surgery, who had a disease of the central nervous system, had been admitted for neurological or cardiac surgery, had Parkinson's disease, a history of alcoholism or drug-dependence, were admitted with a diagnosis of delirium, or were receiving antipsychotic medication. We also excluded patients readmitted during the study period.

The institutional review board of the São João Hospital approved the study.

Testing procedure

Translation and back-translation of the Nu-DESC

With permission from the authors, the Nu-DESC (Appendix 2) was translated according to the guidelines suggested by The Translation and Cultural Adaptation group (ISPOR TCA task force), which has proposed a model for good practice in the translation process.²³

A group of experienced intensive care nurses, the study author and a professional translator translated the source text of the English version of the Nu-DESC to Portuguese. The final Portuguese version was given to a professional translator for back-translation to English, without providing access to the original English version. The group who had made the original translation compared the retranslated version of the instruments to the original and identified and corrected discrepancies. The retranslated version was sent to the authors of the Nu-DESC who approved the Portuguese version.

We asked ten experienced nurses specialized in intensive care to examine the translated version and to identify any unclear words, concepts, or elements that they found difficult to understand. The findings of this debriefing process were incorporated to improve the performance of the translated instrument. The translated Portuguese version of the Nu-DESC is shown in Appendix 1.

Delirium evaluation

We simultaneously and independently evaluated each patient admitted to the SICU and included in the study for delirium, using the Portuguese versions of both the Nu-DESC and the ICDSC within 24 hours of admission by both the research staff physician and one bedside nurse.

Both nurses and research staff physicians performed daily ratings during the patient's entire SICU stay.

The primary care nurse assessed the patients with the Nu-DESC every morning between 8 and 10 a.m. An independent and trained physician member of the research team assessed all patients using the Nu-DESC and was blinded to previous test procedures results.

Another member of staff research and regular nurse made further evaluation using ICDSC for reference.

Use of the ICDSC was already standard practice in nursing care and all SICU nurses had been involved in the process of translation/back-translation of the Nu-DESC, and were trained in its use.

The information used in this study to rate the instruments included information obtained from the patient during the last 24 hours, the primary nurses' evaluation, and the nurses' patient review chart. The tests were rated independently and separately by the research physician and the bedside nurse, both blinded to each other's scores.

Instruments

ICDSC

This scale includes 8 items based on DSM-IV criteria of delirium and additional features of delirium that can be integrated into the daily nurse-scoring assessment and can be carried out by the ICU bedside nurses regularly. On the ICDSC, a patient is given a score from 0 to 8; scores of 4 or above have a sensitivity of 99% and a specificity of 64% for identifying delirium. In this study patients were considered to have delirium if they had ICDSC scores of 4 and above.

Nu-DESC

The Nu-DESC assesses 5 dimensions of cardinal features of delirium: orientation, behavior, communication, perceptions, and psychomotor activity. We rated the symptoms on a 3-point scale (0, 1 or 2), and a cumulative score of 2 or higher, out of 10, denotes the presence of delirium.

Statistical analysis

We calculated sensitivity, specificity and predictive values of Nu-DESC ratings, with respective 95% confidence intervals. We also performed receiver operating characteristic (ROC) curve analyses.

We evaluated the reliability of the questionnaire by calculating Cronbach's alpha.²⁴

We estimated agreement between nurses and the research staff physician using weighted kappa, with linear weighting, and proportions of agreement with respective 95% confidence intervals for each Nu-DESC dimension. We calculated agreement between nurses and the research staff physician for total score using Intraclass Correlation Coefficient (ICC).²⁵

Results

Patient characteristics

We performed eighty-one concomitant physician-nurse evaluations of the Nu-DESC and of the ICDSC over 133 patient evaluation days, representing 86% of SICU occupation for the study period. Table 1 describes the baseline characteristics of the sample. Based on the ICDSC, we diagnosed delirium in 12 of the 78 patients (15%).

Sensitivity, specificity and ROC curve analyses

Nu-DESC correctly identified all twelve delirium cases. With a positive Nu-DESC score of 2 or more points, sensitivity was 100% (95% CI = [75%,100%]). Nu-DESC diagnosed 9 false positive; specificity was 86% (95% CI = [76%,93%]), positive predictive value was 57% (95% CI = [36%,76%]) and negative predictive value was 100% (95% CI = [94%,100%]). Fig. 1 shows ROC curve analysis. There were no significant differences between area under the curve (AUC) of physician's (AUC = 0.985, 95% CI = [0.928, 0.999]) and nurses' ratings (AUC = 0.979, 95% CI = [0.918, 0.998]).

Reliability and agreement

The Cronbach's alpha of the Nu-DESC checklist was similar between staff physician and nurses (0.81, 95%CI = [0.73, 0.87] vs. 0.79, 95%CI = [0.71, 0.85]). In the 78 paired observations performed, proportions of agreement between nurses and the research staff physician for the Nu-DESC items were all high (> 0.96) and weighted kappa ranged from 0.79 to 0.93. For all patient evaluations, agreement between nurses and research staff physician was total (Table 2). ICC for agreement between nurses and the research staff physician for the total Nu-DESC score was 0.98 with a 95% confidence interval from 0.96 to 0.99.

Discussion

This study suggests that the Portuguese version of the Nu-DESC is an accurate and reliable tool to assess delirium in critical care patients. This study also suggests that nurses as well as critical care physicians can rate the Nu-DESC as quickly and accurately. The instrument shows high sensitivity, which is important for a screening/monitoring instrument and a particularly high specificity not specifically different than those of the ICDSC. The Nu-DESC possessed values similar to those of other published studies.^21,26,27

This study has several strengths. The instrument used was formally translated and back-translated using recommended guidelines and the authors of the original Nu-DESC study approved the faithfulness of the new version to the original. Also, this study presented an original design, with independent paired evaluations between a single physician and multiple bedside nurses. The results presented here suggest that the evaluations are equivalent in accuracy between physicians and nurses, indicating equivalent clinical assessment ability to detect delirium. This probably also reflects the involvement in the planning, baseline, and education phases amongst all the SICU staff, in order to cover all aspects relevant to the project. Our results are concordant with those of nursing studies that have evaluated the effect of educational efforts focused on the use of validated delirium screening tools.²⁸ A group of researchers, nurses experienced in intensive care, and professional translators translated the Nu-DESC, contributing to preserve the meaning of the words and concepts specific to the SICU context. Previously, the authors noted that, with minimal training, nurses' compliance using sedation and delirium instruments is excellent. In the study of Pun et al.,²⁹ the authors concluded that the data agreement for delirium assessment between bedside nurses and a reference-standard rater was very high.

In a study that validated Nu-DESC for use in ICUs, Luetz et al.¹⁴ suggested it may be a good alternative for detecting delirium in critically ill patients due to its high sensitivity but also high specificity (sensitivity: 83%, specificity: 83%) when compared to DSM-IV criteria. In addition to being similar to those presented here, their results were very similar to those obtained in the original validation study by Gaudreau et al.¹³ in an oncology inpatients setting. Our results strengthen the Nu-DESC's general reliability, even for populations as different as oncology and ICU patients.

This study also presents its limitations. The sample size allowed limited power to detect small differences. Clearly, the results presented here should be reproduced in larger samples, but are promising nonetheless.

We chose the ICDSC over the CAM-ICU as the gold standard in this study, since several studies had shown that the two instruments presented similar psychometric characteristics and the ICDSC was already in use in our setting. It has also been shown to be highly valid, as compared to DSM-IV evaluation by psychiatrists, and highly reliable, and is often preferred by nurses over other more burdensome delirium tools.

Conclusion

The Portuguese translation of the Nu-DESC showed good correlation with the original version and could be applicable to Portuguese critical care settings. Inter-rater reliability between staff physician and nurses was very good for item-by-item comparison, as well as for the diagnosis of delirium. Psychometric characteristics of the Nu-DESC were similar to those of the ICDSC, but the Nu-DESC has the significant advantage of being quicker to rate than the ICDSC because it involves fewer items. Thus, the Nu-DESC is promising as a good screening and monitoring instrument, with high sensitivity, but also as an accurate diagnostic instrument with high specificity for assessment of delirium in critical care settings.

Key messages

• The Portuguese translation of the Nu-DESC showed good correlation with the original version and could be applicable to Portuguese critical care settings.

• The Portuguese version of the Nu-DESC is an accurate and reliable tool to assess delirium in critical care patients.

• Nurses as well as critical care physicians can rate the Nu-DESC accurately.

• The Nu-DESC shows high sensitivity and high specificity for delirium assessment.

Authors' contributions

FA participated in conception, design, acquisition of the data, analysis of the data, statistical analysis, critical revision of the manuscript and supervision.

DV and MN participated in conception, design, acquisition of the data, analysis of the data and critical revision of the manuscript.

CS and JG has been involved in drafting the manuscript, analysis of the data and critically revising it for important content.

All authors read and approved the final manuscript.

Conflicts of interest

The authors declare no conflicts of interest.

Acknowledgements

Post-anesthesia Care Unit nurses team.

References

• 1. Ely EW, Shintani A, Truman B, et al. Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. Jama. 2004;291:1753-62.
• 2. Leslie DL, Zhang Y, Holford TR, et al. Premature death associated with delirium at 1-year follow-up. Arch Intern Med. 2005;165:1657-62.
• 3. Olin K, Eriksdotter-Jönhagen M, Jansson A, et al. Postoperative delirium in elderly patients after major abdominal surgery. Br J Surg. 2005;92:1559-64.
• 4. Thomason JW, Shintani A, Peterson JF, et al. Intensive care unit delirium is an independent predictor of longer hospital stay: a prospective analysis of 261 non-ventilated patients. Crit Care. 2005;9:R375-81.
• 5. Zakriya K, Sieber FE, Christmas C, et al. Brief postoperative delirium in hip fracture patients affects functional outcome at three months. Anesth Analg. 2004;98:1798-802.
• 6. Aakerlund LP, Rosenberg J. Postoperative delirium: treatment with supplementary oxygen. Br J Anaesth. 1994;72:286-90.
• 7. Deiner S, Silverstein JH. Postoperative delirium and cognitive dysfunction. Br J Anaesth. 2009;103(Suppl 1):i41-6.
• 8. Girard TD, Pandharipande PP, Ely EW. Delirium in the intensive care unit. Crit Care. 2008;12(Suppl 3):S3.
• 9. Quimet S, Kavanagh B, Gottfried SB, et al. Incidence, risk factors and consequences of ICU delirium. Intensive Care Med. 2007;33:66-73.
• 10. Bergeron N, Dubois MJ, Dumont M, et al. Intensive Care Delirium Screening Checklist: evaluation of a new screening tool. Intensive Care Med. 2001;27:859-64.
• 11. Ely EW, Gautam S, Margolin R, et al. The impact of delirium in the intensive care unit on hospital length of stay. Intens Care Med. 2001;27:1892-900.
• 12. Ely EW, Inouye SK, Bernard GR, et al. Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU). Jama. 2001;286:2703-10.
• 13. Gaudreau JD, Gagnon P, Harel F, et al. Fast, systematic, and continuous delirium assessment in hospitalized patients: the nursing delirium screening scale. J Pain Symptom Manage. 2005;29:368-75.
• 14. Luetz A, Heymann A, Radtke FM, et al. Different assessment tools for intensive care unit delirium: which score to use? Crit Care Med. 2010;38:409-18.
• 15. Lutz A, Radtke FM, Franck M, et al. [The Nursing Delirium Screening Scale (NU-DESC)]. Anasthesiol Intensivmed Notfallmed Schmerzther. 2008;43:98-102.
• 16. Johnson J. Identifying and recognizing delirium. Dement Geriatr Cogn Disord. 1999;10:353-8.
• 17. Milisen K, Foreman MD, Abraham IL, et al. A nurse-led interdisciplinary intervention program for delirium in elderly hip-fracture patients. J Am Geriatr Soc. 2001;49:523-32.
• 18. Lundstrom M, Edlund A, Karlsson S, et al. A multifactorial intervention program reduces the duration of delirium, length of hospitalization, and mortality in delirious patients. J Am Geriatr Soc. 2005;53:622-8.
• 19. Jacobi J, Fraser GL, Coursin DB, et al. Clinical practice guidelines for the sustained use of sedatives and analgesics in the critically ill adult. Crit Care Med. 2002;30:119-41.
• 20. American Psychiatric Association. Diagnostic and statistical manual mental disorders IV (DSM IV); 4th edition. Washington, DC; 1994.
• 21. Radtke FM, Franck M, Schneider M, et al. Comparison of three scores to screen for delirium in the recovery room. Br J Anaesth. 2008;101:338-43.
• 22. Radtke FM, Franck M, MacGuill M, et al. Duration of fluid fasting and choice of analgesic are modifiable factors for early postoperative delirium. Eur J Anaesthesiol. 2010;27:411-6.
• 23. Wild D, Grove A, Martin M, et al. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8:94-104.
• 24. Cronbach LJ. Coefficient Alpha and the internal structure of tests. Psychometrika. 1951;16:297-334.
• 25. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420-8.
• 26. Leung JM, Leung VW, Leung CM, et al. Clinical utility and validation of two instruments (the Confusion Assessment Method Algorithm and the Chinese version of Nursing Delirium Screening Scale) to detect delirium in geriatric inpatients. Gen Hosp Psychiatry. 2008;30:171-6.
• 27. Radtke FM, Franck M, Schust S, et al. A comparison of three scores to screen for delirium on the surgical ward. World J Surg. 2010;34:487-94.
• 28. Devlin JW, Marquis F, Riker RR, et al. Combined didactic and scenario-based education improves the ability of intensive care unit staff to recognize delirium at the bedside. Crit Care. 2008;12:R19.
• 29. Pun BT, Gordon SM, Peterson JF, et al. Large-scale implementation of sedation and delirium monitoring in the intensive care unit: a report from two medical centers. Crit Care Med. 2005;33:1199-205.

*

Recebido do Departamento de Anestesiologia-Unidade de Terapia pós-Anestesia; Centro Hospitalar de São João, Portugal.

Publication Dates

• Publication in this collection
  18 Dec 2013
• Date of issue
  Dec 2013

History

• Received
  12 July 2012
• Accepted
  06 Sept 2012