SciELO - Scientific Electronic Library Online

 
vol.63 issue6Comparison of laryngeal mask airway supreme and laryngeal mask airway proseal for laryngopharyngeal trauma and postoperative morbidity in childrenProspective, randomized clinical trial of laryngeal mask airway Supreme® used in patients undergoing general anesthesia author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


Revista Brasileira de Anestesiologia

Print version ISSN 0034-7094

Rev. Bras. Anestesiol. vol.63 no.6 Campinas Nov./Dec. 2013

http://dx.doi.org/10.1016/j.bjan.2012.09.002 

SCIENTIFIC ARTICLE

 

Delirium assessment in postoperative patients: validation of the portuguese version of the nursing delirium screening scale in critical care*

 

 

Fernando AbelhaI,II; Dalila VeigaII; Maria NortonII; Cristina SantosIII; Jean-David GaudreauIV,V,VI

IAnesthesiology and Perioperative Care Unit, Surgical Department, School of Medicine, University of Porto, Porto, Portugal
IIDepartment of Anesthesiology, Centro Hospitalar de São João, Porto, Portugal
IIIClinical and Health Services Research, School of Medicine, University of Porto, Porto, Portugal
IVDepartment of Psychiatry, L'Hôtel-Dieu de Québec, Québec, QC, Canada
VCentre Hospitalier Universitaire de Québec, Québec, QC, Canada
VISchools of Pharmacy and Medicine and Department of Psychiatry, School of Medicine, Laval University, Québec, QC, Canada

Corresponding author

 

 


ABSTRACT

BACKGROUND AND OBJECTIVES: The aim of this study was to validate the Portuguese version of the Nursing Delirium Screening Scale (Nu-DESC) for use in critical care settings.
METHODS: We simultaneously and independently evaluated all postoperative patients admitted to a surgical Intensive Care Unit (SICU) over a 1-month period for delirium, using the Portuguese versions of both the Nu-DESC and the Intensive Care Delirium Screening Checklist (ICDSC) within 24 hours of admission by both the research staff physician and one bedside nurse. We determined the diagnostic accuracy of the Nu-DESC using sensitivity, specificity and ROC curve analyses. We assessed reliability between nurses and the research staff physician for Nu-DESC by intraclass correlation coefficient (ICC). We assessed agreement and reliability between Nu-DESC and ICDSC by overall and specific proportions of agreement and by kappa statistics.
RESULTS: Based on the ICDSC, we diagnosed delirium in 12 of the 78 patients. Reliability between nurses and the staff physician for total Nu-DESC score was high. Agreement between nurses and staff physician in the delirium diagnosis was perfect. The proportion of overall agreement between Nu-DESC and ICDSC in the delirium diagnosis was 0.88 and the kappa ranged from 0.79 to 0.93. Nu-DESC Sensitivity was 100 and specificity was 86%.
CONCLUSIONS: The Portuguese version of the Nu-DESC appears to be an accurate and reliable assessment and monitoring instrument for delirium in critical care settings.

Keywords: Delirium; Postoperative care; Critical care


 

 

Introduction

Postoperative delirium is associated with higher mortality, prolonged duration of intensive care unit (ICU) stay, more frequent medical complications and higher health care costs.1-6 Many risk factors have been described for the development of delirium,7-9 many of them being constantly present in these populations (e.g. high analgesic doses, heavy sedation). Delirium is often the presenting sign of a more serious underlying condition, whether physical abnormality or drug toxicity. The reported incidence of delirium in the ICU ranges widely from 16% to 89%, with numerous studies reporting incidence values close to 75%.1,4,10,11

Detection of delirium in critical care patients can be particularly difficult, which could partly explain the wide variations in measured incidence, as multiple factors can hamper making a clear clinical diagnosis e.g. lack of verbal communication for intubated patients, fast-paced and technologically advanced environment. Specific instruments have been developed to take these particularities into account such as the Confusion Assessment Method for the ICU (CAM-ICU).12 These are widely used despite being more burdensome to rate than round-the-clock observational instruments that can be rated quickly and show promise for use in critical care settings such as the Nursing Delirium Screening Scale (Nu-DESC).13-15 Since early recognition and treatment of delirium is of crucial importance in reducing its duration and severity,16-18 current practice guidelines recommend that ICU patients be routinely assessed for delirium using a validated instrument.19 This would help to better identify the causes of delirium and prepare strategies for prevention of delirium in critical care settings.

Operationalized criteria derived from the Diagnostic and Statistical Manual of Mental Disorders IV (DSM-IV)20 are often used as the gold standard for diagnosing delirium although validated instruments such as the Confusion Assessment Method (CAM) and the Intensive Care Delirium Screening Checklist (ICDSC) are also widely used in delirium studies. The ICDSC was validated using operationalized DSM-IV criteria as the gold standard.10 Gaudreau et al.13 developed the Nursing Delirium Screening Scale (Nu-DESC) to assess delirium on a continuous 24-hour basis. The Nu-DESC can be fully integrated into routine clinical care and can be completed in less than 1 minute. It has already been used for delirium assessment in the recovery room and in critical care settings.21,22 This instrument has shown very good discriminative power, with high sensitivity and specificity and high levels of agreement with DSM-IV classification. Recently, Luetz et al.14 validated the Nu-DESC for the ICU and found that it presented high sensitivity (82%-83%) and specificity (81%-83%).

This study aimed to prospectively validate the Portuguese version of the Nu-DESC for use in critical care settings.

 

Methods

Subjects and setting

This prospective study was carried out in the post anaesthesia care unit (PACU) of the Hospital São João, Porto, Portugal, an 1100-bed community teaching hospital in Porto, Portugal. Within the PACU, there is a surgical intensive care unit (SICU) with 5 beds, where critically ill patients are admitted, closely monitored, and treated. We included in the study all consecutive Portuguese-speaking adult patients admitted to the SICU after non-cardiac and non-intracranial surgery between 1 May 2011 and 31 May 2011 (31 days) who were expected to remain in the hospital postoperatively for more than 48 hours.

We excluded patients who did not provide or were incapable of providing informed written consent before surgery, who had a disease of the central nervous system, had been admitted for neurological or cardiac surgery, had Parkinson's disease, a history of alcoholism or drug-dependence, were admitted with a diagnosis of delirium, or were receiving antipsychotic medication. We also excluded patients readmitted during the study period.

The institutional review board of the São João Hospital approved the study.

Testing procedure

Translation and back-translation of the Nu-DESC

With permission from the authors, the Nu-DESC (Appendix 2) was translated according to the guidelines suggested by The Translation and Cultural Adaptation group (ISPOR TCA task force), which has proposed a model for good practice in the translation process.23

A group of experienced intensive care nurses, the study author and a professional translator translated the source text of the English version of the Nu-DESC to Portuguese. The final Portuguese version was given to a professional translator for back-translation to English, without providing access to the original English version. The group who had made the original translation compared the retranslated version of the instruments to the original and identified and corrected discrepancies. The retranslated version was sent to the authors of the Nu-DESC who approved the Portuguese version.

We asked ten experienced nurses specialized in intensive care to examine the translated version and to identify any unclear words, concepts, or elements that they found difficult to understand. The findings of this debriefing process were incorporated to improve the performance of the translated instrument. The translated Portuguese version of the Nu-DESC is shown in Appendix 1.

Delirium evaluation

We simultaneously and independently evaluated each patient admitted to the SICU and included in the study for delirium, using the Portuguese versions of both the Nu-DESC and the ICDSC within 24 hours of admission by both the research staff physician and one bedside nurse.

Both nurses and research staff physicians performed daily ratings during the patient's entire SICU stay.

The primary care nurse assessed the patients with the Nu-DESC every morning between 8 and 10 a.m. An independent and trained physician member of the research team assessed all patients using the Nu-DESC and was blinded to previous test procedures results.

Another member of staff research and regular nurse made further evaluation using ICDSC for reference.

Use of the ICDSC was already standard practice in nursing care and all SICU nurses had been involved in the process of translation/back-translation of the Nu-DESC, and were trained in its use.

The information used in this study to rate the instruments included information obtained from the patient during the last 24 hours, the primary nurses' evaluation, and the nurses' patient review chart. The tests were rated independently and separately by the research physician and the bedside nurse, both blinded to each other's scores.

Instruments

ICDSC

This scale includes 8 items based on DSM-IV criteria of delirium and additional features of delirium that can be integrated into the daily nurse-scoring assessment and can be carried out by the ICU bedside nurses regularly. On the ICDSC, a patient is given a score from 0 to 8; scores of 4 or above have a sensitivity of 99% and a specificity of 64% for identifying delirium. In this study patients were considered to have delirium if they had ICDSC scores of 4 and above.

Nu-DESC

The Nu-DESC assesses 5 dimensions of cardinal features of delirium: orientation, behavior, communication, perceptions, and psychomotor activity. We rated the symptoms on a 3-point scale (0, 1 or 2), and a cumulative score of 2 or higher, out of 10, denotes the presence of delirium.

Statistical analysis

We calculated sensitivity, specificity and predictive values of Nu-DESC ratings, with respective 95% confidence intervals. We also performed receiver operating characteristic (ROC) curve analyses.

We evaluated the reliability of the questionnaire by calculating Cronbach's alpha.24

We estimated agreement between nurses and the research staff physician using weighted kappa, with linear weighting, and proportions of agreement with respective 95% confidence intervals for each Nu-DESC dimension. We calculated agreement between nurses and the research staff physician for total score using Intraclass Correlation Coefficient (ICC).25

 

Results

Patient characteristics

We performed eighty-one concomitant physician-nurse evaluations of the Nu-DESC and of the ICDSC over 133 patient evaluation days, representing 86% of SICU occupation for the study period. Table 1 describes the baseline characteristics of the sample. Based on the ICDSC, we diagnosed delirium in 12 of the 78 patients (15%).

 

 

Sensitivity, specificity and ROC curve analyses

Nu-DESC correctly identified all twelve delirium cases. With a positive Nu-DESC score of 2 or more points, sensitivity was 100% (95% CI = [75%,100%]). Nu-DESC diagnosed 9 false positive; specificity was 86% (95% CI = [76%,93%]), positive predictive value was 57% (95% CI = [36%,76%]) and negative predictive value was 100% (95% CI = [94%,100%]). Fig. 1 shows ROC curve analysis. There were no significant differences between area under the curve (AUC) of physician's (AUC = 0.985, 95% CI = [0.928, 0.999]) and nurses' ratings (AUC = 0.979, 95% CI = [0.918, 0.998]).

 

 

Reliability and agreement

The Cronbach's alpha of the Nu-DESC checklist was similar between staff physician and nurses (0.81, 95%CI = [0.73, 0.87] vs. 0.79, 95%CI = [0.71, 0.85]). In the 78 paired observations performed, proportions of agreement between nurses and the research staff physician for the Nu-DESC items were all high (> 0.96) and weighted kappa ranged from 0.79 to 0.93. For all patient evaluations, agreement between nurses and research staff physician was total (Table 2). ICC for agreement between nurses and the research staff physician for the total Nu-DESC score was 0.98 with a 95% confidence interval from 0.96 to 0.99.

 

Discussion

This study suggests that the Portuguese version of the Nu-DESC is an accurate and reliable tool to assess delirium in critical care patients. This study also suggests that nurses as well as critical care physicians can rate the Nu-DESC as quickly and accurately. The instrument shows high sensitivity, which is important for a screening/monitoring instrument and a particularly high specificity not specifically different than those of the ICDSC. The Nu-DESC possessed values similar to those of other published studies.21,26,27

This study has several strengths. The instrument used was formally translated and back-translated using recommended guidelines and the authors of the original Nu-DESC study approved the faithfulness of the new version to the original. Also, this study presented an original design, with independent paired evaluations between a single physician and multiple bedside nurses. The results presented here suggest that the evaluations are equivalent in accuracy between physicians and nurses, indicating equivalent clinical assessment ability to detect delirium. This probably also reflects the involvement in the planning, baseline, and education phases amongst all the SICU staff, in order to cover all aspects relevant to the project. Our results are concordant with those of nursing studies that have evaluated the effect of educational efforts focused on the use of validated delirium screening tools.28 A group of researchers, nurses experienced in intensive care, and professional translators translated the Nu-DESC, contributing to preserve the meaning of the words and concepts specific to the SICU context. Previously, the authors noted that, with minimal training, nurses' compliance using sedation and delirium instruments is excellent. In the study of Pun et al.,29 the authors concluded that the data agreement for delirium assessment between bedside nurses and a reference-standard rater was very high.

In a study that validated Nu-DESC for use in ICUs, Luetz et al.14 suggested it may be a good alternative for detecting delirium in critically ill patients due to its high sensitivity but also high specificity (sensitivity: 83%, specificity: 83%) when compared to DSM-IV criteria. In addition to being similar to those presented here, their results were very similar to those obtained in the original validation study by Gaudreau et al.13 in an oncology inpatients setting. Our results strengthen the Nu-DESC's general reliability, even for populations as different as oncology and ICU patients.

This study also presents its limitations. The sample size allowed limited power to detect small differences. Clearly, the results presented here should be reproduced in larger samples, but are promising nonetheless.

We chose the ICDSC over the CAM-ICU as the gold standard in this study, since several studies had shown that the two instruments presented similar psychometric characteristics and the ICDSC was already in use in our setting. It has also been shown to be highly valid, as compared to DSM-IV evaluation by psychiatrists, and highly reliable, and is often preferred by nurses over other more burdensome delirium tools.

 

Conclusion

The Portuguese translation of the Nu-DESC showed good correlation with the original version and could be applicable to Portuguese critical care settings. Inter-rater reliability between staff physician and nurses was very good for item-by-item comparison, as well as for the diagnosis of delirium. Psychometric characteristics of the Nu-DESC were similar to those of the ICDSC, but the Nu-DESC has the significant advantage of being quicker to rate than the ICDSC because it involves fewer items. Thus, the Nu-DESC is promising as a good screening and monitoring instrument, with high sensitivity, but also as an accurate diagnostic instrument with high specificity for assessment of delirium in critical care settings.

Key messages

• The Portuguese translation of the Nu-DESC showed good correlation with the original version and could be applicable to Portuguese critical care settings.

• The Portuguese version of the Nu-DESC is an accurate and reliable tool to assess delirium in critical care patients.

• Nurses as well as critical care physicians can rate the Nu-DESC accurately.

• The Nu-DESC shows high sensitivity and high specificity for delirium assessment.

Authors' contributions

FA participated in conception, design, acquisition of the data, analysis of the data, statistical analysis, critical revision of the manuscript and supervision.

DV and MN participated in conception, design, acquisition of the data, analysis of the data and critical revision of the manuscript.

CS and JG has been involved in drafting the manuscript, analysis of the data and critically revising it for important content.

All authors read and approved the final manuscript.

 

Conflicts of interest

The authors declare no conflicts of interest.

 

Acknowledgements

Post-anesthesia Care Unit nurses team.

 

References

1. Ely EW, Shintani A, Truman B, et al. Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. Jama. 2004;291:1753-62.         [ Links ]

2. Leslie DL, Zhang Y, Holford TR, et al. Premature death associated with delirium at 1-year follow-up. Arch Intern Med. 2005;165:1657-62.         [ Links ]

3. Olin K, Eriksdotter-Jönhagen M, Jansson A, et al. Postoperative delirium in elderly patients after major abdominal surgery. Br J Surg. 2005;92:1559-64.         [ Links ]

4. Thomason JW, Shintani A, Peterson JF, et al. Intensive care unit delirium is an independent predictor of longer hospital stay: a prospective analysis of 261 non-ventilated patients. Crit Care. 2005;9:R375-81.         [ Links ]

5. Zakriya K, Sieber FE, Christmas C, et al. Brief postoperative delirium in hip fracture patients affects functional outcome at three months. Anesth Analg. 2004;98:1798-802.         [ Links ]

6. Aakerlund LP, Rosenberg J. Postoperative delirium: treatment with supplementary oxygen. Br J Anaesth. 1994;72:286-90.         [ Links ]

7. Deiner S, Silverstein JH. Postoperative delirium and cognitive dysfunction. Br J Anaesth. 2009;103(Suppl 1):i41-6.         [ Links ]

8. Girard TD, Pandharipande PP, Ely EW. Delirium in the intensive care unit. Crit Care. 2008;12(Suppl 3):S3.         [ Links ]

9. Quimet S, Kavanagh B, Gottfried SB, et al. Incidence, risk factors and consequences of ICU delirium. Intensive Care Med. 2007;33:66-73.         [ Links ]

10. Bergeron N, Dubois MJ, Dumont M, et al. Intensive Care Delirium Screening Checklist: evaluation of a new screening tool. Intensive Care Med. 2001;27:859-64.         [ Links ]

11. Ely EW, Gautam S, Margolin R, et al. The impact of delirium in the intensive care unit on hospital length of stay. Intens Care Med. 2001;27:1892-900.         [ Links ]

12. Ely EW, Inouye SK, Bernard GR, et al. Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU). Jama. 2001;286:2703-10.         [ Links ]

13. Gaudreau JD, Gagnon P, Harel F, et al. Fast, systematic, and continuous delirium assessment in hospitalized patients: the nursing delirium screening scale. J Pain Symptom Manage. 2005;29:368-75.         [ Links ]

14. Luetz A, Heymann A, Radtke FM, et al. Different assessment tools for intensive care unit delirium: which score to use? Crit Care Med. 2010;38:409-18.         [ Links ]

15. Lutz A, Radtke FM, Franck M, et al. [The Nursing Delirium Screening Scale (NU-DESC)]. Anasthesiol Intensivmed Notfallmed Schmerzther. 2008;43:98-102.         [ Links ]

16. Johnson J. Identifying and recognizing delirium. Dement Geriatr Cogn Disord. 1999;10:353-8.         [ Links ]

17. Milisen K, Foreman MD, Abraham IL, et al. A nurse-led interdisciplinary intervention program for delirium in elderly hip-fracture patients. J Am Geriatr Soc. 2001;49:523-32.         [ Links ]

18. Lundstrom M, Edlund A, Karlsson S, et al. A multifactorial intervention program reduces the duration of delirium, length of hospitalization, and mortality in delirious patients. J Am Geriatr Soc. 2005;53:622-8.         [ Links ]

19. Jacobi J, Fraser GL, Coursin DB, et al. Clinical practice guidelines for the sustained use of sedatives and analgesics in the critically ill adult. Crit Care Med. 2002;30:119-41.         [ Links ]

20. American Psychiatric Association. Diagnostic and statistical manual mental disorders IV (DSM IV); 4th edition. Washington, DC; 1994.         [ Links ]

21. Radtke FM, Franck M, Schneider M, et al. Comparison of three scores to screen for delirium in the recovery room. Br J Anaesth. 2008;101:338-43.         [ Links ]

22. Radtke FM, Franck M, MacGuill M, et al. Duration of fluid fasting and choice of analgesic are modifiable factors for early postoperative delirium. Eur J Anaesthesiol. 2010;27:411-6.         [ Links ]

23. Wild D, Grove A, Martin M, et al. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8:94-104.         [ Links ]

24. Cronbach LJ. Coefficient Alpha and the internal structure of tests. Psychometrika. 1951;16:297-334.         [ Links ]

25. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420-8.         [ Links ]

26. Leung JM, Leung VW, Leung CM, et al. Clinical utility and validation of two instruments (the Confusion Assessment Method Algorithm and the Chinese version of Nursing Delirium Screening Scale) to detect delirium in geriatric inpatients. Gen Hosp Psychiatry. 2008;30:171-6.         [ Links ]

27. Radtke FM, Franck M, Schust S, et al. A comparison of three scores to screen for delirium on the surgical ward. World J Surg. 2010;34:487-94.         [ Links ]

28. Devlin JW, Marquis F, Riker RR, et al. Combined didactic and scenario-based education improves the ability of intensive care unit staff to recognize delirium at the bedside. Crit Care. 2008;12:R19.         [ Links ]

29. Pun BT, Gordon SM, Peterson JF, et al. Large-scale implementation of sedation and delirium monitoring in the intensive care unit: a report from two medical centers. Crit Care Med. 2005;33:1199-205.         [ Links ]

 

 

Corresponding author:
Fernando Abelha
E-mail: fernando.abelha@gmail.com (F. Abelha)

Received 12 July 2012; accepted 6 September 2012

 

 

* Received from the Department of Anesthesiology - Post-Anesthesia Care Unit; Centro Hospitalar de São João, Porto, Portugal.

 

 

APPENDIX 1

Items of “The nursing delirium screening scale” (Portuguese version).

Os itens do “The nursing delirium screening scale”.
Características e descrições [classificação dos sintomas 0-2]

1. Desorientação:

Manifestação de desorientação verbal ou comportamental de não estar orientado no tempo, no lugar ou na percepção das pessoas presentes.

2. Comportamento inadequado:

Comportamento inadequado face ao lugar e/ou para com a pessoa, por exemplo, puxar por tubos ou pensos, tentar sair da cama quando tal está contraindicado, etc.

3. Comunicação inadequada:

Comunicação inadequada face ao lugar e/ou para a pessoa, por exemplo, incoerência, falta de comunicação, discurso absurdo ou ininteligível.

4. Ilusões / alucinações:

Ver ou ouvir coisas inexistentes no local; distorções de objetos visuais.

5. Atraso psicomotor:

Reação atrasada, poucas ou nenhumas ações/palavras, por exemplo, quando estimulado, o doente tem uma resposta atrasada e/ou o doente não consegue ser despertado.

 

APPENDIX 2

Items of “The nursing delirium screening scale” (English version).

The nursing delirium screening scale items, features and descriptions [symptoms rating (0–2)]

Verbal or behavioral manifestation of not being oriented to time or place or misperceiving persons in the environment

2. Inappropriate behavior

behavior inappropriate to place and/or for the person, e.g. pulling at tubes or dressings, attempting to get out of bed when that is contraindicated and so on

3. Inappropriate communication

Communication inappropriate to place and/or for the person, e.g. incoherence, noncommunicativeness, nonsensical or unintelligible speech

4. Illusions/ hallucinations

Seeing or hearing things that are not there; distortions of visual objects

5. Psychomotor retardation

Delayed responsiveness, few or no spontaneous action/words, e.g. when the patient is prodded, reaction is deferred and/or the patient is unarousable.

 

List of abbreviations

95%CI - 95% confidence intervals

AUC - area under the curve

CAM-ICU - confusion assessment method for the intensive care unit

DSM-IV - Diagnostic and Statistical Manual of Mental Disorders IV

ICC - intraclass coefficient

ICDSC - Intensive Care Delirium Screening Checklist

ICU - intensive care unit

Nu-DESC - The Nursing Delirium Screening Scale

PACU - post-anaesthesia care unit

ROC - relative operating characteristic

SICU - surgical intensive care unit

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License