Validation of sedation scores in mechanically ventilated children admitted to a tertiary pediatric intensive care unit

Abstracts

OBJETIVOS: O uso de escalas de sedação é fundamental em unidades de terapia intensiva pediátrica. A escala Comfort-Behavior é validada para avaliação de crianças, contudo, é um instrumento extenso. A escala de avaliação da atividade motora está validada para adultos, é mais simples do que a anterior e possível de ser usada em crianças. Nenhuma dessas escalas está validada na língua portuguesa. O objetivo primário deste estudo foi validar as duas escalas traduzidas para o português em crianças submetidas à ventilação mecânica. Os objetivos secundários foram avaliar o nível de sedação dos pacientes em ventilação mecânica de unidades de terapia intensiva pediátrica terciária e comparar o desempenho das duas escalas nesta população. MÉTODOS: Após a tradução para o português, as escalas foram aplicadas em 26 pacientes por dois médicos, simultaneamente. Obteve-se um total de 116 observações por escala. RESULTADOS: O coeficiente de correlação intraclasse foi 0,90 (IC95% 0,85 - 0,93) para a escala Comfort-Behavior e 0,94 (IC 95% 0,92 - 0,96) para a avaliação da atividade motora. O alfa de Crombach para o observador A ao aplicar a escala Comfort-Behvior foi 0,81 e para o observador B, 0,92. O coeficiente de Spearman para o observador A foi 0,86 e para o observador B, 0,91. As aplicações das escalas revelaram pacientes bastante sedados, atingindo pontuações baixas em ambas. CONCLUSÕES: A validação das escalas Comfort-Behavior e avaliação da atividade motora para o português foi realizada com sucesso. Ambas foram adequadas para emprego em crianças em ventilação mecânica. Nas aplicações avaliadas, o nível de sedação observado na unidade estudada foi alto.

Unidades de terapia intensiva pediátrica; Respiração artificial; Monitorização fisiológica; Medição da dor; Analgésicos; Comportamento infantil; Criança

ORIGINAL ARTICLE

Validation of sedation scores in mechanically ventilated children admitted to a tertiary pediatric intensive care unit

Carolina Friedrich Amoretti^I; Gleiber Oliveira Rodrigues^II; Paulo Roberto Antonacci Carvalho^III; Eliana de Andrade Trotta^IV

^IPhysician, Pediatric Intensive Care Resident from the Hospital de Clínicas de Porto Alegre (RS), Brazil

^IIPhysician, Pediatric Intensive Care Resident from the Hospital de Clínicas de Porto Alegre (RS), Brazil

^IIIPhD, Associated Professor from the Pediatrics Department of the Faculdade de Medicina da Universidade Federal do Rio Grande do Sul - UFRGS - Porto Alegre (RS), Brazil

^IVMasters, Assistant Professor from the Pediatric Department of the Faculdade de Medicina da Universidade Federal do Rio Grande do Sul - UFRGS - Porto Alegre (RS), Brazil

Author for correspondence

ABSTRACT

OBJECTIVES: Sedation scores are important tools for use in pediatric intensive care units. The Comfort-Behavior scale is a valid method for the assessment of children although it is considered an extensive scale. The motor activity assessment scale is validated for an adult population. We considered it simpler then the one above and suitable for application in children. None of these scores had been translated into Portuguese. Our objective was to apply both scales in Portuguese to a pediatric population under mechanical ventilation. Secondary objectives were to evaluate the sedation level of children on mechanical ventilation in tertiary pediatric intensive care units and to compare the Comfort-Behavior and motor activity assessment scales in this population.

METHODS: After translating the scales into Portuguese, both were simultaneously applied to 26 patients by 2 pediatricians. Each scale was applied 116 times in total.

RESULTS: The intraclass correlation coefficient was 0.90 (0.85 - 0.93 CI 95%) for the Comfort-Behavior and 0.94 (0.92 - 0.96 CI 95%) for the motor activity assessment scale. When applying the Comfort-Behavior scale, the Crombach's alpha was 0.81 for observer A and 0.92 for observer B. The Spearman coefficient was 0.86 for observer A and 0.91 for observer B. These patients were found to be deeply sedated, showing low values in both scales.

CONCLUSIONS: The scales were successfully translated into Portuguese and both were adequate to assess pain and sedation in the pediatric population under mechanical ventilation. Sedation level was high in this sample of applications.

Keywords: Intensive care units, pediatric; Respiration, artificial; Monitoring, physiologic; Pain measurement/methods; Analgesics/standards; Child behavior; Child

INTRODUCTION

Most patients admitted to the pediatric intensive care unit (ICU) need sedation and analgesia to optimize treatment. Sedation lessens agitation and permits better synchronization with mechanical ventilation, reduces oxygen demand and controls anxiety or pain caused by disease or by the unit environment.⁽¹⁾ For an adequate assessment of the patient's sedation level, a practical, objective and easy to use tool is required. Although the clinical opinion of physicians and nurses is important, a scale application is required to estimate the neurophysiologic effects of sedatives and other interventions as well as to permit a comparison between samples of patients.

The validated sedation scales for an adult population are readily found in literature. However, specific scales for the pediatric population are scarce. The Comfort scale was described by Ambuel et al. in 1992 and is validated for critically ill pediatric patients.⁽²⁾ However, as this scale describes behavior and physiological parameters (heart rate and arterial pressure) and the latter usually are controlled in an intensive care environment, in 2005 the Comfort-Behavior scale was validated as an alternative for the first one. It was comprised only of behavior variables, and further had an item on crying for a better assessment of the child out of mechanical ventilation (Appendix 1).⁽³⁾ In international literature the Comfort-Behavior scale was validated comparing it to the Comfort scale.⁽⁴⁾ This is considered a difficult scale to apply as it is lengthy; and not validated for the Portuguese language.

As opposed to the complexity of the Comfort-Behavior scale, there are simpler and more objective validated scales for adults. The Motor Activity Assessment Scale (MAA) ranks patients in the intensive care unit (ICU) by a cognitive spectrum using clear and concise descriptions for each category (Appendix 2).⁽⁵⁾ The scale was initially validated for adult, surgical patients under mechanical ventilation. This scale has not yet been validated for the pediatric population or for the Portuguese language.

Considering the growing use of sedation in the PICU of the Hospital de Clínicas de Porto Alegre and the need for a specific protocol to assess the patients' level of sedation, this study was carried out for the primary purpose of validating the Comfort-Behavior and the Motor Activity Assessment Scale for the Portuguese language for children submitted to mechanical ventilation. Secondary purposes were to assess the level of sedation and compare performance of the MAA scale with the Comfort-Behavior for the pediatric population under mechanical ventilation staying at the unit under study.

METHODS

After approval by the Research Ethics Committee of the Hospital de Clínicas de Porto Alegre a cross sectional, observational study was carried out at the PICU of the Hospital de Clínicas de Porto Alegre a tertiary reference academic general hospital with 13 beds. There, children from the 28^th day to 16 years of age are admitted for a variety of clinical and surgical treatments, with the exception of cardiac surgery and trauma. Due to the observational characteristics of the study a written informed consent was not required.

All patients admitted to the PICU and under mechanical ventilation were eligible for the study. Patients with traumatic brain injury eventually admitted to the unit, as well as those with hypoxic-ischemic encephalopathy, ischemic or hemorrhagic stroke, cerebral paralysis, alterations of the muscle function, metabolic or induced coma by undue use of drugs or medication (intoxication, attempted suicide or negligence) were excluded from this sample.

Tools

Comfort-Behavior Scale

As previously described, the Comfort-Behavior scale derives from the Comfort scale, the most extensively used and a basis for the majority of recent comparative studies. This scale presents eight variables each one with six scores. Physiological variables, heart rate (HR) and arterial pressure (AP) are used to more objectively evaluate the level of distress; however, the correct use of these variables assumes that the basic values for HR and AP are adjusted every day. Because these variables are medicated and controlled in the PICU, their utility in the assessment of the sedation adequacy is questionable. The Comfort-Behavior scale refers only to behavior variables using in addition, an item regarding crying for better assessment of children out of mechanical ventilation. Ista et al. evaluated the validity of applying physiological variables in the scale (Comfort vs. Comfort-Behavior) and concluded that, without the physiological variables, the internal consistency measured by the Crombach's alpha increased from 0.78 to 0.84.⁽⁴⁾ After publication of this study, which concluded that the adjusted scale is a reliable alternative to the Comfort scale, the latter has been increasingly used in the PICU. The same study also established cut-off points in the Comfort-B scale, based upon application by trained nurses and a comparison to the Nurse Interpretation Sedation Scale (NISS) was made. Whereupon it was suggested that a score < 10 would correspond to super-sedation and > 23 to low sedation. Intermediate scores (11-22) could not predict the degree of sedation, requiring a more careful observation. This scale is considered complex because of the number of variables at each level, in addition to being an extensive scale.

Motor Activity Assessment Scale (MAA)

Developed in an ICU in Utah, it uses seven levels of consciousness characterized by response to pain and degree of cooperation. Each assessment must strictly follow the explanation corresponding to each category, leaving only a small margin for different interpretations (Appendix 2). It does not rank but classifies the sedation level. Thus, the MAA standardizes and facilitates recording and assessment of the level of patient's sedation, in addition, its application takes little time.

Protocol

Translation of the scales into the Portuguese language

The project started with the translation and retrotranslation of the scales into the Portuguese language. As such, the scales were translated from English into the Portuguese language and then again translated into English based upon their Portuguese version. After a comparison was made between the two scales (original version and translated version into English based upon the scale in Portuguese) by a person whose native language is English.

Application of the scales

Application was performed by two resident physicians in a pediatric intensive care unit, once a day (in the morning) for each patient included in the study, by sequential order of the beds (from "A" to "M") Application of the two scales was performed in a blind and sequential way, however simultaneously by the two observers. For each application of the two scales, a report with the patient's data was filled out with age, gender, diagnosis, mechanical ventilation parameters (MV), sedation used and doses of vasoactive drugs.

Sample calculation and statistical analysis

The calculated sample for this study was of 113 observations for each scale, considering the identification of a minimum correlation coefficient of 0.3 with a two-tailed alpha of 0.05 and a statistical power of 90%.

After collection, data were stored in a database structured in the Excel 2002, Microsoft Office program to be analyzed in SPSS 12.0. Frequencies of demographic data and medians and interquartiles for qualitative variables were described whenever appropriate.

Reproducibility of the scales was assessed by means of the interclass correlation coefficient (ICC). An ICC > 0.75 corresponds to an excellent correlation, between 0.4 and 0.75 the ICC is considered moderate and, under 0.4 the ICC is considered weak. Internal consistency of the Comfort-Behavior scale was assessed by using the Crombach's alpha coefficient in the assessment of the MAA scale, no statistical tests were performed because it is a classifying scale. Because there is no gold standard for the assessment of sedation in children, and the Comfort-Behavior scale is validated for pediatrics, concurrent validity was tested with the Spearman coefficient comparing assessments of each scale.

RESULTS

From May 15 to September 15 of 2005, 464 assessments were recorded (116 paired scores for each scale), in a total of 26 patients. Median of age was 6 months (2 - 12 months, IQ25-IQ75). Nineteen boys and 7 girls were observed and the more frequent diagnoses were those of respiratory disease (84%), while postoperative and sepsis were responsible for 8% (n=2) each (Table 1). Combination of midazolan with fentanyl was the most often used continued sedation and analgesia design and 100% of the assessed children were under the regimen of sedation in continuous infusion.

The interclass correlation coefficient was of 0.90 (CI 95% 0.86 - 0.93; p < 0.01) for the Comfort-Behavior scale and of 0.95 (CI 95% 0.92 - 0.96; p < 0.01) for the MAA scale, disclosing an excellent reproducibility of the Portuguese version of both scales. The Crombach coefficient for observer A when applying the Comfort-Behavior scale was of 0.81 and for observer B of 0.92, showing a very good internal consistency. (Table 2). In assessing concurrent validity, the Spearman coefficient for observer A was of 0.88 and for observer B of 0.91 with p<0.01 for both.

In this sample, 55% of patients had Comfort-Behavior scores of less than 10. The scores most often given were: 7 by observer 1 (17.2%) and 6 by observer 2 (34.5%). Using cut-off points previously described in literature to categorize results in the Comfort-Behavior scale, scores of < 10 were found in 62% of the times, when the scale was applied by observer 1 vs. 67% when it was applied by observer 2, with kappa of 0.69. Only one score over 23 was fond. Regarding the MAA scale, 42% of patients were considered in a non-responsive state and 15% in a responsive state only to pain stimuli. The score most often given was zero for the two observers, with kappa of 0.64.

DISCUSSION

To date there were no validated scales in the Portuguese language. This work carried out the validation of scales used into Portuguese. For this purpose the referential of translation and retro-translation of the tools was used and tests of reproducibility (interclass correlation coefficient) were performed and internal consistency among the 116 applications carried out by each of the two observers in each scale. Results of the ICC and of Crombach's alpha are excellent, validating this version of the scales translated into Portuguese.

On occasion, interpretation of results of applications of the Comfort-Behavior scale, that range from 8 to 40 may not be precise and not present a clinical relevance. In the endeavor to solve this issue, the Ista et al.,⁽⁴⁾ study previously mentioned, suggests a cut-off points for classification of patients in super sedated (score 6-10) sedated (from 11 to 22) and less sedated ( more than 23). Taking these cut-off points as valid and analyzing our sample from this aspect, mean of scoring is on the limit between the two first cut-off points, independent from variation. This discloses a sample of quite sedated patients. When analyzing the most frequently found diagnoses in this population, we found 72% with pulmonary disease. This may reflect a need for handling (deeper sedation to better tolerate high MV parameters) or a tendency of the studied PICU team to maintain patients under high sedation. About 35% of patients were between the 11 and 22 scores in this scale, scores that represent a population needing special attention and that cannot be easily classified as adequately sedated. Literature suggests that these patients needed another type of assessment. In these cases use of the bispectral index (BSI) may explain the patients' level of consciousness and the level of sedation and analgesia required.^(6,7)

Studies seeking to compare Comfort and Comfort-B with new scales are not unusual^(6,8) since these scales are considered long and complex there is an ongoing search for alternatives. The initial consideration for the choice of the MAA scale was based precisely on its simplicity, believing that this would lead to a more coherent and reproducible analysis, in addition to being a scale applicable to pediatric patients. Data analyzed disclosed that this scale correlates well with the sedation levels of the Comfort-Behavior scale (with a Spearman coefficient).

CONCLUSION

The Comfort-B and MAA scales in Portuguese have proven to be adequate for assessment of sedation and analgesia of children in the ICU under mechanical ventilation due to diverse diseases, and are equivalent among themselves.

Use of these sedation and analgesia scales for assessment of therapeutic adequacy for patients is a growing practice in pediatrics, especially in an environment such as the PICU. The existence of different types of scales discloses the lack of international standardization. Furthermore, existing scales require adequacy of the variables according to the population cared for in each specific unit (postoperative, use or not of mechanical ventilation, etc,) The present study validated, for the Portuguese language, two scales with a different profile for use in children under mechanical ventilation in a Brazilian PICU in an effort to stimulate the practice of this type of monitoring.

ACKNOLEDGEMENTS

We wish to thank Dr. Patrícia Miranda Lago for her support in the thematic preparation and incentive during data collection.

REFERÊNCIAS

• 1. De Jonghe B, Cook D, Appere-de-Vecchi C, Guyatt G, Meade M, Outin H. Using and understanding sedation scoring systems: a systematic review. Intensive Care Med. 2000;26(3):275-85.
• 2. Ambuel B, Hamlett KW, Marx CM, Blumer JL. Assessing distress in pediatric intensive care environments: the COMFORT scale. J Pediatr Psychol. 1992;17(1):95-109.
• 3. van Dijk M, Peters JW, van Deventer P, Tibboel D. The COMFORT Behavior Scale: a tool for assessing pain and sedation in infants. Am J Nurs. 2005;105(1):33-6.
• 4. Ista E, van Dijk M, Tibboel D, de Hoog M. Assessment of sedation levels in pediatric intensive care patients can be improved by using the COMFORT "behavior" scale. Pediatr Crit Care Med. 2005;6(1):58-63. Comment in: Pediatr Crit Care Med. 2005;6(1):91-2.
• 5. Devlin JW, Boleski G, Mlynarek M, Nerenz DR, Peterson E, Jankowski M, et al. Motor Activity Assessment Scale: a valid and reliable sedation scale for use with mechanically ventilated patients in an adult surgical intensive care unit. Crit Care Med. 1999;27(7):1271-5. Comment in: Crit Care Med. 1999;27(7):1384-5. Crit Care Med. 2000;28(8):3124.
• 6. Twite MD, Zuk J, Gralla J, Friesen RH. Correlation of the Bispectral Index Monitor with the COMFORT scale in the pediatric intensive care unit. Pediatr Crit Care Med. 2005;6(6):648-53; quiz 654. Comment on: Pediatr Crit Care Med. 2005;6(6):715-8.
• 7. Mondello E, Siliotti R, Noto G, Cuzzocrea E, Scollo G, Trimarchi G, Venuti FS. Bispectral Index in ICU: correlation with Ramsay Score on assessment of sedation level. J Clin Monit Comput. 2002;17(5):271-7.
• 8. Brunow de Carvalho W, Lucas da Silva PS, Paulo CS, Fonseca MM, Belli LA. Comparison between the Comfort and Hartwig sedation scales in pediatric patients undergoing mechanical lung ventilation. Sao Paulo Med J. 1999;117(5):192-6.

Publication Dates

• Publication in this collection
  05 Feb 2009
• Date of issue
  Dec 2008

History

• Accepted
  04 Nov 2008
• Received
  28 Jan 2008