Interobserver agreement in a checklist of care in enteral nutrition therapy

This is a reliability study that preceded a clinical trial (NCT03497221), carried out at a university hospital in southern Brazil (June and July 2017). A checklist of 25 items related to care in ENT was performed by a nurse (reference standard) and nine research assistants (RA). Assessments were carried out concurrently and independently. Agreement was tested using the Statistical Package for the Social Sciences, version 21.0. Kappa values (k) were considered as poor (0 to 0.19), relative (0.20 to 0.39), moderate (0.40 to 0.59), substantial (0.60 and 0.79), almost perfect (0.80 to 0.99), and perfect (1). The study was approved by an Institutional Review Board (number 16-0534).

Results

Three hundred fifty-one observations were made in duplicate; the lowest number of observations was with RA, 5 (n = 35) and the highest with RA, 8 (n = 45). Items related to ENT were assessed in three blocks: identification of infusion bottles and infusion pump; support materials for administering the therapy; care for patients using ENT. There was almost perfect or perfect agreement in all observation pairs, with lowest Kappa for RA 6 (k = 0.890; 95% CI = 0.86, 0.92) and the highest for RA 3 (k = 0.965; 95% CI = 0.93, 0.99).

Conclusion

Interobserver agreement, when applying a checklist containing 25 items, was excellent, which minimizes the occurrence of measurement bias in subsequent steps.

Nutrition therapy; Enteral nutrition; Checklist; Bias

Checklist items	RA 1 n=38 k (CI): 0.89 (0.86– 0.92)	RA 2 n=38 k (CI): 0.91 (0.86– 0.92)	RA 3 n=42 k (CI): 0.96 (0.93– 0.99)	RA 4 n=41 k (CI): 0.95 (0.92– 0.97)	RA 5 n=35 k (CI): 0.93 (0.90– 0.96)	RA 6 n=41 k (CI): 0.89 (0.86– 0.92)	RA 7 n=39 k (CI): 0.94 (0.92–0.97)	RA 8 n=45 k (CI): 0.94 (0.92– 0.97)	RA 9 n=38 k (CI): 0.94 (0.92– 0.98)
Identification of the enteral diet bottle being administered
Same as the patients’ bracelet	0.79 (0.69– 0.99)	0.89 (0.77– 1 *)	0.73 (0.66– 0.93)	1.00 (0.85– 1*)	0.94 (0.81– 1*)	1.00 (0.85– 1*)	0.94 (0.81– 1*)	0.95 (0.82– 1*)	0.94 (0.81– 1*)
Expiration (up to 3h)	1.00 (0.85– 1*)	0.91 (0.77– 1*)	0.89 (0.75– 1*)	0.91 (0.78– 1*)	0.94 (0.81– 1*)	1.00 (0.85– 1*)	0.95 (0.81– 1*)	0.96 (0.82– 1*)	0.95 (0.81– 1*)
Identification of the water bottle for cleaning utensils and ENT
Same as the patients’ bracelet	0.81 (0.77– 1*)	1.00 (0.85– 1*)	0.80 (0.60– 0.89)	1.00 (0.85– 1*)	0.95 (0.77– 1*)	1.00 (0.85– 1*)	1.00 (0.85– 1*)	1.00 (0.86– 1*)	1.00 (0.85– 1*)
Expiration (24h)	0.94 (0.81– 1*)	1.00 (0.85– 1*)	1.00 (0.85– 1*)	1.00 (0.85– 1*)	0.91 (0.77– 1*)	0.95 (0.81– 1*)	1.00 (0.85– 1*)	1.00 (0.86– 1*)	1.00 (0.85– 1*)
Identification of the water bottle for hydration
Same as the patients’ bracelet	1.00 (0.85–1*)	1.00 (0.85– 1*)	0.25 (0.7- 0.46)	1.00 (0.85– 1*)	0.90 (0.77– 1*)	0.94 (0.81– 1*)	1.00 (0.85– 1*)	1.00 (0.86– 1*)	1.00 (0.85– 1*)
Expiration (24h)	1.00 (0.85– 1*)	1.00 (0.85– 1*)	0.94 (0.82– 1*)	1.00 (0.85– 1*)	0.89 (0.77– 1*)	0.85 (0.74– 1*)	1.00 (0.85– 1*)	1.00 (0.86– 1*)	1.00 (0.85– 1*)
Identification of the infusion pump used for ENT
No dirt	0.93 (0.81– 1*)	0.82 (0.77– 1*)	0.92 (0.75– 1*)	0.86 (0.78– 1*)	0.87 (0.77– 1*)	0.74 (0.78– 1*)	0.92 (0.81– 1*)	0.87 (0.79 - 1*)	0.91 (0.81– 1*)

Checklist items	RA 1 n=38 k (CI): 0.89 (0.86– 0.92)	RA 2 n=38 k (CI): 0.91 (0.86– 0.92)	RA 3 n=42 k (CI): 0.96 (0.93– 0.99)	RA 4 n=41 k (CI): 0.95 (0.92– 0.97)	RA 5 n=35 k (CI): 0.93 (0.90– 0.96)	RA 6 n=41 k (CI): 0.89 (0.86– 0.92)	RA 7 n=39 k (CI): 0.94 (0.92–0.97)	RA 8 n=45 k (CI): 0.94 (0.92– 0.97)	RA 9 n=38 k (CI): 0.94 (0.92– 0.98)
Identification of the equipment used for ENT administration (blue)
At the bedside	1.00 (0.85– 1*)	1.00 (0.85– 1*)	0.73 (0.66– 0.93)	1.00 (0.85– 1*)	1.00 (0.85– 1*)	0.84 (0.70– 0.99)	1.00 (0.85– 1*)	1.00 (0.86– 1*)	1.00 (0.85– 1*)
Expiration (24h)	0.88 (0.77–1*)	0.90 (0.77– 1*)	0.93 (0.78– 1*)	0.89 (0.78– 1*)	1.00 (0.85– 1*)	0.94 (0.81. 1*)	0.93 (0.81– 1*)	0.94 (0.82– 1*)	1.00 (0.85– 1*)
No diet dirt	0.95 (0.81–1 *)	0.80 (0.69– 0.99)	0.94 (0.67– 0.96)	0.96 (0.81– 1*)	0.95 (0.77– 1*)	0.65 (0.52– 0.81)	0.67 (0.54– 0.84)	0.96 (0.82– 1*)	0.90 (0.77– 1*)
Identification of the tip of the blue equipment for ENT protected with cover
When not infusing the diet	0.57 (0.45-0.75)	0.76 (0.69– 0.99)	1.00 (0.67– 0.96)	0.87 (0.78– 1 *)	0.94 (0.81– 1 *)	0.93 (0.90– 0.96)	1.00 (0.85– 1 *)	0.94 (0.82– 1 *)	1.00 (0.85– 1 *)
Identification of the syringe used for ENT (Oralpak^®)
At the bedside	1.00 (0.85–1 *)	1.00 (0.85– 1 *)	1.00 (0.85– 1 *)	1.00 (0.85– 1 *)	1.00 (0.85– 1 *)	1.00 (0.85– 1 *)	0.94 (0.81– 1 *)	1.00 (0.86– 1 *)	1.00 (0.85– 1 *)
Expiration (6h)	1.00 (0.85–1 *)	0.86 (0.77– 1 *)	0.90 (0.78– 1 *)	0.94 (0.81– 1 *)	0.89 (0.77– 1 *)	0.94 (0.81– 1 *)	0.87 (0.73– 1 *)	1.00 (0.86– 1 *)	0.93 (0.90– 0.96)
Clean and without residues	0.90 (0.77–1 *)	0.90 (0.77– 1 *)	1.00 (0.75– 1*)	0.79 (0.67– 0.96)	0.91 (0.77– 1 *)	0.66 (0.56– 0.85)	0.96 (0.81– 1 *)	0.96 (0.82– 1 *)	1.00 (0.85– 1 *)
Labeled (name and medical record number)	0.95 (0.81–1 *)	1.00 (0.85– 1 *)	0.95 (0.78– 1 *)	0.96 (0.81– 1 *)	0.85 (0.70– 0.99)	0.92 (0.78– 1 *)	0.96 (0.81– 1 *)	1.00 (0.86– 1 *)	1.00 (0.85– 1 *)
Labeled (bed, date and shift)	0.87 (0.77–1 *)	0.94 (0.81– 1 *)	1.00 (0.75– 1 *)	0.94 (0.81– 1 *)	0.78 (0.70– 0.99)	0.95 (0.81– 1 *)	0.95 (0.81– 1 *)	0.95 (0.82– 1 *)	1.00 (0.85– 1 *)
Identification of the plastic cup used to sanitize devices used in ENT
Expiration (6h)	0.94 (0.81–1 *)	0.89 (0.77– 1 *)	1.00 (0.82– 1 *)	1.00 (0.85– 1 *)	1.00 (0.85– 1 *)	0.95 (0.74– 1 *)	1.00 (0.85– 1 *)	1.00 (0.86– 1 *)	0.94 (0.81– 1 *)
Clean and without residues	0.81 (0.73–1 *)	0.91 (0.77– 1 *)	0.94 (0.77– 1 *)	0.95 (0.81– 1 *)	1.00 (0.85– 1 *)	0.95 (0.81– 1 *)	0.90 (0.77– 1 *)	0.95 (0.82– 1 *)	0.95 (0.81– 1 *)
Dry	0.82 (0.69–0.99)	0.90 (0.77– 1 *)	1.00 (0.78– 1 *)	0.89 (0.74– 1 *)	0.95 (0.81– 1 *)	0.88 (0.74– 1 *)	1.00 (0.85– 1 *)	0.96 (0.82– 1 *)	0.90 (0.77– 1 *)

Checklist items	RA 1 n=38 k (IC): 0.89 (0.86– 0.92)	RA 2 n=38 k (IC): 0.91 (0.86– 0.92)	RA 3 n=42 k (IC): 0.96 (0.93– 0.99)	RA 4 n=41 k (IC): 0.95 (0.92– 0.97)	RA 5 n=35 k (IC): 0.93 (0.90– 0.96)	RA 6 n=41 k (IC): 0.89 (0.86– 0.92)	RA 7 n=39 k (IC): 0.94 (0.92–0.97)	RA 8 n=45 k (IC): 0.94 (0.92– 0.97)	RA 9 n=38 k (IC): 0.94 (0.92– 0.98)
Observation and identification of the fixation of ENT related to conservation and validity
Clean	0.76 (0.69– 0.99)	0.64 (0.77– 1*)	0.80 (0.60– 0.89)	0.87 (0.81– 1*)	1.00 (0.85– 1*)	0.45 (0.70– 0.99)	0.89 (0.81– 1*)	0.73 (0.66– 0.93)	0.28 (0.69– 0.99)
No oil	0.69 (0.65– 0.95)	0.63 (0.57– 0.87)	0.81 (0.75– 1*)	0.57 (0.63– 0.92)	0.53 (0.47– 0.77)	0.20 (0.44– 0.72)	0.63 (0.58– 0.87)	0.61 (0.56– 0.83)	0.68 (0.61– 0.91)
Not detached	0.27 (0.57– 0.87)	0.47 (0.61– 0.91)	1.00 (0.82– 1*)	0.84 (0.78– 1*)	0.87 (0.81– 1*)	0.41 (0.63– 0.92)	0.87 (0.81– 1*)	0.85 (0.79– 1*)	0.47 (0.61– 0.91)
Not pulled	0.72 (0.77– 1*)	0.16 (0.61– 0.91)	1.00 (0.82– 1 *)	0.65 (0.81– 1 *)	0.70 (0.74– 1 *)	0.74 (0.78– 1 *)	0.65 (0.81– 1 *)	0.41 (0.62– 0.90)	0.44 (0.69– 0.99)
Dated (up to 24h previous)	1.00 (0.85–1*)	1.00 (0.85– 1 *)	1.00 (0.82– 1 *)	0.94 (0.81– 1 *)	1.00 (0.85– 1 *)	0.78 (0.70– 0.99)	0.94 (0.81– 1 *)	1.00 (0.86– 1 *)	1.00 (0.85– 1 *)
Observation and identification of the elevated head of patients’ beds = > 30º
In diet/water administration by ENT	0.42 (0.37– 0.67)	0.94 (0.81– 1 *)	0.89 (0.64– 0.92)	0.90 (0.78– 1 *)	0.83 (0.70– 0.99)	0.76 (0.67– 0.96)	0.85 (0.73– 1 *)	0.86 (0.78– 1 *)	0.94 (0.81– 1 *)

Escola Paulista de Enfermagem, Universidade Federal de São Paulo R. Napoleão de Barros, 754, 04024-002 São Paulo - SP/Brasil, Tel./Fax: (55 11) 5576 4430 - São Paulo - SP - Brazil
E-mail: actapaulista@unifesp.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] Corresponding author: Ana Paula Almeida Corrêa. Email: anacorrea.enf@gmail.com