Cumulative sum analysis applied to the teaching of spinal anesthesia

Oliveira Filho, Getúlio Rodrigues de; Pederneiras, Sérgio Galluf; Garcia, Jorge Hamilton Soares

doi:10.1590/S0034-70942001000300004

Abstracts

BACKGROUND AND OBJECTIVES: Objective performance measurements are necessary during the learning process of anesthetic techniques. This study aimed at describing the cusum (cumulative sum) analysis to build subarachnoid puncture learning curves. METHODS: The successful subarachnoid space identification during the first attempt, in the first interspace approached, and the success of 275 spinal blocks performed by 5 residents during the initial six-month training period were used to build learning curves through the cusum analysis. Acceptable failure rates were taken from a sample of 264 spinal blocks performed by experienced anesthesiologists. The number of blocks needed for proficiency was calculated for each resident, for each attribute. RESULTS: A wide variability in the number of blocks needed for attaining proficiency was observed, depending on the resident or on the attribute. However, most residents attained proficiency after 50 blocks. All residents achieved a 90% success rate after 30 blocks. CONCLUSIONS: Cusum analysis may be used for objectively measuring performance during the learning phase of spinal anesthesia. A minimum of 50 blocks is required in order to attain proficiency in identifying subarachnoid space.

ANESTHESIOLOGY: teaching; ANESTHETIC TECHNIQUES, Regional: spinal block

JUSTIFICATIVA E OBJETIVOS: Medidas objetivas de desempenho são necessárias para o ensino de técnicas anestésicas. Este estudo teve por objetivos aplicar e descrever o método da soma cumulativa dos desvios para a construção de curvas de aprendizado da punção subaracnóidea. MÉTODO: O sucesso da punção subaracnóidea na primeira tentativa, no primeiro espaço abordado, e o sucesso da anestesia de 275 bloqueios subaracnóideos realizados por médicos em especialização (ME) durante os primeiros seis meses de treinamento foram utilizados para a construção de curvas de aprendizado, utilizando o método das somas cumulativas de desvios. As taxas aceitáveis de falha foram derivadas de uma amostra de 264 bloqueios subaracnóideos realizados por anestesiologistas. O número necessário de bloqueios para se obter proficiência foi calculado, para cada ME, para cada atributo. RESULTADOS: Houve uma grande variabilidade em relação ao número de bloqueios necessários para se atingir a proficiência, dependendo do atributo e do ME. Contudo, a maioria dos ME atingiu proficiência para os atributos de sucesso durante a primeira tentativa ou no primeiro espaço inter-espinhoso abordado, após 50 bloqueios. Uma taxa de sucesso da anestesia de 90% foi obtida por todos os ME após 30 bloqueios. CONCLUSÕES: O método da soma cumulativa de desvios pode ser utilizado para medir objetivamente o desempenho de médicos em especialização, durante a fase de aprendizado de anestesia subaracnóidea. Um número mínimo de 50 bloqueios é necessário para se obter proficiência em identificar o espaço subaracnóideo.

ANESTESIOLOGIA: ensino; TÉCNICAS ANESTÉSICAS, Regional: subaracnóidea

JUSTIFICATIVA Y OBJETIVOS: Medidas objetivas de desempeño son necesarias para la enseñanza de técnicas anestésicas. Este estudio tuvo por objetivo aplicar y describir el método de la suma cumulativa de los desvíos para la construcción de curvas de aprendizaje de la punción subaracnóidea. MÉTODO: El suceso de la punción subaracnóidea en la primera tentativa, en el primer espacio abordado, y el suceso de la anestesia de 275 bloqueos subaracnóideos realizados por médicos en especialización (ME) durante los primeros seis meses de entrenamiento fueron utilizados para la construcción de curvas de aprendizaje, utilizando el método de las sumas cumulativas de desvíos. Las tasas aceptables de falla fueron derivadas de una muestra de 264 bloqueos subaracnóideos realizados por anestesistas. El número necesario de bloqueos para obtenerse proficiencia fue calculado, para cada ME, para cada atributo. RESULTADOS: Hubo una grande variabilidad en relación al número de bloqueos necesarios para alcanzar la proficiencia, dependiendo del atributo y del ME. No obstante, la mayoría de los ME alcanzó proficiencia para los atributos de suceso durante la primera tentativa o en el primero espacio inter-espinoso abordado, después 50 bloqueos. Una tasa de suceso de la anestesia de 90% fue obtenida por todos los ME después de 30 bloqueos. CONCLUSIONES: El método de la suma cumulativa de desvíos puede ser utilizado para medir objetivamente el desempeño de los médicos en especialización, durante la etapa de aprendizaje de anestesia subaracnóidea. Un número mínimo de 50 bloqueos es necesario para obtenerse proficiencia en identificar el espacio subaracnóideo.

SCIENTIFIC ARTICLE

Cumulative sum analysis applied to the teaching of spinal anesthesia^* * Received from CET/SBA Integrado de Anestesiologia da SES-SC, Florianópolis, SC

Uso da soma cumulativa dos desvios para avaliação da proficiência no ensino do bloqueio subaracnóideo

Uso de la suma cumulativa de los desvíos para evaluación de la proficiencia en la enseñanza del bloqueo subaracnóideo

Getúlio Rodrigues de Oliveira Filho, TSA, M.D.^I; Sérgio Galluf Pederneiras, TSA, M.D.^II; Jorge Hamilton Soares Garcia, TSA, M.D.^III

^IResponsável pelo CET/SBA

^IIInstrutor Co-responsável pelo CET/SBA

^IIIInstrutor do CET/SBA

^{Correspondence} Correspondence to: Dr. Getúlio Rodrigues de Oliveira Filho Address: Rua José Cândido Silva 179/402 ZIP: 88075-250 City: Florianópolis, Brazil E-mail: grof@th.com.br

SUMMARY

BACKGROUND AND OBJECTIVES: Objective performance measurements are necessary during the learning process of anesthetic techniques. This study aimed at describing the cusum (cumulative sum) analysis to build subarachnoid puncture learning curves.

METHODS: The successful subarachnoid space identification during the first attempt, in the first interspace approached, and the success of 275 spinal blocks performed by 5 residents during the initial six-month training period were used to build learning curves through the cusum analysis. Acceptable failure rates were taken from a sample of 264 spinal blocks performed by experienced anesthesiologists. The number of blocks needed for proficiency was calculated for each resident, for each attribute.

RESULTS: A wide variability in the number of blocks needed for attaining proficiency was observed, depending on the resident or on the attribute. However, most residents attained proficiency after 50 blocks. All residents achieved a 90% success rate after 30 blocks.

CONCLUSIONS: Cusum analysis may be used for objectively measuring performance during the learning phase of spinal anesthesia. A minimum of 50 blocks is required in order to attain proficiency in identifying subarachnoid space.

Key Words: ANESTHESIOLOGY: teaching; ANESTHETIC TECHNIQUES, Regional: spinal block

RESUMO

JUSTIFICATIVA E OBJETIVOS: Medidas objetivas de desempenho são necessárias para o ensino de técnicas anestésicas. Este estudo teve por objetivos aplicar e descrever o método da soma cumulativa dos desvios para a construção de curvas de aprendizado da punção subaracnóidea.

MÉTODO: O sucesso da punção subaracnóidea na primeira tentativa, no primeiro espaço abordado, e o sucesso da anestesia de 275 bloqueios subaracnóideos realizados por médicos em especialização (ME) durante os primeiros seis meses de treinamento foram utilizados para a construção de curvas de aprendizado, utilizando o método das somas cumulativas de desvios. As taxas aceitáveis de falha foram derivadas de uma amostra de 264 bloqueios subaracnóideos realizados por anestesiologistas. O número necessário de bloqueios para se obter proficiência foi calculado, para cada ME, para cada atributo.

RESULTADOS: Houve uma grande variabilidade em relação ao número de bloqueios necessários para se atingir a proficiência, dependendo do atributo e do ME. Contudo, a maioria dos ME atingiu proficiência para os atributos de sucesso durante a primeira tentativa ou no primeiro espaço inter-espinhoso abordado, após 50 bloqueios. Uma taxa de sucesso da anestesia de 90% foi obtida por todos os ME após 30 bloqueios.

CONCLUSÕES: O método da soma cumulativa de desvios pode ser utilizado para medir objetivamente o desempenho de médicos em especialização, durante a fase de aprendizado de anestesia subaracnóidea. Um número mínimo de 50 bloqueios é necessário para se obter proficiência em identificar o espaço subaracnóideo.

Unitermos: ANESTESIOLOGIA: ensino; TÉCNICAS ANESTÉSICAS, Regional: subaracnóidea

RESUMEN

JUSTIFICATIVA Y OBJETIVOS: Medidas objetivas de desempeño son necesarias para la enseñanza de técnicas anestésicas. Este estudio tuvo por objetivo aplicar y describir el método de la suma cumulativa de los desvíos para la construcción de curvas de aprendizaje de la punción subaracnóidea.

MÉTODO: El suceso de la punción subaracnóidea en la primera tentativa, en el primer espacio abordado, y el suceso de la anestesia de 275 bloqueos subaracnóideos realizados por médicos en especialización (ME) durante los primeros seis meses de entrenamiento fueron utilizados para la construcción de curvas de aprendizaje, utilizando el método de las sumas cumulativas de desvíos. Las tasas aceptables de falla fueron derivadas de una muestra de 264 bloqueos subaracnóideos realizados por anestesistas. El número necesario de bloqueos para obtenerse proficiencia fue calculado, para cada ME, para cada atributo.

RESULTADOS: Hubo una grande variabilidad en relación al número de bloqueos necesarios para alcanzar la proficiencia, dependiendo del atributo y del ME. No obstante, la mayoría de los ME alcanzó proficiencia para los atributos de suceso durante la primera tentativa o en el primero espacio inter-espinoso abordado, después 50 bloqueos. Una tasa de suceso de la anestesia de 90% fue obtenida por todos los ME después de 30 bloqueos.

CONCLUSIONES: El método de la suma cumulativa de desvíos puede ser utilizado para medir objetivamente el desempeño de los médicos en especialización, durante la etapa de aprendizaje de anestesia subaracnóidea. Un número mínimo de 50 bloqueos es necesario para obtenerse proficiencia en identificar el espacio subaracnóideo.

INTRODUCTION

Spinal block learning starts by mastering lumbar puncture techniques. Teaching starts with an intensive supervision which progressively decreases as the resident improves performance, until finally it is no longer needed. The transition point between intensive supervision and no supervision must be objectively defined through quantitative performance evaluation tools, that is, through the level of proficiency.

The ideal evaluation tool should be easily computed and provide data to follow the evolution of each resident, identifying performance levels apart of acceptable standards and measuring the results of corrective interventions.

The cusum (cumulative sum) analysis (CSA) is a statistical method used to measure the proficiency of residents during the learning process of procedures such as tracheal intubation, spinal and epidural anesthesia and central venous and arterial catheterization ¹.

The method is easily computed and requires the calculation of three parameters: sigma deviation (s), upper control limit (UCL), calculated as from an unacceptable failure rate (p₀) and lower control limit (LCL), calculated as from an acceptable failure rate (p₁). To calculate CSA, one starts from a zero point, which represents the first procedure. At every success obtained in a subsequent procedure, the values is subtracted from the previous CSA and at every failure, the value of 1 - s is added. So a curve representing the learning process is obtained, which goes up when there are failures and goes down when successes are achieved. When the CSA line goes beyond UCL, performance is considered inadequate. While the CSA line is between LCL and UCL no statistical decision may be made and additional information is needed. When the CSA line goes beyond UCL coming from above, performance is considered acceptable for not being significantly different from p₁ ².

So, this study aimed at building learning curves for spinal puncture techniques, using the cusum analysis to determine the minimum number of procedures performed by residents to attain proficiency.

METHODS

The Medical Ethics Committee, Hospital Governador Celso Ramos, approved this study. Prospective data on the first 275 spinal blocks performed by 5 residents during the first six training months were collected (Group R). Residents were called A, B, C, D and E. During the same period, a control group was formed (Group A) with a sample of 264 spinal blocks performed by 22 experienced anesthesiologists.

In Group R, blockades were performed under the supervision of instructors duly qualified by the Brazilian Society of Anesthesiology. As to lumbar puncture, residents were oriented about patients positioning, antisepsis technique, placement of sterile drapes, interspace choice, local anesthesia, needle positioning and spinal space identification by CSF observation.

All blockades were performed with disposable Quincke needles. Age, gender, palpation quality of spinous processes (easy, difficult, impossible) spinal space access route (median or paramedian), position during lumbar puncture (lateral or sitting), positioning conditions (satisfactory, unsatisfactory) and lumbar puncture needle size were recorded for all patients.

The following parameters were recorded and coded as 1 if present and zero if absent:

1. Attribute 1: success on the first attempt - spinal space location with just one skin puncture.

2. Attribute 2: success on the first interspace - spinal space location in the first interspace approached although with more than one skin puncture.

3. Attribute 3 - success of anesthesia - induction of complete surgical anesthesia

To define control limits for attributes 1, 2 and 3, the following rates were derived from a sample of 264 spinal blocks performed by 22 anesthesiologists working in two hospitals during the same period (Group A).

1. Success rate for the first attempt - number of blockades where the spinal interspace was located with just one skin puncture divided by the total number of punctures multiplied by 100.

2. Success rate for the first space - number of blockades where spinal space was located by one or more skin punctures, but in the first interspace, divided by the total number of blockades and multiplied by 100.

3. Success rate for anesthesia - number of blockades where the surgical procedure was performed with no need for blockade supplementation with opioids or general anesthesia, divided by the total number of blockades and multiplied by 100.

For attribute 1, p₁ was established as 0.35 because failure rate in Group A was 35%, and p₀ was arbitrarily defined as 0.5. Similarly, p₀ was established as 0.1 and p₁ as 0.2 for attribute 2. For attribute 3, Group A's failure rate was 5%, resulting in p₀ = 0.05 and p₁ was arbitrarily defined as 0.2. For attribute 3, the same success rate of 90% was adopted and defined by a consensus of specialists¹, resulting in p₀ = 0.1 and p₁ = 0.2. Values of a (type I error) and b (type II error) were established as 0.1.

Variables for CSA were computed using the following formulas:

a = ln[(1-b)/a]

b = ln[(1- a)/b]

P = ln (p₁/p₀)

Q = ln [(1 - p₁)/(1- p₀)]

LCL = -b/(P + Q)

UCL = a/(P + Q)

s = Q/(Q + P)

Learning curves for each attribute were calculated for each resident based on cusum analysis of deviations. Proficiency in a certain attribute (acceptance of null hypothesis) was identified by CSA values permanently below LCL. Number of blockades executed until proficiency was achieved, was compared among residents by the Chi-square and Fisher exact tests.

Patients demographic data and procedure technical characteristics were compared among groups by Student t (continuous variables) or chi-square (categorical variables) tests.

RESULTS

Groups A and R did not differ as to age. Group R had more male patients. In Group A, the paramedian approach and the sitting position were more frequently used than in Group R. 25G needles were more frequently used by Group R. There were no differences between groups as to 22 and 27G needles (Table I). Group R did not use 26 and 29G needles. Anatomic references quality (spinal processes) was equally distributed among groups (Table II).

Thumbnail

Brazilian Journal of Anesthesiology, 2001; 51: 3: 208 - 217

Cumulative sum analysis applied to the teaching of spinal anesthesia