The importance of clinical experience for the measurement of scoliosis curve in children by Cobb technique

ABSTRACT

Scoliosis is defined as a deformity with lateral deviation of the spine in the coronal plane, torsion of the spine and trunk, and disturbances in the sagittal profile. This postural alteration is evaluated by anteroposterior incidence radiography using the Cobb method. The objective of this study was to verify the influence of evaluator experience on inter- and intra-rater reliability of the Cobb angle of scoliosis curvatures in children. In total, 39 patients aged 7 to 18 years with idiopathic scoliosis were included in this study. The exams were evaluated by two physical therapists, a chiropractor and a physical therapy student. Each evaluator rated each exam twice and the second evaluation occurred after seven days, characterizing the intra-rater reliability. Furthermore, the first evaluations provided the inter-rater reliability. Statistical analysis was performed with intraclass correlation coefficient (ICC), Bland-Altman analysis, descriptive analysis of mean absolute deviation, standard error of measurement, and minimum detectable chance. Correlations ranged from good (ICC>0.5) for intra-rater reliability among professionals to weak (ICC=0.4) for the inexperienced evaluator. The inter-rater reliability of the professional’s evaluations was good (ICC=0.6) and the same analysis with the presence of an inexperienced evaluator was weak (ICC=0.3). Evaluations among professionals showed less variability of measurements and standard deviation values compared to the inexperienced evaluator. The measurement of the angles of the scoliosis through the Cobb method carried out by experienced professionals showed better agreement as well as intra- and inter-rater reliability, lower standard deviation, and variability among the measurements.

Keywords |
Spine; X-Rays; Reproducibility of Results

RESUMO

A escoliose é definida como uma deformidade com desvio lateral da coluna no plano coronal, torsão da coluna e do tronco e distúrbio no perfil sagital. Essa alteração postural é avaliada por meio de radiografia de incidência anteroposterior, utilizando-se o método de Cobb. O objetivo do estudo é verificar a influência da experiência do avaliador sobre a confiabilidade intraexaminador e interexaminador do ângulo Cobb em curvaturas escolióticas de crianças. Foram incluídas na pesquisa 39 crianças portadoras de escoliose idiopática, com idade entre 7 e 18 anos. Os exames foram avaliados por dois fisioterapeutas, um quiropraxista e um estudante de fisioterapia - cada um avaliando duas vezes cada exame. A segunda avaliação ocorreu após sete dias, para confiabilidade intraexaminador. Ademais, as primeiras avaliações forneceram dados para confiabilidade interexaminador. A análise estatística foi realizada com coeficiente de correlação intraclasse (CCI), análise de Bland e Altman e análise descritiva do desvio absoluto médio, erro-padrão de medição e mínima mudança detectável. Observou-se boa confiabilidade (CCI>0,5) para as análises intraexaminadores entre os profissionais, e confiabilidade fraca (CCI=0,4) para o avaliador inexperiente. A confiabilidade interexaminador dos profissionais foi boa (CCI=0,6), e com a presença do avaliador inexperiente foi fraca (CCI=0,3). As avaliações entre os profissionais apresentaram menor variabilidade das medidas e valores de desvio-padrão quando comparadas com as do avaliador inexperiente. A mensuração dos ângulos da escoliose por meio do método de Cobb realizada por profissionais experientes apresentou melhores índices de concordância e de confiabilidade intra e interexaminadores e menor desvio-padrão e variabilidade entre as medidas.

Descritores |
Coluna Vertebral; Raios X; Reprodutibilidade dos Testes

RESUMEN

La escoliosis se define como una alteración con curvatura lateral de la columna vertebral en el plano coronal, torsión de la columna vertebral y del tronco y trastorno en el perfil sagital. Esta alteración postural se evalúa mediante radiografía anteroposterior, utilizando el método de Cobb. El presente estudio tiene como objetivo verificar la influencia de la experiencia del evaluador para la fiabilidad intraexaminador e interexaminador del ángulo de Cobb en las curvaturas escolióticas de los niños. El estudio incluyó a 39 niños con escoliosis idiopática entre 7 y 18 años de edad. Los exámenes fueron evaluados por dos fisioterapeutas, un quiropráctico y un estudiante de fisioterapia, siendo que cada uno evaluó cada examen dos veces. Tras siete días, ocurrió una segunda evaluación para la fiabilidad intraexaminador. Además, las primeras evaluaciones proporcionaron datos para la fiabilidad interexaminador. El análisis estadístico se realizó con el coeficiente de correlación intraclase (ICC), con el análisis de Bland y Altman y con el análisis descriptivo de la desviación media absoluta, del error estándar de medición y del cambio mínimo detectable. Se observó una alta fiabilidad (ICC>0,5) en los análisis intraexaminadores entre los profesionales, y una baja fiabilidad (ICC=0,4) en los de evaluadores inexpertos. La fiabilidad interexaminador de los profesionales fue buena (ICC=0,6), y la presencia del evaluador inexperto fue baja (ICC=0,3). Las evaluaciones entre los profesionales mostraron una menor variabilidad de las medidas y valores de desviación estándar en comparación con los del evaluador inexperto. La medición de los ángulos de escoliosis utilizando el método de Cobb que había sido realizada por profesionales con experiencia mostró mejores índices de concordancia y fiabilidad intra e interexaminadores y una menor desviación estándar y variabilidad entre las mediciones.

Palablas clave |
Columna Vertebral; Rayos X; Reproducibilidad de los Resultados

INTRODUCTION

Scoliosis is defined as a lateral deviation with deformity of the spine in the coronal plane, as well as backbone and trunk torsion and sagittal profile disorders¹. Regarding its typology, idiopathic, congenital, neuromuscular and degenerative scoliosis are cited, each type predominant in a certain age group and with different peculiarities. Adolescent idiopathic scoliosis (AIS) is the most common variant found in healthy children, exhibiting a curvature greater than 10° according to the Cobb angle technique, with a prevalence of 2% to 3%. The prevalence of curvatures larger than 20° is between 0.3% and 0.5%, while curvatures larger than 40° Cobb are found in less than 0.1% of the population². Late diagnosis and treatment can result in severe deformities, affecting physical appearance, cardiopulmonary function and psychological well-being.

Postural change evaluation is performed by anteroposterior incidence X-ray (AP), using the Cobb method-recommended by the Scoliosis Research Society, considered gold standard for scoliosis measurement³ and used for decision making in curvature progression and treatment success⁴. This technique measures the amplitude curve by means of measuring and estimating the angle between lines respectively drawn, tangent to the vertebral superior and inferior end plate to be measured^{5), (6}. Some factors such as terminal vertebra selection, both caudal and cranially belonging to the curvature; wide radiographic markers; different evaluator measurement; inaccurate conveyers; construction and technical design error; squareness or angle design error, interfere with this assessment⁴.

Five progression degrees are considered clinically relevant in many studies. However, intra- and inter-rater variability in determining the Cobb angle in x-rays differs between 4° and 8°, respectively^{7)- (9}. Despite some theoretical advantages, little inter-rater reliability has been demonstrated in the last vertebrae selection to scoliosis evaluation¹⁰. In addition to these difficulties when trying to measure the Cobb angle in children, the evaluation becomes more complex due to bone immaturity, incomplete ossification and anomalous development of the terminal vertebrae, making it difficult to trace a straight line on the vertebral plateau to form the angle¹¹. Such challenges and the evaluator’s inexperience in radiological evaluation may contribute to the angular variability found in the studies, however, the actual interference of the evaluator’s experience in such evaluations is unknown. Thus, this study sought to investigate the evaluator’s experience influence on Cobb angle measurement of intra- and inter-reliability at the spine of children with scoliosis curvatures.

METHODOLOGY

Panoramic radiographs of the spine of 39 children between 7 and 18 years old with idiopathic scoliosis were evaluated. The radiographs were carried out in a hospital in Porto Alegre during the study. S-shaped curvatures were found in eight children. In these cases, only the main curvature was analyzed. In the remaining children, we analyzed single curvature. The patients were only subjected to the radiation already required for their outpatient treatment.

The radiographs were examined by four evaluators (A, B, C and D). They re-evaluated each exam after seven days, featuring intra-rater reliability. Moreover, first evaluation of reliability provided inter-rater data analysis. The evaluations were performed using the application software Matlab 7.9 in accordance with the method described by Cobb¹² and were also conducted independently. Each evaluator needed to define the vertebrae to be marked without knowledge of the other evaluator’s results. Patients subjected to some kind of spinal surgery, those with some congenital anomaly and those whose radiographs had characteristics incompatible with idiopathic scoliosis were excluded.

Out of four evaluators, three professionals, two physical therapists and a chiropractor (evaluators A, B and C), with a minimum of 5 years of experience in Orthopedics were asked to analyze spine X-rays. The Physical Therapy student (D evaluator) was in the fifth semester of an undergraduate degree. All underwent a three-hour training consisting of radiograph analysis using the application software Matlab 7.9 before carrying out the evaluations. Patients were only included after the signature of an Informed Consent Form by their parents or guardians.

For analysis purposes, data normality was verified using the Shapiro-Wilk test. The data were analyzed using the intraclass correlation coefficient (ICC) and Bland-Altman analysis. We also conducted a descriptive analysis, with mean, standard deviation (SD), mean absolute deviation (MAD) estimate, minimal detectable change (MDC) and standard error of measurement (SEm). According to Fleiss¹³, an ICC<0.4 value is considered low, between 0.4 and 0.75 good, and excellent when ICC>0.75. We adopted a level of significance of 0.05 in all tests, and all statistical analyses were performed using SPSS version 17.0.

RESULTS

The sample was composed of 39 radiographies, with average patient age 12.94±3.03 years, 48.52±13.93kg body mass, height, and BMI of 1.50±0.15m 21.08±3.84kg/m². Table 1 shows the mean and standard deviation (SD) of each evaluator on the 1^st and 2^nd days of evaluation. Evaluator D showed higher SD values than the others.

Table 1
Mean and standard deviation (SD) of Cobb angle measurement for scoliosis in the four evaluators’ measurements (A, B, C and D)

Scoliosis (n=39)	1º Evaluation	2º Evaluation
	Mean (SD)	Mean (SD)
Evaluator A	10.4±3.4°	10.8±3.7°
Evaluator B	11.9±4.4°	12.0±4.6°
Evaluator C	10.5±4.2°	9.8±4.3°
Evaluator D (inexperienced)	13.8±7.6°	13.2±7.3°

Intra-rater reliability

Table 2 shows high reliability rates, with ICC higher than 0.553 among professionals (evaluators A, B and C), while evaluator D’s ICC indicated low reliability. The SEm and MDC values were around 2.3° and 4.6°, respectively, in the professionals’ measurements, whereas evaluator D showed significantly higher values. Each evaluator’s estimate of MAD in intra-rater reliability measures showed similar values, however, the MAD standard deviation of evaluator D was significantly higher.

Table 2
Intra-rater reliability results (evaluators A, B, C and D) in Cobb angle of scoliosis measurement

Scoliosis	ICC	P	SEm	MDC	MAD
(n=39)	(95% CI)
Evaluator A	0.553	<0.001*	2.4°	4.7°	−0.3±3.4°
	(0.292-0.738)
Evaluator B	0.711	<0.001*	2.4°	4.7°	−0.04±3.4°
	(0.512-0.837)
Evaluator C	0.735	<0.001*	2.1°	4.2°	0.6±3.0°
	(0.550-0.852)
Evaluator D (inexperienced)	0.451	0.002*	5.3°	10.5°	0.5±7.8°
	(0.162-0.669)

• * Significant reliability; ICC: intraclass correlation coefficient; SEm: standard error of measurement; MDC: minimum detectable change; MAD: mean absolute deviation.

Inter-rater reliability

The inter-rater reliability shown in Table 3 was evaluated with the first evaluation values. High reliability was observed among the professionals, whereas evaluator D’s ICC indicated low reliability. Concerning EPM and MMD values, greater measurement error values were observed in analysis with evaluator D’s measurements.

Table 3
Inter-rater reliability results of Cobb angle measurements of scoliosis between evaluators A, B, C and D (D corresponding to the inexperienced evaluator).

Scoliosis	ICC	p	SEm	MDC	MAD
(n=39)	(95% CI)
All evaluators	0.298	<0.001*	4.1°	8.1°	3.8±3.7°
	(0.141-0.483)
Evaluators A, B and C	0.648	<0.001*	2.4°	3.8°	2.9±2.9°
	(0.485-0.781)

• *Significant reliability. ICC: intraclass correlation coefficient; SEm: standard error of measurement; MDC: minimum detectable change; MAD: mean absolute deviation.

Inter-rater agreement

Table 4 shows the evaluator’s agreement (A×B; A×C, A×D, B×C, B×D and C×D) estimated by Bland-Altman analysis and with the first evaluation values being used. Differences farthest from zero and larger standard deviations can be observed in the student’s analysis, and consequently higher upper and lower limits compared with the professionals.

Table 4
Bland-Altman analysis results of agreement identification between all evaluators (D corresponding to the inexperienced evaluator)

Evaluators	Average difference (°)	Upper limit (°)	Lower Limit (°)	Standard deviation (°)	Outliers
A×B	−0.69	6.71	−8.08	3.77	1
A×C	−0.01	6.64	−6.67	3.40	0
A×D	−1.37	14.88	−17.62	8.29	1
B×C	0.67	7.59	−6.24	3.53	1
B×D	−0.68	16.74	−18.11	8.89	0
D×C	−1.36	14.73	−17.45	8.21	0

DISCUSSION

This study showed lower ICC and higher SD in their measures regarding intra-rater reliability, featuring higher variability in the intra-rater reliability evaluation. It may express evaluator D’s inexperience and unfamiliarity with the technique compared to the professionals, which indicates experience is an important factor for the proper use of the Cobb method in the evaluation of scoliosis.

Low reliability was found in all evaluators’ measurements concerning the inter-rater reliability evaluation. However, we found high and significant reliability when only considering the professional’s evaluations, which supports the initial hypothesis that the evaluator’s experience can be a factor that explains the differing intra- and inter-rater reliability levels found in studies using the Cobb method.

Regarding inter-rater agreement, we emphasize that the evaluations have more values approximated to zero than to the mean difference, and lower variability between measurements among professionals in our study, considering SD and upper and lower limits. Results contrary to ours were found by Ritter et al. ⁽¹⁴, in which the low agreement for both experienced and inexperienced evaluators was assigned to the difficulties of evaluating the method and not due to the person’s experience. Nevertheless, few studies examine the correlation between evaluators with regard to experience.

Studies on reliability are still much divergent in the literature. Lenke et al. ⁽¹⁵, for example, found high intra- and inter-rater reliability demonstrated by Kappa values of 0.92 and 0.83, respectively; Ogon et al. ⁽¹⁶ found values of 0.73 and 0.62 for the same variables. Experience and, therefore, the judgment of the person who measures may be one possible explanation for this variability between studies. This is because the selection of the terminal vertebrae belonging to the curvature is an important parameter to measure idiopathic scoliosis curvature⁵, as the error values are lower when the vertebrae are predetermined⁴. Agreeing with this, it has been shown that the Cobb method may be more prone to error due to the selection of different end plate vertebrae⁶ and the estimation of their different slopes^{6), (17}. That is why inexperienced evaluators could present greater difficulty in that choice and, consequently, higher variability in their measures.

Analyzing the values of SEm and MDC is a goal of estimating ICC values, since these variables have error values inherent to the measurement. In this study, experienced evaluators had values in agreement with those reported in literature¹⁸. However, when the inexperienced evaluator’s values were added, a significantly higher error value in all reliability aspects was observed, which reinforces the importance of professional experience to measurement, as it enables a more accurate analysis with less error values between measurements.

Table 2 shows variability of −0.3° to 0.6° of intra-rater reliability among the professionals, which is lower than 4.6° found in Ylikoski and Tallroth¹⁹. Intra-rater variability increased modestly when we analyzed evaluator D’s measures. However, its standard deviation was substantially higher than those found among the other evaluators, showing that evaluation is less reliable when performed by someone inexperienced. Despite the absence of evaluator inexperience reports, the literature also presents greater intra-rater variability values such as 9.6°¹¹ and 10.4°⁷.

Considering that scoliosis treatments are determined based on the angular extent of spine deviation⁷, knowing precisely the inherent variability in this test is important, since, in clinical practice, a variation of more than 5° indicates curvature progression⁶. Given this context, the training in the method is advisable for the evaluators of radiographs regarding the evaluation of scoliosis. Traditionally, this is already common in clinical settings. However, regarding the monitoring of idiopathic scoliosis diagnosed patients, tests are often analyzed by different professionals, which is not ideal since the literature indicates higher inter-rater than intra-rater reliability values³. Nevertheless, the evaluation of scoliosis by the Cobb method is often carried out in the study environment by researchers themselves, not always trained in the method, which may explain the differences found.

CONCLUSION

The evaluator’s experience influences on the Cobb angle measurements of scoliosis curvatures in children due to the better agreement rates of experienced professionals’ measures, and the intra- and inter-reliability and lower variability compared to the inexperienced evaluator’s measures.

REFERÊNCIAS

• ¹
  Weiss HR, Negrini S, Rigo M, Kotwicki T, Hawes MC, Grivas TB, et al. Indications for conservative management of scoliosis (guidelines). Scoliosis Spinal Disord. 2006;1(1):1. doi: 10.1186/1748-7161-1-5
  » https://doi.org/10.1186/1748-7161-1-5
• ²
  Weinstein SL. Natural history. Spine. 1999;24(24):2592-600.
• ³
  Rosenfeldt MP, Harding IJ, Hauptfleisch JT, Fairbank JT. A comparison of traditional protractor versus Oxford Cobbometer radiographic measurement: intraobserver measurement variability for Cobb angles. Spine. 2005;30(4):440-3. doi: 10.1097/01.brs.0000153401.78638.cb
  » https://doi.org/10.1097/01.brs.0000153401.78638.cb
• ⁴
  Gstoettner M, Sekyra K, Walochnik N, Winter P, Wachter, R, Bach CM. Inter- and intraobserver reliability assessment of the Cobb angle: manual versus digital measurement tools. Eur Spine J. 2007;16:1587-92. doi: 10.1007/s00586-007-0401-3
  » https://doi.org/10.1007/s00586-007-0401-3
• ⁵
  Mehta SS, Modi HN, Srinivasalu S, Chen T, Suh SW, Yang JH, Song HR. Interobserver and intraobserver reliability of Cobb angle measurement: endplate versus pedicle as bony landmarks for measurement: a statistical analysis. J Pediatr Orthop. 2009;29(7):749-54. doi: 10.1007/s00247-009-1464-6
  » https://doi.org/10.1007/s00247-009-1464-6
• ⁶
  Zhang J, Lou E, Le LH, Hill DL, Raso JV, Wang Y. Automatic Cobb measurement of scoliosis based on fuzzy Hough Transform with vertebral shape prior. J Digit Imaging. 2009;22(5):463-72. doi: 10.1007/s10278-008-9127-y
  » https://doi.org/10.1007/s10278-008-9127-y
• ⁷
  Carman DL, Browne RH, Birch JG. Measurement of scoliosis and kyphosis radiographs. Intraobserver and interobserver variation. J Bone Joint Surg Am. 1990;72(3):328-33.
• ⁸
  Geijer H, Beckman KW, Jonsson B, Andersson T, Persliden J. Digital radiography of scoliosis with a scanning method: initial evaluation. Radiology. 2001;218(2):402-10. doi: 10.1148/radiology.218.2.r01ja32402
  » https://doi.org/10.1148/radiology.218.2.r01ja32402
• ⁹
  Gross C, Gross M, Kuschner S. Error anlaysis of scoliosis curvature measurement. Bull Hosp Jt Dis Orthop Inst. 1983;43(20):171-7.
• ¹⁰
  Potter BK, Rosner MK, Lehman Jr RA, Polly Jr DW, Schroeder TM, Kuklo TR. Reliability of end, neutral and stable vertebrae identification in adolescent idiopathic scoliosis. Spine. 2005;30:1658-63. doi: 10.1097/01.brs.0000170290.05381.9a
  » https://doi.org/10.1097/01.brs.0000170290.05381.9a
• ¹¹
  Loder RT, Urquhart A, Steen H, Graziano G, Hensinger RN, Schlesinger A, et al. Variability in Cobb angle measurements in children with congenital scoliosis. J Bone Joint Surg Br. 1995;77(5):768-70.
• ¹²
  Cobb J. Outline for the study of scoliosis. Instr Course Lect. 1948;5:261-75.
• ¹³
  Fleiss JL. The design and analysis of clinical experiments. New York: John Wiley & Sons; 1986.
• ¹⁴
  Ritter R, Nagasse Y, Ribeiro I, Yamazato C, Oliveira FMD, Kusabara R. Comparison of Cobb angle measurement in scoliosis by residents and spine experts. Coluna/Columna. 2016;15(1):13-6. doi: 10.1590/S1808-185120161501147274
  » https://doi.org/10.1590/S1808-185120161501147274
• ¹⁵
  Lenke LG, Betz RR, Harms J, Bridwell KH, Clements DH, Lowe TG, et al. Adolescent idiopathic scoliosis: a new classification to determine extent of spinal arthrodesis. J Bone Joint Surg Am. 2001;83(8):1169-81. doi: 10.2106/00004623-200108000-00006
  » https://doi.org/10.2106/00004623-200108000-00006
• ¹⁶
  Ogon M, Giesinger K, Behensky H, Wimmer C, Nogler M, Bach CM, et al. Interobserver and intraobserver reliability of Lenke's new scoliosis classification system. Spine. 2002;27(8):858-62.
• ¹⁷
  Allen S, Parent E, Khorasani M, Hill DL, Lou E, Raso JV. Validity and reliability of active shape models for the estimation of cobb angle in patients with adolescent idiopathic scoliosis. J Digit Imaging. 2008;21(2):208-18. doi: 10.1007/s10278-007-9026-7
  » https://doi.org/10.1007/s10278-007-9026-7
• ¹⁸
  Troyanovich SJ, Harrison DE, Harrison DD, Holland B, Janik TJ. Further analysis of the reliability of the posterior tangent lateral lumbar radiographic mensuration procedure: Concurrent validity of computer- aided X-ray digitization. J Manipulative Physiol Ther. 1998;21(7):460-7.
• ¹⁹
  Ylikoski M, Tallroth K. Measurement variations in scoliotic angle, vertebral rotation, vertebral body height, and intervertebral disc space height. J Spinal Disord. 1990;3(4):387-91.

• Study conducted at Escola de Educação Física, Fisioterapia e Dança (Esefid). Universidade Federal do Rio Grande do Sul (UFRGS) - Porto Alegre (RS), Brazil.
• Finance Source: none
• approved by the Research Ethics Committee of Hospital Mãe de Deus de Porto Alegre, under protocol # 534/21.

Publication Dates

• Publication in this collection
  16 Sept 2019
• Date of issue
  Jul-Sep 2019

History

• Received
  05 Feb 2018
• Accepted
  06 June 2019