Interobserver agreement on non-contrast computed tomography interpretation for diagnosis of urolithiasis in patients with acute flank pain

Abstracts

OBJECTIVE: To evaluate the interobserver agreement on non-contrast computed tomography interpretation by a group of experienced abdominal radiologists, for the study of urolithiasis in patients presenting acute flank pain. MATERIALS AND METHODS: Prospective study of 52 patients submitted to non-contrast enhanced helical computed tomography. The images were subsequently analyzed by three independent observers, with the interobserver agreement assessed by means of the kappa (kappa) statistical method. The following parameters were analyzed: a) presence, localization and measurement of ureteral calculi; b) intrarenal calyceal system dilatation; c) perirenal fat heterogeneity; d) ureteral dilatation; e) ureteral wall edema (halo sign). RESULTS: Ureteral calculi were found in 40 of 52 patients (77%). The interobserver agreement was almost perfect as regards identification of ureteral calculi (kappa = 0.89) and ureteral dilatation (kappa = 0.87), substantial for calyceal system dilatation (kappa = 0.77), and moderate for perirenal fat heterogeneity (kappa = 0.55) and ureteral wall edema (kappa = 0.56). CONCLUSION: Non-contrast-enhanced abdominal computed tomography presents high reproducibility in the evaluation of urolithiasis and secondary signs of the calyceal system obstruction.

Lithiasis; Ureteral calculi; Flank pain; Computed tomography

OBJETIVO: Avaliar a reprodutibilidade da tomografia computadorizada sem contraste na avaliação da litíase ureteral e os sinais secundários de obstrução do sistema coletor em pacientes com cólica renal aguda. MATERIAIS E MÉTODOS: Estudo prospectivo de 52 pacientes com diagnóstico clínico de cólica renal aguda submetidos a exame de tomografia computadorizada de abdome sem contraste. Os exames foram realizados com técnica helicoidal e posteriormente analisados por três observadores independentes, com a concordância interobservador avaliada pelo método estatístico kappa (kapa). Foram analisados os parâmetros: a) presença, localização e mensuração dos cálculos ureterais; b) dilatação do sistema coletor intra-renal; c) heterogeneidade da gordura perirrenal; d) dilatação ureteral; e) edema da parede ureteral (sinal do halo). RESULTADOS: Foram encontrados 40 cálculos ureterais na tomografia computadorizada (77%). A concordância interobservador para a identificação do cálculo ureteral e da dilatação ureteral foi quase perfeita (kapa = 0,89 e kapa = 0,87, respectivamente), substancial para dilatação do sistema coletor intra-renal (kapa = 0,77) e moderada para heterogeneidade da gordura perirrenal e para edema da parede ureteral (kapa = 0,55 e kapa = 0,56, respectivamente). CONCLUSÃO: A tomografia computadorizada de abdome sem contraste apresenta elevada reprodutibilidade na avaliação da litíase ureteral e dos sinais secundários de obstrução do sistema coletor.

Litíase; Cálculos ureterais; Dor no flanco; Tomografia computadorizada

ORIGINAL ARTICLES

Interobserver agreement on non-contrast computed tomography interpretation for diagnosis of urolithiasis in patients with acute flank pain^*

Luís Ronan Marquez Ferreira de Souza^I; Salomão Faintuch^I; Daniel Bekhor^II; Dario Ariel Tiferes^I; Suzan Menasce Goldman^III; Jacob Szejnfeld^IV

^IMD, Radiologist, Doctor in Radiology by Department of Diagnostic Imaging of Universidade Federal de São Paulo/Escola Paulista de Medicina, Titular Member of Colégio Brasileiro de Radiologia (Brazilian College of Radiology)

^IIMD, Radiologist, Master in Radiology by Department of Diagnostic Imaging of Universidade Federal de São Paulo/Escola Paulista de Medicina

^IIIAdjunct Doctor Professor, Head for Department of Diagnostic Imaging Genitourinary Sector at Universidade Federal de São Paulo/Escola Paulista de Medicina

^IVAdjunct Professor, Private Docent at Department of Diagnostic Imaging of Universidade Federal de São Paulo/Escola Paulista de Medicina

^{Mailing address}

ABSTRACT

OBJECTIVE: To evaluate the interobserver agreement on non-contrast computed tomography interpretation by a group of experienced abdominal radiologists, for the study of urolithiasis in patients presenting acute flank pain.

MATERIALS AND METHODS: Prospective study of 52 patients submitted to non-contrast enhanced helical computed tomography. The images were subsequently analyzed by three independent observers, with the interobserver agreement assessed by means of the kappa (k) statistical method. The following parameters were analyzed: a) presence, localization and measurement of ureteral calculi; b) intrarenal calyceal system dilatation; c) perirenal fat heterogeneity; d) ureteral dilatation; e) ureteral wall edema (halo sign).

RESULTS: Ureteral calculi were found in 40 of 52 patients (77%). The interobserver agreement was almost perfect as regards identification of ureteral calculi (k = 0.89) and ureteral dilatation (k = 0.87), substantial for calyceal system dilatation (k = 0.77), and moderate for perirenal fat heterogeneity (k = 0.55) and ureteral wall edema (k = 0.56).

CONCLUSION: Non-contrast-enhanced abdominal computed tomography presents high reproducibility in the evaluation of urolithiasis and secondary signs of the calyceal system obstruction.

Keywords: Lithiasis; Ureteral calculi; Flank pain; Computed tomography.

INTRODUCTION

The technological development in the field of computed tomography (CT) has resulted in an increase in its diagnostic accuracy and, at the same time, in a decrease in time necessary for examination. The acceptance of CT as a diagnosis method for evaluating urinary tract lithiasis has increased, considering that this method suppresses the disadvantages presented by other imaging methods, such as excretory urography and ultrasound^(1–5): the use of ionic contrast agent, the difficulty for evaluating the whole ureter as well as the interposition of abdominal loops. On the other hand, the advantages of CT for lithiasis evaluation are: short images acquisition times, there is no need for the use of contrast media, and presents high sensitivity in cases of urinary lithiasis⁽⁶⁾. As far it is known, the unique exceptions are calculi resulting from the use of protease inhibitors like Indinavirâ⁽¹⁾.

Since the publication in 1995 of a study developed by Smith in 1994⁽⁷⁾, CT has been considered as the method of choice for diagnosis of urolithiasis. Provided the appro priate technique is applied, CT presents high sensitivity (96%-100%), specificity (95%-100%) and accuracy (96%098%)^(1,8).

There are few studies demonstrating the CT reproducibility in our environment⁽⁸⁾, analyzing the level of agreement amongst experienced radiologists in the interpretation of CT aiming at evaluating the tract urinary lithiasis.

In this context, the objective of this study has been to evaluate the reproducibility of the noncontrast-enhanced CT in the diagnosis of urolithiasis and secondary signs of calyceal system obstruction in patients presenting acute renal colic.

MATERIALS AND METHODS

A retrospective study was performed with 52 patients who were referred to the Emergency Department of Hospital São Paulo – Universidade Federal de São Paulo/Escola Paulista de Medicina (Unifesp/EPM), in the period between February and July, 2002, with diagnosis of acute renal colic.

The patients' ages ranged between 17 and 75 years (mean age 37 years). Thirty-nine (75%) patients were male and 13 (25%) were female.

All the patients were submitted to non-contrast-enhanced CT, and those who presented clinical symptoms, imaging findings or laboratory evidences of other chronic urinary tract diseases, like pyelonephritis, renal tuberculosis and nephrocalcinosis, were excluded from this study. Also, patients undergoing treatment with protease inhibitors (Indinavir^®) were not included, because of the relation between the utilization of this drug with the production of calculi which could not be identified by CT.

CT equipment utilized were Philips (Philips Medical Systems; Eindhoven, Holland) Secura Release 1.3 and Tomoscan AV-EV1 models, with helical acquisition technique (collimation and reconstruction interval: 5 mm; pitch 1 to 1.5), from the kidney superior pole to the pubic symphysis, with moderate vesical repletion and breath-holding. Non-contrast-enhanced images acquisition was performed with 120 kV and 200 m.

Later, a blind random review of the images was accomplished by three independent radiologists with more-than-five-year experience in Abdominal Radiology. The following parameters were taken into consideration: a) presence, localization and measurement of ureteral calculi; b) intrarenal calyceal system dilatation; c) perirenal fat heterogeneity; d) ureteral dilatation; e) ureteral wall edema (halo sign).

The direct visualization of a hyperdense image with calcareous density (> 311 UH) inside the calyceal system was considered as the primary tomographic sign of calculi presence.

The calculi localization along ureter was classified into: ureteropelvic junction, upper/proximal third (above sacroiliac joints), lower/distal third (below sacroiliac joints) and in the three renal thirds ureterovesical joint⁽⁹⁾.

The calculi measurement was performed in a workstation utilizing osseous window in an axis perpendicular to the ureter, i.e. in tomographic axial slices.

The intrarenal calyceal system dilatation was diagnosed affecting the upper, middle and lower thirds of the kidneys, characterized in axial slices. Ureteral dilatation was considered positive when the ureter presented an axial diameter > 4 mm⁽³⁾.

This research project has been analyzed and approved by Unifesp/EPM Committee of Ethics in Research.

The interobserver agreement statistical analysis (a comparison of variation among the three radiologists) in the CT evaluation was based on Kappa index (k)⁽¹⁰⁾, as per Chart 1.

RESULTS

Cases of ureteral calculi were consensually found by the three radiologists in 40 (77%) of the 52 patients submitted to CT. Calculi localization was as follows: 30% in the upper third, 5% in the middle third, 18% in the lower third and 47% in the ureterovesical joint. The ureteral calculi size ranged between 0.20 cm and 1.40 cm (mean 0.50 cm). From the 40 calculi identified, 14 (36%) were < 0.40 cm.

From 12 patients with acute renal colic who did not present any renal calculus (23% of the sample), one presented with an infected renal cyst and four were diagnosed as patients who had been evaluated after the renal calculus passage, because their CT images demonstrated an unilateral calyceal system dilatation on the side reported as symptomatic, without any other imaging findings. From the remaining seven patients, five could not be diagnosed and two were diagnosed with other etiologies not related to the urinary tract (appendicitis and ovarian disorder).

The interobserver agreement on identification of ureteral calculi and dilatation was almost perfect (k = 0.89 e k = 0.87, respectively), substantial for intrarenal calyceal system dilatation (k = 0.77) and moderate for perirenal fat heterogeneity and ureteral wall edema (k = 0.55 and k = 0.56, respectively) (Tables 1 and 2).

DISCUSSION

The facility in identifying ureteral calculi, the non-necessity of a contrast agent, the relatively low cost, short acquisition times and good acceptance by patients have made CT the method preferred by American Radiologists, later adopted in the other countries ^(1–11).

Authors like Rosen et al.⁽¹¹⁾ indicate that the utilization of CT in patients assisted in an emergency environment increases the level of self-confidence of the physician, besides reducing the hospital stay and surgical periods for the majority of patients.

In the tomographic study interpretation, the primary sign of urolithiasis is the identification of ureteral calculi. In our study, the interobserver agreement related to the identification of ureteral calculi was almost perfect (k = 0.89).

The results of the present study have evidenced a higher rate of interobserver agreement than the study by Freed et al.⁽¹²⁾, evaluating the agreement among three experienced observers — one resident (last year) in Radiology and one Urologist specialized in lithiasis. This study has indicated an interobserver agreement of k = 0.67–0.71 (substantial), k = 0,65–0,67 between observers and the resident (substantial), and k = 0.33–0.46 (moderate) between observers and the Urologist. Based on data similar to those presented in our study, these authors say that CT presents a high accuracy in the detection or urolithiasis with an excellent level of agreement between the experienced observers and the resident, and that this method can be successfully utilized in a teaching environment (hospital-schools), since the imaging clinical findings suggest a low level of difficulty for them to be interpreted⁽¹²⁾.

As regards secondary findings of calyceal system obstruction in the analysis of perirenal fat heterogeneity, there was a moderate interobserver agreement (k = 0.55). On the other hand, as regards the analysis of the calyceal system by CT, there was a substantial agreement (k = 0.77).

Regarding the other secondary signs evaluated, there was an almost perfect agreement in the study of ureteral dilatation (k = 0.87). In the analysis of ureteral wall edema there was a moderate agreement (k = 0.56).

These findings reflect a good level of reproducibility in the daily practice of these secondary signs which are considered as the most significant and sensitive signs, obtaining a good level of agreement among the experienced radiologists. A variation was identified between observers and observer 3, in a paired evaluation, which would not even affect the averages applied to evaluate the method reproducibility. This variation, when analyzed alone, has evidenced an equivocal initial appraisal of one of the Radiologists concerning the signs of perirenal fat heterogeneity and ureteral wall edema, reflecting the significance of definite concepts and an appropriate terminology for an ideal utilization of the method⁽²⁾.

In an analysis carried out by Holdgate and Chan⁽¹³⁾, in 127 CT studies with diagnosis of ureterolithiasis, the emergency physicians diagnostic skill was evaluated in comparison with the radiological reports. They have observed that the level of agreement was substantial for evaluation of calculi and renal alterations (k > 0,75), but with low accuracy for evaluation of secondary signs of calyceal system obstruction and for differential diagnosis.

In our department, the mean images acquisition time, in a helical tomography protocol aimed at evaluating urolithiasis, is of seven minutes (examination room time). This data consolidates even more the indication of CT as a first line study in the assessment of patients presenting acute renal colic, when the necessity of an accurate and fast diagnosis changes the clinical conduct⁽¹⁶⁾.

Different imaging methods are available for diagnosis of ureterolithiasis in patients with acute renal colic, however, CT has surpassed all the other methods, due the possibility of evaluation of the whole ureter without the interposition of abdominal loops, besides the non-necessity of ionic contrast-enhancement; resulting in high diagnostic accuracy and speed, associated with a high reproducibility^17,18, according to the data reported in the present study.

REFERENCES

Received September 13, 2005.

Accepted after revision February 21, 2006.

• 1. Tamm EP, Silverman PM, Shuman WP. Evaluation of the patient with flank pain and possible ureteral calculus. Radiology 2003;228:319329.
• 2. Souza LRMF, Faintuch S, De Nicola H, et al A tomografia computadorizada helicoidal no diagnóstico da litíase ureteral. Rev Imagem 2004;26: 315321.
• 3. Smith RC, Levine J, Rosenfeld AT. Helical CT of urinary tract stones. Epidemiology, origin, pathophysiology, diagnosis, and management. Radiol Clin North Am 1999;37:911952.
• 4. Sourtzis S, Thibeau JF, Damry N, Raslan A, Vandendris M, Bellemans M. Radiologic investigation of renal colic: unenhanced helical CT compared with excretory urography. AJR Am J Roentgenol 1999;172:14911494.
• 5. Ege G, Akman H, Kuzucu K, Yildiz S. Acute ureterolithiasis: incidence of secondary signs on unenhanced helical CT and influence on patient management. Clin Radiol 2003;58:990994.
• 6. Lanoue MZ, Mindell HJ. The use of unenhanced helical CT to evaluate suspected renal colic. AJR Am J Roentgenol 1997;169:15791584.
• 7. Smith RC, Rosenfield AT, Choe KA, et al Acute flank pain: comparison of non-contrast-enhanced CT and intravenous urography. Radiology 1995; 194:789794.
• 8. Galvão Filho MM, D'Ippolito G, Hartmann LG, et al O valor da tomografia computadorizada helicoidal sem contraste na avaliação de pacientes com dor no flanco. Radiol Bras 2001;34:129134.
• 9. Erdodru T, Aker O, Kaplancan T, Erodlu E. Predictive role of non-contrast spiral computerized tomography on spontaneous passage of ureteral stones. Int Braz J Urol 2002;28:516521.
• 10. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1997;33:159174.
• 11. Rosen MP, Sands DZ, Longmaid HE 3rd, Reynolds KF, Wagner M, Raptopoulos V. Impact of abdominal CT on the management of patients presenting to the emergency department with acute abdominal pain. AJR Am J Roentgenol 2000;174: 13911396.
• 12. Freed KS, Paulson EK, Frederick MG, et al Interobserver variability in the interpretation of unenhanced helical CT for the diagnosis of ureteral stone disease. J Comput Assist Tomogr 1998;22:732737.
• 13. Holdgate A, Chan T. How accurate are emergency clinicians at interpreting noncontrast computed tomography for suspected renal colic? Acad Emerg Med 2003;10:315319.
• 14. Duarte RJ, Prado NG, Santos Jr MW. Qual a avaliação mínima necessária para pacientes com suspeita de ureterolitíase? Urologia contemporânea 2004; Número 1. Disponível em: URL:http://www.urologiacontemporanea.com.br Acessado em 18/7/2005.
• 15. Caserta NMG. Litíase urinária e nefrocalcinose. In: Prando A, Prando D, Caserta NMG, Bauab Jr T, editores. Urologia: diagnóstico por Imagem. 1Ş ed. São Paulo: Sarvier, 1997;8289.
• 16. Goldman SM, Faintuch S, Ajzen SA, et al Diagnostic value of attenuation measurements of the kidney on unenhanced helical CT of obstructive ureterolithiasis. AJR Am J Roentgenol 2004;182: 12511254.
• 17. Freitas RMC, Silva LC, Santos JLS, Tavares Júnior WC. Avaliação dos métodos de imagem no diagnóstico da urolitíase: revisão da literatura. Radiol Bras 2004;37:291294.
• 18. Smith RC, Varanelli M. Diagnosis and management of acute ureterolithiasis: CT is truth. AJR Am J Roentgenol 2000;175:36.

*

Study developed at Department of Diagnostic Imaging of Universidade Federal de São Paulo/Escola Paulista de Medicina, São Paulo, SP, Brazil.

Publication Dates

• Publication in this collection
  21 Nov 2006
• Date of issue
  Oct 2006

History

• Accepted
  21 Feb 2006
• Received
  13 Sept 2005