Comparison between the NASCET method and subjective visual impression in the evaluation of internal carotid artery stenosis

Comparação entre o método NASCET e impressão visual subjetiva na avaliação de estenose da artéria carótida interna

Abstracts

OBJECTIVE: To evaluate the accuracy of subjective visual impression (SVI) of an experienced neuro-radiologist in the measurement of the degree of internal carotid artery (IC) stenosis evaluated by digital angiography (DGA). METHOD: Ten symptomatic patients with internal carotid stenosis greater than 70% in a previous duplex scan were submitted to DGA. The degree of stenosis in both sides (symptomatic and asymptomatic) were evaluated by the same neuro-radiologist who gave his SVI and applied the NASCET method immediately after. Both methods were compared using the intraclass correlation coeficient (r) and its 95% confidence interval (95% ci). For each method, the sample (20 ICs) was also divided in surgical (stenosis between 70 and 99%) and non surgical ICs, using kappa concordance coeficient (k) to compare the results. RESULTS: The results comparing the 20 values obtained by each method are: r = 0.90 (95% ci: 0.77 -- 0.96). Dividing the sample in surgical and non surgical ICs, k = 0.857, p < 0.0001; sensitivity = 100% (39.6% - 100%); specificity = 93.8% (67.7% - 99.7%); positive predictive value = 80% (29.9% - 98.9%); negative predictive value = 100% (74,7% - 100%). CONCLUSION: The SVI may be used by at least some experienced neuroradiologists as a preliminary tool to evaluate the degree of IC stenosis with DGA, but a standardised and well established method should be routinely performed.

carotid stenosis; angiography diagnostic use; comparative study; cerebrovascular disease

OBJETIVO: Avaliar a acurácia da impressão visual subjetiva (IVS) de um neurorradiologista experiente na mensuração do grau de estenose da artéria carótida interna (CI), avaliado pela angiografia digital (AGD). MÉTODO: Dez pacientes com estenose sintomática da CI maior que 70% ao duplex scan foram submetidos à AGD. O grau de estenose nos lados sintomático e assintomático foi avaliado pelo mesmo neurorradiologista, que aplicou sua IVS e logo após o método NASCET. Os métodos foram comparados utilizando-se o coeficiente de correlação intra-classe (r) e seu intervalo de confiança 95% (ic 95%). Para cada método, a amostra foi dividida em cirúrgica -- CI com estenose de 70% a 99% - e não cirúrgica, e o coeficiente kappa de concordância (k) foi usado para comparar os resultados. RESULTADOS: Comparando-se os 20 valores obtidos por cada método, r = 0,90 (ic 95%: 0,77 -- 0,96). Após dicotomização da amostra, obteve-se k = 0,857, com p < 0,0001; sensibilidade = 100% (39,6% - 100%); especificidade = 93,8% (67,7% - 99,7%); valor preditivo positivo = 80% (29,9% - 98,9%); e valor preditivo negativo = 100% (74,7% - 100%). CONCLUSÃO: A IVS pode ser usada por pelo menos alguns neurorradiologistas experientes como método de avaliação preliminar do grau de estenose da CI por AGD. Um método padronizado e bem estabelecido cientificamente deve ser, entretanto, aplicado rotineiramente.

estenose carotídea; angiografia uso diagnóstico; estudo comparativo; transtornos cerebrovasculares

COMPARISON BETWEEN THE NASCET METHOD AND SUBJECTIVE VISUAL IMPRESSION IN THE EVALUATION OF INTERNAL CAROTID ARTERY STENOSIS

Marco Oliveira Py¹, Charles André¹, Feliciano Silva de Azevedo²

ABSTRACT - OBJECTIVE: To evaluate the accuracy of subjective visual impression (SVI) of an experienced neuro-radiologist in the measurement of the degree of internal carotid artery (IC) stenosis evaluated by digital angiography (DGA). METHOD: Ten symptomatic patients with internal carotid stenosis greater than 70% in a previous duplex scan were submitted to DGA. The degree of stenosis in both sides (symptomatic and asymptomatic) were evaluated by the same neuro-radiologist who gave his SVI and applied the NASCET method immediately after. Both methods were compared using the intraclass correlation coeficient (r) and its 95% confidence interval (95% ci). For each method, the sample (20 ICs) was also divided in surgical (stenosis between 70 and 99%) and non surgical ICs, using kappa concordance coeficient (k) to compare the results. RESULTS: The results comparing the 20 values obtained by each method are: r = 0.90 (95% ci: 0.77 ¾ 0.96). Dividing the sample in surgical and non surgical ICs, k = 0.857, p < 0.0001; sensitivity = 100% (39.6% - 100%); specificity = 93.8% (67.7% - 99.7%); positive predictive value = 80% (29.9% - 98.9%); negative predictive value = 100% (74,7% - 100%). CONCLUSION: The SVI may be used by at least some experienced neuroradiologists as a preliminary tool to evaluate the degree of IC stenosis with DGA, but a standardised and well established method should be routinely performed.

KEY WORDS: carotid stenosis, angiography diagnostic use, comparative study, cerebrovascular disease.

Comparação entre o método NASCET e impressão visual subjetiva na avaliação de estenose da artéria carótida interna

RESUMO - OBJETIVO: Avaliar a acurácia da impressão visual subjetiva (IVS) de um neurorradiologista experiente na mensuração do grau de estenose da artéria carótida interna (CI), avaliado pela angiografia digital (AGD). MÉTODO: Dez pacientes com estenose sintomática da CI maior que 70% ao duplex scan foram submetidos à AGD. O grau de estenose nos lados sintomático e assintomático foi avaliado pelo mesmo neurorradiologista, que aplicou sua IVS e logo após o método NASCET. Os métodos foram comparados utilizando-se o coeficiente de correlação intra-classe (r) e seu intervalo de confiança 95% (ic 95%). Para cada método, a amostra foi dividida em cirúrgica ¾ CI com estenose de 70% a 99% - e não cirúrgica, e o coeficiente kappa de concordância (k) foi usado para comparar os resultados. RESULTADOS: Comparando-se os 20 valores obtidos por cada método, r = 0,90 (ic 95%: 0,77 ¾ 0,96). Após dicotomização da amostra, obteve-se k = 0,857, com p < 0,0001; sensibilidade = 100% (39,6% - 100%); especificidade = 93,8% (67,7% - 99,7%); valor preditivo positivo = 80% (29,9% - 98,9%); e valor preditivo negativo = 100% (74,7% - 100%). CONCLUSÃO: A IVS pode ser usada por pelo menos alguns neurorradiologistas experientes como método de avaliação preliminar do grau de estenose da CI por AGD. Um método padronizado e bem estabelecido cientificamente deve ser, entretanto, aplicado rotineiramente.

PALAVRAS-CHAVE: estenose carotídea, angiografia uso diagnóstico, estudo comparativo, transtornos cerebrovasculares.

Since the beginning of the twentieth century, Chiari (1906) and Hunt (1914) have shown, in autopsies, the role of extracranial internal carotid artery (IC) in the mechanism of stroke, relating it to IC occlusion¹. On October 1951, Carrea et al. performed the first carotid reconstruction, influenced by Miller Fisher's studies about carotid occlusion and stroke^1,2. In 1954, Eastcott et al. made the first endarterectomy, with atherosclerotic plaque resection and reconstruction of the common carotid artery and the IC and their anastomosis². Endarterectomy became a popular procedure during the 1970 and 1980 decades^1-4. Nevertheless, the procedure lacked scientific proof of efficacy and reports of severe complications appeared frequently².

Preliminary results of important multicentric trials on symptomatic patients were published in the 1990 decade, notably the North American Symptomatic Carotid Endarterectomy Trial (NASCET)⁵ and the European Carotid Surgery Trial (ECST)⁶.These trials demonstrated no benefit from endarterectomy in mild IC stenosis ¾ less than 30%. On the other hand, in severe IC stenosis ¾ equal or greater than 70% - benefits of the procedure were clear^5,6. Recently, final results of both trials were published^7,8. ECST⁷ concluded that endarterectomy should only be indicated if the degree of IC stenosis was equal or greater than 80%. NASCET⁸ found that endarterectomy was reliable if this degree was equal or greater than 70% and in selected patients with 50% to 69% stenosis.

Discussion emerged about different cut-points between these two trials. It became clear that differences could be explained by distinct methods to measure IC stenosis, although both trials have used digital angiography (DGA) criteria^3,9. The NASCET method calculated the degree of IC stenosis by dividing the vessel lumen diameter in the narrowest zone of stenosis by the diameter of the IC distal to the stenosis, in an area free of angles⁵. The ECST proceeded by dividing the same artery lumen diameter in the narrowest zone of stenosis by the estimated original lumen diameter of the carotid bulb⁶. Both techniques were criticized^10,11. The NASCET method, for instance, may find negative values for IC stenosis, if the remaining carotid bulb lumen diameter is greater than that of the distal IC; and ECST uses a subjective criteria to evaluate the original carotid bulb diameter¹⁰.Apparent discrepancies in cut-points of NASCET and ECST were compared and it was found that there is a straight correspondence between their results, as showed in Table 1^3,9. For instance, a 60% degree of stenosis in NASCET corresponds to a 80% stenosis in ECST³. Other authors suggested a new method to measure IC stenosis¹⁰. They consider carotid bulb diameter as 1.2 the diameter of common carotid artery in a region three to five centimeters proximal to the bulb. Then, the following formula is applied: (1 ¾ D / N) x 100%, where D is the stenosed lumen diameter and N the estimated normal carotid bulb diameter¹⁰. There is no agreement in literature on the best method to measure IC stenosis. Some authors even suggest that, in clinical practice, most radiologists do not use any published method, preferring only a subjective visual impression (SVI) of the degree of IC stenosis¹².

In the present study, the authors compare a well established and widely used^9,11 angiographic method ¾ NASCET⁵ - to measure the degree of IC stenosis with the SVI of an experimented neuroradiologist (FSA). The main objective is to calculate their correlation and concordance rates.

METHOD

This is a prospective, longitudinal study with consecutive patients and data collection between June 1997 and September 1999. Ten consecutive patients from a university hospital selected to another study¹³ - agreed to participate. In summary, the inclusion criteria were: age up to 80 years; history of transient ischemic attack (TIA) or mild stroke ¾ Rankin scale¹⁴ up to grade two ¾ in the ipsilateral IC territory; a previous DS showing IC stenosis equal or greater than 70% in one or both extracranial IC; and informed consent. The exclusion criteria included kidney, liver or lung failure or cancer leading to expected survival less than five years; extensive previous stroke; severe diseases that could increase the risk of the endarterectomy; any additional contraindication to surgery. Addering to the ECST criteria, the maximum interval between the neurological event and DGA was six months^6,7.

DGA was made in a Stenoscop (General Electric) machine, using the Seldinger technique by femoral artery catheterization and performed by the same examiner (FSA). All four cervical arteries and the intracranial vessels were studied. The neuroradiologist first gave his SVI and then immediatly applied the NASCET method⁵.

For statistical analysis, a graphic measuring the dispersion of the values through a given line and comparing the obtained values in each measurement method was created, as suggested¹⁵. If there is a perfect correlation between comparing values, all points will be on the given line; the more distant from the line, the more discordant are the values. Intraclass correlation coefficient (r) was also calculated to the numeric results, measuring the correlation between SVI and the NASCET method¹⁵. The 95% confidence interval (95% ci) was calculated, determining limits of correlation with a 5% error. Sample size and variability of results are taken into account in 95% ci calculation.

To increase the utility of the study in clinical practice, we divided the sample in surgical ¾ 70% to 99% IC stenosis - and non-surgical ¾ less than 70% stenosis or total occlusion. Kappa concordance coefficient (k) and its p value were calculated to the obtained groups. Kappa coefficient measures observed concordance, excluding concordance by chance^16,17. Sensitivity, specificity, positive predictive value and negative predictive value were also calculated for SVI, with their 95% ci. Values of "r" and "k" were classified as bad (less than 0.4), good (0.4 to 0.75) or excellent (greater than 0.75), following Landis and Koch's criteria¹⁸.

Besides global comparison of the results, an analysis of each case was made, considering differences greater than 10% between the results were considered to indicate discordance, and differences less than 10% as indicating concordance. This level of difference (10%) was chosen because it is outside the range of measurement error¹⁹ and because meaningful increments in stroke risk occur between decile levels²⁰.

RESULTS

Ten patients ¾ 20 ICs ¾ were studied. Their clinical presentation and course have been fully described¹³. Table 2 shows the main numeric values obtained. Figure 1 shows a graphic with plotted values of both measurement methods.

Intraclass correlation coefficient (r) was 0.90 (95% ci = 0.77 ¾ 0.96). Dividing the sample in surgical and non-surgical ICs, k = 0.857 (p < 0.0001); sensitivity of SVI was 100% (39.9% - 100%); specificity = 93.8% (67.7% - 99.7%); positive predictive value = 80% (29.9% - 98.9%); and negative predictive value = 100% (74.7% - 100%).

DISCUSSION

There is still controversy in the international literature about the best way to measure the degree of extracranial IC stenosis using DGA. The two main studies on endarterectomy ¾ NASCET⁵ and ECST⁶¾ used different methods to perform this measurement. There is indeed a way to compare the results of each of such studies (Table 1)³. Alternative forms to measure the degree of IC stenosis have already been described¹⁰. Some authors emphasize that most radiologists do not use any of these methods in daily practice, depending instead only on their SVI¹².

Using material generated in another study¹³, the authors made a direct comparison between SVI of an experienced neuro-radiologist (FSA) and a well established method to measure IC stenosis ¾ the NASCET method⁵.There was an excellent correlation between the obtained numeric values. There was also little dispersion of values in the graphics with plotted results (Fig 1). When the sample was arbitrarily divided in surgical and non-surgical IC, concordance was also shown to be excellent, the same applying to sensitivity, specificity, positive and negative predictive values. Nineteen calculations were higly correlated (differences less than 10% between the studied methods). One patient, however, would be referred to endarterectomy by the SVI method but not if the NASCET criteria were used. In this case, the SVI found an 80% degree of IC stenosis and the NASCET method found a 54% stenosis (Fig 2). We must comment, however, that following NASCET's final results⁸ and using only anatomic criteria, this patient would possibly be submitted to endarterectomy anyway.

We must recognize that the results presented here have large confidence intervals, reflecting the small sample size. Thus, this study should be continued to increase the number of studied patients and reduce the chance of error.

In conclusion, the SVI may accurately measure the degree of extracranial IC stenosis, studied by DGA, as compared to the NASCET method. SVI, as performed by at least some experienced neuroradiologists, may be used as a preliminary tool to evaluate IC stenosis, studied by DGA. Nevertheless, a standardised and well established method to perform this measurement is imperative in daily clinical practice and should always be applied.

Acknowledgements ¾ We are in debt with Professor Ronir Raggio Luiz for his planning of the statistical analysis.

Received 1 February 2001, received in final form 4 May 2001. Accepted 14 May 2001.

Dr. Marco Oliveira Py - Rua Gal. Ribeiro da Costa 38/401 - 22010-050 Rio de Janeiro RJ ¾Brasil. E-mail: [e-mail]

• 1. Robertson JT. Carotid endarterectomy: a saga of clinical science, personalities, and evolving technology. The Willis lecture. Stroke 1998;29:2435-2441.
• 2. Gorelick PB. Carotid endarterectomy: where do we draw the line? Stroke 1999;30:1745-1750.
• 3. Donnan GA, Davis SM, Chambers BR, Gates PC. Surgery for prevention of stroke. Lancet 1998;351:1372-1373.
• 4. Biller J, Feinberg WM, Castaldo JE et al. Guidelines for carotid endarterectomy: a statement for healthcare professionals from a special writing group of the stroke council, American Heart Association. Circulation 1998;97:501-509.
• 5. North American Symptomatic Carotid Endarterectomy Trial Collaborators. Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. N Engl J Med 1991;325:445-453.
• 6. European Carotid Surgery Trialists' Collaborative Group. MRC European carotid surgery trial: interim results for symptomatic patients with severe (70-99%) or mild (0-29%) carotid stenosis. Lancet 1991;337:1235-1243.
• 7. European Carotid Surgery Trialists' Collaborative Group. Randomized trial of endarterectomy for recently symptomatic carotid stenosis: final results of the MRC European Carotid Surgery Trial (ECST). Lancet 1998;351:1379-1387.
• 8. Barnett HJM, Taylor DW, Eliasziw M et al, for the North American Symptomatic Carotid Endarterectomy Trial Collaborators: benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. N Engl J Med 1998;339:1415-1425.
• 9. Rothwell PM, Gibson RJ, Slattery J, Sellar RJ, Warlow CP. Equivalence of measurements of carotid stenosis: a comparison of three methods on 1001 angiograms. European Carotid Surgery Trialists' Collaborative Group. Stroke 1994;25:2435-2439.
• 10. Bladin CF, Alexandrov AV, Murphy J, Maggisano R, Norris JW. Carotid stenosis index: a new method of measuring internal carotid artery stenosis. Stroke 1995;26:230-234.
• 11. Rothwell PM, Gibson RJ, Slattery J, Warlow CP. Prognostic value and reproducibility of measurements of carotid stenosis: a comparison of three methods on 1001 angiograms. European Carotid Surgery Trialists' Collaborative Group. Stroke 1994;25:2440-2444.
• 12. Young GR, Humphrey PRD, Shaw MDM, Nixon TE, Smith ETS. Comparison of magnetic resonance angiography, duplex ultrasound, and digital subtraction angiography in assessment of extracranial internal carotid artery stenosis. J Neurol Neurosurg Psychiatry 1994;57:1466-1478.
• 13. Py MO, André C, Domingues RC, Azevedo FS, Salomão RF. Internal carotid artery stenosis: comparison of duplex scan and magnetic resonance angiography with digital subtraction angiography. Arq Neuropsiquiatr 2001;59:665-671.
• 14. Rankin J. Cerebral vascular accidents in patients over the age of 60. Scott Med J 1957;2:200-215.
• 15. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307-310.
• 16. Koran LM. The reliability of clinical methods, data and judgments (first of two parts). N Engl J Med 1975;293:642-646.
• 17. Koran LM. The reliability of clinical methods, data and judgments (second of two parts). N Engl J Med 1975;293:695-701.
• 18. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159-174.
• 19. Eliasziw M, Fox AJ, Sharpe BL, Barnett HJ. Carotid artery stenosis: external validity of the North American Symptomatic Carotid Endarterectomy Trial measurement method. Radiology 1997;204:229-233.
• 20. Morgenstern LB, Fox AJ, Sharpe BL, Eliasziw M, Barnett HJ, Grotta JC. The risks and benefits of carotid endarterectomy in patients with near occlusion of the carotid artery. North American Symptomatic Carotid Endarterectomy Trial (NASCET) Group. Neurology 1997;48:911-915.

¹

From the Hospital Universitário Clementino Fraga Filho, Universidade Federal do Rio de Janeiro, Rio de Janeiro RJ, Brazil:

Neurological Service, Department of Internal Medicine;

²

Department of Radiology.

Publication Dates

• Publication in this collection
  05 Oct 2001
• Date of issue
  Sept 2001

History

• Accepted
  14 May 2001
• Received
  01 Feb 2001