Reproducibility of carotid ultrasound measurements in the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil) at baseline

Abstract

Carotid artery assessment by ultrasound is a non-invasive evaluation of subclinical atherosclerosis and a predictor of cardiovascular events. However, ultrasound examinations are operator-dependent. In the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil), ultrasound images have been acquired from more than 10,000 participants. In this article, we describe the reproducibility of carotid intima-media thickness (CIMT), carotid plaque detection, and carotid plaque score (defined as the number of arterial sites with plaques) using ELSA-Brasil protocol, in a subset of 118 participants. Two board-certified radiologists and a trained technician read carotid images. We calculated intra- and inter-observer intraclass correlation (ICC) for CIMT values. We also present kappa coefficients for plaque detection and weighted kappa coefficients for carotid plaque score. Participants were aged 58.2±6.6 years, and 60 (50.8%) were men. For common carotid artery CIMT measurements, intra- and inter-observer ICC values were very good to excellent, ranging from 0.90 (95% confidence interval [95%CI]: 0.72–0.95) to 0.98 (95%CI: 0.97–0.99). For carotid plaque, intra- (0.96 [95%CI: 0.96–0.96]) and inter- (0.99 [95%CI: 0.99–0.99]) observer weighted kappa coefficients were very good. Intra- and inter-observer Kappa coefficients for the presence of plaques by site were good to very good, ranging from 0.69 to 1.00. In conclusion, we found very good reproducibility for carotid plaque score and CIMT measurements in the ELSA-Brasil at baseline. These results are comparable to the best findings from similar large cohorts that analyzed carotid ultrasound data.

Plaque; Intima-media; Epidemiology; Ultrasound; Imaging

Introduction

Ultrasonographic evaluation of carotid artery plaques and carotid intima-media thickness (CIMT) are increasingly used to assess the extent and severity of subclinical atherosclerosis (¹–³). The ultrasound exam is widely-available, safe, and noninvasive; however, its reproducibility depends on machine and operator factors (¹,²). B-mode image quality varies with interrogation angle, machine capabilities, and fine adjustment of instrument controls (¹,²). Disagreement among readers regarding the transition line from common carotid to carotid bulb may lead to divergent CIMT measurements and disagreement as to the location of plaques (¹).

In the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil) (⁴–⁷) study, ultrasound images have been acquired for more than 10,000 participants at baseline, with strict protocols for image acquisition and reading (⁴).The aim of this article is to describe the intra- and inter-observer reproducibility of CIMT, carotid plaque detection, and carotid plaque score, using ELSA-Brasil protocol for ultrasound imaging acquisition and reading.

Material and Methods

ELSA-Brasil is a prospective cohort of 15,105 civil servants from 6 different Brazilian cities (São Paulo, Belo Horizonte, Rio de Janeiro, Porto Alegre, Salvador, and Vitoria), aged 35 to 74 years. It was designed to evaluate cardiovascular diseases and diabetes, as well as their biological and social determinants (⁴–⁷). The baseline assessments occurred from August 2008 to December 2010. Approvals of the ethics committees were granted, and all individuals provided their written informed consent (⁴,⁵).

All participants were examined by trained staff (⁴,⁵). The images were obtained by a Toshiba Aplio XG ultrasound device (Japan) with a linear 7.5 MHz transducer, between August 2008 and December 2010 (⁴,⁵). Images were taken of the longitudinal axis of the common carotid artery (CCA), bulb, and origin of the internal carotid artery (ICA), observing three electrocardiographically gated cardiac cycles. Images were also obtained of the short (transverse) axis of all visible segments, including the bulb and ICA, up to the mandibular branch. All the images obtained were sent to an examination reading center in São Paulo (⁴,⁵). The definitions of CIMT and plaques assumed by the ELSA-Brasil protocol (⁴,⁵) are those adopted by the Mannheim consensus (²) and the Echocardiography American Society (¹).

Measurement of CIMT

To measure CIMT, each CCA was identified along its longitudinal axis, using standard brightness and contrast. CIMT was semi-automatically calculated using Medical Imaging Applications software (MIA-LLC, USA), analyzing three electrocardiographically gated cardiac cycles (⁴,⁵). Only the far wall of the CCA (1 cm proximal to the carotid bulb, with 1 cm in length) was measured, and minimum, mean, and maximum values for each CCA were obtained (⁴,⁵). The current Brazilian recommendations for ultrasound evaluation of carotid atherosclerotic disease adopts the ELSA-Brasil CIMT values as standard for the Brazilian adult population (⁸).

Measurement of plaque

The near and far walls of the CCA, carotid bulb, and origin of the ICA bilaterally were evaluated longitudinally and transversally for plaque detection, following the Mannheim consensus criteria (²). Plaques were evaluated by visual judgment, without measuring the thickness of the lesions (⁹–¹⁴). All 12 combinations of side (left, right), wall (near, far), and segment (CCA, bulb, ICA) were evaluated for the presence or absence of plaques. The count of sites that presented plaques yielded a score ranging from 0 to 12 (¹³,¹⁴).

Study sample

A random subsample of 118 participants from the ELSA-Brasil São Paulo site with valid left and right carotid ultrasound images was selected for this study (⁴,⁶,⁷). The size of this subsample was similar to that of other reproducibility studies (⁹,¹¹,¹³,¹⁴). The same certified technician (DPM) assessed image quality (¹⁵,¹⁶) for all acquisitions.

For the purpose of the present study, readings were performed offline between October 2016 and March 2017, at the reading center in São Paulo. Two board-certified radiologists (PJSN, EHSS) read the plaque images using the same protocol. One radiologist (PJSN) and a trained technician with more than 5 years of experience (DPM) read the CIMT images. Readings for intra-observer agreement were performed with a minimum interval of 3 months between the first and second readings. All staff were blinded to the participants' clinical information.

Statistical analysis

We calculated intraclass correlation (ICC) for CIMT to estimate intra- and inter-observer agreement (¹⁵–¹⁷). ICC estimates were based on single-rating, absolute-agreement, two-way models. We verified residual distribution and potentially influential measurements using DFFITS, residual Q-Q plots, residual x-fitted plots, and Shapiro-Wilk normality tests. ICC coefficients were slightly higher when using datasets after the exclusion of influential points, compared to the use of the entire dataset. To use the same set of observations throughout the paper (enhancing comparability among measurements) and to report conservative results, avoiding bias toward the exclusion of observations with greater disagreement, we opted to report the results from models using the entire dataset as our main results.

We also report kappa coefficients (¹⁷) for intra- and inter-observer agreement in the presence of plaque in each of the twelve possible sites in the carotid arteries and weighted kappa coefficients for intra- and inter-observer agreement in plaque score. ICC values <0.50, between 0.50 and 0.75, between 0.76 and 0.90, and >0.90 are indicative of poor, moderate, good, and excellent reliability, respectively (¹⁵). Kappa coefficients ≤0.20, between 0.21 and 0.40, between 0.41 and 0.60, between 0.61 and 0.80, and >0.80 are classified as poor, fair, moderate, good, and very good reliability, respectively (¹⁷). We report 95% confidence intervals (95%CI). Whenever the 95%CI for kappa coefficients reached 1.0, this result was confirmed by bootstrapping. All analyses were performed using R software, boot, irr, and psych packages.

Results

The studied sample was aged 58.2±6.6 years, 60 (50.8%) were men, 57 (48.3%) had hypertension, 32 (27.1%) had diabetes, 22 (18.6%) were current smokers, and the mean body mass index was 27.3±4.4 kg/m². Table 1 shows the intra- and inter-observer agreement for CIMT and plaque measurements. The mean CIMT and plaque scores had excellent intra- and inter-observer agreement. Maximum and minimum CIMT had very good to excellent intra- and inter-observer agreement. Kappa coefficients for the presence of plaques by site ranged from 0.69 to 1.00 (good to very good).

Table 1
Intraclass correlation and kappa coefficients (95%CI) for intra- and inter-observer agreement in carotid intima-media thickness (CIMT) and plaque measurements.

CIMT measurements	Intraclass correlation coefficients (95%CI)
	Intra-observer	Inter-observer
Left CCA, minimum	0.93 (0.89–0.95)	0.91 (0.81–0.95)
Left CCA, mean	0.98 (0.97–0.98)	0.98 (0.97–0.99)
Left CCA, maximum	0.95 (0.93–0.97)	0.94 (0.73–0.98)
Right CCA, minimum	0.92 (0.88–0.95)	0.93 (0.84–0.97)
Right CCA, mean	0.97 (0.96–0.98)	0.98 (0.97–0.98)
Right CCA, maximum	0.96 (0.95–0.97)	0.90 (0.72–0.95)

Plaque measurements	Kappa coefficients (95%CI)
	Intra-observer	Inter-observer
Plaque score	0.96 (0.96–0.96)	0.99 (0.99–0.99)
Presence of plaque by site
Left CCA, near wall	0.87 (0.72–1.00)	0.95 (0.86–1.00)
Left CCA, far wall	0.77 (0.60–0.93)	0.93 (0.83–1.00)
Left carotid bulb, near wall	0.90 (0.81–0.98)	0.84 (0.73–0.95)
Left carotid bulb, far wall	0.81 (0.71–0.92)	0.93 (0.86–1.00)
Left ICA, near wall	0.84 (0.67–1.00)	0.90 (0.76–1.00)
Left ICA, far wall	0.79 (0.65–0.94)	0.97 (0.91–1.00)
Right CCA, near wall	0.76 (0.49–1.00)	1.00 (1.00–1.00)
Right CCA, far wall	0.76 (0.53–0.99)	0.94 (0.81–1.00)
Right carotid bulb, near wall	0.83 (0.69–0.96)	0.92 (0.83–1.00)
Right carotid bulb, far wall	0.81 (0.69–0.92)	0.98 (0.94–1.00)
Right ICA, near wall	0.85 (0.64–1.00)	0.76 (0.53–0.99)
Right ICA, far wall	0.69 (0.47–0.90)	0.81 (0.64–0.99)

• CCA: common carotid artery; ICA: internal carotid artery.

Discussion

The present analyses show excellent intra- and inter-observer reproducibility for the carotid plaque score and mean CIMT measurements in the ELSA-Brasil. Mean CIMT and plaque scores were more reliable measurements in this study than maximum and minimum CIMT, perhaps because the former reduces the impact of extreme values.

CIMT measurement is challenging, and protocols among studies vary. Far wall measurements are usually preferred because their accuracy has been validated against CCA histological specimens (¹⁸). The phase of the cardiac cycle also differs, and influences the values obtained. Due to expansion of the systolic lumen diameter, which leads to thinning of CIMT during systole, CIMT values obtained from end-systole are lower than those obtained in end-diastole (¹⁹). Another factor to be considered is that computed assisted CIMT measurements are more accurate than manual evaluations (²⁰). All these considerations were used to design the ELSA-Brasil protocol, with a semi-automated CIMT measurement in the far CCA wall, analyzing three electrocardiographically gated cardiac cycles (⁴). It is likely that these factors contributed to the good outcomes of the ELSA-Brasil on CIMT reproducibility.

There is also a lack of agreement on the best way to analyze carotid plaques (¹–³). Plaque thickness, score, area, or volume (by 3-dimensional ultrasound imaging) may be more accurate in predicting cardiovascular risk than the presence of plaques (³,⁹,¹¹–¹⁴). However, plaque score analyses are practical because visual judgement alone is sufficient (⁹,¹¹,¹³,¹⁴). When analyzing individual arterial sites for the presence of carotid plaques, we found some degree of intra- and inter-observer disagreement, albeit small in most cases. However, the ICC for the plaque score was excellent (0.959 and 0.961 for intra- and inter-observer agreement, respectively). This is mainly because, for some plaques, radiologists are unequivocal about their presence, but they may disagree about their exact segment location. This reinforces the plaque score as a reliable measurement of plaque burden, compared to the analysis of individual sites.

Other large cohorts, like the ARIC (Atherosclerosis Risk in Communities cohort) (¹⁰,¹²) and MESA (Multi-ethnic Study of Atherosclerosis cohort) (¹³,¹⁴) published their reproducibility results for plaque data and CIMT. They assessed the presence or absence of plaques in each segment of the carotid arteries (CCA, bulb, and ICA) on both sides, as well as measured CIMT in the far CCA wall. Our results were in agreement with the outcomes presented in that work.

Li et al. (¹⁰) analyzed carotid images from the ARIC, a population-based cohort of subjects between 45 and 64 years of age from four U.S. communities with baseline assessment between 1987 and 1989. The authors found ICC for a mean CIMT of 0.70 (¹⁰,¹²). For the presence of plaques, kappa coefficients for intra- and inter-observer agreement were 0.76 and 0.56, respectively (¹⁰,¹²). Possible explanations for these coefficients are associated with limitations in the equipment and software available at the time of the ARIC image acquisition and reading (¹⁰,¹²). Since the early 1990s, there have been important improvements in ultrasound technology, resulting in high-quality images and improved readings.

Tattersall et al. (¹⁴) analyzed carotid images of participants from the MESA study (¹⁴). MESA enrolled individuals aged 45–84 years, from different ethnic backgrounds (White, African-American, Hispanic, and Chinese-American), from 6 U.S. field centers, with baseline assessment between 2000 and 2002 (¹³,¹⁴). Analyzing the mean CIMT with semi-automated border detection software, (¹⁴) the authors showed that the ICC values for intra- and inter-observer agreement were 0.99 and 0.95, respectively (¹⁴). For carotid plaque presence and score, overall kappa coefficients for intra- and inter-observer agreement were 0.83 and 0.89, respectively (¹⁴). These results are very close to ours, suggesting a reliable methodology in both cohorts.

In conclusion, we found very good reproducibility for carotid plaque score and CIMT measurements in the ELSA-Brasil study. These results are comparable to the best findings from similar large cohorts that analyzed carotid ultrasound data.

Acknowledgments

The ELSA-Brasil baseline study was supported by the Brazilian Ministry of Health (Science and Technology Department) and the Brazilian Ministry of Science and Technology (Financiadora de Estudos e Projetos and CNPq National Research Council) (grants 01 06 0010.00 RS, 01 06 0212.00 BA, 01 06 0300.00 ES, 01 06 0278.00 MG, 01 06 0115.00 SP, 01 06 0071.00 RJ).

References

• ¹
  Stein JH, Korcarz CE, Hurst RT, Lonn E, Kendall CB, Mohler ER, et al. Use of carotid ultrasound to identify subclinical vascular disease and evaluate cardiovascular disease risk: a consensus statement from the American Society of Echocardiography Carotid Intima-Media Thickness Task Force endorsed by the Society for Vascular Medicine. J Am Soc Echocardiogr 2008; 21: 93–111, doi: 10.1016/j.echo.2007.11.011.
  » https://doi.org/10.1016/j.echo.2007.11.011
• ²
  Touboul PJ, Hennerici MG, Meairs S, Adams H, Amarenco F, Bornstein N, et al. Mannheim carotid intima-media thickness and plaque consensus (2004-2006-2011). An update on behalf of the advisory board of the 3rd, 4th and 5th watching the risk symposia, at the 13th, 15th and 20th European Stroke Conferences, Mannheim, Germany, 2004, Brussels, Belgium, 2006, and Hamburg, Germany, 2011. Cerebrovasc Dis 2012; 34: 290–296, doi: 10.1159/000343145.
  » https://doi.org/10.1159/000343145
• ³
  Naqvi TZ, Lee MS. Carotid intima-media thickness and plaque in cardiovascular risk assessment. JACC Cardiovasc Imaging 2014; 7: 1025–1038, doi: 10.1016/j.jcmg.2013.11.014.
  » https://doi.org/10.1016/j.jcmg.2013.11.014
• ⁴
  Mill JG, Pinto K, Griep RH, Goulart A, Foppa M, Lotufo PA, et al. [Medical assessments and measurements in ELSA-Brasil]. Rev Saude Publica 2013; 47(Suppl 2): 54-62 [in Portuguese], doi: 10.1590/S0034-8910.2013047003851.
  » https://doi.org/10.1590/S0034-8910.2013047003851
• ⁵
  Aquino EM, Barreto SM, Bensenor IM, Carvalho MS, Chor D, Duncan BB, et al. Brazilian Longitudinal Study of Adult Health (ELSA-Brasil): objectives and design. Am J Epidemiol 2012; 175: 315–324, doi: 10.1093/aje/kwr294.
  » https://doi.org/10.1093/aje/kwr294
• ⁶
  Santos IS, Bittencourt MS, Oliveira IR, Souza AG, Meireles DP, Rundek T, et al. Carotid intima-media thickness value distributions in The Brazilian Longitudinal Study of Adult Health (ELSA-Brasil). Atherosclerosis 2014; 237: 227–235, doi: 10.1016/j.atherosclerosis.2014.09.004.
  » https://doi.org/10.1016/j.atherosclerosis.2014.09.004
• ⁷
  Santos IS, Alencar AP, Rundek T, Goulart AC, Barreto SM, Pereira AC et al. Low impact of traditional risk factors on carotid intima-media thickness: the ELSA-Brasil Cohort. Arterioscler Thromb Vasc Biol. 2015; 35: 2056–2060, doi: 10.1161/ATVBAHA.115.305765.
  » https://doi.org/10.1161/ATVBAHA.115.305765
• ⁸
  Freire CM, Alcantara ML, Santos SN, Amaral SI, Veloso O, Porto CL, et al. Recomendação para a Quantificação pelo Ultrassom da Doença Aterosclerótica das Artérias Carótidas e Vertebrais: Grupo de Trabalho do Departamento de Imagem Cardiovascular da Sociedade Brasileira de Cardiologia-DIC - SBC Arq Bras Cardiol 2015; 28: e1–e64.
• ⁹
  Hollander M, Bots M, Iglesias del Sol A, Koudstall PJ, Witteman JCM, Grobbee DE, et al. Carotid plaques increase the risk of stroke and subtypes of cerebral infarction in asymptomatic elderly the Rotterdam Study. Circulation 2002; 105: 2872–2878, doi: 10.1161/01.CIR.0000018650.58984.75.
  » https://doi.org/10.1161/01.CIR.0000018650.58984.75
• ¹⁰
  Li R, Duncan BB, Metcalf PA, Crouse JR, Sharrett R, Tyroler HA, et al. B-mode-detected carotid artery plaque in a general population. Stroke 1994; 25: 2377–2383, doi: 10.1161/01.STR.25.12.2377.
  » https://doi.org/10.1161/01.STR.25.12.2377
• ¹¹
  Plichart M, Celermajer DS, Zureik M, Helmer C, Jouven X, Ritchie K, et al. Carotid intima-media thickness in plaque-free site, carotid plaques and coronary heart disease risk prediction in older adults. The Three-City Study. Atherosclerosis 2011; 219: 917–924, doi: 10.1016/j.atherosclerosis.2011.09.024.
  » https://doi.org/10.1016/j.atherosclerosis.2011.09.024
• ¹²
  Nambi V, Chambless L, Folsom AR, He M, Hu Y, Mosley T, et al. Carotid intima-media thickness and presence or absence of plaque improves prediction of coronary heart disease risk: the ARIC (Atherosclerosis Risk in Communities) study. J Am Coll Cradiol 2010; 55: 1600–1607, doi: 10.1016/j.jacc.2009.11.075.
  » https://doi.org/10.1016/j.jacc.2009.11.075
• ¹³
  Gepner AD, Young R, Delaney JA, Tattersall MC, Blaha MJ, Post WS, et al. Comparison of coronary artery calcium presence, carotid plaque presence, and carotid intima-media thickness for cardiovascular disease prediction in the Multi-Ethnic Study of Atherosclerosis. Circ Cardiovasc Imaging 2015; 8: pii: e002262, doi: 10.1161/CIRCIMAGING.114.002262.
  » https://doi.org/10.1161/CIRCIMAGING.114.002262
• ¹⁴
  Tattersall MC, Gassett A, Korcarz CE, Gepner AD, Kaufman JD, Liu KJ, et al. Predictors of carotid thickness and plaque progression during a decade: the Multi-Ethnic Study of Atherosclerosis. Stroke 2014; 45: 3257–3262, doi: 10.1161/STROKEAHA.114.005669.
  » https://doi.org/10.1161/STROKEAHA.114.005669
• ¹⁵
  Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15: 155–163, doi: 10.1016/j.jcm.2016.02.012.
  » https://doi.org/10.1016/j.jcm.2016.02.012
• ¹⁶
  Fleiss J. Statistical Methods for Rates and Proportions. New York: Ed. Wiley; 1981.
• ¹⁷
  Altman D. Practical Statistics for Medical Research. London: CRC Press; 1993.
• ¹⁸
  Pignoli P, Tremoli E, Poli A, Oreste P, Paoletti R. Intimal plus medial thickness of the arterial wall: direct measurement with ultrasound imaging. Circulation 1986; 74: 1399–1406, doi: 10.1161/01.CIR.74.6.1399.
  » https://doi.org/10.1161/01.CIR.74.6.1399
• ¹⁹
  Polak JF, Johnson C, Harrington A, Wong Q, O'Leary DH, Burke G, et al. Changes in carotid intima-media thickness during the cardiac cycle: the Multi-Ethnic Study of Atherosclerosis. J Am Heart Assoc 2012; 1: e001420, doi: 10.1161/JAHA.112.001420.
  » https://doi.org/10.1161/JAHA.112.001420
• ²⁰
  Freire CM, Ribeiro AL, Barbosa FB, Nogueira AI, de Almeida MC, Barbosa MM, et al. Comparison between automated and manual measurements of carotid intima-media thickness in clinical practice. Vasc Health Risk Manag 2009; 5: 811–817.

Publication Dates

• Publication in this collection
  05 Aug 2019
• Date of issue
  2019

History

• Received
  19 Mar 2019
• Accepted
  4 June 2019