Repeatability and reproducibility of automatic segmentation of retinal layers in healthy subjects using Spectralis optical coherence tomography

Repetibilidade e reprodutibilidade da segmentação automática de camadas retinianas em sujeitos saudáveis com tomografia de coerência óptica Spectralis

ABSTRACT

Purpose:

To evaluate the repeatability and reproducibility of automatic segmentation in healthy subjects using a Spectralis optical coherence tomography (OCT) system.

Methods:

A total of 60 eyes from 60 patients were included in this prospective study. Spectral-domain optical coherence tomography images were generated using the Spectralis OCT system. An automated algorithm was used to segment the macular retina into nine layers and evaluate the thickness of each layer in the foveal, inner, and outer Early Treatment Diabetic Retinopathy Study (ETDRS) subfield rings. The eyes were imaged three times by an examiner to assess intraobserver repeatability and imaged once by a second examiner to assess interobserver reproducibility. The first scan was used for reference, whereas the second and third scans were collected using the device’s follow-up mode. Intraclass correlation coefficients (ICCs) of repeatability and reproducibility were analyzed.

Results:

The examiners achieved high repeatability and reproducibility for all parameters. Good agreement was found for all parameters in all ETDRS subdivisions with an ICC of >0.96 for all measurements.

Conclusion:

It is possible to obtain cross-sections from the same location using the device’s follow-up mode, making it virtually impossible to distinguish between repeated measurements taken with the device by different examiners.

Keywords:
Retina; Macula lutea; Tomography, optical coherence; Reproducibility of results; Image processing, Computer-assisted

RESUMO

Objetivo:

Avaliar a repetibilidade e a reprodutibilidade da segmentação automática com tomografia de coerência óptica Spectralis em indivíduos saudáveis.

Métodos:

Foram incluídos neste estudo prospectivo um total de 60 olhos de 60 pacientes. As imagens de tomografia de coerência óptica de domínio espectral (SD-OCT) foram geradas com Spectralis OCT (Heidelberg Engineering, Heidelberg, Alemanha). Um algoritmo automatizado foi utilizado para segmentar a retina macular em nove camadas e quantificar a espessura de cada camada em anéis de subcampo ETDRS fovetais, internos e externos. Os olhos de cada paciente foram imaginados três vezes pelo primeiro examinador para avaliar a repetibilidade intra-observador e uma vez pelo segundo examinador para avaliar a reprodutibilidade entre observadores. A primeira verificação foi definida como a varredura de referência, enquanto a segunda e a terceira varredura foram as varreduras de seguimento e foram realizadas com o uso do modo de acompanhamento, respectivamente. O coeficiente de correlação intraclasse (ICC) de repetibilidade e reprodutibilidade foi analisado. Repetibilidade e reprodutibilidade das medidas obtidas foram analisadas utilizando o coeficiente de correlação intraclasse.

Resultados:

Os examinadores alcançaram alta repetibilidade e reprodutibilidade para todos os parâmetros. Foi encontrado um bom acordo para todos os parâmetros em todas as subdivisões de ETDRS com um coeficientes de correlação intraclasse superior a 0,96 para todas as medições.

Conclusão:

É possível obter seções transversais do mesmo local usando o modo de acompanhamento, o que torna praticamente impossível distinguir entre medições repetidas do dispositivo independentemente do examinador.

Descritores:
Retina; Mácula lútea; Tomografia de coerência óptica; Reprodutibilidade dos testes; Processamento de imagem assistida por computador

INTRODUCTION

Optical coherence tomography (OCT) is a well-known tool for the high-resolution assessment of retinal pathologies. Spectral-domain OCT (SD-OCT) is a more recent technique that allows the imaging of ocular structures with faster scan rates than OCT^(1-2). The evaluation of intraretinal layer thickness is becoming increasingly important for the diagnosis and monitoring of various ocular diseases⁽³⁾. However, few studies have assessed the repeatability and reproducibility of automated total retinal thickness measurements using different SD-OCT instruments in healthy individuals^(4,5) or those with ocular pathology^(6-11).

The aim of this study was to investigate the repeatability and reproducibility of automatic segmentation of nine intraretinal layers in healthy subjects using a SD-OCT system.

METHODS

Subjects

This study followed the tenets of the Declaration of Helsinki and was approved by the Ethics Committee of Afyon Kocatepe University. Healthy subjects (28 males, 32 females) who were admitted to the ophthalmology department for regular ophthalmological examinations with no history of ocular disease except for refractive errors were included, and each signed an informed consent form. The exclusion criteria were ocular surface disorder, corneal disease, previous corneal surgery, contact lens wear, history of ocular trauma or inflammation, and systemic disease.

Measurement device

Retinal OCT imaging was performed using the Spectralis OCT system (Heidelberg Engineering GmbH, Heidelberg, Germany). Segmentation of the retinal layers from each SD-OCT scan was performed using the built-in Spectralis mapping software the Heidelberg Eye Explorer (version 6.0). Spectralis segmentation software was used to obtain the following thickness measurements: total retinal thickness (Retina), retinal nerve fiber layer (RNFL), ganglion cell layer (GCL), inner plexiform layer (IPL), inner nuclear layer (INL), outer plexiform layer (OPL), outer nuclear layer (ONL), and retinal pigment epithelium (RPE). The automatic segmentation tool of the posterior pole scan also provides the thickness of the inner retinal layer (IRL), measured from the internal limiting membrane to the external limiting membrane (ELM), and that of the outer retinal layer (ORL), measured from the ELM to the Bruch membrane (Figure 1). The Spectralis mapping software generates automated retinal thickness measurements based on analyses of the central, inner ring, and outer ring subfields as defined by the Early Treatment Diabetic Retinopathy Study (ETDRS)⁽¹²⁾.

Figure 1
Segmented view of the retinal layers created using the Heidelberg Spectralis automatic segmentation analysis program.

Measurement technique

Measurements were taken according to the guidelines of the respective device manufacturers. Before imaging, all eyes were subjected to an ocular examination including visual acuity testing, auto-refraction, intraocular pressure, and ophthalmoscopic examination. After enrollment, all eyes were imaged without mydriasis. Three repeated measurements were performed within a short period of time on the same day by a single examiner (EC) to test the intraobserver repeatability. The first examination was marked as the reference in the follow-up mode, a special feature of the Heidelberg SD-OCT system that allows scanning of the same part of the retina during follow-up visits. SD-OCT imaging was also performed once by another examiner (MCS) on the same day to test interobserver reproducibility. The thicknesses of all layers within the central ETDRS zones of 0-1000 and 1000-3000 µm (inner ring) and 3000-6000 µm diameter (outer ring) were recorded for each scan (Figure 2).

Figure 2
1: central area; 2-5: inner ring: 6-9, outer ring.

Statistical analysis

All of the statistical analyses were performed by Statistical Package for the Social Sciences software (version 17.0 for Windows; SPSS Inc., Chicago, IL, USA). Descriptive statistics are shown as mean ± standard deviation. The intraobserver repeatability was measured with three OCT images obtained by the same operator, while the interobserver reproducibility was measured with two OCT images obtained by two different operators. The overall mean thickness and intraclass correlation coefficients (ICCs) were calculated to evaluate the repeatability and reproducibility of the thickness measurements. The overall mean thickness of the nine intraretinal layers along the central macular 6-mm scan length was determined as the mean of the three measurements by the same examiner. The paired ICCs were analyzed to evaluate the agreement of the thickness measurements between the two SD-OCT instruments.

RESULTS

Sixty volunteers (28 men, 32 women) aged 20-40 years (mean age, 30.26 ± 6.31 years) participated in this study. Table 1 shows the repeatability and reproducibility of the thickness measurements for intraretinal layers measured using the Spectralis OCT system. There were no significant differences among the three thickness measurements obtained by the same examiner. ICCs obtained for the intraobserver repeatability and interobserver reproducibility tests were >0.964 for the entire retina and for all of the intraretinal layers in all of the subfields (Table 1).

Table 1
Intraobserver and interobserver agreement for retinal layer thickness

	T-a1 (µm)	T-a2 (µm)	T-a3 (µm)	T-b (µm)	ICCa1-2	ICCa1-3	ICCa-b
Central
RNFL	011.62 ± 02.48	011.59 ± 02.51	011.69 ± 02.55	011.24 ± 02.38	1.000	1.000	0.971
GCL	013.74 ± 03.93	013.55 ± 03.71	013.92 ± 03.98	013.43 ± 03.86	0.982	0.989	0.975
IPL	019.94 ± 03.31	019.90 ± 03.27	019.86 ± 03.25	019.82 ± 03.11	1.000	0.997	1.000
INL	015.84 ± 04.52	015.69 ± 04.41	015.88 ± 04.49	015.83 ± 04.34	0.989	1.000	1.000
OPL	021.58 ± 04.50	021.65 ± 04.61	021.55 ± 04.57	021.54 ± 04.39	0.991	1.000	1.000
ONL	091.97 ± 09.51	091.84 ± 09.52	091.68 ± 09.42	092.36 ± 09.51	1.000	0.974	1.000
RPE	016.22 ± 02.19	016.22 ± 02.13	016.10 ± 02.19	016.22 ± 02.19	1.000	0.985	1.000
IRL	181.07 ± 22.70	182.36 ± 23.26	182.79 ± 22.88	181.85 ± 22.21	1.000	1.000	1.000
ORL	085.91 ± 06.04	084.64 ± 05.18	084.82 ± 06.04	084.34 ± 05.71	1.000	1.000	1.000
İnner ring
RNFL	011.62 ± 02.48	011.59 ± 02.51	011.69 ± 02.55	011.24 ± 02.38	1.000	1.000	0.971
GCL	013.74 ± 03.93	013.55 ± 03.71	013.92 ± 03.98	013.43 ± 03.86	0.982	0.989	0.975
IPL	019.94 ± 03.31	019.90 ±0 3.27	019.86 ± 03.25	019.82 ± 03.11	1.000	0.997	1.000
INL	015.84 ± 04.52	015.69 ± 04.41	015.88 ± 04.49	015.83 ± 04.34	0.989	1.000	1.000
OPL	021.58 ± 04.50	021.65 ± 04.61	021.55 ± 04.57	021.54 ± 04.39	0.991	1.000	1.000
ONL	091.97 ± 09.51	091.84 ± 09.52	091.68 ± 09.42	092.36 ± 09.51	1.000	0.974	1.000
RPE	016.22 ± 02.19	016.22 ± 02.13	016.10 ± 02.19	016.22 ± 02.19	1.000	0.985	1.000
IRL	181.07 ± 22.70	182.36 ± 23.26	182.79 ± 22.88	181.85 ± 22.21	1.000	1.000	1.000
ORL	085.91 ± 06.04	084.64 ± 05.18	084.82 ± 06.04	084.34 ± 05.71	1.000	1.000	1.000
Outer ring
RNFL	035.87 ± 03.24	036.08 ± 03.81	035.56 ± 03.20	036.21 ± 03.42	0.989	0.987	0.971
GCL	037.83 ± 03.25	037.30 ± 03.08	038.11 ± 03.46	037.85 ± 03.30	0.975	0.980	1.000
IPL	030.41 ± 02.45	030.35 ± 02.34	030.45 ± 02.45	030.40 ± 02.49	1.000	1.000	1.000
INL	033.61 ± 02.01	033.58 ± 02.00	033.44 ± 01.97	035.04 ± 02.32	1.000	0.998	0.964
OPL	027.72 ± 02.17	027.69 ± 02.10	027.57 ± 02.05	027.70 ± 02.17	1.000	0.996	1.000
ONL	058.89 ± 06.51	058.98 ± 06.66	058.75 ± 06.49	058.83 ± 06.51	1.000	1.000	1.000
RPE	012.69 ± 01.23	013.03 ± 01.25	012.88 ± 01.23	013.05 ± 01.47	0.979	0.996	0.979
IRL	222.18 ± 10.51	222.78 ± 10.44	222.10 ± 10.50	223.20 ± 09.42	1.000	1.000	1.000
ORL	077.17 ± 01.97	077.42 ± 02.79	078.47 ± 01.86	076.75 ± 01.78	1.000	1.000	1.000

• T-a1= mean thickness of first measurement by examiner 1; T-a2= mean thickness of second measurement by examiner 1; T-a3= mean thickness of third measurement by examiner 1; T-b= mean thickness of first measurement by examiner 2; ICC a1-2= intraclass correlation coefficients of repeatability (T-a1-T-a2); ICC a1-3= intraclass correlation coefficients of repea­tability (T-a1-T-a3); ICCa-b= intraclass correlation coefficients of reproducibility; RNFL= retinal nerve fiber layer; GCL= ganglion cell layer IPL; inner plexiform layer; INL= inner nuclear layer; OPL= outer plexiform layer; ONL= outer nuclear layer; RPE= retinal pigment epithelium; IRL= inner retinal layer; ORL= outer retinal layer.

DISCUSSION

Repeatability and reproducibility are very important for assessing measurement fluctuations for an instrument and when an instrument is used by multiple operators. Measurements of the intraretinal layers are significant morphometric parameters in the diagnosis of retinal and neurological diseases as well as for monitoring disease progression⁽¹³⁾. In this study, we evaluated the repeatability and reproducibility of measurements for nine intraretinal layers (RNFL, GCL, IPL, INL, OPL, ONL, RPE, IRL, ORL) determined by an automated algorithm applied to images obtained using the SD-OCT system with the follow-up mode. The follow-up mode is also called AutoRescan and implements active eye tracking to perform OCT scans automatically on the retina at the same point as in the previous review⁽¹⁴⁾. In our study, for both intraobserver and interobserver comparisons, ICCs were high for all layers. The intraobserver and interobserver test results indicated that the SD-OCT system produced excellent repeatable and reproducible measurements for all of the intraretinal layers. The repeatability of measurements in the present study is consistent with those in previous reports involving different OCT devices.

Previous studies reported a higher degree of repeatability for macular thickness with Spectralis OCT measurements. Fiore et al. demonstrated that the eye tracking mechanism and follow-up function improved reproducibility, ensuring the same scanning location was selected on following visits and that moving artifacts were reduced⁽¹⁵⁾. Ctori et al. reported excellent repeatability and reproducibility for the thickness of each of eight individual retinal layers located centrally or at 2° or 5° eccentricity away from the foveal measurements obtained using the SD-OCT segmentation software in a young healthy cohort⁽¹⁶⁾. What differentiates our study from previous studies is its assessment of repeatability and reproducibility in all subdivisions (wide area) of ETDRS. Debuc et al. reported the repeatability and reproducibility of thickness measurements for six intraretinal layers using automated custom software developed for the Stratus OCT with healthy subjects. They found that ICCs were >0.75 for all layers except for the OPL and OS/RPE⁽¹⁷⁾. Wang et al. reported the repeatability of thickness measurements for nine intraretinal layers determined by manual segmentation of ultra-high-resolution OCT (UHR-OCT) images with an axial resolution of approximately 2 µm with healthy subjects. The ICC was >0.90 in most of the intraretinal layers⁽¹⁸⁾. Liu et al. reported the repeatability and reproducibility of thickness measurements for eight-retinal layers using a custom-built UHR-OCT instrument (~3 µm resolution) and a commercial RTVue100 OCT (~5 µm resolution) instrument with 20 normal subjects. They found ICCs of >0.80 for all layers except for the inner and outer segments⁽¹³⁾. Our results are consistent with those of these studies, showing that the thickness measurements of higher-axial-resolution OCT instruments have better repeatability for all retinal layers.

There are some limitations to the present study. First, it included only healthy subjects. Diseased retinal structures may vary substantially among patients, which is likely to increase the frequency of segmentation errors. Thus, the repeatability and reproducibility values may be reduced in diseased retinas. In future studies, we will apply our new method to evaluate a variety of retinal diseases. However, the purpose of the current study was to determine the clinical significance of any changes in the repeatability and reproducibility of the intraretinal measurements.

In conclusion, the current study evaluated measurements of retinal layer thickness in the macula area. The measurements obtained using the follow-up mode of the SD-OCT system demonstrated high repeatability and reproducibility with excellent ICCs.

• Funding: No specific financial support was available for this study.
• Approved by the following research ethics committee: Afyon Kocatepe University (# 2017/4).

REFERENCES

• ¹
  Nassif N, Cense B, Park B, Pierce M, Yun S, Bouma B, et al. In vivo high resolution video-rate spectral-domain optical coherence tomography of the human retina and optic nerve. Opt Express. 2004;12(3):367-76.
• ²
  Wojtkowski M, Bajraszewski T, GorczyÅska I, Targowski P, Kowalczyk A, Wasilewski W, et al. Ophthalmic imaging by spectral optical coherence tomography. Am J Ophthalmol. 2004;138(3):412-9.
• ³
  Cesareo M, Ciuffoletti E, Martucci A, Balducci C, Cusumano A, Ricci F, et al. Automatic segmentation of posterior pole retinal layers in patients with early stage glaucoma using spectral domain optical coherence tomography. J Clin Exp Ophthalmol. 2016; 7:538. doi: 10.4172/2155-9570.1000538.
  » https://doi.org/10.4172/2155-9570.1000538
• ⁴
  Tan CS, Li KZ, Lim TH. A novel technique of adjusting segmentation boundary layers to achieve comparability of retinal thickness and volumes between spectral domain and time domain optical coherence tomography. Invest Ophthalmol Vis Sci. 2012; 53(9):5515-9.
• ⁵
  Wolf-Schnurrbusch UE, Ceklic L, Brinkmann CK, Iliev ME, Frey M, Rothenbuehler SP, et al. Macular thickness measurements in healthy eyes using six different optical coherence tomography instruments. Invest Ophthalmol Vis Sci. 2009;50(7):3432-7.
• ⁶
  Krebs I, Smretschnig E, Moussa S, Brannath W, Womastek I, Binder S. Quality and reproducibility of retinal thickness measurements in two spectral-domain optical coherence tomography machines. Invest Ophthalmol Vis Sci. 2011;52(9):6925-33.
• ⁷
  Comyn O, Heng LZ, Ikeji F, Bibi K, Hykin PG, Bainbridge JW, et al. Repeatability of Spectralis OCT measurements of macular thickness and volume in diabetic macular edema. Invest Ophthalmol Vis Sci. 2012;53(12):7754-9.
• ⁸
  Patel PJ, Chen FK, Ikeji F, Xing W, Bunce C, Da Cruz L, et al. Repeatability of stratus optical coherence tomography measures in neovascular age-related macular degeneration. Invest Ophthalmol Vis Sci. 2008;49(3):1084-8.
• ⁹
  Patel PJ, Chen FK, Ikeji F, Tufail A. Intersession repeatability of optical coherence tomography measures of retinal thickness in early age-related macular degeneration. Acta Ophthalmol. 2011;89(3):229-34.
• ¹⁰
  Pinilla IM, Garcia-Martin EM, Fernandez-Larripa SM, Fuentes-Broto LP, Sanchez-Cano AI, Abecia EM. Reproducibility and repeatability of cirrus and spectralis fourier-domain optical coherence tomography of healthy and epiretinal membrane eyes. Retina. 2013; 33(7):1448-55.
• ¹¹
  Diabetic Retinopathy Clinical Research Network Writing Committee, Bressler SB, Edwards AR, Chalam KV, Bressler NM, Glassman AR, Jaffe GJ, Melia M, Saggau DD, Plous OZ. Reproducibility of spectral-domain optical coherence tomography retinal thickness measurements and conversion to equivalent time-domain metrics in diabetic macular edema. JAMA Ophthalmol. 2014;132(9):1113-22. doi: 10.1001/jamaophthalmol.2014.1698.
  » https://doi.org/10.1001/jamaophthalmol.2014.1698
• ¹²
  Photocoagulation for diabetic macular edema. Early Treatment Diabetic Retinopathy Study Report No.1. 1. Early Treatment Diabetic Retinopathy Study research group. Arch Ophthalmol. 1985;103(12):1796-806.
• ¹³
  Liu X, Shen M, Huang S, Leng L, Zhu D, Lu F. Repeatability and reproducibility of eight macular intra-retinal layer thicknesses determined by an automated segmentation algorithm using two SD-OCT instruments. PLoS One. 2014;9(2):e87996.
• ¹⁴
  Heidelberg Engineering. Spectralis HRA + OCT User manual software version 6.0. Heidelberg: Germany; 2015.
• ¹⁵
  Fiore T, Lupidi M, Androudi S, Giansanti F, Fruttini D, Cagini C. Repeatability of retinal macular thickness measurements in healthy subjects and diabetic patients with clinically significant macular edema: Evaluation of the follow-up system of Spectralis optical coherence tomography. Ophthalmologica. 2015;233(3-4):186-91.
• ¹⁶
  Ctori I, Huntjens B. Repeatability of foveal measurements using spectralis optical coherence tomography segmentation software. PLoS One. 2015;10(6):e0129005. doi: 10.1371/journal.pone.0129005.
  » https://doi.org/10.1371/journal.pone.0129005
• ¹⁷
  DeBuc DC, Somfai GM, Ranganathan S, Tátrai E, Ferencz M, Puliafito CA. Reliability and reproducibility of macular segmentation using a custom-built optical coherence tomography retinal image analysis software. J Biomed Opt. 2009;14(6):064023. doi: 10.1117/1.3268773.
  » https://doi.org/10.1117/1.3268773
• ¹⁸
  Wang Y, Jiang H, Shen M, Lam BL, DeBuc DC, Ye Y, et al. Quantitative analysis of the intraretinal layers and optic nerve head using ultra-high resolution optical coherence tomography. J Biomed Opt. 2012;17(6):066013. doi: 10.1117/1.JBO.17.6.066013.
  » https://doi.org/10.1117/1.JBO.17.6.066013

Publication Dates

• Publication in this collection
  Nov-Dec 2017

History

• Received
  25 Apr 2017
• Accepted
  17 July 2017