A robust fully automatic lumen segmentation method for in vivo intracoronary optical coherence tomography

Introduction: Intravascular optical coherence tomography (IVOCT) is an in-vivo imaging modality based on the introduction of a catheter in a blood vessel for viewing its inner wall using electromagnetic radiation. One of the most developed automatic applications for this modality is the lumen area segmentation, however on the evaluation of these methods, the slices inside bifurcation regions, or with the presence of complex atherosclerotic plaques and dissections are usually discarded. This paper describes a fully-automatic method for computing the lumen area in IVOCT images where the set of slices includes complex atherosclerotic plaques and dissections. Methods: The proposed lumen segmentation method is divided into two steps: preprocessing, including the removal of artifacts and the second step comprises a lumen detection using morphological operations. In addition, it is proposed an approach to delimit the lumen area for slices inside bifurcation region, considering only the main branch. Results: Evaluation of the automatic lumen segmentation used manual segmentations as a reference, it was performed on 1328 human IVOCT images, presenting a mean difference in lumen area and Dice metrics of 0.19 mm2 and 97% for slices outside the bifurcation, 1.2 mm2 and 88% in the regions with bifurcation without automatic contour correction and 0.52 mm2 and 90% inside bifurcation region with automatic contour correction. Conclusion: This present study shows a robust lumen segmentation method for vessel cross-sections with dissections and complex plaque and bifurcation avoiding the exclusion of such regions from the dataset analysis.


Introduction
Optical coherence tomography (OCT) is based on the technology of low-coherence interferometry and provides cross-sectional images of tissue samples with far better resolution than any other form of imaging modality in vivo.This modality has many similarities with ultrasound imaging, but instead of using sound, it uses the scattering of electromagnetic radiation in the near infrared frequency as the signal source.Intravascular optical coherence tomography (IVOCT) is an in vivo high-resolution imaging technique based on the introduction of a catheter inside the blood vessel for viewing its inner wall.Due to the resolution of 10 μm to 20 μm, this modality allows the visualization of tissues on the microscopic scale (Prati et al., 2010), enabling the visualization of atherosclerotic plaques.Several past studies evaluated the usefulness of IVOCT (Suter et al., 2010;Yabushita et al., 2002), showing many advantages over other modalities such as intravascular ultrasound (IVUS).Finn et al. (2011) analyzed the differences between both IVOCT and IVUS modalities.
An IVOCT image sequence is composed of a large number of slices, which means that manual segmentations of structures are time-consuming and operator dependent.Many automatic methods have been developed to segment and calculate the lumen area such as Moraes et al. (2013), Sihan et al. (2008;2009), Tsantis et al. (2012) and Ughi et al. (2012a).However, only healthy and non-bifurcation slices are considered at the evaluation stage.Another important automatic application of IVOCT is the assessment of stent apposition and coverage, where the area formed by a set of struts is calculated (Ughi et al., 2012b;Wang et al., 2012;2015;Xu et al., 2011).Among other automatic applications for this modality are methods to quantify calcified tissue (Wang et al., 2010), methods for neointimal tissue analysis (Tung et al., 2013;Ughi et al., 2014) and methods to characterize atherosclerotic plaques (Ughi et al., 2013;van Soest et al., 2010).
A lumen segmentation method is important to indicate stenosis, and is also useful as a first step to analyze the vessel wall.The main reason for the IVOCT image acquisition is to analyze atherosclerotic plaques present in the vessel wall.Several studies (Diletti et al., 2011;Gonzalo et al., 2009;Ingebrigtsen et al., 2004) analyze the correlation between bifurcations and lesion formation.While Zarins et al. (1983) suggests that flow separation has been proposed as hemodynamic potentiator of lesion formation, providing a motive for lumen segmentation in these regions.
Here we propose a fully automated method to segment the coronary lumen area in human IVOCT datasets, which include complex plaque slices (Kronzon and Tunick, 2006), arterial dissections and side branches.The proposed method is based on a priori information of OCT frames, mathematical morphology operations and distance transforms to correct the lumen area in bifurcation slices.An evaluation was performed by comparing manual segmentation by an expert with results of the proposed automatic method in a data set of 9 different patients, comprising 1188 slices outside bifurcation region and 141 slices inside bifurcation region.The manual segmentation was performed, restricting the lumen area to the main branch; this paper thus shows the difference between the automatic approach with lumen correction and without it for bifurcation region.The tests with non-bifurcation slices and with complex plaques and artery dissections resulted in a mean difference of 0.19 ± 0.13 mm 2 .While the results for bifurcation slices show a mean difference between manual and automatic segmentation of 1.2 ± 0.83 mm 2 without automatic correction and 0.52 ± 0.81 mm 2 with automatic correction.

Image data
Coronary images were collected by a Fourier-Domain OCT (FD-OCT) system C7-XR (OCT Intravascular Imaging System, St. Jude Medical, St. Paul, Minnesota) at the Heart Institute, University of São Paulo Medical School (INCOR-HC FMUSP).The study protocol was approved by the institutional review board (CAPPesq) under registration number 0243/08.Image sets were taken from nine different patients immediately before stenting.During this research, we considered 1328 frames from 9 pullbacks with 20 mm/s speed and a data frame rate of 100 frames/s.The total number of frames was 1328 and they were divided into bifurcation region (141 frames) and non-bifurcation region (1188 frames).In all the datasets, we observed the presence of plaques and artery dissections.The input images to the proposed method were in DICOM format with dimension 1024 × 1024 pixels in Cartesian coordinates.

Method to segment the vessel lumen
Briefly, the lumen contour from each IVOCT frame is obtained after two processes based on Macedo et al. (2015b): i) preprocessing and ii) lumen contour detection, as shown in Figure 1.The pre-processing stage includes speckle noise removal, catheter and the guide-wire; moreover, a binarization is performed to highlight the intimal layer.The second process includes A-lines sweeping to find high gradients on the first Figure 1.Proposed methodology for automatic obtention of a lumen contour.The proposed methodology receives, as input, each frame from a whole OCT image set and performs pre-processing to highlight the intima layer, followed by morphological operations and the application of edge detection step to obtain the lumen contour as output.Adapted from Macedo et al. (2015b).
vessel wall and a subtraction operation to crop the lumen area.An addition to this method is the lumen contour correction method in the case of bifurcation slices.The bifurcation slices were identified manually by an expert and the correction method was utilized only for this type of slice.

Pre-processing
The objective of this stage is to obtain a binary image with a highlighted intimal layer detailing the recesses at the luminal-intimal border.
For this stage, the input data is a 2D image represented by a Cartesian coordinate system.This image is transformed into a polar coordinate system.Our catheter segmentation technique exploits the fact that the Dragonfly catheter diameter is 2.7 French (F) ~ 0.90 mm and the spatial resolution of the image is known.Given the radius of the catheter sheath region, this region can easily be removed from the image.As the IVOCT image has intrinsic speckle noise, we performed a bilateral filtering (Tomasi and Manduchi, 1998) to smooth the image while preserving the edges.The Otsu method (Otsu, 1979) was used to separate the region with high gradient magnitude, which consists mainly of the intimal layer.More details can be found in Macedo et al. (2015b).
The Guide Wire (GW) is characterized by a bright reflection immediately followed by a shadow.The a priori information used for GW removal is that GW diameter is 0.35 mm; to remove it, an area constraint was applied (only regions with non-significant size area can be removed).

Lumen contour detection
Each A-line is scanned from the bottom to the top of the image, until the inferior border of the intimal layer (intimal-medial border) is found.After this, the region below the intimal layer has value zero and the intimal layer plus lumen region has value one.To segment only the lumen region, a subtraction between the image containing intimal layer plus lumen and the first image containing only the intimal layer is performed.The latter operation is able to preserve recesses on the luminal-intimal border, which can be generated by thrombus or dissections.
To eliminate holes due to the binarization operation and artifact shadows, a sequence of five dilations followed by five erosions was applied; the number of operations is related to the maximum angle a shadow of any artifact can occupy in the image.Since the GW shadow is always approximately 25 degrees, on the first line of the polar image, its identification is performed searching for a horizontal gap of 20-35 pixels on the polar image.If a gap on the first line is identified, all corresponding A-lines are filled from the top to the lumen border.Finally, the lumen border points are detected by a Sobel edge detection method (Sobel, 1990) and the resulting polar image is transformed into an image in Cartesian coordinates.The method explained in Macedo et al. (2015b) used a morphological gradient to detect borders while this current method uses Sobel method.

Automatic contour correction for bifurcation frames
The method described above detects the whole lumen area for each slice, however the lumen area in a slice inside a bifurcation region comprises more than one branch, so we propose an additional automatic approach to delineate only the main branch of a coronary.This approach is performed only for bifurcation frames previously chosen by an expert.
The objective of this approach is to detect the respective center and contour points of the main branch using the Distance Transform (DT).According to Borgefors (1986), the DT is an operation that converts a binary image into a gray-level image where all pixels have a value corresponding to the distance to the nearest feature pixel (in this case of study, a contour pixel).For the DT, given a digital image with a regular lumen contour in Cartesian coordinates, the brightest point corresponds to the center of the lumen contour.However, if the contour is inside a bifurcation region, two situations can happen: (i) the brightest point will be located in the center of the main branch or (ii) there will be a relevant point situated at the center of each branch.DT of a regular contour and contour within a bifurcation frame are shown in Figure 2. If the number of center points is higher than one, the chosen point is the closest point to the catheter (center of the 2D image).Once the center of the main branch is established, the distance between all contour points and the center is calculated and the minimum distance is chosen to correct the vessel border.This minimum distance is used as the radius of an imaginary circle used as a frontier, all contour points inside this circle will be preserved and the contour points outside this circle will be replaced by the corresponding circle points.Figure 3 shows the result of the correction method in a bifurcation region.

Validation experiments
In the present study, one independent expert observer, blinded to automatic segmentation results, performed a manual segmentation of the image sets previously described using a contour tracing process with the ImageJ software (ImageJ, 2015).These manual segmentations were used as reference for the method validation.Each slice was visually classified as being a slice in a Bifurcation Region (BR) or in a Non-Bifurcation Region (NBR).For slices in BR, the manual segmentation prioritized the tracked arterial area.Examples of manual segmentations are shown in Figure 4.For BR slices, we compared results with and without automatic contour correction.
A set of six metrics was used to measure the accuracy of the proposed automatic method for lumen segmentation, considering the manual segmentation as a reference.The measures are: a) Volume overlap error (VOE): (1) b) RMS symmetric surface distance (RMSSSD): (2) c) Dice similarity index (DSI):  where, V m is the volume segmented by the manual method, V a is the volume segmented by the automatic method, A and M are the lumen areas from the automatic segmentation and manual segmentation in square millimeters, respectively, a is a point on the lumen contour from the manual method, m is a point on the lumen contour from the automatic method, N a and N m are the quantity of contour points produced by the automatic and manual methods respectively, N is the number of samples and, finally, TP is the number of true positives (number of pixels correctly identified as being within the lumen contour), TN is the number of true negatives (number of pixels that were correctly identified as being outside the lumen contour), FP is the number of false positives (number of pixels which are outside the lumen contour but were erroneously identified as being inside the lumen contour) and FN is number of false negatives (number of pixels which are inside the lumen contour but were erroneously identified as being outside the lumen contour).

Results
The proposed algorithm was applied to 9 human IVOCT datasets without any user interaction and without the presence of stents.Slices corresponding to the trunk of the coronary were discarded.Slices with residual blood in a significant quantity were also discarded.For NBR frames without automatic correction, the use of the proposed method provided a mean and standard deviation lumen area of 5.6 ± 3.1 mm 2 versus 5.4 ± 3.0 mm 2 from the manual method, resulting in 0.17 mm 2 of absolute difference.The slope produced from linear regression was 1.02 with a y-axis interception 0.07, R 2 =0.998 and p<0.001.In the case of BR IV-OCT frames, the proposed method without automatic lumen correction method provided a mean and standard deviation of lumen area of 6.4 ± 2.5 mm 2 compared with 5.3 ± 2.3 mm 2 for the manual method.Using the automatic lumen correction method for BR IVOCT frames, we obtained a mean and standard deviation lumen area of 5.1 ± 2.2 mm 2 from the proposed method compared with 5.3 ± 2.2 mm 2 for the manual method.Table 1 shows all the evaluation metrics for the proposed automatic method.
On the top of Figure 5 is shown the relative difference between the manual and automatic method for NBR slices according to Bland-Altman statistics, and the respective linear regression graphics are on the bottom of Figure 5.
The manual segmentation of a slice took approximately 60 seconds.On the other hand, the processing time of automatic lumen segmentation for a slice took approximately 15 seconds with our software running on a machine with CPU i7-3.46GHz and 32 GB of RAM.The proposed method was implemented in MATLAB 2013b (MathWorks, Natick, MA). Figure 6 shows examples of the proposed segmentation method for the case of artery dissections, bifurcations and complex plaques.

Discussion
Results in the literature (Sihan et al., 2009(Sihan et al., , 2008;;Tsantis et al., 2012) using 10 to 20 IVOCT datasets, with NBR slices without challenges, reported an absolute difference of mean lumen area of 0.1 mm 2 , whereas our approach obtained an absolute difference of mean lumen area of 0.17 mm 2 using 9 datasets.Tsantis et al. (2012) obtained a MDA of 0.11 ± 0.7 mm 2 , while Ughi et al. (2015), considering the presence of plaques and bifurcations, obtained 0.02 mm 2 of MDA.Our method obtained a MDA of 0.19±0.13mm 2 for NBR slices with plaques and dissections, and 0.52 ± 0.81 mm 2 for BR slices with automatic correction.Dice similarity index was computed in Moraes et al. (2013), Tung et al. (2011), Ughi et al. (2015) and Wang et al. (2010) presenting values between 96% and 97.8%, while our method reached DSI of 97.2% for NBR slices, 88.2% for BR slices without automatic correction and 90.5% for BR slices with automatic correction.Another metric that can be compared is VOE, whose value ranges from 93.8% in Moraes et al. (2013) and 95.4% in Tsantis et al. (2012), whereas our approach computed VOE of 94.7% for NBR slices, and 83.8% and 79.9% for BR slices, with and without lumen correction respectively.In this case, the Hausdorff distance measures gross distortions between two contours.Our method achieved 0.15 ± 0.09 mm of HD without bifurcations while Wang et al. (2010) presented 0.07 ± 0.05 mm, which although being smaller, discarded slices with dissections and with substantial luminal blood in contact with the arterial wall.The metric RMSSSD means the symmetric distance point-to-point between both contours.Tung et al. (2011) achieved 2.44 ± 3.41 mm  99.8 ± 0.1 98.9 ± 0.8 99.2 ± 0.9 of RMSSSD while our method obtained 0.04 ± 0.03 mm for the same metric without NBR slices.Even for BR slices, our results were much better than those in the Tung et al. (2011).Finally, we note that the value of accuracy metric from the confusion matrix was not able to differentiate errors on BR slices without correction because all results of this measure are greater than 98%, so the difference between the three set of tests are not significant.
Our proposed method for NBR slices showed similar results to those presented by previously published methods.The higher values of error for our methods can be explained by our inclusion of slices containing dissections and plaques, whereas other methods discard this type of slices from the statistical analysis.This is shown in Figure 6-f, which presents irregular shape and consequently the segmentation method had more difficulty to find the right border.
As expected, the results for BR slices are worse than for the NBR slices.However, the additional proposed method to correct the lumen contour for BR slices decreases the error (MDA) from 1.2 mm 2 to 0.52 mm 2 , despite of the complexity and subjectivity involved in these regions.One of the reasons is the fact that the correction method cuts the lumen area more than it should.
In the manual segmentation, the expert excludes the side branches by delineating only the main branch.However, our algorithm considers the entire lumen area, which makes for a poor comparison.In order to further reduce the error in BR slices, the stateof-the-art of automatic segmentation methods must be improved.Methods such as Sihan et al. (2009) manually correct the lumen contour for BR slices and Ughi et al. (2015) can automatically discard the side branch area, but this is not always achievable.Methods based on spline fitting can well quantify generically the lumen area; however this approach is not able to identify artery dissections on the lumen border, while the method presented herein can.In terms of limitation, our method was not tested on images with the presence of stents; the strut brightness could impact negatively in our scheme, particularly in the binarization step.This limitation may be solved in the future by developing methods to extract the struts and fill the strut shadows.Although slices with poor luminal blood were tested as shown in Figure 6a, the slices with substantial luminal blood in contact with the arterial wall were discarded for statistical analysis.The performance was improved by 75% in comparison with manual segmentation.However other automatic methods present an improvement of 99%, so in the future we can consider a larger effort to reduce the processing time.The segmentation of the lumen area was performed completely without user interaction, but the additional step to correct contours to limit the main branch area was performed only for manually chosen BR slices.
In summary, considering an artery segment, it is desirable to quantify the entire lumen rather than quantifying only artery stretches between junctions.We presented a fully automated methodology that is able to detect the lumen borders in IVOCT images, even for slices in bifurcation regions and vessel wall with dissections.The automated method was validated using the estimations of expert observers as gold standard.The results suggest that this method can be a useful tool for blood flow analysis and computer-guided diagnosis.An automatic BR classifier as in Macedo et al. (2015a, b) would be an interesting future development since the automatic correction could be performed only for BR slices.

Figure 2 .
Figure 2. Results of the distance transform method in a lumen area: (a) Lumen area in a non-bifurcation region; (b) Lumen area in a bifurcation region.

Figure 3 .
Figure 3. Representation of the automatic lumen correction method: a) A lumen contour in a bifurcation region; b) The result of automatic correction of lumen contour (red line) and the imaginary circle (blue dotted line).

Figure 4 .
Figure 4. Examples of manual segmentation: (a) Frame outside a bifurcation region; (b) Frame inside a bifurcation region with a manual correction prioritizing the main branch.

Figure 5 .
Figure 5.Comparison of automatic and manual segmentation.Bland-Altman graphic (superior) and linear regression (inferior).(a) NBR slices without automatic correction; (b) BR slices without automatic correction; (c) BR slices with automatic correction.

Figure 6 .
Figure 6.Examples of the proposed automatic segmentation (yellow line): (a) Residual blood in the vessel lumen area (arrow); (b) Artery dissection (asterisk); (c) Severe stenosis; (d) Regular slice; (e) Artery dissection (asterisk); (f) Complex plaques with recession (arrow); (g) Complex plaque; (h) Bifurcation with a small side branch (asterisk) and the result of the automatic contour correction (green line); (i) Bifurcation slice with a significant side branch (asterisk) with the result of the automatic contour correction (green line).

Table 1 .
Evaluation metrics of the automatic algorithm using the manual segmentation as reference.