Algorithm for predicting macular dysfunction based on moment invariants classification of the foveal avascular zone in functional retinal images

Introduction: A new method for segmenting and quantifying the macular area based on morphological alternating sequential filtering (ASF) is proposed. Previous studies show that persons with diabetes present alterations in the foveal avascular zone (FAZ) prior to the appearance of retinopathy. Thus, a proper characterization of FAZ using a method of automatic classification and prediction is a supportive and complementary tool for medical evaluation of the macular region, and may be useful for possible early treatment of eye diseases in persons without diabetic retinopathy. Methods: We obtained high-resolution retinal images using a non-invasive functional imaging system called Retinal Function Imager to generate a series of combined capillary perfusion maps. We filtered sequentially the macular images to reduce the complexity by ASF. Then we segmented the FAZ using watershed transform from an automatic selection of markers. Using Hu’s moment invariants as a descriptor, we can automatically classify and categorize each FAZ. Results: The FAZ differences between non-diabetic volunteers and diabetic subjects were automatically distinguished by the proposed system with an accuracy of 81%. Conclusion: This is an innovative method to classify FAZ using a fully automatic algorithm for segmentation (based on morphological operators) and for the classification (based on descriptor formed by Hu’s moments) despite the presence of edema or other structures. This is an alternative tool for eye exams, which may contribute to the analysis and evaluation of FAZ morphology, promoting the prevention of macular impairment in diabetics without retinopathy.


Introduction
The retina is the only place in the body where blood vessels can be viewed noninvasively in vivo.Technological evolution has been leading to the development of digital imaging systems over the last two decades, revolutionizing the ability to take images of the ocular fundus.Essentially, modern digital imaging systems provide images with sufficient resolution for most clinical scenarios.
We described the reliability of the digital assessment of perifoveal capillary networks as an indicator of macular perfusion (Faria et al., 2014).This knowledge can be applied to various retinal conditions, such as diabetic macular edema or ischemia and age-related macular degeneration.Moreover, the assessment of foveal perfusion map reflects the microcirculation status in the brain, heart, and kidney (Blonde, 2012;Flammer et al., 2013;Klein et al., 2006;McGeechan et al., 2009;Sun et al., 2009;Taylor et al., 2007).
Foveal avascular zone (FAZ) is the central structure of the macula and it is important for the macular function.Many previous studies in 1970s correlated macular ischemia and loss of visual acuity based on the analysis of angiographies (Bresnick et al., 1975;1976;Kohner and Henkind, 1970).More recent studies have demonstrated that, in diabetics, extensive ischemia of the macular region enlarges the FAZ and this is the main feature of maculopathy, in which profound and irreversible visual loss may occur (Arend et al., 1995;Sim et al., 2013).Particularly, in cases of diabetic maculopathy, the extent of ischemia is masked by coexisting macular edema.
A recent research described features of optical coherence tomography that also associated the macular ischemia with impaired visual function (Sim et al., 2014).
In 2015, we developed a new method for automatic segmentation of the FAZ using watershed transform (Silva et al., 2015) applied to the original image from two markers: the first defined by the global minimum contour of the alternating sequential filtering (ASF) image (filter to reduce the complexity of the original intensities), and the second defined by an inscribed circle of the first marker.The growing regions procedure derived from these two markers resulted in two areas precisely bounded by the FAZ, with an island of relatively darker tones around them.The aim of the present work was to demonstrate the reliability of this algorithm in the automatic segmentation of FAZ.In addition, we defined a descriptor for its characterization and classification in order to detect abnormality in macula image.For this task, we added a set of eye fundus images from diabetics to the database, and implemented a FAZ descriptor based on Hu's moments.We established the distinction of non-diabetic or diabetic pattern (without retinopathy) after training performed by a classifier, highlighting extra-trees compared to tested k-nearest neighbourhood (KNN), random forests, support vector machines and neural networks.

Study population
We invited persons with type 1 diabetes at the hospital of the University of Campinas to participate in the study.The criteria for inclusion in the study were a history of sudden onset of severe hyperglycemia and weight loss, spontaneous sustained ketosis or ketonuria, age between 18 to 45 years, living with diabetes mellitus (to allow enough time for the development of diabetic retinal disease), and freedom from any other endocrine and metabolic disease, as well as nondiabetic renal, liver and cardiac disease.In this study, we included 10 persons with diabetes for at least 10 years and without retinopathy, i.e., early treatment diabetic retinopathy study level 1 (Klein et al., 1984).This was confirmed by examination by indirect ophthalmoscopy and biomicroscopy.The exclusion criteria were pregnancy, high degree myopia, chorioretinitis scars, posterior uveitis, glaucoma, and previous ocular surgery, because such events may influence the development of diabetic retinopathy (Conrath et al., 2005).We invited 10 staff members from the hospital of the University of Campinas to participate as control group, with a mean age of 30 years (range 25-35).All volunteers presented at the time of evaluation best corrected visual acuity of 20/20 in both eyes.Therefore, it is very likely that ischemic changes present in macular area were very mild.This was important because there is a strong correlation between visual acuity and macular ischemia (Samara et al., 2017).Our research was carried out in conformity with the tenets of the "Declaration of Helsinki" and approved by the local Ethics Committee (approval number 745/2011).The informed consent was obtained from participants prior to conducting the ophthalmic examination.

Image acquisition
We described the image acquisition in previous work (Faria et al., 2014).Briefly, we acquired retinal images using a Retinal Function Imager (RFI) that applies fundus illumination with predominantly green component to highlight blood vessels (Grinvald et al., 2004;Izhaky et al., 2009).We acquired each image using a 20-degree aperture camera, focused on the fovea and with a resolution of 4.27 micron/pixel.After image recording, we can create "differential images" by dividing each single retinal image by the average of the entire series in order to extract the motion signal from the stationary retinal background.In the present work, four to five image series of 1024×1024 pixels per frame were used from each volunteer to get a capillary perfusion map (CPM) by RFI.

Automatic segmentation of foveal avascular zone
After acquisition of the retinal images via RFI, we can apply a sequence of image processing operators, based on dilation and erosion (Dougherty and Lotufo, 2003;Silva et al., 2015).Dilation and erosion of a pixel (p) are, respectively, the maximum and the minimum intensities determined on its neighborhood.Morphological opening of p is achieved by erosion followed by dilation to eliminate small noises as narrow isthmuses or thin protrusions.In contrast, we can achieve morphological closing by dilation followed by erosion (dual of opening) to fill crevices in the borders, thus joining the adjacent regions.ASF consists in an iterative morphological openings and closings applied on an increasing neighborhood (disk-shaped in our case) for each p (Dougherty and Lotufo, 2003;Silva et al., 2015).These authors used ASF typically as a restoration procedure to eliminate noise and mixed granularity.In this work, we applied ASF to simplify the images, enhancing the separation of regions where the intensities are slightly greater compared to surrounding areas, i.e., with high probability of occurrence of FAZ.For segmentation, they applied ASFs for n iterations on a combined CPM series.Increasing disks around each pixel p for 1, 4, we can consider 10 iterations for analyses as shown in Figure 1.
Figure 2 shows each step of FAZ segmentation.Figure 2a represents the result of 10 ASF iterations from Figure 1a. Figure 2b represents the FAZ region  by a 3D graph of Figure 2a with the intensity of a pixel representing an altitude on the topographic surface.The aim is to approximate the intensities of the darker regions of lower values or even to remove them, defining the global minima or regions completely surrounded by the other lighter regions (dark islands).The binary image generated in Figure 2c represents this lower region and works as a marker over the FAZ.FAZ segmentation was refined using the watershed transform (Silva et al., 2015), which corresponds to the boundaries of adjacent catchment basins formed by the accumulation of water over regional minima or regions with markers.For automatic FAZ segmentation, we define inner and outer markers (seeds labeled as internal or external for growth of regions) as a binary image (Figure 2d) and we obtain the area between both limits (Figure 2e) by watershed transform.As a result, the automatic FAZ segmentation can be obtained (Figure 2f) from a combined CPM series of high-resolution retinal image.The segmentation obtained from the proposed algorithm achieved an accuracy of 99% compared to an average of three manual segmentations.We described details of this implementation in recent publication (Silva et al., 2015).

Automatic classification of retinal images
Next, we processed each FAZ segmented region for extraction of features.Specifically, Hu's moments are interesting by the invariance to translation, rotation and scale.Seven invariant moments are described using the normalized central moments of second-order and third order (Hu, 1962).Firstly, an object represented by a binary digital image f(x,y) is considered.Then, f(x,y)=1 is associated to a coordinate (x,y) belonging to the object and f(x,y)=0 to a background pixel (x and y are line and column values, respectively).Thus, x and y are the averages of the lines and columns, respectively, of all pixels belonging to the object, ( ) x y is the centroid or mass center.We compute the spatial moments by Equation 1 and use Equation 2to calculate the invariant central moments with respect to translations.

( )
Invariants η ij with respect to both translation and scale can be defined from central moments (Equation 2) by dividing through a properly scaled zero-th central moment, according to Equation 3, considering i + j ≥ 2.
Finally, Equations 4 to 10 describe the seven Hu's moments, invariants with respect to translation, scale and rotation.3 3 4 We calculated and depicted these seven values for the FAZ region of non-diabetic and diabetic persons in the multidimensional space (up to seven dimensions) for machine training algorithms.After tests using several classifier algorithms, we chose extra-trees (Geurts et al., 2006) for presenting one of the best accuracies and area under the receiver operating characteristic (ROC) curve (according to Table 2).The idea is to use all of Hu's moments to compose a descriptor in a multidimensional space and to automatically separate the samples into two classes.We evaluate the descriptor of an unclassified FAZ based on the determination of the smallest distances in relation to all points already labeled in this space.From the set of Hu's moment invariants as descriptor, the classifier provides the automatic distinction of an unknown FAZ in two possible classes: non-diabetic and diabetic.
We partitioned the dataset randomly into k f mutually exclusive subsets (folds) that contained a representative number of each class.We retained of the k f parts, a single subset as the validation data for testing the model, and used the remaining k f -1 as training data.We expressed the evaluation statistics of the k f -fold-cross-validation results, with each of the k f subsets used exactly once as the validation data (Kohavi, 1995), as 'mean' ± 'standard deviation'.In other words, we obtained a single estimation from the average and standard deviation of k f results for each subject (non-diabetic or diabetic).We adopted two values for k f : k f =38 (total of samples) or leave-one-out-validation, and k f =5 or 5-fold-cross-validation.

Results
We chose for the automatic classification 38 images from all the 20 individuals.We calculated Hu's moment invariants from 10 non-diabetic (20 images) and 10 diabetic (18 images) volunteers for all pixels of FAZ region and alternatively for pixels belonging only to the contour of FAZ region.We estimated the quality of the detection of abnormality in the FAZ through the accuracy (sum of true positive and true negative, divided by the sum of true positive, true negative, false positive and false negative classifications) from leave-one-out and 5-fold-cross-validation.We also considered the area under the ROC curve (AUC) from 5-fold-cross-validation.In addition, we took into account the results of the multidimensional classifier expressed as the average and standard deviation for each volunteer (non-diabetic or diabetic).
Tables 1 and 2 show the performance of the tested classifiers at detecting the abnormality of each FAZ segmentation.We compared the following classifiers: k-nearest neighbourhood (Shakhnarovich et al., 2006) with two odd values of parameter k, random forests (Breiman, 2001), extra-trees (Geurts et al., 2006), linear support vector machines (Wu et al., 2004), and multi-layer perceptron neural network (MLP) (Kingma and Ba, 2015).We used all seven moments of Hu as descriptor in Table 1.In Table 2, we use only the moments I 2 , I 4 , and I 7 because they are the most representative to serve as descriptor, reaching the best accuracy from all combinations of reduction of the moment invariants.We obtained the most outstanding set of results with extra-trees, in which the automatic classification of a new fundus image classification was correct with about 81% accuracy (using both leave-one-out and 5-fold-cross validations) and 89% AUC, considering only moments I 2 , I 4 and I 7 as descriptors of FAZ contours (Figure 3).This result is quite significant considering the relatively small number of segmented images in the database and the great difficulty of distinguishing such classes even by an expert.
Table 1.Accuracy and area under the receiver operating characteristic curve (AUC) of multidimensional classifiers for automatic identification of the type of subject (non-diabetic or diabetic), using all seven moments of Hu as descriptor of the segmented regions of each foveal avascular zone or contours of these regions.In this table, k is the number of nearest neighbors of k-nearest neighborhood (KNN) algorithm.MLP is the multi-layer perceptron neural network.SVM means support vector machine network.The best results by type of experiment and validation are highlighted in bold.

Accuracy leave-one-out validation
Accuracy

Accuracy leave-one-out validation
Accuracy

Accuracy leave-one-out validation
Accuracy

Accuracy leave-one-out validation
Accuracy

Discussion
There are few works found in the literature on retina classification in diabetes based on FAZ shape analysis.According to Table 3, all related studies (Alipour et al., 2012;Ballerini, 1999a;1999b;Hani et al., 2010) used subjects with diabetic retinopathy.Alipour et al. (2012) and Ballerini (1999a;1999b) used images obtained by an invasive method of contrast injection.In relation to the segmentation, selections of initial point or image cropping were carried out in some cases (Ballerini, 1999a;1999b;Hani et al., 2010).Area (Alipour et al., 2012;Hani et al., 2010) or distinct formulation of moments (Ballerini, 1999a;1999b) were used as features of each FAZ.As there is no similarity in the image databases used, it is not possible to compare the results objectively.However, we added notes about validation in the last column of Table 3.
In this study, we propose a new method for automatic FAZ classification using the CPM tool of an RFI apparatus based on an algorithm of morphological operator.Using a high-resolution retinal image CPM tool, we initiated the segmentation with ASF, followed by the definition of global minima and watershed, thus generating a simpler image.This process enabled us to automatically determine the statistics of each FAZ.The algorithm applied in this technology is able to automatically identify correctly in 81% of the cases whether a certain FAZ is or is not abnormal, with confirmation of the diagnosis of diabetes from the clinical chart.To the best of our knowledge, the main contributions of this work are: simple and direct automatic segmentation of FAZ without vessel identification, initial selection of point or cropping of image; use of a subset of Hu's moment invariants as descriptor and multiple classifiers (especially extra-trees) that allowed satisfactory results (>70%) in almost all tests.The proposed technique may be useful in the early prognosis of retinopathy, and may signal de need of ophtalmological care by telemedicine systems, especially in regions with insufficient medical care.
Overall, in this paper, we described a novel computational algorithm to automatically evaluate the retina macular region.Future works can be conducted with a larger diabetic population to improve the training database of this work.The predictive results of this  algorithm can also be applied to track the progression of retinopathy in diabetic patients.

Figure 1 .
Figure 1.Example of n iterations of alternating sequential filtering (ASF) applied to a combined capillary perfusion map (CPM) series using disk-shaped neighborhood on each pixel p.(a) Original image after application of the CPM tool; (b) Only one iteration (n=1) of ASF on the previous image; (c) Four sequential iterations (n=4) of ASF on the same initial image generating a simpler image, but still several catchment basins remained; (d) More iterations (n=10) resulted in a very simple image with only one global minimum in the center.

Figure 2 .
Figure 2. Foveal avascular zone (FAZ) localization and segmentation.(a) Alternating sequential filtering (ASF) image (n=10); (b) 3D representation of topographical surface of this image in greyscale as a relief altitude; (c) Region determined from the global minima of the image in (b), used as automatic internal marker of FAZ; (d) Circle with 75% of the maximum inscribed circle radius, and contour of Figure 2c; (e) FAZ segmentation after watershed transform applied on the original capillary perfusion map (CPM) image of Figure 1a using connected components of Figure 2d as markers; (f) The segmentation border was drawn on the original CPM image.

Figure 3 .
Figure 3. Receiver operating characteristic with area under the curve (AUC) of 0.89±0.11from extra-trees classifier for 5-fold-cross-validation, using moments I 2 , I 4 and I 7 as descriptor of each automatically segmented foveal avascular zone (FAZ) contour.

Table 2 .
Accuracy and area under the receiver operating characteristic curve (AUC) of multidimensional classifiers for automatic identification of the type of subject (non-diabetic or diabetic), using moments I 2 , I 4 and I 7 as descriptor of the segmented regions of each foveal avascular zone or contours of these regions.The best result by type of experiment and validation is highlighted in bold.In this table, k is the number of nearest neighbors of k-nearest neighborhood (KNN) algorithm.MLP is the multi-layer perceptron neural network.SVM means support vector machine network.

Table 3 .
Comparison of related works found in the literature for retina classification, with or without diabetic retinopathy (DR), based on foveal avascular zone (FAZ) segmentation, characterization and classification.