Mature pomegranate recognition methods in natural environments using machine vision

The use of machine vision to recognize mature pomegranates in natural environments is of major significance in improving the applicability and work efficiency of picking robots. By analyzing the color characteristics of color images of mature pomegranates under different illumination conditions, the feasibility of the YCbCr color model for pomegranate image recognition under different illumination conditions was proven. First, the Cr component map of pomegranate image is selected and then the pomegranate fruit is segmented by the kernel fuzzy C-means clustering algorithm to obtain the pomegranate image. Contrast experiments of pomegranate image segmentation under different illumination conditions were then performed using the proposed kernel fuzzy C-means clustering algorithm, the fuzzy C-means clustering algorithm, the Otsu algorithm and the threshold segmentation algorithm. Results of the experiments verified the effectiveness and superiority of the proposed algorithm.


INTRODUCTION
With rising labor costs, rapid development of intelligent control technology and information technology, and increasing processing speed of computers, studies on fruit and vegetable picking robots have attracted more and more attention. Among them, the picking robots based on machine vision have become a research focal point in both domestic and foreign agriculture engineering fields (WANG et al, 2015;ZHAO et al,2016;BULANON & KATAOKA T, 2010). The prerequisite for the proper operation of any picking robot is to achieve the correct identification and the position of the fruit. In order to correctly identify the fruit, the key is to separate the fruit from complex backgrounds such as those containing branches and leaves under complex lighting conditions. Therefore, determination of the exact spatial location of the fruit is necessary for the robot to complete fruit picking.
In order to improve the accuracy of fruit recognition, domestic and foreign scholars have carried out a significant volume of research. In the literature , the I component map of the nighttime litchi image was selected and the Otus algorithm was used to segment the image and remove the background. The fuzzy C-means (FCM) clustering algorithm was used to obtain the nighttime litchi fruit image. One study (JI et al. 2016) proposed a Retinex algorithm based on a guided filter for nighttime apple image enhancement and this achieved certain results. Another study (JIA et al., 2015) employed the particle swarm optimization algorithm-based independent component analysis and

RURAL ENGINEERING
night noise reduction algorithm to achieve nighttime recognition for an apple picking robot.
Based on the color information in the images, the localization and recognition system of grapes in the natural environment was studied (REIS et al., 2012). XU et al. (2005) studied the citrus recognition method based on color information. XIONG et al. (2011), identified lychee fruits and fruit stalks by selecting the Cr component map in the YCbCr color model of the lychee image, combined with one-dimensional random signal histogram and fuzzy C-means clustering algorithm. The above studies provided a good theoretical basis for the operation of picking robots. However, the above studies were more focused on daytime front-lighting and backlighting conditions, with little or no involvement in nighttime and daytime insufficient illumination conditions. While more attention has been paid to the picking of apples, lychees, grapes, citrus and other fruits, the pomegranate has hardly been studied.
As a proven unsupervised clustering analysis method, the FCM clustering algorithm can retain more information from the original image. Therefore, it has been widely used in areas such as medical image processing (ZENG et al., 2008), pattern classification (YANG & HUANG, 2015), and fruit image segmentation XIONG et al., 2011;BETANZOSA et al., 2000). However, the standard FCM clustering algorithm does not fully consider the spatial information of the pixels and; is therefore, sensitive to the noise information in the image (CHUANG et al., 2006).
In order to improve the robustness of the FCM clustering algorithm, many researchers have introduced the kernel method into various clustering algorithms (YU et al., 2015;KANG et al., 2010;MA et al., 2007;GONG et al., 2013;ZHANG & LI, 2017;LEI & OUYANG, 2018;CHAIRA & RAY, 2005). The fundamental concept is to map each sample to the high-dimensional kernel space by using nonlinear mapping. The nonlinear problem in the input space is linearized in the high-dimensional feature space (BETANZOSA et al., 2000), thus improving the performance of the clustering algorithms.
In this paper, we selected mature pomegranates as a subject and analyze the RGB, LAB, and YCbCr color models of pomegranate images under different illumination conditions. By selecting the Cr component map in YCbCr color model, the improved kernel fuzzy C-means clustering algorithm (KFCM) is adopted to recognize pomegranate images under sufficient daytime illumination, insufficient daytime illumination, and nighttime conditions.

Image acquisition and color feature analysis Color image acquisition
This paper used mature pomegranates as research subjects and studied the recognition of mature pomegranates in the natural environment. The image collection times were from 10:00 to 11:00 in the morning, from 18:30 to 19:30 in the afternoon and from 20:00 to 21:00 in the evening on September 6 to 12, 2018. More than 400 images were collected, including 212 from 9:00 to 11:00, 120 from 18:00 to 19:30 and 93 from 20:00 to 21:30. The machine vision system included a CCD camera, an image acquisition card, and a computer. The DH-HV3100FC digital camera was used, the image acquisition card was a DH-VT123, the resolution of acquired visible light image was 1080 ×1440 pixels, and the format was 24-bit true color image in JPG format. The image was adjusted to 256 × 256 pixels before applying the algorithms. Experimental platform: The computer used was the Intel(R) Core(TM) i5-3230M CPU @2.60 GHz, configured with 4 GB memory. The operating system was Windows 10, and the programming language was MATLAB (2014R).

Color feature analysis
Comparing figure 1(a), figure 1(b) and figure 1(c), it can be seen that the color of the pomegranate fruits in the pomegranate images is relatively vivid; however, there are a lot of dark areas in the image background, and the clarity of the whole image is not high. The edges are relatively blurred. When the daytime illumination is insufficient, the dark areas in the background are less than the nighttime; however, the color of the fruit is not as clear as that observed at nighttime. There are shadows, and the edges are also blurred. During the shooting in natural conditions, the background of the acquired images is complex and variable, making it difficult to accurately extract the pomegranate fruit.
In this paper, various color models (RGB, LAB, YCbCr, YIQ, HSV) were used to analyze the color features of pomegranate images collected under natural conditions. Figure 2 shows  In the same way, the 15 color components analysis was performed on the pomegranate images taken when the daytime illumination was insufficient. It was reported that in the images of the A component, the Cr component, and the H component there was a relatively large contrast between the fruit and the background. These three components can be selected to analyze the pomegranate images with insufficient daytime illumination.
Under night conditions, 15 analyzes of color components were also performed on pomegranate images . It was reported that the fruit and the background had relatively large contrast in the images of the A component, the Cr component, the I component, and the H component. Consequently, these four components can be selected to analyze the pomegranate image under nighttime flash conditions.
In summary, the Cr component image and the A component image provided better suppression of the effects of the sky background and uneven light in the analysis of the color characteristics of the pomegranate images with sufficient daytime illumination, insufficient daytime illumination and nighttime conditions. From the real-time consideration of the algorithm, the conversion of the RGB color model to the Lab color model is nonlinear while the conversion of the RGB color model to the YCbCr model is linear. Hence, the YCbCr color model was finally selected for pomegranate image segmentation. The conversion formula from the RGB color model to the YCbCr model is (1)

Kernel fuzzy C-means clustering algorithm Fuzzy C-means clustering algorithm
The classical FCM clustering algorithm iteratively optimizes the objective function based on the weighted similarity measure of image pixels and the cluster centers to obtain the minimum value of the objective function and determine the optimal cluster. The objective function of the FCM algorithm is usually expressed as follows: (2) where m is the weighting factor (Fuzzy coefficient). Note that . In this paper, m=2; u ij indicates the membership degree of the i-th sample belonging to the j-th class, , ; X i indicates the i-th sample, V j indicates the jth clustering center, || x i -v j || is the distance norm between the sample X i and the cluster center V j . Euclidean distance is used in this paper. The iterative formulas for the fuzzy membership degree U ij and the cluster center V j can be obtained by using the Lagrangian conditional extreme value optimization theory.

Kernel fuzzy C-means clustering algorithm
Using the kernel method, the objective function of KFCM is (5) Defined by the Mercer core (6) (6) Using the Gaussian kernel function, we have K (x, x)=1. Then, equation (6) can be rewritten as (7) Apply (7) to (5), the objective function of the KFCM algorithm can be obtained as (8) By finding the optimal solution to equation (8), we can derive the iterative formulae for obtaining the membership degree and the cluster center (9) (10) The corresponding iterative calculation of the KFCM algorithm is described below.
Step 1: Based on prior knowledge, set the iteration stop threshold e, the maximum number of iterations t max , initialize the membership degree matrix U and the iteration counter t = 0; Step 2: Calculate the fuzzy cluster center using formula (10); Step 3: Calculate the new membership degree matrix using equation (9); Step 4: If or v t = v t+1 t > t max , the iteration ends, output U and V; otherwise t = t + 1, return to Step 2 and continue with the iteration.

RESULTS AND DISCUSSION
To verify the accuracy of the proposed kernel fuzzy C-means clustering algorithm for pomegranate image recognition, image recognition experiments for daytime and nighttime conditions were conducted. Subsequently, 150 sufficient daytime illumination, 100 insufficient daytime illumination, and 80 nighttime LED illumination images were randomly selected (Table 1).

Recognition test for a pomegranate fruit
The fruit identified by the algorithm in the study was a pomegranate. The test results are shown in table 1. The standard for the identification of pomegranate fruit images is for more than 90% of all pomegranate fruit images to be identified correctly.
The test results in table 1 show that the algorithm can correctly identify mature pomegranates in a natural environment. Using the same algorithm to identify pomegranate under various lighting conditions leads to different results. The recognition capability is better at night than during the day. The recognition capability is better with sufficient daytime illumination than with a lack of illumination. The reasons are sufficient daytime illumination, insufficient illumination, among others. The fruit under the backlight is black, which is close to the background color, affecting the accuracy of the pomegranate image recognition. Under night conditions, the illuminated mature pomegranate can be clearly distinguished from the background, and there is no backlight acquisition image, which effectively improves the success rate of the recognition. However, under night conditions, an external light source in the background of the image would cause an identification error for the pomegranate image.

Analysis of algorithms
For comparison, the threshold segmentation algorithm, the Otsu algorithm, and the FCM clustering algorithm were compared with the KFCM clustering algorithm proposed in this paper, and the segmentation effect diagram is shown in figure 3, 4, 5.
It can be seen from figures 3, 4, 5 that the FCM algorithm cannot segment the pomegranate image under insufficient daytime illumination while   It can be seen from table 2 that for pomegranate images with sufficient daytime illumination and nighttime conditions, the FCM and Otsu algorithms have the longest running time, the threshold segmentation method was faster, and the KFCM algorithm was the fastest of the four algorithms. The KFCM algorithm takes slightly

CONCLUSION
In order to quickly and accurately identify fruit images used to guide the operation of fruit picking robots, this paper first converts the color image of pomegranate from RGB color model to YCbCr model, extracts the Cr component in YCbCr model and applies it to image recognition. Secondly, based on the FCM algorithm, the kernel function is introduced and the KFCM clustering algorithm is applied to the segmentation of pomegranate images. The recognition experiments for pomegranate images under various illumination conditions are performed using this algorithm. Results demonstrated that the recognition accuracy is 90.3%. Results also showd that the algorithm achieves a good real-time performance in terms of pomegranate image recognition. The average segmentation of a pomegranate image takes 0.048 s in sufficient daytime, 0.187 s in relatively insufficient daytime, and 0.086 s in nighttime.

DECLARATION OF CONFLICT OF INTERESTS
The authors declare no conflict of interest. The founding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, and in the decision to publish the results.