Acessibilidade / Reportar erro

Extraction of navigation line based on improved grayscale factor in corn field

Obtenção de linhas de tráfego em lavoura de milho baseada no fator de escala de cinza melhorado

ABSTRACT:

Focusing on the problem that corn plant in different growth periods is grayed out by known methods, the gray scale difference of different part is large or the soil discrimination degree is not high, the navigation path is low in accuracy and speed. This paper proposed a new method for extracting cornfield navigation baselines, which is used to control walking of agricultural robots. Design method included image segmentation, navigation point extraction, and navigation path fitting. Image segmentation is based on a new grayscale factor combined with median filtering, OSTU method and morphological operations to achieve the separation of crops and soil. The extraction of the navigation point is based on the binary image vertical projection map to obtain the region of interest, and the navigation point coordinates are determined by calculating the relative center point of the white pixel points of the sampling line in the region of interest. The Hough transform is used to fit the navigation point obtained by the vertical projection map to determine the navigation path, and then the control parameters are obtained. The gray scale factor that is improved in this paper combined with the vertical projection map can extract the target ridge with an accuracy rate of 92%, and the accuracy of extracting the navigation line is more than 90%. When conducting navigation tracking experiments in corn field, the maximum error is 5cm.

Key words:
agricultural robots; improved grayscale factor; navigation baselines; vertical projection; hough transform

RESUMO:

Depois de usar o método conhecido como escala de cinza para plantas de milho, a diferença de escala de cinza entre diferentes partes da planta é grande ou a diferenciação do solo não é alta, a precisão do trajeto de navegação é baixa e a velocidade é lenta. Neste trabalho foi proposto um novo método de extração da linha de referência de navegação para campos de milho, que é usado para controlar a caminhada de robôs agrícolas. Os métodos de projeto incluem: segmentação de imagem, extração de ponto de navegação e encaixe de linha de navegação. A segmentação por imagem é baseada na separação de culturas e solos com base no novo fator de escala de cinza combinado com a filtragem mediana, método de Otsu e operação morfológica. A extração de pontos de navegação é baseada em um mapa de projeção vertical de imagem de dois valores para obter a área de interesse, e as coordenadas do ponto de navegação são determinadas por cálculo do ponto central relativo no pixel branco da linha amostral na área de interesse. O fator de escala de cinza melhorado nesta pesquisa irá extrair o centro da entre linha de plantio alvo com uma taxa de precisão de 92%. A precisão de extração da linha de navegação é mais de 90%. O erro máximo foi de 5cm quando o experimento de rastreamento de navegação em tempo foi é realizado em campos de milho.

Palavras-chave:
robôs agrícolas; fator de escala de cinza; linha de navegação; projeção vertical; transformação de Hough

INTRODUCTION:

In recent years, with the development of a series of high-tech technologies such as modern information technology, biotechnology, and engineering technology, precision agriculture has rapidly developed and popularized. Automatic navigation of agricultural machinery is the core technology of precision agriculture (HU et al., 2015HU, J.; GAO, L.; BAI, X.; LI, T. et al. Review of research on automatic guidance of agricultural vehicles. Transactions of the Chinese Society of Agricultural Engineering, 31, n.10, p.1-10, 2015. Available from: <Available from: https://www.ingentaconnect.com/content/tcsae/tcsae/2015/00000031/00000010/art00001 >. Accessed: Oct. 15, 2018. doi: 10.11975/j.issn.1002-6819.2015.10.001.
https://www.ingentaconnect.com/content/t...
), which has been widely used in agricultural production activities such as farm furrows, pesticide spraying, and harvesting (ÅSTRAND et al., 2002ÅSTRAND, B.; BAERVELDT, A. J. An Agricultural Mobile Robot with Vision-Based Perception for Mechanical Weed Control. Autonomous Robots, 13, n.1, p.21-35, 2002. Available from: <Available from: https://link_springer.gg363.site/article/10.1023/A:1015674004201 >. Accessed: Oct. 09, 2018. doi: 10.1023/A:1015674004201.
https://link_springer.gg363.site/article...
; JI et al., 2011JI, R.; QI, L. Crop-row detection algorithm based on Random Hough Transformation. Mathematical & Computer Modelling, 54, n.3, p.1016-1020, 2011. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S08957177100052 >. Accessed: Oct. 25, 2018. doi: 10.1016/j.mcm.2010.11.030.
https://www.sciencedirect.com/science/ar...
; WU et al., 2019WU, J.; DENG, M.; FU, L.; MIAO, J., 2019, Cham. Vanishing Point Conducted Diffusion for Crop Rows Detection. Springer International Publishing. p.404-416. Available from: <Available from: https://link_springer.gg363.site/chapter/10.1007/978-3-030-02804-6_54 >. Accessed: Oct. 19, 2018. doi: 10.1007/978-3-030-02804-6_54.
https://link_springer.gg363.site/chapter...
). Machine vision, as the main research content of automatic navigation of agricultural robots, is widely used in the detection of crop lines in various crops. For example, wheat (ZHAO et al., 2016ZHAO, T.; NOGUCHI, N.; YANG, L.; ISHII, K. et al. Fast edge detection method for wheat field based on visual recognition. 2016. 32-37p. Available from: <Available from: http://xueshu.baidu.com/usercenter/paper/show?paperid=c72d212b891efcc6fa561d632a467a07&site=xueshu_se&hitarticle=1&sc_from=gxu >. Accessed: Sep. 09, 2018. doi: 10.6041/j.issn.1000-1298.2016.11.005.
http://xueshu.baidu.com/usercenter/paper...
), soybeans (KISE et al., 2005KISE, M.; ZHANG, Q.; MáS, F. R. A Stereovision-based Crop Row Detection Method for Tractor-automated Guidance. Biosystems Engineering, 90, n. 4, p. 357-367, 2005. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S1537511004002260 >. Accessed: Nov. 25, 2018. doi: 10.1016/j.biosystemseng.2004.12.008.
https://www.sciencedirect.com/science/ar...
), cucumbers (REN et al., 2009REN,Y.;TAN,Y.;YANG,H. et al. Visua lnavigation of cucumber picking robot based on fuzzy control, Journal of Jiangsu University(Natural Science Edition), 30, n.7, p.343-346, 2009. Available from: <Available from: http://www.cnki.com.cn/Article/CJFDTotal-JSLG200904007.htm >. Accessed: Oct. 15, 2018.
http://www.cnki.com.cn/Article/CJFDTotal...
), potatoes (GARCíA-SANTILLáN et al., 2018GARCíA-SANTILLáN, I.; PELUFFO-ORDOñEZ, D.; CARANQUI, V.; PUSDá, M. et al., 2018b, Computer Vision-Based Method for Automatic Detection of Crop Rows in Potato Fields. Springer. p.355-366. Available from: <Available from: https://link_springer.gg363.site/chapter/10.1007/978-3-319-73450-7_34#citeas >. Accessed: Oct. 18, 2018. doi:10.1007/978-3-319-73450-7_34.
https://link_springer.gg363.site/chapter...
), corn (SURIYAKOON et al., 2017; ZHANG et al., 2018ZHANG, X.; LI, X.; ZHANG, B.; ZHOU, J. et al. Automated robust crop-row detection in maize fields based on position clustering algorithm and shortest path method. Computers and Electronics in Agriculture, 154, p.165-175, 2018. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S0168169918306586 >. Accessed: Sep.09, 2018. doi: 10.1016/j.compag.2018.09.014.
https://www.sciencedirect.com/science/ar...
; SONG et al., 2017SONG, Y.; LIU, Y.; LIU, L.; ZHU, D. et al. Extraction Method of Navigation Baseline of Corn Roots Based on Machine Vision. Transactions of the Chinese Society for Agricultural Machinery, 48, n.2, p.38-44, 2017. Available from: <Available from: http://xueshu.baidu.com/usercenter/paper/show?paperid=ba7eeb1b6b8c9f5dabfa4d71d4dbf624&site=xueshu_se&hitarticle=1&sc_from=gxu >. Accessed: Dec. 05, 2018. doi: 10.6041/j.issn.1000-1298.2017.02.005.
http://xueshu.baidu.com/usercenter/paper...
; DIAO et al., 2015DIAO, Z.; ZHAO, M.; SONG, Y.; WU, B. et al. Crop line recognition algorithm and realization in precision pesticide system based on machine vision. Transactions of the Chinese Society of Agricultural Engineering, 31, n.7, p.47-52, 2015. Available from: <Available from: https://www.ingentaconnect.com/content/tcsae/tcsae/2015/00000031/00000007/art00007 >. Accessed: Oct. 16, 2018.
https://www.ingentaconnect.com/content/t...
; MONTALVO et al., 2012MONTALVO, M.; PAJARES, G.; GUERRERO, J. M.; ROMEO, J. et al. Automatic detection of crop rows in maize fields with high weeds pressure. Expert Systems with Applications, 39, n.15, p.11889-11897, 2012. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S0957417412003806 >. Accessed: Dec. 25, 2018. doi: 10.1016/j.eswa.2012.02.117.
https://www.sciencedirect.com/science/ar...
; JIANG et al., 2015JIANG, G.; WANG, Z.; LIU, H. Automatic detection of crop rows based on multi-ROIs. Expert Systems with Applications, 42, n.5, p.2429-2441, 2015. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S0957417414006575#f0005 >. Accessed: Oct. 05, 2018. doi: 10.1016/j.eswa.2014.10.033.
https://www.sciencedirect.com/science/ar...
; GUIJARRO et al., 2011GUIJARRO, M.; PAJARES, G.; RIOMOROS, I.; HERRERA, P. J. et al. Automatic segmentation of relevant textures in agricultural images. Computers & Electronics in Agriculture, 75, n.1, p.75-83, 2011. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S0168169910001924 >. Accessed: Nov. 05, 2018. doi: 10.1016/j.compag.2010.09.013.
https://www.sciencedirect.com/science/ar...
), cotton (ZHAI et al., 2016ZHAI, Z.; ZHU, Z.; DU, Y.; SONG, Z. et al. Multi-crop-row detection algorithm based on binocular vision. Biosystems Engineering, 150, p.89-103, 2016. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S1537511015301884 >. Accessed: Sep. 09, 2018. doi: 10.1016/j.biosystemseng.2016.07.009.
https://www.sciencedirect.com/science/ar...
) and rice (CHOI et al., 2015CHOI, K. H.; SANG, K. H.; SANG, H. H.; PARK, K. H. et al. Morphology-based guidance line extraction for an autonomous weeding robot in paddy fields. Computers & Electronics in Agriculture, 113, n.C, p.266-274, 2015. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S0168169915000563 >. Accessed: Oct. 15, 2018. doi: 10.1016/j.compag.2015.02.014.
https://www.sciencedirect.com/science/ar...
) etc other crop line detection and recognition.

At present, the control of corn diseases and insect pests is mainly based on manual spraying of pesticides, which has some problems such as low efficiency, uneven spraying and harm to human body. Therefore, the application of robotics and machine vision technology to corn pesticide spraying is currently the developmental trend. However, the machine visual navigation method studied by many scholars is only suitable for the detection of crop rows during specific growth periods of corn, and can not meet the needs of visual navigation at all stages of the corn growth cycle. Especially after the corn flowering, because the crop line is difficult to extract, it iwas difficult for agricultural robot to realize the automatic navigation. How to accurately extract the navigation lines between corn rows during each growth cycle is the key to realize the automatic navigation of agricultural robots, while the precise extraction of navigation lines depends on the successful segmentation of corn and soil. Many scholars had conducted extensive research on the segmentation of corn crop images. SURIYAKOON et al. (2017) used excess red grayscale factor to separate corn from soil. SONG et al. (2017SONG, Y.; LIU, Y.; LIU, L.; ZHU, D. et al. Extraction Method of Navigation Baseline of Corn Roots Based on Machine Vision. Transactions of the Chinese Society for Agricultural Machinery, 48, n.2, p.38-44, 2017. Available from: <Available from: http://xueshu.baidu.com/usercenter/paper/show?paperid=ba7eeb1b6b8c9f5dabfa4d71d4dbf624&site=xueshu_se&hitarticle=1&sc_from=gxu >. Accessed: Dec. 05, 2018. doi: 10.6041/j.issn.1000-1298.2017.02.005.
http://xueshu.baidu.com/usercenter/paper...
), DIAO et al. (2015DIAO, Z.; ZHAO, M.; SONG, Y.; WU, B. et al. Crop line recognition algorithm and realization in precision pesticide system based on machine vision. Transactions of the Chinese Society of Agricultural Engineering, 31, n.7, p.47-52, 2015. Available from: <Available from: https://www.ingentaconnect.com/content/tcsae/tcsae/2015/00000031/00000007/art00007 >. Accessed: Oct. 16, 2018.
https://www.ingentaconnect.com/content/t...
) used differently improved 2G-B-R grayscale factor to separate corn crops from soil. MONTALVO et al. (2012MONTALVO, M.; PAJARES, G.; GUERRERO, J. M.; ROMEO, J. et al. Automatic detection of crop rows in maize fields with high weeds pressure. Expert Systems with Applications, 39, n.15, p.11889-11897, 2012. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S0957417412003806 >. Accessed: Dec. 25, 2018. doi: 10.1016/j.eswa.2012.02.117.
https://www.sciencedirect.com/science/ar...
) separated corn plants from soil using a traditional grayscale factor 2G-B-R and double Otsu. JIANG et al. (2015JIANG, G.; WANG, Z.; LIU, H. Automatic detection of crop rows based on multi-ROIs. Expert Systems with Applications, 42, n.5, p.2429-2441, 2015. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S0957417414006575#f0005 >. Accessed: Oct. 05, 2018. doi: 10.1016/j.eswa.2014.10.033.
https://www.sciencedirect.com/science/ar...
), GUIJARRO et al. (2011GUIJARRO, M.; PAJARES, G.; RIOMOROS, I.; HERRERA, P. J. et al. Automatic segmentation of relevant textures in agricultural images. Computers & Electronics in Agriculture, 75, n.1, p.75-83, 2011. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S0168169910001924 >. Accessed: Nov. 05, 2018. doi: 10.1016/j.compag.2010.09.013.
https://www.sciencedirect.com/science/ar...
) used genetic algorithm to improve the grayscale factor, and then used it to separate corn from soil.

The above methods can successfully separate the corn crop from soil in a specific scene, and extract the crop rows. However, during the different growth cycles of corn, the large difference of grayscale in different parts of corn, or the little gray difference of corn plants and soil, both which could affect the segmentation of corn plants and soil. However, the grayscale factors proposed in the above literature do not considered these situations, so the segmentation effect is not ideal. In order to achieve the segmentation of corn plants and soil in various periods, a new gray-scale factor is proposed in this paper, and on the basis of the new grayscale factor, the overall algorithm of corn field navigation line extraction is proposed. Firstly, in order to achieve the segmentation of corn plants and soil in different growth periods, this paper used the improved grayscale factor, combined with morphological processing and binary algorithm to separate soil and corn plants. Secondly, the region of interest (ROI) is intercepted by the binary image vertical projection map. Third, the coordinates of the discrete points of the navigation are determined by calculating the relative coordinates of the white points of the sampled pixel. Finally, The Hough transform was used to fit the navigation points to obtain the navigation line, which is used to control the automatic walking of the corn spray robot, and achieved good results.

Corn field image pretreatments

Intercepting the region area of interest

If the whole image captured by the camera is taken as the research object, it will inevitably lead to a large amount of calculation and interference, which will eventually affect the real-time and accuracy of autonomous navigation. In the actual intelligent navigation process, the region of interest only accounts for a part of the image. Because in the course of the actual investigation, it is reported that not all corn crop rows are straight lines, artificial corn crop rows have a certain degree of arc, especially in some mountain corn fields.in order to reduce the amount of image calculation, and improve navigation accuracy. In this paper, the idea of “turning the curve into straight” was taken, the image that is captured by camera is intercept out a part as the research object, after many interceptions, the decomposition of a curved crop row into multiple straight crop lines. After many field shooting results, it was reported that the last 190 pixel rows of the captured image are intercepted as the image region of interest can not only get good experimental results, but also greatly reduce the processing time of the image.

Propose a new grayscale factor

The images captured by the camera were stored in RGB format. In the RGB color space, the G component of crops and weeds has a large proportion, while the R and B component of soil background have a large proportion. Therefore, the appropriate grayscale factor is very important for the segmentation of crops and soil. In this paper, a new grayscale factor is proposed based on the commonly used graying factor.

At present, most of the graying of farmland plants adopts the 2G-R-B method, which can increase the weight of green plants, highlight the green characteristics of plant targets, enhance the contrast between target and background, and have a good effect on the segmentation of green plant and soil. Using this grayscale factor to process the image; although, the green plant and the soil can be well separated, the grayscale discrimination of different part of the same corn plant is high, which cause many holes in the binarized image, therefore this method affecting the extraction accuracy of navigation discrete points. The algorithm is such as formula (1).

grayx,y= 0 2G-R-B<0255 2G-R-G>2552Gx,y-Rx,y-Bx,y else (1)

In the formula: gray (x,y) Grayscale factor;

G(x, y) -- Color space green component;

R(x, y) -- Color space red component;

B(x, y) -- Color space blue component;

Another common method is G-B. In the farmland image, the G value in the R, G, and B components of the green plant is significantly different from the G value of the soil background, so a pair of grayscale images can be obtained by using the two components G and B. However, the disadvantage of using this grayscale factor is that the grayscale image is relatively fuzzy, and the feature point discrimination is insufficient. The algorithm is such as formula (2).

grayx,y=Gx,y-Bx,y G>R0 else (2)

In the formula: gray (x, y) -- Grayscale factor;

G(x, y) -- Color space green component;

B(x, y) -- Color space blue component;

Based on the advantages of the above two grayscale factors, a new gray-scale factor is proposed in this paper. Because the R component of the soil is large, this paper through increasing the R component to limit the G component to distinguish between soil and green plants, and can well extract the ridges from the plant background. At the same time, G(x, y) - B(x, y) method is used to determine the gray value which could reduce the gray level difference of different parts of the same plant. The algorithm is such as formula (3).

grayx,y=Gx,y-Bx,y else 0 G>R or G>B (3)

In the formula: gray (x, y) -- Grayscale factor;

G(x, y) -- Color space green component;

R(x,y) -- Color space red component;

B(x, y) -- Color space blue component;

Noise removal and image binarization

The main content of this paper is the intelligent navigation of corn spray robot. When the robot works in the field, there is a situation in which the camera lens is contaminated by dew, pesticides, etc., thereby affecting the sharpness of the captured images. In this paper, median filtering is used to deal with noise pollution, and the Otsu method is used to binarize the image. Due to uneven coverage of green leaves and interference of plastic mulch and other debris, there will be some white spots and holes after the binarization of grayscale image. These white spots and holes can affect the accuracy of navigation discrete points, and even causes the occurrence of extreme deviation points, which could affect navigation line accuracy. Therefore, it is necessary to perform morphological processing on the binarized image, which can remove incoherent structure while ensuring the basic shape feature of the image. This paper uses morphological closure operations to process binary images, smooth object contours, break narrow gaps, and eliminate smaller holes. and eliminate smaller holes the process is as in formula (4).

Ag=AgΘg(4)

1. Firstly, the structure element g is used to expand the image A, as in formula (5);

Ag=xg+xI x(5)

2. The structured element g is then used to erode the expanded image, as in formula (6);

BΘg=xg+x A(6)

Get the navigation discrete points

In this paper, an algorithm based on vertical projection map is proposed to obtain the discrete points. Firstly, the image region of interest (ROI) is further extracted by the column coordinates of the lowest trough on both sides of the intermediate peak in the vertical projection of the binary image, thereby reducing the influence of surrounding ridges and other factors on the target ridge, and speeding up processing speed of the image. Then, taking the intermediate pixel column of the intercepted region of interest as a reference, the pixel value of the left side of the referencewas negative, and the value of the pixel column of the right side is positive, and then calculating the algebraic sum of the coordinates of the white phase points on the left and right sides of the reference. Finally, the result iwas divided by the number of pixels of the sample row of white pixels to determine the coordinates of the navigation discrete points. Pixel vertical projection is a simple statistical method that can achieve the target ridge extraction. (7) is vertical projection method formula. (8) is navigation discrete point solution formula.

f(x)=y=0K-1f(x,y) x=0,1,2,H-1 (7)

In the formula: f(x, y) -- The value of the binary image point (x, y) (0 or 1);

K -- Image height;

H -- Image width;

x=i=0L-1xi-L+j=L+1Nxj-LK(8)

In the formula: L -- The column in which the baseline is located;

N -- The number of columns of the binary image;

K -- Represents the total number of white pixels in a sampling line;

In order to speed up the fitting of navigation lines, this paper takes one sample line every 10-pixel rows to calculate a navigation discrete point.

Fit the navigation line

Considering that the Hough transform has strong anti-interference ability, which can reduce the influence of error points and improve the fitting accuracy. Therefore, this paper used Hough transform to fit the navigation discrete points obtained by the vertical projection map to determine the navigation line. The Hough transform converts the point (x, y) in the plane Cartesian coordinate system into a corresponding sinusoid d=xcosθ+ysinθ . The specific algorithm is as follows:

Initialize an accumulator that corresponds to the square associated with the parameter space coordinates, initially set to 0.

2) For each navigation discrete point (xk, yk) obtained, ( be equal to the allowed subdivision value on the ( axis, and then solve the corresponding ( value by formula ρ=xkcosθ+yksinθ . The value of ( is rounded to the nearest integer value of the ( axis, and the corresponding ((i, (j) accumulator is incremented by 1.

3) Find the maximum value Q in the accumulator, which means that there are Q points on the line xcosθj+ysinθj=ρi that in the xy plane, thereby find the navigation path.

Control system and extraction of navigation parameters

The corn spray robot control system scheme is shown in figure 1. The whole system is composed of human-computer interaction layer and motion control layer. The human-computer interaction layer is composed of a laptop computer, the motion control layer is composed of a controller and a driver. The human-computer interaction layer is mainly responsible for sending commands and other operations. During the working process, the computer transports the images captured by the camera through USB, performs image processing, plans the navigation path of the robot, and sends instructions to the motion controller. The motion control layer acquires a motion command issued by the host computer to drive the DC motor to rotate.

Figure 1
Structure of corn spray robot control system.

When the robot walks in the field, the control command is mainly obtained from the navigation path: Angle deviation β and displacement deviation d. The angle deviation refers to the angle β between the navigation path and the axis direction of the robot. The displacement deviation refers to the distance d between the navigation path and the camera, as shown in figure 2.

Figure 2 -
Angle deviation and displacement deviation. 1. Trackcar 2. Camera 3. Hough transform path.

Experiment and analysis

In order to verify the advantages of the grayscale factor proposed in this paper, and the real-time of algorithm for seeking and fitting navigation discrete points. Making experiments in corn fields with different growth cycle and growth environment. Finally, the feasibility of the method is verified by the navigation and tracking experiment.

Experimental platform

The construction of the experimental platform is shown in figure 3. It mainly includes the main control computer, the tracked vehicle part, the camera and its support, the STM 32F407ZGT6 control chip, the DC motor JCF76R-24120-210R and its driver. Experimental data is mainly processed on platforms Python2 and Opencv2. The crawler chassis was produced by Shenzhen Shen lian Technology Co., Ltd. The camera used the industrial camera MER-132-43U3C-L from Daheng Image Co., Ltd, which transmits data by USB3.0. The camera is mounted on the bracket, the angle and height of the camera can be adjusted by the bracket, and the camera is mounted on the crawler vehicle chassis through the bracket.

Figure 3 -
Robot experiment platform. 1. Industrial camera 2. Support 3. Main control computer. 4. Trackcar 5. Motor driver 6. STM control chip 7. Lithium battery.

Farmland experiment

One hundred pictures are collected in four different environments and processed on the master computer. In order to judge the accuracy of the navigation line that is extracted through the method proposed in this paper, the navigation datum is manually fitted as the measurement datum to judge the accuracy of the navigation line. Taking the starting point of the experimental vehicle as the coordinate origin, the artificial coordinate system is established, the coordinates of maize rhizomes between rows are measured, and the gradient is calculated. Finally, the datum equation is solved; therefore, the artificially fitted navigation datum line was obtained. The angle between manually navigation datum line and the navigation line obtained by the algorithm proposed in this paper is the navigation angle, which is used to measure the accuracy of the navigation datum line obtained by the algorithm proposed in this paper. The processes and results are shown in figure 4 to figure 7. The effect is shown in table 1.

Figure 4
- Corn field experiment of covering the mulch film. a. Original image covered by plastic film b. The result of intercepting region of interest c. The processing result by using improved graying factor d. The processing result by using the grayscale factor (1) e. The processing result by using the grayscale factor (2) f. The result of grayscale image contaminated by salt and pepper noise g. the filtered resulted of median filter h. The result of binarization of figure g i. The result of binarization of figure d j. The result of binarization of figure e k. The result of the morphological closure of figure h l. The vertical projection map of figure k m. The vertical projection map of figure i n. Navigation discrete points o. The result of the navigation discrete point fitting through Hough transform.

Figure 5 -
Corn field experiment of no mulch cornfield. a. Originalimage without the plastic film b. The result of intercepting region of interest c. The processing result by using improved graying factor. d. The processing result by using the grayscale factor (1) e. The result of binarization of figure c f. The result of binarization of figure d g. The result of the morphological closure of figure e h. The vertical projection map of figure g i. Navigation discrete points j. The result of the navigation discrete point fitting through Hough transform.

Figure 6 -
Corn field experiment of mountain area. a. Originalimage of the mountain corn seedlings b. The result of intercepting region of interest c. The processing result by using improved graying factor d. the result of the morphological closure operation after the figure c binarization e. The region of interest intercepted by the vertical projection map of figure d f. Navigation discrete points g. The result of the navigation discrete point fitting through Hough transform.

Figure 7
- Corn field experiment of flowering. a. Original image of flowering corn field b. The result of intercepting region of interest c. The processing result by using improved graying factor d. the result of the morphological closure operation after the figure c binarization e. The region of interest intercepted by the vertical projection map of figure d f. Navigation discrete points g. The result of the navigation discrete point fitting through Hough transform.

Table 1
Results of Navigation Line Extraction in Different Environments.

It can be seen from the comparison of figures 4c, 4d, 4e, 4h, 4i, 4j: When the image is processed by the grayscale factor 2G-R-B, the large difference of grayscale in different parts of corn, which makes some holes when the image is binarized, and morphological processing could not fill all the holes, which seriously affects the accuracy of the navigation discrete points. When the image is processed by the grayscale factor G-B, the grayscale image is relatively fuzzy overall, and some plant leaves of the target ridge are lost after binarization, at the same time, there are more holes, which will affect the accuracy of the navigation points and navigation path. When the image is processed by the grayscale factor proposed in this paper, the difference in plant and soil in the grayscale image can be increased while the difference in grayscale in different parts of the same plant can be reduced. The image after the grayscale processing is relatively clear, and the feature points of the target ridge are not lost in the binarized image. Moreover, the number of holes is less and the area is small, and the small holes can be processed by the morphological closing formula (4). The result is shown in figure 4k. Details of the plant leaves in the target ridge are completely preserved, and the small holes on both sides of the ridge are well filled, which can attenuate the interference of the target ridge lacking seedlings and uneven growth to a certain extent. As shown in figure 4l, the target ridge is highlighted in the vertical projection map. By comparing figure 4m and figure 4l, it can be seen that after processing the image by using the grayscale factor proposed in this paper, the peaks and troughs in the vertical projection map of the binary image are less, which reduces the extraction target ridge cycle algorithm operation time. As shown in figure 4n, by using the vertical projection map to intercept the binary image, the region of interest (target ridge) in the image can be extracted well, which not only reduces other ridge disturbances, but also reduces the amount of image calculation. It can be seen from the result of the navigation discrete points shown in figure 4n, that the navigation discrete points calculated by the method (8) are well distributed in the central area of the ridge. The Hough transform fitting navigation discrete points to get the navigation line as shown in figure 4o, which has achieved good results. From the experimental comparisons of figure 4, figure 5, figure 6 and figure 7, it can be seen that for corn fields with different growth periods and environments, the grayscale factor proposed in this paper well separates plants and soil. After the morphological operation, the holes of the target ridge can be basically filled, which makes the vertical projection curve of the binary image smoother and reduced the number of peaks and troughs in the vertical projection. From figure 8, it can be seen that the navigation line obtained through method that proposed in this paper is well distributed in the target row.

Figure 8
- Navigation line extraction results. a. Thenavigation line of cornfield covered with plastic film b. The navigation line of cornfield without plastic film c. The navigation line of mountain cornfield d. The navigation line of flowering cornfield.

From table 1, we can see that the accuracy of navigation line is more than 90%, and the processing time of each frame is less than 110 ms, which can be applied to the extraction of navigation lines in different growth periods of corn field.

Grayscale method analysis

MONTALVO et al., (2012MONTALVO, M.; PAJARES, G.; GUERRERO, J. M.; ROMEO, J. et al. Automatic detection of crop rows in maize fields with high weeds pressure. Expert Systems with Applications, 39, n.15, p.11889-11897, 2012. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S0957417412003806 >. Accessed: Dec. 25, 2018. doi: 10.1016/j.eswa.2012.02.117.
https://www.sciencedirect.com/science/ar...
),SONG et al., (2017SONG, Y.; LIU, Y.; LIU, L.; ZHU, D. et al. Extraction Method of Navigation Baseline of Corn Roots Based on Machine Vision. Transactions of the Chinese Society for Agricultural Machinery, 48, n.2, p.38-44, 2017. Available from: <Available from: http://xueshu.baidu.com/usercenter/paper/show?paperid=ba7eeb1b6b8c9f5dabfa4d71d4dbf624&site=xueshu_se&hitarticle=1&sc_from=gxu >. Accessed: Dec. 05, 2018. doi: 10.6041/j.issn.1000-1298.2017.02.005.
http://xueshu.baidu.com/usercenter/paper...
),DIAO et al., (2015DIAO, Z.; ZHAO, M.; SONG, Y.; WU, B. et al. Crop line recognition algorithm and realization in precision pesticide system based on machine vision. Transactions of the Chinese Society of Agricultural Engineering, 31, n.7, p.47-52, 2015. Available from: <Available from: https://www.ingentaconnect.com/content/tcsae/tcsae/2015/00000031/00000007/art00007 >. Accessed: Oct. 16, 2018.
https://www.ingentaconnect.com/content/t...
), Suriyakoon and Ruangpayoongsak., (2017)SURIYAKOON, S.; RUANGPAYOONGSAK, N., 2017b, Leading point based interrow robot guidance in corn fields. IEEE. p.8-12. Available from: <Available from: https://ieeexplore_ieee.gg363.site/abstract/document/7935032 >. Accessed: Dec. 19, 2018. doi: 10.1109/ICCRE.2017.7935032.
https://ieeexplore_ieee.gg363.site/abstr...
, JIANG et al., (2015JIANG, G.; WANG, Z.; LIU, H. Automatic detection of crop rows based on multi-ROIs. Expert Systems with Applications, 42, n.5, p.2429-2441, 2015. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S0957417414006575#f0005 >. Accessed: Oct. 05, 2018. doi: 10.1016/j.eswa.2014.10.033.
https://www.sciencedirect.com/science/ar...
), GUIJARRO et al. (2011GUIJARRO, M.; PAJARES, G.; RIOMOROS, I.; HERRERA, P. J. et al. Automatic segmentation of relevant textures in agricultural images. Computers & Electronics in Agriculture, 75, n.1, p.75-83, 2011. Available from: <Available from: https://www.sciencedirect.com/science/article/pii/S0168169910001924 >. Accessed: Nov. 05, 2018. doi: 10.1016/j.compag.2010.09.013.
https://www.sciencedirect.com/science/ar...
) proposed different grayscale factor algorithms to extract crop line. In order to verify the processing effect of the grayscale factor proposed in this paper, the farmland image taken by the test vehicle is taken as the research object. The judgment basis is whether the target ridge can be extracted. The vehicle runs at a speed of 0.3m/s. One frame of image is extracted every 1s. A total of 100 frames of image are collected as the research object. The method comparison results are shown in table 2. From table 2, we can see that the grayscale algorithm proposed in this paper can effectively separate the target ridge from the plant.

Table 2
Comparison of time-consuming and accuracy of different algorithms.

Navigation tracking

The navigation tracking experiment was carried out in the corn field shown in figure 5. The corn is planted in wide and narrow rows, wide row spacing of 65 cm and narrow row spacing of 30 cm. The crawler-mounted car is 45cm wide and the robot can work in a wide-row planting environment. According to the image taken by the camera during walking, the main control computer processes the acquired image in real time to obtain a navigation path, and controls the steering with the displacement deviation and the angular deviation shown in figure 3. When the experimental platform drives at 0.3m/s, the error map is shown in figure 9. The maximum error is less than 5cm, and the crop line will not be encountered. Therefore, the method proposed in this paper can meet the work requirements.

Figure 9
Navigation path tracking error map.

CONCLUSION:

In this paper, corn fields with different growth periods and growth environments are taken as research objects. In the image preprocessing stage, only 190-pixel rows are taken as regions of interest, which reduces the amount of image processing and speeds up the processing. For the image graying process, a new graying factor is proposed in this paper. The image processed by the new gray factor not only retains the feature points of the target ridge, but also weakens the influence of plant coverage unevenness to some extent, so that the binary image has fewer holes, which makes it successful to intercept the region of interest through the vertical projection. The navigation points are obtained based on the vertical projection map of the binary image, and the navigation path is well fitted through the Hough transform. It also takes good results in the process of tracking the navigation path of corn field.

In this paper, a new grayscale factor is proposed, which can be used to well separate corn plants from soil and meet the automatic navigation requirements of corn spray robots in the different growth cycles and environments.

1) This paper proposds that the vertical projection map and Hough transform are used to extract the navigation discrete points and navigation paths, which improved the shortcomings of the traditional Hough transform fitting straight line, to a certain extent, and can satisfies the real-time requirements of the robot.

2) When the robot walks at a speed of 0.3 ms-1, the maximum error is less than 5 cm, which can satisfy the application of the robot in agriculture.

ACKNOWLEDGEMENTS

The work was supported by National Natural Science Foundation of China (No.1703116), Natural Science Foundation of Guangxi (No.2017GXNSFBA198228), Guangxi science and technology project (No.AD19110034), Guangxi Young Teacher Promotion Program (No.T3010098018), Guangxi Advanced Manufacturing Laboratory Project (No. 16-380-12S001).

REFERENCES

  • CR-2019-0699.R1

Publication Dates

  • Publication in this collection
    11 May 2020
  • Date of issue
    2020

History

  • Received
    12 Sept 2019
  • Accepted
    22 Jan 2020
  • Reviewed
    29 Mar 2020
Universidade Federal de Santa Maria Universidade Federal de Santa Maria, Centro de Ciências Rurais , 97105-900 Santa Maria RS Brazil , Tel.: +55 55 3220-8698 , Fax: +55 55 3220-8695 - Santa Maria - RS - Brazil
E-mail: cienciarural@mail.ufsm.br