Deep learning with aerial surveys for extensive livestock hotspot recognition in the Brazilian Semi-arid Region

ABSTRACT In the Brazilian Semi-arid Region, extensive livestock farming with ecoproductive management is the most efficient way to maintain and increase the production of goat products (e.g., meat) with of not depleting environmental resources. This set of actions (induced goat migration and pasture closure) is part of Livestock 4.0, in which Industry 4.0 feed areas are efficiently managed using artificial intelligence and deep learning properly monitored by the producer and the consumer. The objective of this work was to identify pasture areas with Opuntia ficus-indica (Mill, Cactaceae) forage palm species for breeding and production of Capra aegagrus-hircus goats (Lineu, Bovidae) using aerial survey images captured by drones classified using deep learning techniques. The methodological steps of the Industry Architecture Reference Model 4.0 were adapted to the field situation (Semi-arid Region) including (A) study area delimitation, (B) image collection (by drones), (C) deep learning training, convolutional neural network (CNN) training, (D) training accuracy analysis, and (E) automatic goat production evaluation and validation. The area classification based on the forage palm density allowed us to measure the environmental degradation caused by livestock. Stimulated goat migration reduced this degradation as well as increased goat biomass and volume production.


INTRODUCTION
In areas where primary resources are scarce, locating hotspots for agricultural production is fundamental.Deteriorated areas, such as areas in semiarid climates, in which there is a scarcity of water and a negative water balance (evapotranspiration is greater than precipitation) (Santana;Encinas, 2016), locating and evaluating areas of interest for economic-ecological management, and their aggregation to a production chain (e.g., livestock) can increase local revenues and have positive socioenvironmental impacts (Santana, 2016).
In semiarid regions, extensive livestock farming with ecoproductive management is the most efficient way to maintain and increase production (e.g., meat) while not depleting environmental resources (Santana;Encinas, 2016).One example is goat production with stimulated management, in which the producer enforces goat migration to areas with a greater density of direct nutritional resources (e.g., forage cactus) (Magalhães et al., 2021).To identify and classify areas for possible management, the efficient analysis of the area through trained aerial classification is needed.This set of actions is part of Livestock 4.0, in which Industry 4.0 feed areas are efficiently managed using artificial intelligence and machine learning and duly monitored by the producer and the consumer (Stumpenhausen, 2018).
Finding areas and feeding goats with the Opuntia ficusindica (Mill, Cactaceae) forage cactus in semiarid regions are sources of increased production; the plant tissue of this palm has approximately 85% water while the other 25% has essential nutrients for nutrition, increasing goat biomass (Magalhães et al., 2021), such as the Capra aegagrus-hircus (Lineu, Bovidae) species, which is most consumed and exported goat in terms of human nutrition (Santana;Encinas, 2016).
The use of deep learning to recognize areas of interest for ecoproduction (hotspots) is an emerging natural practice, as its use in the classification of aerial images allows for surveys over long distances and in remote locations, reducing cost and time (Lopez-Jimenez et al., 2019).The aim of this method is to identify a specific species in a diverse environment (Lee et al., 2017), measure forest density (Sun et al., 2017) and identify phytopathology (Barbedo, 2019).This method can be used for morphological and phenological recognition (Gyires-Toth et al., 2019), exotic species eradication (Lopez-Jimenez et al., 2019), plant parameter alternatives (Pearline;Kumar;Harini, 2019) and fauna identification (Miao et al., 2019).Ecoproduction is based on using natural primary resources in cycles of productive efficiency associated with environmental conservation (Santana, 2017).
When performing large scale sampling in small areas, unmanned aerial vehicles (UAVs) are the main tools used for the ecoproductive classification of biotic assessments (Lopez-Jimenez et al., 2019), social analysis (Suel et al., 2019), local measurements (Quevedo et al., 2019), and geomorphology studies (Moor et al., 2019), among others.There are several popular models of professional drones on the market that are affordable for scientific projects and industrial analysis (< U$ 400) and have intuitive handling and a flight autonomy of 30 min to travel in a 1 km² area at a real-time altitude of 182 m (SZ DJI Technology, 2018).
For these types of images and their analysis, a convolutional neural network (CNN) is recommended because it is a class of feedforward artificial neural networks with a range of multilayer perceptrons designed using the least amount of preprocessing; CNNs are ideal for 2D images (RGB) with shift invariance and space invariance (Sun et al., 2017;Pearline;Kumar;Harini, 2019).
Therefore, the hypothesis of this work was as follows.The automatic identification of natural food sources (cactus) for goats and the management of the goat herd in identified areas results in an optimization of the food supply and the conservation of native flora.Thus, the objective of this work was to identify pasture areas in a semiarid climate with the Opuntia ficus-indica (Mill, Cactaceae) forage cactus species for the rearing and production of Capra aegagrus-hircus (Lineu, Bovidae) goats through an aerial survey of images captured by drones and perform classification using a deep learning technique.

MATERIAL AND METHODS
The methodological steps of the Reference Model for Industry Architecture 4.0 (Heidel et al., 2017) were adapted to the field situation (semiarid region) and include (A) study area delimitation, (B) image collection, (C) deep learning training, (D) training accuracy analysis, and (E) automatic goat production evaluation and validation.

Study area
Data collection was carried out in an area under the BSh Semiarid Climate and Caatinga ecosystem (Santana, 2017) in backland of the State of Pernambuco (7°35'00" S and 39°42'22" W), where there is a predominance of cactus (Opuntia ficus-indica Mill, Cactaceae) due to its introduction in the region in previous periods and an extensive management of Capra aegagrus-hircus goats (Lineu, Bovidae) (Figure 1).The specific study area (image collection location) was in the rural area of Floresta City, PE, Brazil (08°36'04" S and 38°34'07" W).

Collection of images
The images and information were collected (every 15 days from July 2018 to July 2019) by a camera attached to a DJI Phantom 4 RTK drone (SZ DJI Technology Co., Ltd., Shenzhen, Guangdong, China).The camera records at 5 cm pixel -1 , with an image resolution of 5472 × 3648 and H.264 video, 4K: 3840 × 2160 30 p.The flights were performed manually from 0 to 100 m above the surface close to noon.In the image capture areas, the drone hovered in the air for 5 s for complete stabilization and image capture of the region of interest (ROI).The fragments (patches) of the image for analysis (RGB channels) that contained the forage cactus were separated at a minimum resolution of 32 × 32 pixels.A total of 51,499 palm fragments were separated for classification (palm class, Figure 2).This was completed for the formation of the 'no palm' class by separating 51,499 'no palm' fragments (32 x 32 pixels).

Training: Deep Learning
Palm identification in the images was performed by training a convolutional neural network (CNN) (Goodfellow;Bengio;Courville, 2016).Φ w : I → y .The input was an RGB image, I, the output was a predicted class label represented by a multinomial distribution, y , and the network parameters were defined by w.
The Φ of the modified version of the LeNet-5 network (Chen et al., 2018) was used and configured, as shown in Figure 3.The sequence of actions is given as follows: (1) (i) input a 3-channel image with a resolution of 32 × 32; (ii) apply 6 convolution filters, each with a size of 5 × 5; (iii) perform max pooling with a 2 × 2 kernel; (iv) apply a set of 16 kernel convolutions with a size of 5 × 5; (v) perform max pooling with a 2 × 2 kernel; (vi) flatten the features to a one-dimensional vector of size 400; (vii) apply three fully connected layers with 120, 84 and 2 nodes; (viii) obtain the CNN output in a vector of real numbers (logits); and (ix) apply the LogSoftMax function to convert the logits into a normalized probability distribution (Equation 1): To train the Φ w network, the internal parameters, w, (weights and biases) were adjusted so that the output fits the real (true) data.In the training process, the images of the dataset were input into Φ w in batches, and the outputs were compared with the true labels, y, under a loss function (Equation 2) (Goodfellow;Bengio;Courville, 2016): where is a batch element and c is the class index.Once the loss was calculated for a given lot, the internal parameters were adjusted using the previous propagation algorithm (gradient descent optimization).
Network training was performed with the Adam optimizer on an Intel Core i7 machine with an NVIDIA GeForce 1080 GPU.The hyperparameters were defined as follows: learning rate of 0.01, number of epochs of 150, and batch size of 2500 (Gopalakrishnan et al., 2017).

Data augmentation
Sample amplification was performed to increase the number of samples (data augmentation) (i) without enlargement, the fragments were resized to 32 × 32 and normalized; and (ii) vertical and horizontal flipping as well as resizing and normalizing were performed, with a probability of 0.5, both as independent events.The validation set was obtained by amplifying the number of samples by 0.98 and 0.95 for the training and validation sets, respectively (Lopez-Jimenez et al., 2019).When the ROIs were questionable, field visits were carried out to verify the image collected. (2)

Real-time detection and automatic assessment of livestock areas
After training, forage cactus recognition was performed in real time using YOLOv3, an object detection system for real-time images (Redmon;Farhadi, 2018;Birrell et al., 2019).In the field, the drone images were automatically sent to a notebook and analyzed in real time.One image captured 100 m above the surface can cover an area of 2.56 hectares (160 x 160 m, see image capture resolution).From this information, we were able to calculate the cactus density per hectare.Thus, it was possible to delimit the hotspots, areas with at least one Opuntia ficusindica (Mill, Cactaceae) cactus per 5 m 2 of space.
The analysis of the impact of goats on hotspots was recorded by monitoring 50 Capra aegagrus-hircus (Lineu, Bovidae) in five hectares at a time (extensive management) over a period of time.All goats were tagged and registered with a subcutaneous chip (Autag Technology Europe B.V., Moordrecht, Netherlands).These goats were 6 years old (the age at which bone growth and height stabilize and senescence does not occur).During the study period, the 50 goats explored an area of approximately 1,200 hectares (12 km 2 ).
In the first six months (July to December 2018), the goats were released into five hectares of the study area and migrated spontaneously (control group).Then (January to July 2019), the goats were constantly guided to areas that had more than one cactus per 5 m 2 of space.When the area had one cactus every 10 m 2 of space due to foraging, the stimulated migration of goats to areas of higher cactus density was performed (sample group).
Goat mass data (kg ind -1 ) were statistically compared between the sample groups (spontaneous migration and stimulated migration) using Student's t test (95% reliability) (D'agostino;Belanger;D'agostino, 1990).Previously, to confirm the performance of the parametric test, the normality of the distribution was confirmed through the D'Agostino normality test (Zar, 1999).The relationship between goat mass (kg ind -1 ) and forage cactus density (ind ha -2 ) was fitted to nonlinear curves (sigmoidal: growth curves) and performed using regression analysis parameters (D'agostino;Belanger;D'agostino, 1990).

RESULTS AND DISCUSSION
Increasing the number of samples from the original sample (data augmentation) resulted in a faster reduction in training loss (Figure 4A) and an increase in training accuracy (Figure 4B) and validation accuracy (Figure 4C).On the validation test, the accuracy of using the flip technique is better than the accuracy of using the data without amplification.In the first periods of training in other studies (Lopez-Jimenez et al., 2019;Wang et al., 2019), the precision was not different between the flip technique and no data amplification, and the initial weights were more similar and the final weights were more distinct after the updates.In contrast, this distinction was observed in this study by epoch 10 (Figure 4C).This is due either to the characteristics of image accuracy or to the more homogeneous background in the 'without palm' classification in semiarid regions.At the end of each semester, the goats were measured in relation to their body mass on a platform scale (Bench Scale BBA231-3BC60A -Mettler Toledo Ind., Barueri, São Paulo, Brazil), and their volumes were estimated using a 3D scanner (3D Systems Capture Scanner Plus Pro Pack, TEquipment.NET, Long Branch, New Jersey, United States).
The number of false-positive results was higher for the 'without palm' classification (Figure 5A) due to the presence of other plants aggregated with the palm (Figure 6A) and the homogeneity of the pixels where the palm does not appear (Figure 6B).However, in the normalized confusion matrix, the number of falsepositives results did not exceed one percent for both classes (Figure 5B).In Figure 6A, the detection accuracy is highlighted, as the plant in the center of the image was not recognized by the system, which is in line with other studies (Birrell et al., 2019;Lopez-Jimenez et al., 2019).The detection and recognition of areas by the density of forage cactus made it possible to monitor the degradation of an area by the consumption of cacti by goats (from one cactus every 5 m 2 to one cactus every 10 m 2 ).Then, the goats migrated to areas with greater biomass of the studied plant.As shown in Figure 7, goats that remained in the same place for a longer time (spontaneous migration) consumed almost all the available cactus, ultimately reducing the per capita amount of food per goat, causing a loss of mass (Figure 7A) and body volume (Figure 7B) for these goats.This difference was significant (p < 0.001) when observing the mean mass between stimulated migration (86 ± 2 kg) and spontaneous migration (71 ± 3 kg) goats.All sample groups had a normal data distribution (p < 0.05, D'Agostino Test), highlighting the importance of managing livestock production in the semiarid region (Santana;Encinas, 2016).
There was a significant and direct (sigmoidal) proportionality in the relationship between the mass of a goat and the cactus density where it moved, as shown from the points of spontaneous migration at the beginning of the curve and stimulated migration at the end of the curve (Figure 7C).This result also highlights the importance of techniques such as the application of deep learning associated with aerial surveys in dystrophic regions to cause, through increased production, an increase in local revenues and positive socioenvironmental impacts (Encinas; Santana;Muñoz, 2019;Santana;Encinas;Muñoz, 2022).
Therefore, this study demonstrated the need to unite science, technology and society to overcome historical social demands: productivity in an area of environmental dystrophy (Brazilian semiarid).Demand collection, method structure selection, data analysis, technological systematization for the biophysical and environmental assessments of a production scenario, and the positive and real impact in an area of social and environmental vulnerability were actions that highlighted the efficiency of interdisciplinarity in contextual productive solutions.This work aligned the determination of the spatial distribution of plant and animal biomasses through images (Santana;Encinas;Muñoz, 2022), the ability to obtain images by cameras coupled to drones (Pearline;Kumar;Harini, 2019;Quevedo et al., 2019), scientific-technological engagement to search for a contextual and practical solution (Lima et al., 2019;Lima et al., 2022), science applied to production (Santana;Encinas;Muñoz, 2019), the environmental efficiency of primary energy use (Santana;Imaña-Encinas, 2013;Imaña-Encinas et al., 2016;Santana;Encinas, 2018;Imaña-Encinas et al., 2021) and positive social and economic impacts (Santana et al., 2015;Lima et al., 2019;Nascimento et al., 2022).The proposed methods and analyses are interdisciplinary and show the interface of technologies with sociotechnical enterprises, the solidarity economy, cultural innovations, territorial considerations and the sustainability of primary environmental resources (Hafstad, 1957;Lee, 2010).

Figure 2 :
Figure 2: Examples of fragments collected containing images of the forage cactus Opuntia ficus-indica (Mill, Cactaceae) for analysis in the Brazilian semiarid region.

Figure 7 :
Figure 7: Mass of Capra aegagrus-hircus (Lineu, Bovidae) in spontaneous migration and stimulated migration areas, (B) representation of the volumetric distinction between individuals of the two types of migration, and (C) relationship between the mass of Capra sp. and the density of forage cactus Opuntia ficus-indica (Mill, Cactaceae), average of the five hectares studied (n = 50 for each group).

Figure 6 :
Figure 6: Detection and accuracy probabilities of the forage cactus Opuntia ficus-indica (Mill, Cactaceae): (A) 3 m above the surface in an area of high cactus density, and (B) at 100 m in an area with low-density palm.YOLOv3 was used.