AN AUTOMATIC DETECTION METHOD FOR ABNORMAL LAYING HEN ACTIVITIES USING A 3D DEPTH CAMERA

ABSTRACT

With the increasing scale of farms and the correspondingly higher number of laying hens, it is increasingly difficult for farmers to monitor their animals in a traditional way. Early warning of abnormal animal activities is helpful for farmers’ fast response to the negative impact on animal health, animal welfare and daily management. This study introduces an automatic and non-invasive method for detecting abnormal poultry activities using a 3D depth camera. A typical region including eighteen Hy-line brown laying hens was continuously monitored by a top-view Kinect during 49 continuous days. A mean prediction model (MPM), based on the frame difference algorithm, was built to monitor animal activities and occupation zones. As a result, this method reported abnormal activities with an average accuracy of 84.2% and a rate of misclassifying abnormal events of 15.8% (P_FPR). Additionally, it was found that the flock showed a diurnal change pattern in the activity and occupation quantified index. They also presented a similar changing pattern each week.

KEYWORDS
Activity; occupation index; 3D depth camera; MPM; laying hens

INTRODUCTION

Livestock management decisions are mostly based on the observation, judgement, and experience of farmers. However, with the increasing scale of farms and the correspondingly higher number of animals, it is increasingly difficult for farmers to monitor their animals in a traditional way. Moreover, it is impossible for farmers to monitor their animals continuously for a full 24 hours. Modern technology now makes it possible to use cameras, microphones, and sensors sufficiently close to and sometimes on the animal so that they can, in effect, assist farmers’ eyes and ears in everyday farming ( Kashiha et al. 2013a ). These techniques can facilitate the development of “early warning systems”, which shorten the response time to individual animal needs ( Norton & Berckmans 2017 ). Employing such a tool to monitor flocks can help farmers substantially manage their animals and houses more efficiently ( EFSA 2012 ). More detailed individual information can be perceived with image analysis techniques. For example, depth image processing can realize automatic detection of hens’ behaviours ( Lao et al. 2017 ). By using modified cameras, researchers found that problematic events can be detected with an automatic method to predict the distribution index of broilers ( Kashiha et al. 2013b ). A computer vision-based system can be used for automatic detection of dairy cow lying behaviour in free-stall barns ( Porto et al. 2013 ). The health and welfare status of animals is often closely related to their active state and behavioural changes, so a better understanding of animal activities is of great help in the study of animal behaviour, animal welfare, and animal productivity ( Ni et al. 2017 ).

Abnormal animal activity and declining production performance indicate problems in the chicken house. For example, sudden changes in animal activity may be related to animal heat stress, human disturbance, environmental control system failures, and feeding and drinking water system failures, all of which will have varying degrees of influence on production performance: egg production rate, rate of death and elimination, and drinking water and consumption ( Kashiha et al. 2013b ). If these abnormal events can be monitored and identified in real time and problems can be found quickly, it will be of great help to production management. Usually, in animal research and production, the activity index is an important animal activity parameter that is available. Optical flow analysis is a popular method for studying animal movement, which involves detecting the rate of changes in brightness in each area of an image frame ( Dawkins et al. 2012 ). In a large-scale henhouse, however, researchers found that the quality of colour images and videos are easily affected by light levels and the distance between cameras and animals ( Dawkins et al. 2012 ). Therefore, the optical flow method might not be a good choice for monitoring flocks in a low-light level environment. Moreover, 2D digital image segmentation and recognition can be problematic under real farm conditions due to dynamic background restrictions, such as dim or uneven light intensity in the house and varying floor status. These factors can affect the robustness of the algorithm for accurate classification ( Lao et al. 2016 ). While 3D depth cameras can solve this problem, information captured by a depth image sensor differs considerably from that of colour digital images in that each pixel in the depth data reflects the distance between the object and the depth image sensor ( Lao et al. 2016 ). Depth image analysis is a new method that helps detect not only horizontal but also vertical distribution attributes of animals ( Gregersen et al., 2013 ) without restricting the light environment. This feature allows for continuous monitoring of animal behaviours throughout the day. Depth image analysis has been used in the automatic detection of animal lameness ( Van Hertem et al., 2013 ; Viazzi et al., 2013 ). Additionally, this method is non-invasive and contactless, and it can measure animal movement in real time ( Springer & Seligmann 2016 ).

Given the above rationale, this study aims to explore whether the application of 3D depth sensors is competent in contactless continuous 24-h monitoring of laying hen activities. The objectives are as follows: (i) development of the activity monitoring model and (ii) application and testing of the algorithm.

MATERIAL AND METHODS

Animals and house

Experiments were conducted on a small-scale experimental farm (116° E 40° N, Beijing, China) from 7 July 2017 to 7 September 2017. The Hy-line brown laying hens (48-57 weeks) were reared in stacked cages, and the stock was more than 1,000 birds. Due to space limitations, a top-cage area (1.2 m L×1.2 m W× 0.6 m H) including eighteen hens was selected for device installation ( Figure 1 ). The tested cage was modified to an open roof cage covered by Perspex sheets to meet the image collection requirement. There was ad libitum access to food and water during the experiment. The light schedule was from 4:00 a.m. to 8:00 p.m.), and the poultry flock was fed twice a day, once between 7:30 and 8:30 a.m. and again between 4:00 and 6:00 p.m. Temperature and relative humidity parameters were recorded every five minutes. Additionally, production performance data were recorded, such as laying rate, number of dead and culled chickens, feed consumption and water consumption.

FIGURE 1
On-site test platform.

Data collection

Successive 24-hour images were recorded over 49 days. A top-view Kinect camera for Windows V1 (Microsoft Corp., Washington, USA), installed at a height of 1.8 m from the bottom of the cage, was used to monitor the flock ( Figure 2 ). Depth images (the resolution is 640×480 pixels) were recorded in .txt format at one frame per three seconds ( Jana, 2012 ). The Kinect was connected to a mini industrial personal computer (IPC) that stored the depth images for subsequent analysis. NI LabVIEW 2015 (American National Instrument Corp., Texas, USA) was used for preprocessing and analysing depth images based on a VDM (vision development module).

FIGURE 2
The schematic of the experiment platform.

Depth image analysis

Physical activity as defined in this study, quantified by an activity index, is the animal group movement, and it was expressed using the neighbour frame difference in pixel intensity. Original depth images were preprocessed and analysed in LabVIEW software ( Figure 3 ). The activity index can be used to measure animal movement, and the principle of this technique is to calculate the change in pixel intensity I between two adjacent frames ( Bloemen et al. 1997 ). If chickens are inactive, the change is smaller than that when they are active. The preprocessing steps are as following:

FIGURE 3
Flowchart of depth image processing.

Image cropping

To discard irrelevant edges, it is necessary to crop the original image to remove useless pixel points. For example, the original image size is 640×480 pixels, and some areas, such as chicken feeding trough areas including useless pixel points in one image, were removed (after cropping 510*480 pixels).

Morphological processing

Close, dilate and open operations and a 7*7 median filter were applied to depth image processing to obtain a filtered image.

Binarization processing

Depth images were converted to binary images to extract significant features, including chicken pixels and pixel intensity changes (threshold range: 1,200-1,700 mm).

The change in pixels was calculated as ( Costa et al. 2009 ):

(1) $$I_{mov}\left(x,y,t\right)=I\left(x,y,t\right)-I\left(x,y,t-1\right)$$

Where:

I_mov = neighbour frame difference in pixel intensity;

( x , y) = coordinate of images;

t = time index (s),

I = pixel intensity values.

The image activity index was calculated as the ratio between the total change in pixels and the total number of pixels per image ( Bloemen et al. 1997 ):

(2) $$activity\left(t\right)=\frac{{\displaystyle \sum _{x,y\in p}{I}_{mov}\left(x,y,t\right)}}{{\displaystyle \sum _{x,y\in p}1}}\times 100\%$$

Where:

activity = animal activity index (%);

t = time index (s);

I_mov = pixel change of two frames,

( x , y) = image coordinates.

The image occupation index was calculated as the ratio between the pixel values of the area with chickens and the total number of pixels in one image to obtain the animal occupation index ( Bloemen et al. 1997 ):

(3) $$occupation\left(t\right)=\frac{{\displaystyle \sum _{x,y\in p}{I}_{animal}\left(x,y,t\right)}}{{\displaystyle \sum _{x,y\in p}1}}\times 100\%$$

Where:

occupation = animal occupation index (%);

t = time index (s);

I_animal = the total pixel values of areas with chickens,

( x , y) = image coordinate.

Frame difference data were analysed in ways that avoided the influence of increasing the body size of the birds (all birds were of similar size over the experimental period because they had grown to a stable body weight between 48 and 57 weeks old).

STATISTICAL ANALYSIS

Statistically, the experimental data were calculated to obtain the mean value and standard deviation of the activity index as well as the occupation index in different weeks. The mean prediction model (MPM) based on the normal mean value (no human intervention, immune operation and facility breakdown) of both the activity and occupation index of the last week was modelled in the training set (38 days with full 24-hour data per day) to detect abnormal activity or occupation events (including human intervention, immune operation and facility breakdown) ( Du et al. 2018a ). When measured values deviated from the mean value, a warning signal occurred. More than 25% of negative or positive deviation from measured values raised the alarm when it lasted for 30 min or longer ( Kashiha et al. 2013b ). An event logbook was recorded by one trained farmer at the same time and was regarded as the ground truth to evaluate the MPM accuracy in abnormal event warnings in the testing set (11 days with full 24 hours of data per day) ( Du et al. 2018a ). The performance parameters were calculated as ( Lao et al. 2016 ):

(4) $$P_{accuracy}=1-P_{FPR}-P_{FNR}$$

Where:

P_accuracy is the accuracy of abnormal event warnings,

P_FPR is the rate of misclassifying normal events as abnormal events (i.e., false positives), whereas P _FNR is the rate of misclassifying abnormal events as normal events (i.e., false negatives).

RESULTS AND DISCUSSION

Figure 4 and Figure 5 depict the MPM of the activity and occupation quantified index in the training set (no abnormal activities or events included). As shown in these figures, the fluctuation in the amplitude of the activity index was lower than that of the occupation index. A higher activity index and a higher occupation index appeared during the daytime. This can be explained by chickens being active in the daytime along with a high activity index, and they expressed their natural instincts, such as stretching, preening and pacing ( Kuhne et al. 2013 ; Pereira et al. 2013 ). However, chickens tend to be silent at night and gather together for resting and inactivity, which might result in a lower occupation index ( Du et al. 2018b ). Additionally, the flock expressed a regular change pattern of activity and occupation index per day. Each week also showed a similar change pattern. Chickens tended to be more active during the light period from 4:00 a.m. to 8:00 p.m. than during the night period. There were great differences in day and night according to bird activity and the occupation index, which quantitatively confirms diurnal patterns of bird activity. The result of this method of perceiving animal activity patterns is similar to that of a preliminary experiment ( Du et al. 2018a ).

FIGURE 4
Week MPM of the activity index in the training set.

FIGURE 5
Week MPM of the occupation index in the training set.

Then, the MPM method was applied to detect abnormal activity events in the testing set to evaluate its performance (no abnormal occupation index occurred). Abnormal events that occurred in the testing set included human intervention, immune operation and feeding restriction, which could impose a negative impact on chicken activity ( Kashiha et al. 2013b ; Du et al. 2018a ).

As shown in Figure 6 , abnormal activity events were easily detected by the MPM method. In Figure 6(a) , an abnormal activity warning occurred between 4:30 p.m. and 5:00 p.m. and it was caused by artificial feeding restriction. It has been proven that if chickens cannot obtain access to food, they might express a strong demand for food, feel frustrated, and produce more food calls ( Kuhne et al. 2013 ). Figures 6(b) , 6(c) , 6(d) and 6(f) display similar warnings during feeding time, which might be caused by feeding restrictions. In Figure 6(b) and 6(g) , warning events occurred between 2:30 p.m. and 4:30 p.m., which might be related to flock stress caused by artificial immune operations. Figure 6(b) (from 6:30 p.m. to 7:00 p.m.), Figure 6(d) (from 7:30 p.m. to 8:00 p.m.) and Figure 6(g) (from 5:30 p.m. to 6:00 p.m.) present warning events; these were false positive warning events that were not recorded in the farmer logbook. In Figure 6(c) , another warning event occurred between 11:00 p.m. and 11:30 p.m. This might be caused by environmental changes that were related to the ventilation control system. The experimental cage was near the air intake, and a change in night ventilation control strategy might influence flock activity ( Du et al. 2018a ). In Figure 6(d) , an abnormal activity warning occurred between 4:30 a.m. and 5:30 a.m. This might be related to flock stress caused by the manual operation of eliminating weak chickens, as it broke the poultry’s circadian clock ( Hy-Line, 2016 ; Shimmura & Yoshimura, 2013 ). In Figure 6(e) , another abnormal activity warning occurred between 9:00 a.m. and 10:00 a.m. This might be related to flock stress caused by noise from pipe maintenance. Compared with the farmer logbook, this MPM method can detect some abnormal activities and might help farmers trace possible reasons for activity event warnings. For example, as shown in Table 1 , 5 days into the testing set with abnormal activity displayed that the laying rate was lower than that in the neighbouring two days. The lowest laying rate in the testing period was 83.7% (24th July), which might be caused by higher average relative humidity ( Table 1 ). Additionally, abnormal water consumption might exist along with abnormal activity warnings.

FIGURE 6
Results of the MPM in the testing set of (a) 22nd July in week 49, (b) 24th July in week 50, (c) 25th July in week 50, (d) 26th July in week 50, (e) 15th August in week 52, (f) 20th August in week 53, and (g) 23rd August in week 54. A black line denotes a week MPM. A black-dotted line denotes the normal activity index. Red-dotted lines indicate the abnormal activity index.

TABLE 1
Hens’ production performance and environmental parameters in the testing set.

Data set	Date	Laying rate (%)	Dead and culled hen number	Feed-gain ratio (g)	Water consumption (mL)	Environmental parameters
						Average temperature (°C)	Average relative humidity (%)
Testing set	07-22	91.9	1	122.7	28.5	23.0	78.1
	07-24 #	83.7 *	0	134.7	23.2	23.0	96.0
	07-25	92.5	0	121.5	34.5	23.5	81.0
	07-26	90.5 *	0	125.5	30.4	19.5	83.1
	08-15	89.4 *	0	129.5	38.3	22.5	80.2
	08-20	90.5 *	0	127.1	63.4	22.0	66.3
	08-23 #	89.9 *	1	129.6	27.2	23.0	79.2

• ^*
  denotes that the value is lower than that in the neighbouring two days.
• ^#
  denotes an immune operation day.

Whatever type of physical or mathematical model we use, experience shows that the hard work starts when the model is implemented in real livestock houses ( Dawkins et al. 2012 ). One of the main challenges of the MPM method is the lack of enough events in the logbook, especially events recorded at night. Although seven weeks (49-55) of data detected some abnormal activity events through depth image analysis, more abnormally recorded events at daytime and nighttime might be more helpful for the performance assessment of the MPM method and feature description of poultry movement. Additionally, the MPM method could be used to predict animal activity conditions of the next week, which is simple, fast and implementable for real-time application. It was found that 32 out of 38 abnormal activity events were correctly detected (the false positive rate was 15.8%, and the false negative rate was 0.0%) ( Table 2 ). The accuracy was higher than 73.9%, which resulted in a preliminary test ( Du et al. 2018a ). The reference methods show accuracies of 95.2% ( Kashiha et al., 2013b ) and 61.9% ( Pluk et al., 2010 ). Compared with the optical flow method, the MPM method could detect animal movement in full 24 hours without considering the influence of the change in light levels, and it can also help farmers obtain more details, such as the animal activity index and occupation zones, both day and night. Furthermore, more data are needed to verify that this MPM method would be suitable for monitoring a whole animal production period.

TABLE 2
MPM accuracy of abnormal event warnings.

Testing set	Abnormal activities number	Number and P FPR	Number and P FNR	P accuracy
Week 49	2	0 (0.0%)	0 (0.0%)	100.0%
Week 50	24	4 (16.7%)	0 (0.0%)	83.3%
Week 52	3	0 (0.0%)	0 (0.0%)	100.0%
Week 53	5	0 (0.0%)	0 (0.0%)	100.0%
Week54	4	2 (50.0%)	0 (0.0%)	50.0%
Total	38	6 (15.8%)	0 (0.0%)	84.2%

This study attempts to monitor animal activity quantitatively to provide early warning of abnormal events that affect production performance. An in-depth exploration into the occurrence rule of abnormal events has not yet been conducted, which is also the next step in this study. Moreover, different image frames can, to some extent, influence the analysis results, such as one frame per second or one frame per five seconds. However, in this study, deep research has not yet been carried out. The key problem may be the smaller number of chickens in the experiment compared with a similar broiler warning system ( Kashiha et al. 2013b ), which is due to the difficulty of device installation caused by stacked-cage design and the viewing angle of a depth camera. If the problem can be solved in future research, more detailed and associated information may be explored, and the relationship between animal activity and production performance may be well understood.

CONCLUSIONS

An MPM method was proposed to continuously and noninvasively monitor hen activities and quantified occupation indices in real time. It can report abnormal activity events caused by human intervention, immune operation and feeding restriction in daily stacked-cage management. This method can help farmers monitor their animals and rapidly detect abnormal events in a 24-hour day. In the testing stage, MPM was able to detect abnormal activity events with an average accuracy of 84.2%. By using depth image processing, it is possible to eliminate the interference of environmental light and external shielding, such as cages and cables. Compared to colour images, depth images are more suitable for animal behaviour detection and analysis at low illuminance. Additionally, the flock showed a diurnal pattern in the change in activity and occupation index, and each week presented a similar change pattern. Further research will consider the application of this method in other systems, such as enriched cages and no-cage systems.

ACKNOWLEDGEMENTS

This work is funded by the National Key Research and Development Program of China (2016YFD0700204 and 2017YFD0701602) and is supported by the Key Laboratory of Agricultural Engineering in Structure and Environment, which provided research farms, hardware, and technical assistance. Especially, thanks to Xiuyun Bai and Xuming Liu for providing poultry management help.

REFERENCES

• Bloemen H, Aerts JM, Berckmans D, Goedseels V (1997) Image analysis to measure activity index of animals. Equine Veterinary Journal Suppl 16-19.
• Costa A, Borgonovo F, Leroy T, Berckmans D, Guarino M (2009) Dust concentration variation in relation to animal activity in a pig barn. Biosystems Engineering 104: 118-124.
• Dawkins MS, Cain R, Roberts SJ (2012) Optical flow, flock behaviour and chicken welfare. Animal Behaviour 84: 219-223.
• Du XD, Cao YF, Teng GH (2018a) A method based on image and sound processing for monitoring abnormal events in a breeder house. Journal of China Agricultural University 12: 114-121.(in Chinese)
• Du XD, Lao FD, Teng GH (2018b) A Sound source localisation analytical method for monitoring the abnormal night vocalisations of poultry. Sensors 18: 2906.
• EFSA (2012) Scientific Opinion on the use of animal-based measures to assess welfare of broilers. EFSA Journal 10: 2774.
• Gregersen T, Jensen T, Andersen M, Mortensen L, Maselyne J, Hessel E, Ahrendt P (2013) Consumer grade range cameras for monitoring pig feeding behaviour. In: 6th European Conference on Precision Livestock Farming: p360–369.
• Hy-Line (2016) Parent stock management Guides. Available: http://www.hyline.com/aspx/general/dynamicpage.aspx?id=255
  » http://www.hyline.com/aspx/general/dynamicpage.aspx?id=255
• Jana A (2012) Kinect for Windows SDK Programming guide. Birmingham, UK, Packt Publishing.
• Kashiha M, Bahr C, Haredasht SA, Ott S, Moons CPH, Niewold TA, Ödberg FO, Berckmans D (2013a) The automatic monitoring of pigs water use by cameras. Computers and Electronics in Agriculture 90: 164-169.
• Kashiha M, Pluk A, Bahr C, Vranken E, Berckmans D (2013b) Development of an early warning system for a broiler house using computer vision. Biosystems Engineering 116: 36-45.
• Kuhne F, Sauerbrey AFC, Adler S (2013) The discrimination-learning task determines the kind of frustration-related behaviours in laying hens (Gallus gallus domesticus). Applied Animal Behaviour Science 148: 192-200.
• Lao FD, Brown-Brandl T, Stinn JP, Liu K, Teng GH, Xin HW (2016) Automatic recognition of lactating sow behaviors through depth image processing. Computers and Electronics in Agriculture 125: 56-62.
• Lao FD, Du XD, Teng GH (2017) Automatic recognition method of laying hen behaviors based on depth image processing. Transactions of the Chinese Society for Agricultural Machinery 48(1): 155-162(in Chinese).
• Ni J, Liu S, Radcliffe JS, Vonderohe C (2017) Evaluation and characterisation of Passive Infrared Detectors to monitor pig activities in an environmental research building. Biosystems Engineering 158: 86-94.
• Norton T, Berckmans D (2017) Developing precision livestock farming tools for precision dairy farming. Animal Frontiers 7: 18.
• Pereira DF, Miyamoto BCB, Maia GDN, Tatiana Sales G, Magalhães MM, Gates RS (2013) Machine vision to identify broiler breeder behavior. Computers and Electronics in Agriculture 99: 194-199.
• Pluk A, Cangar O, Bahr C, Vranken E, Berg GVD, Berckmans D (2010) Impact of process related problems on water intake pattern of broiler chicken. International Conference of Agricultural Engineering. Cemagref. Proceedings…
• Porto SMC, Arcidiacono C, Anguzza U, Cascone G (2013) A computer vision-based system for the automatic detection of lying behaviour of dairy cows in free-stall barns. Biosystems Engineering 115: 184-194.
• Shimmura T, Yoshimura T (2013) Circadian clock determines the timing of rooster crowing. Current Biology: CB 23, R231-233.
• Springer S, Seligmann GY (2016) Validity of the Kinect for Gait Assessment: A Focused Review. Sensors, 16.
• Van Hertem T, Maltz E, Antler A, Alchanatis V, Schlageter-Tello A, Lokhorst C, Romanini CEB, Viazzi S, Bahr C, Berckmans D, Halachmi I (2013) Automatic lameness detection based on 3D-video recordings. In: European Conference on Precision Livestock Farming.
• Viazzi S, Van Hertem T, Romanini CEB, Bahr C, Halachmi I, Schlageter Tello A, Lokhorst C, Rozen D, Berckmans D (2013) Automatic back posture evaluation in dairy cows using a 3D camera. In: European Conference on Precision Livestock Farming: p 83–91.

Edited by

Area Editor: Héliton Pandorfi

Publication Dates

• Publication in this collection
  25 June 2021
• Date of issue
  May-Jun 2021

History

• Received
  11 Dec 2020
• Accepted
  01 Mar 2021