ABSTRACT
To enhance the automation and efficiency of combine harvesters, this paper proposes a predictive control method based on Long Short-Term Memory (LSTM) neural networks. The method integrates multi-sensor data fusion using an Extended Kalman Filter (EKF) to improve speed measurement accuracy. By considering feeding volume, operational performance indicators, and critical component speeds, an LSTM-based model predicts the optimal operation speed. The predicted speed is then regulated through an incremental proportional-integral-derivative (PID) control control system. Simulation and field experiments validate the effectiveness of the proposed approach, demonstrating improved speed stability and work efficiency. The results indicate that the system enhances operational performance and reduces manual intervention, contributing to the advancement of intelligent agricultural machinery.
harvesting automation; PID control optimization; LSTM neural network; multi-sensor fusion; smart agricultural machinery
INTRODUCTION
The combine harvester is a critical piece of agricultural machinery (Shaikh et al., 2022) that plays a pivotal role in modern farming. The automation and intelligent control of combine harvesters have the potential to reduce the labor intensity of operators significantly, alleviate labor shortage, and enable more efficient and higher-quality harvesting operations. The combine harvester's stable and efficient performance depends on excellent speed control. As a result, agricultural machinery researchers have extensively investigated methods for regulating the operational speed of combine harvesters, with several notable outcomes.
Liu et al. (2023) proposed a feeding volume discrimination method based on association rule mining for the rapid identification of the feeding volume, which serves as the foundation for the realignment of the combine harvester's operational parameters. Omid et al. (2010) designed a fuzzy logic control system that integrates expert knowledge, such as stem weight and seed loss rate in the upper sieve. The system automatically regulated and controlled combine harvesters to improve operational efficiency and reduce seed loss rate. Wang et al. (2023) determined a model for the relationship between factors such as operation speed, feeding volume, threshing drum torque, threshing drum speed, and harvesting loss of wheat combine harvester. Operation speed is regulated by the real-time detection of the torque exerted on the threshing drum, the speed of the threshing drum itself, and the seed loss incurred by the combine harvester, effectively minimizing harvesting loss. Wang et al. (2021) devised an automated monitoring system for operational speed, featuring an adaptive variable domain fuzzy controller for feeding volume and a hydraulic speed control system. The operating speed PID controller regulates the hydraulic speed control system to achieve the optimal feeding volume. Yin et al. (2018) constructed a distributed data acquisition, transmission, and processing system, which achieves the synchronous aggregation of data such as machine performance, operational status, and power consumption. Chen et al. (2017) put forth a Knowledge Discovery in Databases (KDD) multi-parameter fuzzy control strategy (FCS) for the operation speed of combine harvesters. The primary objective of the operation speed multi-parameter FCS is to identify the optimal compromise between the acceptable weight coefficients of the cutting table auger pusher, the conveying chute, and the threshing rotor, in order to achieve the desired low loss rate and high feed volume. Guo et al. (2019) designed an automatic control system for the operation speed of a wheeled, self-propelled, square baler based on closed-loop control, analyzing the control characteristics of continuously variable transmission (CVT) systems. Mathematical models of the CVT transmission ratio and cylinder elongation were established to stabilize the vehicle speed under working conditions.
Despite these advancements, most current combine harvesters operate with fixed operation speed along predefined paths. Most control systems do not correlate the operation speed with the feeding volume and operation performance. Consequently, the quality and efficiency of the harvesting operation cannot be optimized. To address these issues, this paper proposes an operation speed prediction control system for combine harvester based on LSTM neural networks, which takes into account the problems above and combines them with the feeding volume, operational performance metrics (seed impurity rate, crushing rate, loss rate) and critical parts of the rotational speed (conveying chute rotational speed, threshing drum speed, stray churn rotational speed). The objective is to enhance harvesting efficiency and quality, and to provide the combine harvester with an operational speed adjustment regulation.
MATERIAL AND METHODS
System construction
The combine harvester is a nonlinear, large-delay system. This paper proposes a speed prediction control system for the combine harvester based on an LSTM neural network to ensure that the control effectiveness meets real-time control requirements. The central monitoring objectives are:
-
Feeding volume: Combine harvesting is a complex process, with variations in individual components affecting overall harvesting efficiency. The feed volume is the main factor influencing the performance of various components, making its monitoring essential (Sun et al., 2022). By regulating the combine harvester's operation speed to maintain stable feed volume, it ensures an appropriate and consistent flow through different operational stages. Insufficient feeding reduces operational efficiency, while overfeeding leads to clogging of critical components, increased failure rate, and reduction in harvest quality.
-
Operational performance metrics: To perform effective harvesting, combine harvesters must adhere to specific mechanized harvesting standards. The key quality indicators of machine harvesting operations are primarily concerned with seed impurity rate, crushing rate, and loss rate (Wu et al., 2023). These metrics also serve as operational performance indicators for combine harvesters, reflecting the quality of harvesting operations. They are of significant value in regulating the operation speed of combine harvesters.
-
Rotation speed of the main components: The combine harvester operation process comprises a multitude of rotating components (Goossens et al., 2023), the majority of which are located at the front end of the grain harvesting paddle wheel, feeding churns, conveying churns, the middle of the threshing drum and cleaning threshing drum, debris churns, and conveying churns. Each of these components plays a vital role in the harvesting process. This paper focuses on monitoring the conveying chute rotational speed, threshing drum, and stray churn. The rotational speeds of these three rotating parts are monitored, and their respective operating conditions are obtained from the rotational speed information to assess the stability of the combine harvester’s overall performance.
In conclusion, the key metrics for regulating the operation speed of the combine harvester include feeding volume, seed impurity rate, crushing rate, loss rate, conveying chute rotational speed, threshing drum speed, and stray churn rotational speed. The general block diagram of the proposed operation speed prediction control system for the combine harvester is shown in Figure 1.
Acquisition of operation speed of combine harvesters
To enhance the precision of operation speed measurement in combine harvesters, a multi-sensor fusion technology (Zhang et al., 2024) is employed to integrate the BDS/GPS signal (Lu, 2021) with the inertial navigation Inertial Measurement Unit (IMU) signal. This integration leverages the respective strengths of both signals, thereby reducing the error information generated by either signal. This integration is expected to enhance the positioning and operation speed measurement accuracy of combine harvesters. The block diagram illustrating the EKF-based BDS/GPS and IMU sensor information fusion principle is shown in Figure 2.
The EKF is employed to estimate the position and speed of the combine harvester, in conjunction with a time-lapse expression, as shown in [eq. (1)]:
in which
X𝓂 and Y𝓂 - coordinates of the combine harvester;
θ𝓂 - angle between the line connecting the combine harvester to the origin and x-axis at that coordinate;
v𝓂 - linear speed of the combine harvester in this coordinate;
ω𝓂 - angular speed of the combine harvester in this coordinate.
The process is a nonlinear dynamic system, and the expressions of the state and observation equations are shown in [eq. (2)] and [eq. (3)]:
in which
X𝓂 - system state vector of the combine harvester at time 𝓂;
ƒ - transition function in the nonlinear state;
W - process noise;
Z𝓂- measurement;
h - nonlinear mapping function;
V - measurement noise.
The state estimation and covariance matrix are predicted using the EKF algorithm, as described in [eq. (4)] and [eq. (5)]:
in which
ƒ - kinematic model of the combine harvester;
A - Jacobian matrix derived from ƒ via Taylor expansion;
P - noise covariance matrix;
D - process noise covariance perturbation.
The correction updates are applied after the EKF algorithm completes the state estimation and covariance matrix prediction. The Kalman gain is calculated by measuring the covariance matrix with . Then, the Kalman gain is employed to update the system state vector and the noise covariance matrix. The expressions are shown in [eq. (6)] to [eq. (8)]:
in which
K - Kalman gain;
Q - measurement covariance matrix;
B - observation matrix.
The combine harvester was driven at a constant operation speed of 1 m/s, and the respective operation speed measurement results were recorded to compare the accuracy of BDS/GPS, IMU, and BDS/GPS-IMU fusion operation speed measurement. The operation speed measurement curves for each sensor under constant operation speed is presented in Figure 3.
Construction of LSTM neural network-based operation speed prediction model
The LSTM neural network (Yadav & Thakkar, 2024) addresses the gradient problems of the recurrent neural network (RNN) (Theerthagiri & Menakadevi, 2022). It utilizes distinct gating mechanisms to regulate data flow, mitigating the challenges of gradient vanishing and explosion commonly encountered in RNN (Cai et al., 2023). The structure of the LSTM neural network is shown in Figure 4.
The forget gate uses the Sigmoid function to complete the storage and update of data. The expression for the forget gate is shown in [eq. (9)]:
in which
Ft - forget gate;
σ - sigmoid function;
Wƒ - weight matrix of the forget gate;
ht-1 - output of the cell at time t-1;
Xt - input of the cell at time t;
bƒ - bias of the forget gate.
The purpose of the input gate is to incorporate new information. The expression of the input gate is shown in [eq. (10)] and [eq. (11)]:
in which
It - input gate;
- information added at time t;
Wi - weight matrix of the input gate;
Wc - weight matrix of the cell state;
bi - bias of the input gate;
bc - bias of the cell state;
tanh - activation function.
The cell state is updated by filtering the information added through the input gate with the forget gate. The expression for updating the cell state is shown in [eq. (12)]:
in which
Ct - updated cell state.
The output gate is employed to modify the output values of the implicit layer. The expression for the output gate is presented in [eq. (13)] and [eq. (14)]:
in which
Ot - output gate;
Wo - weight matrix of the output gate;
bo - bias of the output gate;
ht - cell output.
Suitable operation speed is an essential guarantee for combine harvesters to achieve high-quality and efficient harvesting operations. To construct an operation speed prediction model based on an LSTM neural network (Ounoughi & Yahia, 2023), it is necessary to combine the current operation speed with factors such as feeding volume, seed impurity rate, crushing rate, loss rate, conveying chute rotational speed, threshing drum speed, and rotational churn speed.
This paper explores a variety of configurations for the time sliding window, the number of nodes in the hidden layer, and the number of iterations in the parameter tuning of the operation speed prediction model, to enhance the precision of the LSTM neural network-based operation speed prediction. The results of comparative tests on the predicted operation speeds with different metrics are shown in Table 1.
As shown in Table 1, the speed prediction models exhibit varying degrees of effectiveness across different parameter values. Gradually increasing the time-sliding window size, the number of nodes in the hidden layer, and the number of iterations revealed no significant increase in model effectiveness. When the sliding window size was fixed, it became evident that the number of nodes in the hidden layer gradually increased in root mean square error (RMSE) after exceeding a specific value. This resulted in a greater discrepancy between the model's predicted and actual values,
diminishing the model's predictive efficacy. When the number of hidden layer nodes was fixed, the RMSE decreased as the number of iterations increased. The optimal model prediction is achieved when the sliding window size is set to 4, the number of hidden layer nodes is 15, and the number of iterations is 60. Figure 5 illustrates the operation speed prediction model fitting under the multi-parameter combination of the sliding window, hidden layer nodes, and number of iterations (4, 15, 60).
Operation speed controller design and simulation
In the combine harvester operation speed prediction control system, closed-loop control (Zaman & Birk, 2024) is utilized to set the predicted operation speed as the target speed. The acquired operation speed is calculated and converted into
a corresponding voltage value, which drives a solenoid valve to implement the closed-loop control of the combine harvester's operation speed. This paper employs incremental PID control (Mao et al., 2024), with the schematic diagram of the closed-loop control system shown in Figure 6.
The complex harvesting environment and the intricate mechanical transmission system of the combine harvester pose significant challenges in accurately modeling the control of its operation speed. As a result, developing an appropriate mathematical model to achieve the desired control effect remains a challenge. Variations in the volume of the combine harvester's feeding mechanism affect the efficiency of the internal working parts and the quality of the harvested grain. Precise, stable control of the operation speed prevents clogging of the working parts and reduces the seed impurity rate, crushing rate and loss rate, thereby ensuring high operational quality and efficiency.
This paper constructs a control model for the operation speed system to guarantee the high quality and efficiency of harvesting operations. The proposed operation speed prediction model is employed to determine the target operation speed, which the speed module utilizes to gather the optimized real-time speed as the target operation speed. The difference between the predicted and actual operation speeds is calculated and input to the incremental PID controller. This controller amplifies the error signal through a proportional amplifier to adjust the operation speed prediction control system. The valve opening is modified as a voltage signal via the solenoid valve to achieve the target operation speed.
The test prototype used in this study is the Haolong 4LZ-8F, equipped with a YC4DR140-T301 engine and a hydrostatic continuously variable transmission with a mechanical gearbox. The transfer function of the system-controlled object is modeled as a proportional and first-order inertia element. The simulation model of the operation speed prediction control system is built in Simulink (Tang & Zhuang, 2019), as shown in Figure 7.
The PID control parameters must be adjusted to accommodate rapid changes in operation speed to ensure high-quality and efficient harvesting operations. This can be achieved by implementing a prediction control system that responds more swiftly to changes, thereby reducing the amplitude of the oscillation of the transition response. The result is shown in Figure 8. As illustrated in Figure 8, when the target operation speed is fixed at 1.1 m/s, the system response time is less than 0.7 s; the operation speed rises more rapidly; and the system stabilizes within one second. Furthermore, there is no obvious overshooting or oscillation, indicating that the control system meets the requirements of the combine harvester to adjust the operation speed in field operations in real time.
RESULTS AND DISCUSSION
The test prototype was a Haolong 4LZ-8F wheeled full-feed combine harvester with the following specifications: length 6060 mm, width 3200 mm, height 3350 mm, cutting width 2.6 m, and rated feeding volume 8 kg/s.
The sensors and devices were installed on the combine harvester, and field experiments were conducted following system debugging. The combine harvester's operation speed was regulated by both manual control and the operation speed prediction control strategy proposed in this paper.
Before harvesting, the field was cleared, and the edges surrounding the selected test plots were trimmed. Distances were measured using a measuring tape, with markers placed at 10 m intervals before and after the test plots. Initially, five groups of manual control-based combine harvester harvesting trials were conducted, with each trial covering a distance of 30 m. No blockage faults were observed during these trials.
Figure 9(a) shows a slight decrease in operation speed at 10 s, accompanied by a sudden surge in feeding volume, which then returns to normal. This phenomenon may be attributed to the higher density of rice at particular location. It can be observed that when the operation speed is maintained within a specific range for harvesting, the corresponding alteration in feeding volume remains largely within a narrow fluctuation range. As shown in Figures 9(b) and 9(c), as the operation speed is increased, the seed impurity rate and crushing rate also rise. The change in the loss rate is notable, with all three rotational speeds fluctuating slightly within the normal range in the absence of a blockage fault.
After completing the manual control trials, five sets of harvesting trials were conducted using the speed prediction control strategy of the operation speed of the combine harvester proposed in this paper. No instances of clogging were observed during the five tests.
Figure 10(a) shows that the operation speed stabilizes rapidly, with minimal fluctuations in feeding volumes. Figures 10(b) and 10(c) demonstrate that the seed impurity and crushing rate stay within a specific range, with some fluctuations. The loss rate increases as the rate of seed impurity and brokenness increases. No significant difference is observed between the fluctuation of three rotational speeds and the manual control; all remain within the normal operation range.
Field experiments were performed using both manual hand control and the proposed automatic control strategy for comparison. The mean values for the respective trials for both the average feed and the average operational performance parameter are presented in Table 2.
At the conclusion of each experimental group, the impurity rate and crushing rate were manually measured by randomly collecting seeds from the silo and placing them in sample bags. The scavenging-entrainment loss rate was calculated manually by determining the total mass of seeds removed from the rear end of the canvas receiving combine and the total mass of seeds in the bin. As shown in Table 2, for a harvesting distance of 30 m, the average feeding volume of the five manually controlled trials was 6.87 kg/s, and the average harvesting time was 30.92 s. In contrast, for the five automatic control trials, the average feeding volume was 7.24 kg/s, and the average harvesting time was 28.11 s. No clogging faults were observed during the tests conducted under distinct control schemes.
Furthermore, the speeds of all three rotational components remained within the normal operation range, exhibiting minimal fluctuations. The three operational performance metrics of the combine harvester exhibited no significant differences, with the scavenging-entrainment loss rate being less than 2%, the crushing rate being less than 1%, and the impurity rate being less than 1.6%. The operation speed prediction control system proposed in this paper demonstrates an improvement in operation efficiency of approximately 9.1% compared to manual adjustment, achieving the expected outcomes while maintaining the requisite operational quality.
CONCLUSIONS
-
This study presents a predictive control system for the operation speed of combine harvesters, integrating multi-sensor fusion and deep learning techniques. The EKF integrates IMU and BDS/GPS signals to enhance operation speed measurement accuracy through multi-sensor fusion. An LSTM-based model is used for predictive speed adjustment. The control strategy is further refined through an incremental PID controller, ensuring real-time speed regulation.
-
Simulation and field experiments validate the effectiveness of the proposed system. The results demonstrate enhanced speed stability, increased feeding volume, and improved harvesting efficiency, as compared to manual operation. The findings highlight the potential of intelligent control techniques in advancing agricultural automation. Future work will explore adaptive learning strategies and extended field validation across different crop types and harvesting conditions.
ACKNOWLEDGMENTS
Fund Projects: Demonstration and Promotion Project of Modern Agricultural Machinery Equipment and Technology of Jiangsu Provincial Department of Agriculture and Rural Affairs (NJ2022-08); Jiangsu Provincial Scientific and Technological Achievement Transformation Project (BA2020054).
REFERENCES
-
Cai, Z., Feng, G., Wang, Q. (2023). Based on the improved PSO-TPA-LSTM model chaotic time series prediction. Atmosphere, 14(11), 1696. https://doi.org/10.3390/atmos14111696
» https://doi.org/10.3390/atmos14111696 -
Chen, J., Ning, X., Li, Y., Yang, G., Wu, P., & Chen, S. (2017). A fuzzy control strategy for the forward speed of a combine harvester based on KDD. Applied Engineering in Agriculture, 33(1), 15 - 22. https://doi.org/10.13031/aea.11299
» https://doi.org/10.13031/aea.11299 -
Goossens, J., Lenaerts, B., Devos, S., Gryllias, K., De Ketelaere, B., & Saeys, W. (2023). Anomaly detection on the cutter bar of a combine harvester using cyclostationary analysis. Biosystems Engineering, 226(C), 169-181. https://doi.org/10.1016/j.biosystemseng.2023.01.013
» https://doi.org/10.1016/j.biosystemseng.2023.01.013 -
Guo, H., Gao, G., Zhou, W., Lü, Q., Hao, L. (2019). Design and test of automatic control system for walking speed of wheeled self-propelled square baler. Nongye Jixie Xuebao/Transactions of the Chinese Society for Agricultural Machinery, 50(12), 107–114. https://doi.org/10.6041/j.issn.1000-1298.2019.12.012
» https://doi.org/10.6041/j.issn.1000-1298.2019.12.012 -
Liu, Y., Dai, D., Tang, C., Wang, X., Wang, S. (2023). Discriminating feed rate of combine harvester by using association rule mining. Elektronika ir Elektrotechnika, 29(3), 4–10. https://doi.org/10.5755/j02.eie.33859
» https://doi.org/10.5755/j02.eie.33859 -
Lu, Y. (2021). Implementation of BDS/GPS dual-mode software receivers. Navigation: Science and Technology, 10, 463-483. https://doi.org/10.1007/978-981-16-1075-2_9
» https://doi.org/10.1007/978-981-16-1075-2_9 -
Mao, Q., Xu, Y., Chen, J., Chen, J., & Georgiou, T. T. (2024). Maximization of gain/phase margins by PID control. IEEE Transactions on Automatic Control, 1-16. https://doi.org/10.1109/tac.2024.3417717
» https://doi.org/10.1109/tac.2024.3417717 -
Omid, M. L., Mobli, H., Alimardani, R., Mohtasebi, S., & Hesamifard, R. (2010). Design of fuzzy logic control system incorporating human expert knowledge for combine harvester. Expert Systems with Applications, 37(10), 7080 - 7085. https://doi.org/10.1016/j.eswa.2010.03.010
» https://doi.org/10.1016/j.eswa.2010.03.010 -
Ounoughi, C., Yahia, S. B. (2023). Sequence to sequence hybrid Bi-LSTM model for traffic speed prediction. Expert Systems with Applications, 236, Article 121325. https://doi.org/10.1016/j.eswa.2023.121325
» https://doi.org/10.1016/j.eswa.2023.121325 -
Shaikh, T. A., Rasool, T., Lone, F. R. (2022). Towards leveraging the role of machine learning and artificial intelligence in precision agriculture and smart farming. Computers and Electronics in Agriculture, 198(C), 107119. https://doi.org/10.1016/j.compag.2022.107119
» https://doi.org/10.1016/j.compag.2022.107119 -
Sun, Y., Liu, R., Zhang, M., Li, M., Zhang, Z., & Li, H. (2022). Design of feed rate monitoring system and estimation method for yield distribution information on combine harvester. Computers and Electronics in Agriculture, 201(C), 107322. https://doi.org/10.1016/j.compag.2022.107322
» https://doi.org/10.1016/j.compag.2022.107322 -
Tang, M., & Zhuang, S. (2019). On speed control of a permanent magnet synchronous motor with current predictive compensation, 12(1), 65. https://doi.org/10.3390/en12010065
» https://doi.org/10.3390/en12010065 -
Theerthagiri, P., Menakadevi, T. (2022). Mobility speed prediction using ARIMA and RNN for random walk mobility model in mobile ad hoc networks. Concurrency and Computation: Practice and Experience, 34(4), Article e6625. https://doi.org/10.1002/cpe.6625
» https://doi.org/10.1002/cpe.6625 -
Wang, W., Gong, Y., Bai, X., Tan, R., & Huang, W. (2021). Investigation on operating speed regulation system of mobile straw granulator. Nongye Jixie Xuebao/Transactions of the Chinese Society for Agricultural Machinery, 52(10), 186-195. https://doi.org/10.6041/j.issn.1000-1298.2021.10.019
» https://doi.org/10.6041/j.issn.1000-1298.2021.10.019 -
Wang, J., Fang, X., Wang, F., Liu, Y., Zhao, B. (2023). Study on the loss control system of wheat combine harvester. INMATEH-Agricultural Engineering, 69(1), 471–480. https://doi.org/10.35633/inmateh-69-44
» https://doi.org/10.35633/inmateh-69-44 -
Wu, J., Tang, Q., Mu, S., Yang, X., Jiang, L., Hu, Z. (2023). Design and test of self-leveling system for cleaning screen of grain combine harvester. Agriculture-Basel, 13(2), 377. https://doi.org/10.3390/agriculture13020377
» https://doi.org/10.3390/agriculture13020377 -
Yadav, H., Thakkar, A. (2024). NOA-LSTM: An efficient LSTM cell architecture for time series forecasting. Expert Systems with Applications, 238(F), Article 122333. https://doi.org/10.1016/j.eswa.2023.122333
» https://doi.org/10.1016/j.eswa.2023.122333 -
Yin, Y., Zhang, Y., Meng, Z., Du, C., Qin, W., & Guo, S. (2018). Design and test of self-leveling system for cleaning screen of grain combine harvesters. IFAC - Papers Online, 51(17), 855-860. https://doi.org/10.1016/j.ifacol.2018.08.087
» https://doi.org/10.1016/j.ifacol.2018.08.087 -
Zaman, A., & Birk, W. (2024). Online estimation of PID controllers and plant dynamics via multi-recursive least squares estimation from closed-loop I/O data. IET control theory & applications (Wiley-Blackwell), 18(6), 688-709. https://doi.org/10.1049/cth2.12605
» https://doi.org/10.1049/cth2.12605 -
Zhang, Y., Zhang, B., Shen, C., Liu, H., Huang, J., Tian, K., & Tang, Z. (2024). Review of the field environmental sensing methods based on multi-sensor information fusion technology. International Journal of Agricultural and Biological Engineering, 17(2), 1-13. https://doi.org/10.25165/j.ijabe.20241702.8596
» https://doi.org/10.25165/j.ijabe.20241702.8596
Edited by
-
Area Editor:
Tiago Rodrigo Francetto
Publication Dates
-
Publication in this collection
02 June 2025 -
Date of issue
2025
History
-
Received
26 Aug 2024 -
Accepted
04 Apr 2025




















