A statistical approach to static and dynamic tests for Global Navigation Satellite Systems receivers used in agricultural operations

The adaptation of the Global Navigation Satellite Systems (GNSS) technology to fit the needs of farmers requires knowledge of the accuracy level delivered by a GNSS receiver in working conditions. To date, no methodology indicates the minimum number of replications to perform a statistical comparison. This study aims to advance knowledge on the methodological approach for evaluating the static and dynamic performance of GNSS receivers commonly used in agricultural operations. For the static test, a supporting frame in the ground carried all the receivers with coordinates properly transported. In the dynamic test, a circular rail with a 9.55 m radius was installed at ground level with a platform driven by an electric motor to carry the receivers at a constant speed. The transversal error of the receiver to the circular reference line was measured. The error with 95 % probability (E95) to receivers without differential correction ranged between 4.22 m and 0.85 m in the static test, and 2.25 m and 0.98 m in the dynamic test. Receivers with differential correction had E95 values below 0.10 m in the static test and 0.16 m in the dynamic test. Receivers with C/A code require five replications at minimum and 13 replications are needed for L1/L2 with differential correction signals in the dynamic test. The static test needs nine replications for C/A and five for L1/L2 with differential correction signals.


Introduction
The use of Global Navigation Satellite Systems (GNSS) in agriculture has evolved, enabling a revolution in georeferenced data collection, which is performed faster, more accurately, and less costly. Precise positioning is demanded in several agricultural applications (Kabir et al., 2016). Accuracy of commercial GNSS receivers is one of the requirements from farmers and depends on operations to be performed in the field. Thus, the accuracy of its signals and its behavior in agricultural settings should be further investigated. Some challenges in agricultural operations involve the selection of an adequate source of differential correction signals, depending on the accuracy required, and the choice for the right option, regarding availability, practical aspects, and costs.
Some agricultural operations, such as autoguidance technology for precision inter-row cultivation, require high accuracy, normally below 0.10 m, and it is only possible with the use of differential correction signals (Machado and Molin, 2011). Real-Time Kinematic (RTK) and Satellite-based Augmentation Systems (SBAS) with signal distributed by geostationary satellites stand out as the main options for users of real-time differential correction. The use of Network RTK can be an option to share costs without degrading positional accuracy (Bae and Kim, 2018). However, for remote agricultural areas in Brazil, there is little or no access to Internet networks, making Network RTK unavailable or costly. Machado et al. (2010) and Machado and Molin (2011) evaluated the performance of GNSS receivers installed on an agricultural vehicle, with circular paths in a constant trajectory, using a GNSS receiver with RTK signal as a reference for accuracy and precision calculations. Carballido et al. (2014) and Kabir et al. (2016) evaluated the performance of multi-GNSS receivers in static and dynamic conditions in different agricultural sites.
Due to limited information on the performance of GNSS receivers and signals for static and dynamic tests, this study aims to advance knowledge on the methodological approach for evaluating the static and dynamic performance of GNSS receivers commonly used in agricultural operations, including statistical issues related to the number of replications.

Materials and Methods
A supporting frame was allocated at the geographic coordinates 22°42'47.8" S, 47°37'44.9" W, altitude of 540 m. Ten different GNSS receivers, 0.50 m apart, were placed on top of the support frame with the geographic coordinates properly established (Figure 1).
Field test procedures are used for evaluating satellite-based auto-guidance systems in agriculture (Sama and Stombaugh, 2014;Kim et al., 2016;Santos et al., 2017). The American Society of Agricultural and Biological Engineers (ASABE) suggests ASABE X605 (2008) for automated satellite guidance system testing during direct and level travel. ISO 12188-2 (2012) is a standard for testing automatic satellite guidance systems based on ASABE X605, which defines a tracking sensor as an instrument or instrumented system designed to perform repeated horizontal distance measurements required for cross-track error calculations. To perform the dynamic tests, a circular metal rail with 9.55 m radius, centered at the same point, similar to the static test, was placed at level ground, with a platform driven by an electric motor to perform the circular path (Figure 2A) of the GNSS receivers at a constant speed (1.0 m s -1 ). The platform stayed 2.0 m behind the electric motor ( Figure 2B) to avoid possible electromagnetic interference to the antennas of GNSS receivers ( Figure 2C).
Only the path error was obtained, based on the perpendicular distance between the receiver position and the segment of the reference line ( Figure 3A). To avoid assuming a perfect circumference on the rail, we used an L1/L2 GPS+GLONASS receiver, model GR-3, with accuracy 0.003 m+ 0.50 ppm with RTK differential correction in a dynamic condition, with 15.0° elevation mask. The receiver collected positioning data at a frequency of 1 Hz in dynamic condition for 3,600 s, generating the reference line (ION STD 101, 1997). The receiver errors were calculated by the difference between the radial value of the reference line radius (R) and the radial distance (D) between the point generated by the receiver and the reference coordinate ( Figure 3B).
The site where the tests were performed was not completely free of obstacles; however, it is considered adequate to conduct the tests, since it replicates real field conditions, especially the boundary of agricultural fields, with the presence of trees. The shading profile is represented by showing that obstacles did not exceed 30.0° ( Figure 4).
We used receivers with different specifications and accuracy levels, commonly applied in agriculture (Table  1). Group A comprises receivers without differential correction and Group B has receivers with SBAS or RTK correction. SBAS is a geostationary satellite system that improves the accuracy, integrity, and availability of GNSS signals. It does not require local infrastructure and offers competitive accuracy as a private service, commonly used for agricultural purposes in Brazil (Rovira-Más et al., 2015;Banville et al., 2018;Li et al., 2018).   Software C7 (CR Campeiro) was installed on the cell phone to capture and save data. For data collection of L1 and L1/L2 receivers, portable computers were used with SST Field Rover II 7.13 software. For C/A code receivers, data was stored by the internal memory of the equipment.
All GNSS receivers were set to collect data at 1.0 s interval (1.0 Hz), logging for 4,200 s with five replications in the static test and seven replications for the dynamic test, keeping 3,600 s intervals between replications to allow complete reconfiguration of satellite constellations. GNSS involves signals from satellites subject to interference and provide positioning results with a range of common errors over time (Souza and Machado, 2016;Silva and Marques, 2016;Ye et al., 2018).
Before analyzing the data, the initial and the final 300 s were eliminated, using 3,600 s (3,600 points collected). Data were generated in decimal geographical coordinates in Datum WGS 84 and converted to the metric UTM system using software QGIS 2.18 (OSGEO, version 2.18.26).
For the static test, North-South (E NS ) (Equation 1) and East-West (E EW ) (Equation 2) errors were calculated, resulting in the radial error (E) (Equation 3), generated by the receiver in relation to the reference point, as described by Machado et al. (2010), using MS Excel.
where: (Xref, Yref) is the reference point coordinate (m); (Xreal, Yreal) is the coordinate of the GNSS receiver (m). An algorithm based on Java language was developed for analyzing data obtained by the dynamic test, using software NetBeans IDE (version 8.1) to create the reference line and calculate the path error. Both were expressed as an error with 95 % probability (E 95 ) and for each receiver, a root mean squared (RMS) error was calculated (Equation  4) (Pérez-Ruiz and Upadhyaya, 2012).
where: RMS t is the RMS error for the t th receiver; N t is the total number of measurement points for the t th receiver; E is the error of the point i th to the t th receiver.
For the statistical analysis, errors were calculated for each point and the SAS general linear models procedure was used to test for significant differences  between receivers using the ANOVA. The Duncan test was used to compare the performance of the different receivers for their E 95 mean accuracy, at 5 % error probability level. The probability transformation suggested by Box and Cox (1964) was applied in all distributions, transforming E values to stabilize or reduce variability between replications. To compare the performance of different receivers with SBAS and RTK signal correction, we did not compare replications with a loss of signal correction. The number of replications may increase accuracy (Zhang et al., 2018) and improve the capacity of a statistical test to detect smaller differences between estimates of the means of treatments (Danilogorskaya et al., 2017). Sensitivity of a statistical test is largely influenced by sample size. The test power was calculated to verify if a supposed difference between accuracies of receivers was genuine or subject to sample error for the static and dynamic tests. The test power corresponds to 1-β, the probability of rejecting the null hypothesis when it is false, and indicates the correct decision probability based on the alternative hypothesis. It is usually interpreted as the chance to detect a real difference between means or proportions. The simulations were made for various combinations of replication numbers with the nominal level of significance α equals to 5 %, admitting a difference between accuracies of receivers equals to the standard error of the mean.

Results and Discussion
In the static test, the error dispersion for C/A receivers was relatively larger than for other receivers ( Figure 5). The L1/L2 receivers without differential correction (ID 6) resulted in a low error dispersion. However, there was a shift of the data towards the south of the reference, generating greater E NS that decreased the accuracy of the receiver. Receiver ID 5 also showed greater dispersion of errors in the 2 nd and 3 rd quadrants. Receivers with SBAS differential correction have similar dispersion of E NS and E WE . A large number of observations in the 2 nd quadrant of receiver ID 9 may be attributed to problems with the SBAS signal correction, observed during the tests. Receiver ID 10, with RTK differential correction, displayed less dispersion of errors when compared to receivers with SBAS.
The L1/L2 receivers without differential correction showed low variation amplitude of E during data collection, within its replications ( Figure 6). However, there was great variation in the temporal stability of the error. In the 3 rd replication of receiver ID 7, with SBAS differential correction, there was a visible increase of E. In contrast, in the 4 th replication, E started with a high value and decreased over time. Lim et al. (2019) described it as a factor associated with the need of providing the same signal environment to all receivers.
No other receiver with differential correction presented such variation in E at the same time, indicating that receiver ID 7 had the error variation due to the quality of the differential correction signal. Receivers IDs 8 and 9 show less E variation than receiver ID 7, and receiver ID 9 lost connection with the signal correction during the 3 rd replication. This may explain the greater dispersion of E in the 2 nd quadrant, shown in Figure 6. The comparison between error dispersions of receiver ID 6 (without differential correction) with receivers ID 7, 8, and 9 showed accuracy improvement provided by SBAS. However, depending on the accuracy level required in agricultural operations, the lack of correction signals may increase the positioning error. Bažec et al. (2020) identified the same situation during the evaluation of nine GNSS receivers. Figure 7 shows the dispersion of errors during the dynamic tests. There was a small error dispersion for the receiver with RTK differential correction in relation to the other receivers. In general, the dynamic test conditions resulted in greater dispersion of errors when compared to the static test. The L1/L2 receiver without differential correction (ID 6) showed high error variation during the time of data collection within replications, which did not occur during the static test, with error values up to 1.45 m.  In addition, a variation of the position errors of receivers with SBAS leads to a greater differential correction in the dynamic tests. Similar to the static tests, there was a lack of correction signal for receivers with SBAS signal correction, which can be visualized in replication number 4 of receiver ID 8 and replication number 1 of receiver ID 9 when there is more variation of positioning errors. Although these two replications presented a large amplitude of errors, the other replications presented variation below 0.250 m. However, when compared to the static test, there is greater error variation in the dynamic tests.
The E 95 of receivers, without differential correction (Group A), varied from 0.850 m to 4.218 m in the static test and from 0.976 m to 2.256 m in the dynamic test (Table 2). In addition, there was no significant difference in accuracy between receiver ID 2, with only the GPS signal, and receiver ID 3, with GPS+GLONASS signals. A smaller positioning error of receiver ID 3 is expected due to a large number of satellite connections (Banville et al., 2018); however, it is only true if the number of visible satellites is the issue, such as in working conditions under the tree canopy. In the static test, the accuracy of receiver ID 1 is lower than that of receivers IDs 5 and 6. Although receiver ID 1 has low accuracy (E 95 = 4.218 m), it is not significantly different from the other C/A receivers. In Group A, receiver ID 6 has the highest accuracy, with E 95 = 0.850 m in the static test and 0.976 m in the dynamic test. In the static test, values of E 95 are close to the RMS value, indicating low data dispersion, whereas the dynamic test shows greater variation between E 95 and RMS.
According to Table 2, the coefficient of variation (CV) between the replication for Group A did not demonstrate any standard. Some CV values were higher (> 50 %) in the static test in relation to the dynamic test (19 %). For Group B, except for receiver ID 7, there was an increase in error variation between replications in the dynamic test. It is considered that the lower the CV estimates, the greater the precision of the experiment and vice versa, and the greater the experimental precision.
It was expected an L1/L2 receiver without differential correction to be more accurate than C/A code and L1 band receivers. This is due to better reception GNSS receivers for agricultural operations Sci. Agric. v.78, n.5, e20190252, 2021 capacity of the signal, as it can receive weak signals from satellites without self-interference, and it has a better structure for positioning data recovery. Considering that ionospheric delay is a relevant error source in multi-GNSS positioning, Su et al. (2019) and Wang et al. (2019) verified that the combination of multi-GNSS could effectively improve the positioning performance.
Receivers of Group A, except receiver ID 6, had greater accuracy in the dynamic test. It was expected greater error variation in the dynamic test in relation to the static test, due to the continuous changes in the configuration of the satellite constellation and response time of the receiver, causing greater accuracy variation of GNSS positioning (ASABE, 2008). When performing the static and dynamic tests in different moments, these factors also change, implying greater dataset variability.
For GNSS L1/L2 with differential correction, there was greater variability of E 95 between replications in the dynamic test in relation to the static test (Table 2). In a static test, GNSS receivers with differential correction had an E 95 lower than 0.100 m and E 95 of receiver ID 7 was significantly different from GNSS receiver ID 10, with RTK differential correction. E 95 of all receivers with correction used in the static test was closer to the mean errors, indicating low variation in positioning.
GNSS receivers with SBAS resulted in E 95 greater than receiver ID 10, with RTK differential correction in the dynamic test and SBAS receivers had higher E 95 in the dynamic test when compared to the static test. Despite greater variation in the positioning of GNSS receivers in the dynamic test, E 95 of receiver ID 10 was higher in the dynamic test.
Comparing L1/L2 GNSS receivers, E 95 of the receiver without differential correction is greater than the other receivers with differential correction for both tests (static and dynamic). This means that the use of an L1/L2 GNSS receiver without differential correction could generate an E 95 of 0.843 m higher than a receiver with SBAS signal correction in a static condition, and 0.976 m higher than a receiver with SBAS differential correction in the dynamic condition. Therefore, a lack of differential correction signals could provide a considerable increase in GNSS signal error during agricultural operations.
The effect of the replication number indicated that data from receivers with differential correction had a power of 81 % in the static and of 44 % in the dynamic test (Figure 8), which means that the tests were performed with a small number of replications. The low value of the test power for the dynamic test   in Group B could be explained by a greater variety of error values in the dynamic than in the static test.
Considering that higher CV implies less experimental accuracy and, consequently, more replications are required to represent GNSS positioning error.
To test receivers with a differential correction to have at least a power of the 80 % (β = 20 %), we carried out seven replications for the static test and 13 replications for the dynamic condition. For receivers with no differential correction (Group A), the test power was 46 % for the static test and 96 % for the dynamic test conditions, showing that nine replications are necessary for the static and five replications for the dynamic test conditions to achieve a minimum test power of 80 %. The results demonstrate that the number of replications suggested in the power test to compare different GNSS with differential correction complies with the number of replications indicated by ION STD 101 (1997).

Conclusion
This study developed a methodology that allowed the behavior analysis of GNSS receivers of different accuracy levels under static and dynamic conditions. The test under dynamic conditions measuring error perpendicular to the path simulates agricultural demands, for example, the use of GNSS for auto-guidance on agricultural operations. The signal of receivers presented greater variability of positioning errors in the dynamic than in the static condition, consequently, we observed the need to increase the number of replications to improve the power of the statistical test. A minimum of five replications is necessary for the dynamic tests to C/A receivers and 13 replications for L1/L2 receivers with differential correction signals. The static tests require a minimum of nine replications for C/A receivers and five replications for L1/L2 receivers with differential correction signals.