THE INFLUENCE OF FLIGHT CONFIGURATION, CAMERA CALIBRATION, AND GROUND CONTROL POINTS FOR DIGITAL TERRAIN MODEL AND ORTHOMOSAIC GENERATION USING UNMANNED AERIAL VEHICLES IMAGERY

A influência da Configuração de Voo, Calibração de Câmara e Pontos de Controle para Geração de Modelos Digitais de Terreno e Ortoimagens Usando Imagens de Veículos Aéreos Não Tripulados

Abstract:

Technological improvement in sensors and the use of computer vision algorithms made possible the generation of high accuracy mapping products (cm level) using data acquired by low-cost Unmanned Aerial Vehicles (UAV). However, the procedure to optimally set the aerial block configuration is not well understood for some users mainly due to the popularization of the UAV and its use by non-specialists. This study aims to contribute to this aspect, investigating and highlighting the influence of flight parameters, camera calibration and number of Ground Control Points (GCP) on generating digital terrain models and orthomosaic. To address this issue, several field experiments and data processing were carried out. The quality was assessed by calculating the Root Mean Square Error (RMSE) together with a bias evaluation (t-Student test at 90% confidence level). The results suggest that an optimum block configuration for accurate and unbiased products is achieved by surveying at rates of 80%/60% (forward and sidelap, respectively), with an average Ground Sample Distance (GSD) of around 1 cm at a flight height of 31 m, using a pre-calibrated camera and 5 GCP at least.

Keywords:
Accuracy assessment; Low-Cost UAV; Unmanned Aerial Vehicle; Optimum Flight Configuration; Structure-from-Motion

1. Introduction

Nowadays, it is possible to find a wide range of commercial sensors and platforms that can be used for photogrammetric purposes. The use of low-cost aerial platforms, such as multi-rotor UAV, is presented as a flexible alternative to operate in urban and hard to access areas. The UAV can, remotely, vertically take-off and land (VTOL), fly at low altitude Above Ground Level (AGL), hover on a waypoint to capture images and follow a pre-programmed flight plan. The drawback of this flexibility is the limitation of Maximum Take-off Weight (MTOW), battery discharge profile, and low accuracy of low-cost Global Navigation Satellite System (GNSS) receiver/antenna and Inertial Navigation System (INS). Since the lightweight aerial platform must access the national aviation system to conduct the survey, there are some rules and regulation that must be followed according to each country (Stöcker et al., 2017).

In this work it is considered a low-cost system the ones with the following characteristics: single-frequency GNSS receiver/antenna that do not allow a positioning post-processing, i.e the raw satellite observations are not store; INS using non-tactical-grade inertial measurement unit based on MEMS (Microelectomechanical System); non-metric camera without calibration certificate; and a time of flight around 30 minutes per battery or less. Examples of these systems in the market are DJI Mavic Pro and DJI Phantom 4 pro. Any other commercial platform that carries sensors with higher measurement quality or more precise techniques are considered non-low-cost systems.

Basically, UAVs applications can be classified as forestry (Dandois, Olano and Ellis, 2015), agriculture (Kitano et al., 2019; Oliveira et al., 2018), archaeology and cultural heritage (Tombari and Remondino, 2013), environmental survey (Jaud et al., 2019), and civil engineering (Greenwood, Lynch and Zekkos, 2019). Among the reasons for this spread in the civil engineering community is due to the possibility of generating digital cartography products faster and at a lower cost when compared to conventional surveying, traditional aerial Photogrammetry and orbital remote sensing (Colomina and Molina, 2014).

However, rigorous procedures and quality control is necessary to use low-cost systems for photogrammetric purposes, since it incorporates undesirable systematic and random errors from the data acquired by the camera, GNSS and INS. Due to the uncertainties from the mentioned sensors, it is not possible to perform a direct image orientation, also there are distortions from the camera lens system that must be corrected. In this case, the method applied to these platforms is known as indirect image orientation, i.e., the precise position and attitude of the perspective center (PC) at the moment of each image acquisition are obtained by applying a photo-triangulation procedure using accurate three-dimensional GCPs (Ghilani and Wolf, 2006). It is recommended that the data from low-quality GNSS and INS be used only as initial approximation parameters to improve the output accuracy and to reduce convergence interactions (Kraus, 2011).

Without knowledge about Photogrammetry, users can generate 3D models and orthomosaics by running a standard photogrammetric batch that results in visually acceptable products. However, these products carry systematic and random errors derived from the sensors assembled on the platform (UAV), and sometimes due to wrong parameters set by the users. Since the goal of Photogrammetry is to derive accurate and reliable 3D metric information from a set of overlapping images, it is vital to assess the quality of the resulting products before using it. In the process of generating digital cartographic products, there are a variety of configuration elements that must be considered, among these: image overlap, flight height, number of GCPs, and camera calibration.

Udin and Ahmad (2014) and Agüera-Vega (2017b) investigated the accuracy of cartographic digital products generated at different flight height. The flight height and camera resolution has influence on the average image GSD, and consequently, on the number of points to be recognized as tie-points. Increasing the flight height and maintaining a fixed focal length will lead to a loss in spatial resolution, on the other hand, less images are needed to map the same area. Udin and Ahmad (2014) analyzed the influence of four different flight height to generate DSMs and orthomosaics. The experiments used 10 GCPs and the results show that the planimetric accuracy is not affected by increasing the height, although, the altimetric accuracy decrease as the height increases. Agüera-Vega (2017b) observed the same pattern as presented on Udin and Ahmad (2014), where the planimetric accuracy is not affected, on the other hand, vertical accuracy becomes worst with higher flight height.

Dandois, Olano and Ellis (2015), Frey et al. (2018), Torres-Sánches et al. (2018) and Chaudhry, Ahmad and Gulzar (2020) studied the influence of image overlap on final product quality. A number of researches focused on this issue, although, few studies are applied to map urban environments and most of the studies focus on mapping vegetation areas. A Larger number of images at different points of view are needed for scene reconstruction while imaging complex environment and objects, such as buildings. This necessity can be an issue, since high overlap images results in large processing time to generate digital products - which is not desirable for some applications, such as fast-growing construction site management. Also, when surveying at lower overlap rate, even though the time of processing decreases dramatically, the generated product could result in occlusions (Oliveira et al., 2018) or gaps (Fraser and Congalton, 2018) over the area. Dandois, Olano and Ellis (2015) concluded that an optimum flight configuration for determining canopy tree heights is using 95%/80% overlap at a flight height of 80 m above canopies. Torres-Sánches et al. (2018) analyzed the accuracy of generating a Digital Surface Model (DSM) and reached an optimum overlap rate of 95% of forward overlap and 60% side overlap at a height of 100 m. Frey et al. (2018) confirmed that both altitude and image overlap are related to the accuracy of the resulting point cloud (best case scenario is 95% image overlap with a GSD smaller than 5.0 cm), in which is similar to the results obtained by (Dandois, Olano and Ellis, 2015). Chaudhry, Ahmad and Gulzar (2020) compared the effects of the image overlap variations (55-85% forward and sidelap) on the number of tie points, point cloud quality, and planialtimetric accuracy over an urban area with heterogeneity of land cover and topographic variations. The authors concluded that decreasing the overlap rate will lead to poorer RMSE values for the vertical component are obtained, on the other hand, the planimetric accuracy presented similar results. Besides the accuracy, decreasing the overlap rate will decrease the processing time. The optimum overlap rate was obtained at 55% forward and 65% sidelap and yielded a RMSE of approximate 15 cm for planimetric and altimetric evaluation.

A number research in geoscience applications have studied the effect of varying the number of GCPs on the accuracy of cartographic digital products. Rangel, Gonçalves and Pérez (2018) concluded that GCPs placed on the external borders improves the planimetric accuracy and GCPs placed on the interior of the surveyed area improves the altimetric accuracy. The experiments using more than 20 GCPs shows that increasing the number of GPCs does not necessarily represent an improvement in accuracy. Agüera-Veja, Carvajal-Ramírez and Martínez-Carricondo (2017) analyzed 3 GCPs combination (3, 5 and 10) over different areas and with different flight height, and a total of 60 different photogrammetric processing were conducted. The experiments results show that the flattest terrain surveyed at the lowest altitude AGL (50 m) and 10 GCPs reached the highest accuracy, with a RMSE of 0.038 m, 0.035 m, and 0.035 m, for X, Y, and Z, respectively. Increasing the number of GCPs from 5 to 10 does not presented significant improvement as increasing from 3 to 5. Similar aspects were also investigated in Pessoa et al. (2020), where different number of GCP as well its distribution was analyzed under 9 different experiments aiming to generate DSM from UAV imagery. The authors report a non-substancial increase of accuracy when applying more than 3 to 12, which yields accuracies of around 2.33 to 3.00 GSD (horizontal) and 5.00 to 6.00 GSD (vertical). Sanz-Ablanedo et al. (2018) tested a total of 3465 different GCP combinations. The authors concluded that the GCPs should be even distributed in a triangular node grid aiming at the minimization of distance between nearest GCPs. According to the authors, 90 to 100 GCPs regularly distributed reaches the accuracy of 2.5×average GSD. Martínez-Carricondo et al. (2018) investigated a total of 300 different combinations (5 distribution type, 12 combinations of GCPs and 5 different replicates). Best results are obtained with 30 GCPs (1.7 GCP per ha) with a stratified distribution (RMSE of 0.045 m and 0.047 m, planimetric and altimetric, respectively). The authors concluded that higher accuracy can be reached using less GCPs if they are placed on the borders and on the center of the area.

Focusing on camera calibration, many studies have been developed to assess the influence of on-the-job self-calibration for applying non-metric cameras for mapping purposes. Przybilla et al. (2019) developed a high accuracy test field to study 3 different digital cameras and 3 different commercial photogrammetric software (Pix4D, Agisoft PhotoScan, and Inpho UASMaster). The aerial survey was conducted on a regular and cross flight pattern arrangement at different heights with an average GSD of 14 mm. The processing was conducted using 5, 12, 22, and 45 GCPs, and the quality was assessed by analyzing the check-points residuals from the BBA. As a result, different cameras and different photogrammetric software presented different accuracies when using 5 GCPs. This result indicates distinctive approaches to conduct the on-the-job self-calibration - regarding residuals distribution and datum definition. The experiments using 12 GCPs reached a 3D accuracy bellow the GSD on check-points with all three software. The authors noted that there is no expressive improvement when the number of GCPs ranges between 12 to 22 GCPs, except for the Zenmuse X5S camera and UASMaster. The authors concluded that surveying at two different heights increases the accuracy when conducting on-the-job self-calibration, especially for the Z component. Bolkas (2019) analyzed DEMs from an urban environment obtained using an on-the-job self-calibration and considering different numbers of GCPs and layout configuration. This research was based on two approaches: (1) by inserting both EOP and IOP as unknowns and (2) by applying the EOP derived from GNSS-RTK survey and inserting the IOP as unknown on the BBA. Based on the first approach, DEM accuracy becomes stable when using 8 to 12 GCPs (separation distance of 85 and 65m, respectively). For this experiment, the RMSE is around 1-2 cm for parking lot areas and road segments, and 3-5 cm for roof and dome shapes. The standard deviation of approximately 2 cm and the variation on distance and number of GCP indicates that the issue is probably related to image orientation rather than camera calibration. With respect to the second approach with the EOP derived from the GNSS-RTK survey, while the IOP remained as unknown on the BBA, to reach the same RMSE of 1-2 cm on parking lot areas and road segments, the required number of GCPs decreases to 4-5 (separation distance 120 m and 105 m, respectively). For complex objects above ground, a minimum of 6 GCPs (separation distance of 105 m) is required to reach a RMSE of 5-7 cm. The author highlighted the impact of weak camera self-calibration when surveying complex environments and suggested that a high number of tie-points is required and reducing the separation between GCPs will improve the accuracy. Oniga et al. (2020) conducted a cross-flight survey over an area that represents an urban environment, with features such as building, parking lot, and green area. The authors analyzed nine different scenarios varying the number of GCPs (4, 8, 20, 25, 50, 75, 100, 125 and 150) arranged in two layout configurations: regular grid and stratified random. The authors summarized that increasing the number of GCPs results in higher accuracy with lower errors and standard deviation on check-points. The errors are approximately five times smaller when comparing 4 GCPs to 20 GCPs experiment, regarding the point cloud and mesh for both GCP layout approaches. By increasing the number of GCPs from 20 to 150, the RMSE decreases 1 cm in planimetry for both approaches, 4.6 cm in altimetry for the stratified distribution, and 5.1 cm in altimetry for the regular grid.

Considering the investigation of isolated parameters by research in literature, as presented previously, the main contribution of this paper is to evaluate how the accuracy of DTMs and orthomosaics is affected by a set of parameters: number of GCPs, camera calibration, flying height and image overlap. This research aims to find the most economic block configuration, with respect to GCP field survey, and to reduce the volume of imagery acquired by the UAV to decrease the processing time to generate unbiased and precise (cm level) digital cartographic products. To assess the quality, 24 sets of DTM and orthomaics were generated using different block configurations and then compared to surveyed check-points (reference). The check-points are analyzed for bias applying the t-Student test at 90% confidence level and for positional accuracy the RMSE value is applied.

This research is structured as follows: First, an overview of the subject is presented along with a review of the state-of-the-art in Section 1. Second, the materials and method for evaluation of flight configuration are presented in Section 2. Third, the experiments using different configurations and the results are discussed in Section 3. Finally, the conclusion of this paper is drawn in Section 4, followed by the acknowledgements.

2. Materials and method for flight configuration evaluation

In this research, the commonly UAV photogrammetric pipeline is applied within different configurations for generating DTM and orthomosaics. The results are then assessed by accuracy level considering a statistical approach. The proposed method to generate an optimum flight configuration is divided in 3 steps (Figure 1): UAV data acquisition (Section 2.1), photogrammetric processing under different configurations of number of GCPs, camera calibration, flight height, and image overlap (Section 2.2), and quality assessment (Section 2.3).

Figure 1:
Outline of the proposed method.

2.1 UAV data acquisition

In the first step, 4609 images with 90% forward and 80% sidelap, with an average GSD ranging from 0.63 cm to 1.19 cm, were acquired with the DJI Phantom 4 flying at an altitude AGL of 21 m (from the highest point located on the east side of the study area (Figure 2)). The UAV is assembled with a 20MP resolution digital camera attached to a gimbal. The UAV was controlled via radio by a mobile phone with android operational system. The flight missions were planned and conducted using DroneDeploy online application. In total, 9 flight missions (3 for each subplot - Figure 2) were planned with respect to the maximum time of flight per battery (20 minutes).

Besides the imagery, the UAV also provided the position and attitude (EOP) at the moment of image acquisition from integrated GNSS antenna/receiver and INS. The EOP were stored as Exchangeable Image File Format (EXIF) in images metadata, which is used as an initial approximation for the photo-triangulation process due to their uncertainties. The direct EOP from EXIF will be applied to reduce the time for the task of image matching.

For the process of photo-triangulation and quality assessment, a set of 89 targets were surveyed by a total station from 12 pre-established landmarks (standard deviation of ± 1 cm and ± 2 cm, in planimetric and altimetric components, respectively) with an average distance of 20 m among targets. Surveyed targets will be used as GCPs and check-points.

To analyze the influence of applying fixed IOP on the photo-triangulation process, the authors conducted an on-the-job self-calibration by using 23 GCPs, 1335 images with 90% forward and 80% sidelap, respectively, at 31 m with an average GSD of 0.93 cm. The aim was to determine the following 8 IOP coefficients: calibrated focal length (f), principal point displacement (x _p and y _p ), radial symmetric (k ₁ , k ₂ and k ₃ ) and decentering distortion (p ₁ and p ₂ ).

2.2 Photogrammetric processing

The photogrammetric processing follows the standard photogrammetric pipeline aiming to generate point cloud, DEM, DTM, and orthomosaic from a set of 2D images. The different processing tasks, including the camera calibration process, were conducted using the Agisoft Metashape software version 1.5.5 due to experiments done using different photogrammetric software, in which Agisoft Metashape presented better results (Sona et al., 2014; Turner, Lucieer and Wallace, 2014; Jaud et al., 2016). The processing is a four-step procedure: (I) camera alignment: in this step, the software estimates the EOP and IOP by finding and matching tie-points through a large set of tie-points extracted from 2D images and using known GCP. For the additional parameters, the authors choose to optimize the eight IOP coefficients. The alignment parameter was selected at “medium”, which causes image downscaling by a factor of 4 (2 times each side). In this step, the authors used direct EOP (from EXIF file) to speed up the processing by selecting pair preselection “reference”. For the processing, relative constraint values of 5 m and 15 m was considered for the EOP (planimetric and altimetric, respectively) and 5 cm for the GCP (both planimetric and altimetric). After executing the bundle block adjustment, both EOP and IOP are obtained, and a sparse point cloud is generated. (II) the second step is to generate a dense point cloud by computing a depth map for each pixel based on image pair and then merging depth maps into the same datum. The resulting point cloud is generated at medium configuration and will be used as input for DEM and DTM generation. (III) The DEM is generated automatically, and the result of the reconstruction task is used as input for generating georeferenced orthomosaic. (IV) finally, the orthomosaic is generated. The resulting GSD were exported with a value of 2.0 cm. The authors chose to decrease the generated orthomosaic GSD to standardize the planimetric evaluation due to different average GSD imagery obtained during data acquisition through the three areas (Figure 2).

For the DTM generation, the resulting point cloud was loaded into the Trimble Business Center software (Trimble Business Center version 5.10, 2018) and then a classification process was conducted aiming the extraction of only points that represent the ground level (terrain). The second step was to filter the resulting point cloud to create a DTM, that later will be used to assess the product accuracy. This approach was chosen instead of measuring the closest point due to the difference in point cloud density and distribution.

2.3 Evaluation and statistical analysis

A suitable metric for the accuracy evaluation of the generated DTM and orthomosaic is necessary in this stage. For this evaluation, two metrics will be used to assess the products quality: the RMSE and the t-Student test at 90% confidence level (Förster and Bernhard, 2014). The RMSE measures the quality of the fit between a reference data (acquired using total station) and the observed coordinates extracted from the cartographic products (Ghilani and Wolf, 2006), as shown in Equation 1.

$RMSE=\sqrt{\frac{1}{n}{\sum }_{i=1}^n(X_{Reference}-X_{Observation}{)}^2}$ (1)

where,

n = number of check-points tested;

X_i = X, Y or Z coordinates extracted from the cartographic product for the ith check-point.

The t-Student test is applied to the check-points to verify the presence of bias and, therefore, evaluate for systematic errors. First, discrepancies are calculated for each component (Equation 2). The second step is to calculate the mean (Equation 3) and the standard deviation (Equation 4).

$∆\mathrm{X}={\mathrm{X}}_i-{\mathrm{X}}_i^r$ (2)

$∆\stackrel{-}{\mathrm{X}}=\frac{1}{n}\sum _{i=1}^n∆{\mathrm{X}}_i$ (3)

$S_{∆\mathrm{X}}=\sqrt{\frac{1}{n-1}\sum _{i=1}^n({∆{\mathrm{X}}_i-∆\stackrel{-}{\mathrm{X}})}^2}$ (4)

The next step for the t-test analysis is to verify if the mean can be considered equal to zero (Equation 5) or different than zero (Equation 6).

$H_0:∆\stackrel{-}{\mathrm{X}}=0,\mathrm{}\mathrm{N}\mathrm{u}\mathrm{l}\mathrm{l}\mathrm{}\mathrm{H}\mathrm{y}\mathrm{p}\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{s}\mathrm{e}\mathrm{s}$ (5)

$H_1:∆\stackrel{-}{\mathrm{X}}\ne 0,\mathrm{A}\mathrm{l}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{n}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{H}\mathrm{y}\mathrm{p}\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{s}\mathrm{e}\mathrm{s}$ (6)

The t-test value is particularly accepted for a small sample size. For equal or larger than 30, t-test distribution remains valid, even if the distribution is not normal (Montgomery, 2009). In this step, it is necessary to calculate t (Equation 7) and test it against the confidence interval (Equation 8). The confidence level of 90% is based on the Cartographic Accuracy Standard (PEC) (Brasil, 1984) and the Cartographic Accuracy Standard of Digital Mapping Products (PEC-PCD) instructions (Brasil, 2016).

$t_X=\frac{∆\stackrel{-}{X}}{S_{∆\mathrm{X}}}\sqrt{n}$ (7)

$\left|t_X\right|<t_{(n-1,\alpha /2)}$ (8)

The population is accepted if t-statistic is lower than t-critical. This hypothesis confirms that the analyzed sample does not show bias (Galo and Camargo, 1994).

3. Experiments, results, and discussion

In this section, first the study region is presented, which includes information about the area, delimitation of subplots, and the surveyed reference points location. Second, the experimental results are described along with a quality assessment and discussion.

The setup for the experiments and the parameters considered for each processing is summarized on Table 1. It is important to highlight that each experiment is tested with on-the-job self-calibration (IOP as unknowns in the BBA) and with pre-self-calibration (fixed IOP in the BBA).

Table 1:
Summary of parameters for experiments.

Experiment	Area	Number of GCP / Number of check-points	Parameters
Camera Calibration	Area 2	23 / 0	On-the-job self-calibration Overlap rate of 90%/80%
GCP	Area 2	0, 3, 4, 5, 8 and 10 / 22	On-the-job self-calibration and pre-self-calibration Overlap rate of 90%/80%
Flight height	Area 1	8 / 16	On-the-job self-calibration and pre-self-calibration Overlap rate of 90%/80%
	Area 2
	Area 3
Image overlap	Area 2	5 / 22	Pre-self-calibration Overlap rate of 90%/80%, 80%60%, 70%/60%, 70%/40%, 60%/60% and 60%/40%

3.1 Study area (test area)

The study area of approximately 11.92 ha is located at the University of Campinas (UNICAMP) (Figure 2), São Paulo, Brazil. This area covers the Faculty of Civil Engineering, Architecture and Urban Planning (FEC), the Brazilian Agricultural Research Corporation (EMBRAPA), four parking lots and it has predominantly low vegetation, with the terrain altitude ranging from 604 m to 645 m.

Figure 2:
Study area and surveyed points distribution.

The selection of the study area is due to the high slope terrain, which simplified the acquisition of images on different flight height. Another reason is based on the location of pre-established landmarks on campus - that allowed an easy densification of GCPs and check-points. The study area was divided in three smaller plots due to the limited UAV time of flight per battery.

For all experiments presented bellow (in Section 3), it was generated orthomosaics and DTMs for the evaluation process. Figure 3 illustrates both generated products.

Figure 3:
Cartographic Products. (a) Ortomosaic. (b) DTM.

3.2 Experiments considering different number of GCPs and camera calibration

This experiment aims to analyze the influence of two different calibration methods by varying the number of GCPs on the photo-triangulation process. The first calibration method uses the GCPs to perform an on-the-job self-calibration, in which both IOP and EOP are adjusted (being considered as unknown in the BBA). The second method considers a pre-self-calibrated IOP on the photo-triangulation while the EOP remains as unknown in the BBA. The pre-self-calibration IOP were obtained through an on-the-job self-calibration by using the data from Area 2 and 23 GCPs.

In total, 12 photogrammetric processing were conducted using 0, 3, 4, 5, 8, and 10 GCPs, with an average GSD imagery of 0.93 cm at a height of 31.0 m AGL. To standardize the analysis, the quality is assessed by using 22 check-points. The GCPs and check-points layout configuration is illustrated in Figure 4.

Figure 4:
GCP and check-points configuration for GCP and camera calibration experiments.

Table 2 shows the results using on-the-job self-calibration. Table 3 shows the results from the photo-triangulation with pre-self-calibration. The tables are presented with the following information: number of GCP used on the photo-triangulation process, RMSE X, Y and Z (in meters), and B stands for bias (“Y” for biased and “N” for unbiased).

Table 2:
Statistics of photo-triangulation varying the number of GCP with on-the-job self-calibration (IOP as unknowns in the BBA).

GCP	RMSE (m)
	X	B	Y	B	Z	B
0	0.457	Y	2.177	Y	10.312	Y
3	0.038	Y	0.027	Y	0.037	Y
4	0.031	Y	0.031	N	0.034	Y
5	0.025	N	0.032	N	0.032	Y
8	0.026	N	0.032	N	0.036	N
10	0.024	N	0.030	N	0.029	N

Table 3:
Statistics of photo-triangulation varying the number of GCP with pre-self-calibration (fixed IOP in the BBA).

GCP	RMSE (m)
	X	B	Y	B	Z	B
0	0.517	Y	2.210	Y	9.556	Y
3	0.041	Y	0.040	Y	0.428	Y
4	0.031	Y	0.037	N	0.371	Y
5	0.025	N	0.032	N	0.045	Y
8	0.026	N	0.033	N	0.030	N
10	0.024	N	0.029	N	0.034	N

According to Table 2 and Table 3, applying direct EOP acquired by low-cost sensors and zero GCP generates products with high RMSE values, especially for the Z component in both cases (RMSE around 10 m). According to Table 2 (on-the-job self-calibration), comparing the experiment using 4 GCPs with the one using 5, placing one GCP on the center of the area indeed increases the altimetric accuracy, with the RMSE value decreasing from 0.371 m to 0.045 m, in which was also observed in (Martínez-Carricondo et al., 2018). Although both approaches presented the same RMSE values for the planimetric evaluation when using 5 GCP, applying a set of calibrated IOP on the photo-triangulation will decrease the number of GCP required to generate a DTM without bias.

Comparing the overall results for both experiments, it can be seen that increasing the number of GCPs does not improve the precision when the number of GCPs ranges from 5 to 10 (pre-self-calibration) and 8 to 10 (on-the-job self-calibration), which was also observed in Pessoa et al. (2020). According to ASPRS (2014), the generated products can be considered as Class II for horizontal and Class III for vertical accuracy with a scale of 1:100. Increasing the number of GCP does reduce the bias on observations. An important aspect regarding the accuracy of generated products is not only the number of GCPs but its distribution and relative distance between then (Martínez-Carricondo et al., 2018), especially regarding the estimation of the sensor attitude. As noted, when comparing both experiments, it is recommended to calibrate the camera before proceeding to the survey. Camera calibration procedure will reduce the labor when surveying the GCPs, as fewer points are required to indirect georeferencing the imagery. To understand the calibration parameters when increasing the number of GCPs, the authors conceived the Table 4 comparing the IOP parameters obtained on the experiments of on-the-job self-calibration.

Table 4:
Results of IOP for photo-triangulation with on-the-job self-calibration (IOP as unknowns in the BBA).

IOP	Number of GCP
	0	4	5	8	10	23
f (mm)	8.598	8.578	8.586	8.586	8.585	8.585
cx (mm)	-0.044	-0.044	-0.044	-0.044	-0.044	-0.044
cy (mm)	0.014	0.014	0.014	0.014	0.014	0.014
k1 * 105 (mm-2)	3.056	3.130	4.107	4.199	4.198	4.260
k2 * 106 (mm-4)	-3.364	-3.369	-3.380	-3.384	-3.382	-3.382
k3 * 108 (mm-6)	4.808	4.813	4.830	4.836	4.832	4.833
p1 * 105 (mm-1)	-2.289	-2.281	-2.278	-2.275	-2.275	-2.276
p2 * 105 (mm-1)	-3.491	-3.482	-3.489	-3.491	-3.493	-3.498

Table 4 shows that the focal length (f) and the principal point displacement (c_x and c_y) are well determined when applying only computer vision algorithms with zero GCP. This statement is based with respect to the experiments of 0 GCP, 10 GCPs and 23 GCPs, that shows a variation of 0.013 mm on the focal length and presented similar results for the principal point displacement. The comparison has shown that a considerable number of GCPs is required to model the radial distortion k ₁ when applying on-the-job self-calibration. Since off-the-shelf cameras express a large amount of distortion (mainly k ₁ ) (Brown, 1971), it is important to highlight that k ₁ is the IOP coefficient that has more influence on image coordinates residuals when measuring coordinates on image-plane.

3.3 Experiments considering different flight height and camera calibration

For this experiment, six photogrammetric processing were carried out by applying photo-triangulation with pre-self-calibration and on-the-job self-calibration. The experiments were carried out under three different flight height with image overlap of 90%/80% (forward and sidelap, respectively). With respect to the first survey at 21 m (Area 1) with an average GSD of 0.63 cm, the second survey consisted of a survey at flight height of 31 m (Area 2) with an average GSD of 0.93 cm, and the last survey was run at 40 m (Area 3) with an average GSD of 1.19 cm. To perform the photo-triangulation, a set of 8 GCPs were used. The number of GCP is due to the experiments of variation of number of GCP, in which a minimum of 8 GCPs were necessary to provide an accurate product using on-the-job self-calibration. The statistical analysis is based on 16 well distributed check-points to assess the products accuracy and t-student test. The GCP and check-points configuration is illustrated on Figure 5a, and the profile of the test area is illustrated of Figure 5b.

Figure 5:
Surveyed areas for height variation experiment.

Proceeding to the statistical analysis, Table 5 shows the results from the experiments at different flight heights by applying on-the-job self-calibration and Table 6 shows the results from the experiments at different flight heights by applying pre-self-calibration on the photo-triangulation process.

Table 5:
Statistics of photo-triangulation at different height with on-the-job self-calibration (IOP as unknowns in the BBA).

Height (m)	RMSE (m)
	X	B	Y	B	Z	B
21	0.008	N	0.008	N	0.029	N
31	0.024	N	0.032	N	0.033	N
40	0.026	N	0.021	Y	0.056	N

Table 6:
Statistics of photo-triangulation at different height with pre-self-calibration (fixed IOP in the BBA)

Height (m)	RMSE (m)
	X	B	Y	B	Z	B
21	0.006	N	0.006	N	0.050	Y
31	0.024	N	0.033	N	0.027	N
40	0.026	N	0.021	N	0.027	N

The altimetric component is related to the focal length, the air-base and flight height ratio between two consecutive images (Kraus, 2011). From this affirmation, increasing the flight height indeed decreases the altimetric accuracy of a 3D point, in which confirms the literature cited at Section “Introduction” (Section 1). The results for the experiment of 40 m reached a RMSE of 0.026 m, 0.021 and 0.056 m, for X, Y, and Z, respectively, and unbiased DTM.

Proceeding to the analysis of height variation by applying pre-self-calibration (Table 6). Results from 31 m and 40 m show similar accuracy for both planimetric (RMSE ranging between 0.021 m and 0.033 m) and altimetric accuracy with a RMSE of 0.027 for both altitudes. Decreasing the flying height to 21 meters, it is noted that worst altimetric accuracy was obtained (altimetric RMSE of 0.050 m). This poorer accuracy might be related to the IOP that was acquired on Area 2 (at flight height of 31 m) and was inserted as fixed on the BBA on the processing at 21 m. Nevertheless, Area 1 has homogeneous features, such as bare ground and vegetation, which can cause problems of image matching, and the Area 2 also has homogeneous features, such as four parking lots.

Another important fact is that comparing the 31 m and 40 m with on-the-job self-calibration, it is observed that raising the flight height does not affect the precision when analyzing the planimetric component. The drawback of surveying at low altitude is that more images are going to be used as input, resulting in a time-consuming task, which is not desirable for some applications.

3.4 Experiments considering different image overlap rate

The influence of image overlap was studied by considering different forward and sidelap rates in the input imagery. For this experiment, the survey was carried out fixing the acquisition rate at 90%/80%, forward and sidelap, respectively. The process of decreasing the forward overlap was conducted extracting one image from the forward sequence, decreasing the rate from 90% until 60% (each image extraction decreases in 10% the forward overlap). The same process was applied to the sidelap, while on the forward overlap decreased in 10% when extracting one image, the sidelap decreases 20%. The process was repeated until the rate of 40% was reached.

For the photo-triangulation, as the GCP experiment achieved better results using the minimum number of 5 GCPs with pre-self-calibration, the experiments were set with 4 GCPs on the borders and 1 on the center of the area and 22 check-points to assess the quality. Table 7 presents the evaluation.

Table 7:
Statistics of photo-triangulation at different overlap rates with pre-self-calibration (fixed IOP in the BBA).

Overlap (%)		RMSE (m)
Forward	Side	X	B	Y	B	Z	B
90	80	0.025	N	0.032	N	0.032	N
80	60	0.025	N	0.032	N	0.038	N
70	60	0.028	N	0.032	N	0.060	Y
70	40	0.027	N	0.033	Y	0.083	Y
60	60	0.026	N	0.033	N	0.043	Y
60	40	0.031	N	0.032	N	0.055	Y

Although better results are obtained at higher overlap rates, as shown in the experiment of 90%/80%, the experiment using 80%/60% overlap rate achieved similar accuracy and unbiased results. Another aspect is regarding to the type of product, in the case of orthomosaics, decreasing the overlap rate does not affect the planimetric precision, similar results are obtained with 90%/80% and 60%/60%. In the case of a DTM, it is required a minimum of 5 GCPs with pre-self-calibrated IOP and the overlap rate should be at least 80%/60% (forward and sidelap, respectively). According to ASPRS (2014), the generated products can be considered as Class II for horizontal and Class III for vertical accuracy with a scale of 1:100. The loss of altimetric accuracy is related to the air-base and flight height ratio, together with less correspondence points.

Besides planimetric and altimetric accuracy, it is important to analyze the time consumption for generating dense point cloud and orthomosaic at different overlap rates. For this analysis, the authors conceived a comparison in Table 8, with the following information: overlap rate (forward and sidelap), number of tie-points, number of point cloud elements, and full processing time.

Table 8:
Image overlap processing information with pre-self-calibration (fixed IOP in the BBA).

Overlap (%)		Number of images	Number of points		Full processing time
Forward	Side		Tie points	Point Cloud
90	80	1335	1.795.827	62.385.746	26 h 8 min
80	60	360	509.816	57.284.618	2 h 33 min
70	60	353	414.915	52.137.894	1 h 3 min
70	40	179	233.482	50.370.437	1 h 2 min
60	60	183	207.276	46.228.425	44 min
60	40	132	145.032	42.172.112	42 min

Table 8 highlights that higher overlap rates result in more processing time due to the high volume of data being processed. The result of using more images as input is a higher number of homologous points being visible from more images, and consequently, more tie-points are generated and used in the photo-triangulation process.

In some applications that requires continuous and rapidly documentation of a site, reducing the overlap rate from 90%/80% to 80%/60% decreases the processing time in approximately 10 times, from 26 hours to 2 hours and 25 minutes, and generates similar results, as shown in Table 6. The drawback of using lower overlap rates is loss in altimetric accuracy and, sometimes, a missing data on final product (Figure 6).

Figure 6:
Overlap influence on orthomosaic generation.

It can be noticed that at high overlap rate of 90%/80% (Figure 6a) the objects above ground does not show misalignments or errors. At 80%/60% (Figure 6b) and lower overlap rates, such as 70%/60% (Figure 6c), 70%/40% (Figure 6d) and 60%/60% (Figure 6e), the generated orthomosaics shows less quality of details (noise) and the borders of objects above ground (construction) are not aligned. Considering the sloping terrain and lower overlap rate of the experiment with 60%/40% (Figure 6f), the generated orthomosaic resulted in a lack of information on reconstruction. According to Oliveira et al. (2018), to generate a true orthophoto (orthomosaic free from occlusion and double mapping with a uniform scale), a procedure to detect and compensate the occlusions by using adjacent images from the flight mission and an accurate DSM followed by a color adjustment is required.

4. Conclusion

The dataset acquired by the UAV allowed the generation of high accuracy products (cm level) and it demonstrated the influence of different block configurations. Although the camera calibration is necessary for a non-metric digital camera, the results show that the FC6310 camera can be used for generating digital cartographic products even when applying photo-triangulation with additional parameters (IOP), since there are required only a few GCPs. In this research, it was concluded that the difference between using calibrated IOP and conducting on-the-job self-calibration is the necessity of surveying few more GCPs.

In summary, after analyzing the results, the authors reached several conclusions when conducting photogrammetric processing under different block configurations: (1) Number of GCP: 5 GCPs for indirect image orientation with pre-self-calibration and 8 GCPs for on-the-job self-calibration. It guides to the conclusion that by using a pre-calibrated IOP, higher is the chance of generating high accuracy and unbiased products; (2) Image Overlap: 80% and 60%, forward and sidelap, respectively, are enough when compared to 90%/80%, decreasing the required processing time significantly (90.24% faster); (3) Flight height: the variation of height acquisition did not present significant difference when flying at low AGL. The features presented in the area must be considered due to the process of image matching, which facilitates the image orientation procedure, and consequently, to obtain a better accuracy on final product. It is worth mentioning that a fixed IOP (mainly focal length) obtained from a certain height can influence the altimetric quality in different heights, as presented in section 3.3 - which can be a drawback; and (4) Camera calibration: in general, calibrating the camera before the flight showed better results due to the reduction of field survey for collecting GCPs.

The mentioned conclusions and remarks can be a useful guide for specialists and non-specialists that have being working with cartographic product generation based on UAV technology. One of the contributions of this paper is the possibility of providing an easy-to-understand list of impacts that the flight configuration and data processing can cause in the final products. This understanding can avoid extra field work, time of processing, and the use of bad quality orthomosaics and DTMs

It is important to highlight that these experiments do not guarantee exactly the same results when different sensors, processing the data using another photogrammetric software or surveying areas with different aspects (in regard to the area size, slope and features above ground). However, the impact of the elements used in the experiments on final product will be similar. Since the study area does not represent the same aspects of a complex urban environment, with high density of construction and tall buildings, or terrains with steep slopes (canyons), the users of this methodology for these types of areas need to consider some possible problems, such as: the presence of shadows and occlusions. Also, camera calibration parameters of off-the-shelf digital cameras varies from one camera to another due to different quality of materials and assembly. This process must be analyzed very carefully since a slight error in focal length can result in errors in the Z component of DTM and DEM.

This study presented the influence of 4 parameters when generating DTM and orthomosaic and how it affects the resulting digital cartographic products when using different configurations of camera calibration, number and layout of GCPs, image overlap rate and flight height. However, this influence can be considered similar for other UAVs and photogrammetric software but with variations in magnitude. For future research, the authors recommend the validation of the proposed optimum block configuration, applying the same platform, software, and photogrammetric pipeline in areas with different characteristics. Also, the authors recommend further quality analyses, such as linear features, quality indicators (completeness) and correction index.

ACKNOWLEDGMENT

The researchers would like to thank the Graduate Program in Civil Engineering (PPGEC) from University of Campinas (UNICAMP) for provided infrastructure, the UNICAMP Support Fund for Education, Research and Extension (FAEPEX) for the scholarship given to the first author (grant number 2859/18), and financial support for equipment acquisition given to the second author (grant numbers 3270/17 and 2212/19) , and Trimble for providing a license for TBC.

REFERENCES

• Agisot, LLC, 2018. Agisoft metashape pro Agisoft LLC, St. Petersburg, Russia, Version 1.5.5, build 9097.
• Agüera-Vega, F. Carvajal-Ramírez, F. and Martínez-Carricondo, P. 2017a. Assessment of photogrammetric mapping accuracy based on variation ground control points number using unmanned aerial vehicle, Measurement: Journal of the International Measurement Confederation, 98, p. 221-227.
• Agüera-Vega, F. Carvajal-Ramírez, F . andMartínez-Carricondo, P . 2017b. Accuracy of digital surface models and orthophotos derived from unmanned aerial vehicle photogrammetry, Journal of Surveying Engineering, 143(2), p. 04016025.
• American Society for Photogrammetry and Remote Sensing. ASPRS Positional Accuracy Standards for Digital Geospatial Data;ASPRS: Bethesda, MD, USA;2014; Volume 81, pp. A1-A26.
• Bolkas, D. 2019. Assessment of GCP number and separation distance for small UAS surveys with and without GNSS-PPK positioning, Journal of Surveying Engineering , 145(3), p. 04019007.
• Brasil. Decreto-lei nº 89.817, de 20 de Junho de 1984. Normas Técnicas da Cartografia Nacional Brasília, DF: Presidência da República, 1984. Disponível em: Disponível em: https://www2.camara.leg.br/legin/fed/decret/1980-1987/decreto-89817-20-junho-1984-439814-publicacaooriginal-1-pe.html Acesso em: 5 de jul. 2020.
  » https://www2.camara.leg.br/legin/fed/decret/1980-1987/decreto-89817-20-junho-1984-439814-publicacaooriginal-1-pe.html
• Brasil. Norma da Especificação Técnica para Controle de Qualidade de Dados Geoespaciais (ET-CQDG) Brasília: DSG, 94 p. 2016. Disponível em: Disponível em: http://www.geoportal.eb.mil.br/portal/images/PDF/ET_CQDG_1a_edicao_2016.pdf Acesso em: 5 de jul. 2020.
  » http://www.geoportal.eb.mil.br/portal/images/PDF/ET_CQDG_1a_edicao_2016.pdf
• Brown, D. C. 1971. Close-range camera calibration, PE&RS, 37(8), p. 855-866.
• Chaudhry, M. H. Ahmad, A. and Gulzar, Q. 2020. Impact of UAV surveying parameters on mixed urban landuse surface modelling, ISPRS International Journal of Geo-Information, 9(11), p.656.
• Colomina, I. and Molina, P. 2014. Unmanned aerial systems for photogrammetry and remote sensing: A review, ISPRS Journal of Photogrammetry and Remote Sensing, 92, p. 79-97.
• Dandois, J. P. Olano, M. and Ellis, E. C. 2015. Optimal altitude, overlap, and weather conditions for computer vision UAV estimates of forest structure, Remote Sensing, 7(10), p. 13895-13920.
• Förster, W. and Bernhard, W. 2004. Mathematical concepts in Photogrammetry, Manual of Photogrammetry 5th ed. (E. M. Mikhail, J. C. McGlone, McGLONE, and J. S. Bethel, editors). Bethesda, Maryland, American Society for Photogrammetry and Remote Sensing , p. 15-180.
• Fraser, B. T. and Congalton, R. G. 2018. Issues in unmanned aerial systems (UAS) data collection of complex forest environments, Remote Sensing , 10(6), p. 908.
• Frey, J. Kovach, K. Stemmler, S. and Koch, B. 2018. UAV photogrammetry of forests as a vulnerable process. a sensitivity analysis for a structure from motion RGB-image pipeline, Remote Sensing , 10(6), p. 912.
• Galo, M. and Camargo, P. O. 1994. Utilização do GPS no controle de qualidade de cartas. In: Anais do 1° Congresso Brasileiro de Cadastro Técnico Multifinalitário, Florianópolis, SC. Tomo 2: 41- 48.
• Ghilani, C. D. and Wolf, P. R. 2006. Photogrammetric Computer Vision Hoboken, NJ.: John Wiley Sons, Inc., 2006.
• Greenwood, W. W. Lynch, J. P. and Zekkos, D. 2019. Applications of UAVs in civil infrastructure, Journal of Infrastructure Systems, 25(2), p. 04019002.
• Jaud, M. et al. 2016. Assessing the accuracy of high-resolution digital surface models computed by Photoscan and Micmac in sub-optimal survey conditions, Remote Sensing , 8(6), p. 465.
• Jaud, M. et al. 2019. Diachronic UAV photogrammetry of a sandy beach in Brittany (France) for a long-term coastal observatory, ISPRS International Journal of Geo-Information , 8(6), p. 267.
• Kitano, B. T. et al. 2019. Corn plant counting using deep learning and UAV images, IEEE Geoscience and Remote Sensing Letters, p. 1-5.
• Kraus, K. 2011. Photogrammetry - Geometry from images and laser scans 2nd ed, Berlin, Walter de Gruyter, 460 p.
• Martínez-Carricondo, P. et al. 2018. Assessment of UAV-photogrammetric mapping accuracy based on variation of ground control points, International Journal of Applied Earth Observation and Geoinformation, 72, p. 1-10.
• Montgomery DC. Introduction to Statistical Quality Control 6th ed. New York: John Wiley & Sons; 2009.
• Oliveira, H. C. et al. 2018. Surface gradient approach for occlusion detection based on triangulated irregular network for true orthophoto generation, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 11(2), p. 443-457.
• Oniga, V. E. et al. 2020. Determining the suitable number of ground control points for UAS images georeferencing by varying number and spatial distribution, Remote Sensing , 12(5), p. 876.
• Pessoa, G. G. et al. 2020. Assessment of UAV-based digital surface model and the effects of quantity and distribution of ground control points, International Journal of Remote Sensing , 42(1), p. 65-83.
• Przybilla H. J. et al. 2019. Investigations on the geometric quality of cameras for UAV applications using the high-precision UAV test field Zollern colliery. In: International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences XLII-2/W13, 2019 ISPRS Geospatial Week 2019, 10-14 June 2019, Enschede, The Netherlands, p. 531-538.
• Rangel, J. M. G. Goncalves, G. R. and Pérez, J. A. 2018. The impact of number and spatial distribution of GCPs on the positional accuracy of geospatial products derived from low-cost UASs, International Journal of Remote Sensing , 39(21), p. 7154-7171.
• Sanz-Ablanedo, E. et al. 2018. Accuracy of unmanned aerial vehicle (UAV) and SfM photogrammetry survey as a function of the number and location of ground control points used, Remote Sensing , 10(10), p. 1606.
• Sona, G. et al. 2014. Experimental analysis of different software packages for orientation and digital surface modelling from UAV images, Earth Science Informatics, 7(2), p. 97-107, 2014.
• Stöcker, C. et al. 2017. Review of the current state of UAV regulations, Remote sensing, 9(5), p. 459.
• Tombari, F. and Remondino, F. 2013. Feature-based automatic 3D registration for cultural heritage applications. In: 2013 Digital Heritage International Congress (DigitalHeritage), 28 Oct.-1 Nov. 2013, Marseille, France, p. 55-62.
• Torres-Sánchez, J. et al. 2018. Assessing UAV-collected image overlap influence on computation time and digital surface model accuracy in olive orchards, Precision Agriculture, 19(1), p. 115-133.
• Trimble, Trimble Business Center, Version 5.10, 2018. [Online]. Available: http://trimble.com
  » http://trimble.com
• Turner, D. Lucieer, A. and Wallace, L. 2014. Direct georeferencing of ultrahigh-resolution UAV imagery, IEEE Transactions on Geoscience and Remote Sensing , 52(5), p. 2738-2745.
• Udin, W. S. Ahmad, A. 2014. Assessment of photogrammetric mapping accuracy based on variation flying altitude using unmanned aerial vehicle. In: IOP Conference Series: Earth and Environmental Science , 18, p. 012027.

ADDITIONAL INFORMATION

• Data Availability

  Some or all data and models that support the findings of this study are available from the corresponding author upon reasonable request, including the points coordinates, orthomosaics, DEMs and DTMs.

Publication Dates

• Publication in this collection
  12 July 2021
• Date of issue
  2021

History

• Received
  13 Oct 2020
• Accepted
  05 June 2021