COMPARISON BETWEEN ABSOLUTE AND RELATIVE POSITIONAL ACCURACY ASSESSMENT - A CASE STUDY APPLIED TO DIGITAL ELEVATION MODELS

Comparação entre métodos de avaliação de acurácia posicional altimétrica absoluta e relativa - um estudo de caso aplicado a Modelos Digitais de Elevação

Abstract

This paper presents a comparative study between the absolute and relative methods for altimetric positional accuracy of Digital Elevation Models (DEM). For the theoretical basis of this research, the definitions of accuracy (exactness) and precision, as well the concepts related to absolute and relative positional accuracy were explored. In the case study, the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) and the Shuttle Radar Topography Mission (SRTM) DEM were used. In the analysis of the absolute accuracy, 6,568 ground control points from GNSS orbital survey were used, collected through relative-static method. In the relative accuracy, it was used as reference DEM with spatial resolution of 5 meters generated by stereophotogrammetrical process for the Mapping Project of Bahia (Brazil). It was concluded that, once the accuracy of the reference DEM is better than the other two evaluated DEM, the results of the classification for the PEC-PCD for the relative evaluation are equal to or better than the absolute evaluation results, with the advantage to being able to verify the pixel population of the evaluated models, which makes it possible to identify outliers, distortions and displacements, including delimiting regions, which is much less likely with a limited set of control points.

Keywords:
quality control; absolute positional accuracy; relative positional accuracy; SRTM; ASTER; DEM

Resumo

Este artigo apresenta um estudo comparativo entre os métodos absoluto e relativo para a avaliação de acurácia posicional altimétrica de Modelos Digitais de Elevação (MDE). Para o embasamento teórico desta pesquisa, foram apresentadas as definições de acurácia (exatidão) e precisão, bem como conceitos relativos à acurácia posicional absoluta e relativa. No estudo de caso foram utilizados os MDE Avance Spaceborne Thermal Emission and Reflection Radiometer (ASTER) Global Digital Elevation Model (GDEM) e Shuttle Radar Topography Mission (SRTM). Na análise da acurácia absoluta foram utilizados 6.568 pontos de controle oriundos de levantamento orbital GNSS, no método relativo-estático. Já na acurácia relativa foram utilizados como referência os MDE com resolução espacial de 5 metros gerados por processo de estereofotogrametria digital para o Projeto de Mapeamento do estado Bahia. Concluiu-se que, uma vez comprovada que a acurácia do MDE de referência é melhor que o avaliado, os resultados da classificação quanto ao PEC-PCD para a avaliação relativa são iguais ou melhores que os resultados da avaliação absoluta, tendo-se a vantagem adicional de ser possível verificar a população de pixels dos modelos avaliados, o que possibilita identificar outliers, distorções e deslocamentos, inclusive delimitando regiões, o que é muito menos provável com um conjunto limitado de pontos de controle.

Palavras-chave:
controle de qualidade; acurácia posicional absoluta; acurácia posicional relativa; SRTM; ASTER; MDE

1. Introduction

The International Organization for Standardization (ISO) gives international norms about geographic information, being developed in its 19,100 series (ISO 19.157, 2013). The Brazilian standards were built based on these norms through the National Spatial Data Infrastructure (INDE), according to Decree No. 6,666, of November 27, 2008 (Brasil, 2008), with one of its objectives being control quality (DSG, 2016), with the aim of providing, in a standardized way, the methods for the evaluation of geographical data.

The construction of these standards, or technical specifications, for the cartography is the responsibility of the Brazilian Army, regarding the series of topographic charts, defined in the scales of 1:250,000 and larger, as determined by Decree-Law Nº. 243, of February 28, 1967, which establishes the guidelines and bases of Brazilian cartography. Therefore, following this obligation, the Army has elaborated the following Technical Specifications for: (CONCAR, 2008, DSG, 2008, DSG, 2015, DSG, 2016):

• Vector Geospatial Data Structuring (Estruturação de Dados Geoespaciais Vetoriais - EDGV);

• Acquisition of Vector Geospatial Data (Aquisição de Dados Geoespaciais Vetoriais - ADGV);

• Geospatial Data Set Products (Produtos de Conjuntos de Dados Geoespacias - PCDG);

• Geospatial Data Representation (Representação de Dados Geoespaciais - RDG); and

• Geospatial Data Quality Control (Controle de Qualidade de Dados Geoespaciais - CQDG).

Among the technical specifications cited previously, the CQDG is the focus of this article, since it standardizes the quality evaluation criteria, establishing the required measures for each element of quality (Omission, Conceptual Consistency, Format Consistency, Absolute Positional Accuracy and Temporal Validity) considering different geospatial dataset (DSG, 2016).

It should be noted that the first edition of the CQDG presents only measures for the evaluation of absolute positional accuracy and does not provide definition of measures for the evaluation of relative positional accuracy. This is the main reason of analyzing and discussing the results of this type measurement, judging their feasibility or not, for the quality control of Digital Elevation Models (DEM).

Another consideration on the relevance of this work is the fact that the relative method, in comparison with the absolute method, can be applied in a global way, since it makes use of the entire population of pixels, obtaining more realistic results, besides identifying regions with greater discrepancies and outliers.

The objective of this work was to compare quantitatively and qualitatively the statistical results of the assessment of the absolute positional accuracy of the Digital Elevation Digital Models (DEM) Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) and Shuttle Radar Topography Mission (SRTM) with the results of the relative positional accuracy, calculated based upon a reference DEM with better accuracy.

2. Theoretical conceptualization

Aiming at the theoretical basis of this work, this chapter discusses the definitions used for digital models from altimetric data and describes the characteristics of the evaluated DEM (ASTER and SRTM). It is also reviewed the concepts of precision and accuracy, as well as the definitions of absolute and relative positional accuracy.

2.1. Definitions of Digital Model for Altimetry

The surface of the Earth can be modeled in three dimensions by means of Digital Elevation Model (DEM), Digital Terrain Model (DTM) or Digital Surface Model (DSM), however there are still dissonances about its definitions.

Li et. al. (2005 apud Santos et al., 2016) considers the DTM as a set of points that represent the spatial distribution of various types of information on the terrain, being the DEM a particularization when this information translates the altitude of the terrain that models the emerged or submerged land without vegetation canopy or buildings.

The PCDG (DSG, 2015), which is of interest in this work, defines an DEM as a cartographic product obtained from a mathematical model that represents a phenomenon, continuously, from adequately structured and sampled data of the real world. A DEM is called the DTM when it is associated with the representation of the bare-earth (exposed soil), disregarding the accidents found above the ground (buildings, bridges, vegetation, etc.). When the accidents on the ground are modeled, then the DEM is called as DSM.

To agree with the PCDG and, consequently, with the norms of the INDE, this work considers the generic name of all altimetric digital models as DEM, without the concern to differentiate the models that represent or not the bare-earth.

2.2. ASTER GDEM

The ASTER data are the result of a cooperative effort between the National Aeronautics and Space Administration (NASA) and the Ministry of International Trade and Industry of Japan (METI), Earth Resources Data Analysis Center (ERSDAC) and the United States Geological Survey (USGS) for the construction of an open access global DEM (USGS, 2014).

To obtain surface elevation data, ASTER sensors placed on TERRA satellite captured stereoscopic images between 2000 and 2010, operating in the bands of the near infrared spectrum. The GDEM is the most complete model of the Earth's surface, covering 99% of its surface, between the latitudes of 83 ° N and 83 ° S (Celestino and Philips, 2016; Santos et al., 2016).

ASTER GDEM v.2 is provided by the USGS Earth Explorer in the GeoTIFF image format in 1°x1° geographic cutouts, with spatial resolution of 1 arc-second (about 30 meters near the equator). The files are in the WGS 84 geographic coordinate reference system, with the geological model EGM96 as altimetric reference (USGS, 2014).

According to Tachikawa et. al. (2011), ASTER GDEM v.2 has an absolute vertical accuracy of 17 meters with a 95% confidence level. The authors also verified that in forest areas the values of the model are about 8 meters above the correct reference and in areas of exposed soil the model tends to be 1 meter below the reference.

2.3. SRTM

The SRTM was a project of NASA and the National Geospatial-Intelligence Agency (NGA), which had the international collaboration of the German Space Agency (Deutsches Zentrum für Luft- und Raumfahrt - DLR) and the Italian Space Agency (Agenzia Spaziale Italiana - ASI) to acquire radar data with the objective of creating the first global altimetric model (USGS, 2015, Santos et al., 2016).

In order to obtain the elevation data, two Synthetic Aperture Radars (SAR) were used by interferometry (InSAR) in the C and X bands. The data were collected aboard the space shuttle Endeavor in a period of 11 days (11 to 22 February 2000) and cover more than 80% of the Earth's surface area, covering latitude 60 ° N to 56 ° S (Forkuor and Maathuis, 2012, Celestino and Philips, 2014).

The SRTM MDE are provided by the USGS EarthExplorer in the GeoTIFF image format in 1°x1° geographic cutouts, with spatial resolution of 1 and 3 arc-seconds (about 30 and 90 meters near the equator line, respectively). The files are in the WGS 84 geographic coordinate reference system, with the geographic model EGM96 as altimetric reference (USGS, 2015).

It should be highlighted that, until recently, the vast majority of researches on SRTM was based on products with a resolution of 3 arc-seconds, since the models with resolution of 1 arc-second were made available to the public openly from November 2014 (LP DAAC, 2014).

The absolute vertical accuracy of the DEM SRTM of 3 arc-seconds is less than 16 meters, although this accuracy may vary, depending on the continent and region (Jozsa et al., 2014). Already for the model with 1 arc-second was obtained an accuracy better than 6 meters in the region of Saudi Arabia (Elkhrachy, 2017) and in the northeast of Brazil (França et. al, 2018).

2.4. Accuracy (or Exactness) versus Precision

The positional accuracy is one of the quality elements standardized in ISO 19.157 (ISO 19.157, 2013) which specifies how the geometric and positional characteristics of geographic data must be evaluated.

Nevertheless, for the evaluation of positional quality, several terms have been used, highlighting accuracy, exactness and precision, sometimes for reasons of idiomatic translation, leading to misinterpretations. Therefore, it is necessary to differentiate the concepts of precision and accuracy, being the latter synonymous of exactness (Monico et al., 2009).

According to Celestino and Philips (2014), the accuracy shows the degree to which spatial information is recorded in its true position, relative to a terrestrial reference or with reference to other adjacent information. They also define precision as the degree of agreement of a series of measurements made on similar conditions, translated by the standard deviation of the observations.

Gemael (1994) states that the term accuracy is related to random and systematic effects and concerns the dispersion of values in relation to the exact value of the observation, whereas the precision is related to random effects and concerns the dispersion of values in relation to the average of the observations.

According to Monico et. al. (2009) the concept of accuracy involves systematic and random errors, while the concept of precision involves only random errors. The accuracy, therefore, can be translated in terms of the following parameters: tendency and precision (Monico et al., 2009). Mathematically the tendency is calculated as the difference between the mean of the observations and the known (or expected) value and the precision is the standard deviation of the observations.

The Accuracy measure proposed by Guass is known as the Mean Square Error (MSE) calculated by (1), where σ² represents the dispersion of the measures (variance or uncertainty) in relation to the average of the observations and b represents the tendency, bias or vice of the estimator (Monico et al., 2009). In other words, the 𝜎² means random errors and b translates systematic errors.

$$MSE=E\left[{\left(\widehat{\theta }-\theta \right)}^2\right]={\sigma }_{\widehat{\theta }}^2+b^2\simeq \frac{{\sum }_{i=1}^N{\epsilon }_i^2}{N}$$ (1)

For large samples, the MSE approximates to the quadratic mean of the errors (𝜀), where 𝜀 correponds to the difference of an observed (or measured) value 𝑧 𝑜 and the reference (known) value 𝑧 𝑟 (Equation 2).

$$\epsilon =z_o-z_r$$ (2)

In this research, it was considered the mathematical formulation for tendency b and precision 𝜎 given by (3) and (4), being the accuracy calculated by the Root Mean Square Error (RMSE), given by (5).

$$b=\frac{\sum _{i=1}^N{\epsilon }_i}{N}$$ (3)

$$\sigma =\sqrt{\frac{\sum _{i=1}^N({\epsilon }_i-b)²}{N}}$$ (4)

$$RMSE=\sqrt{\frac{\sum _{i=1}^N{\epsilon }_i²}{N}}$$ (5)

2.5. Absolute versus Relative Positional Accuracy

The concept of positional accuracy is well established in the cartographic literature and defined in the standards of ISO 19.113, ISO 19.114 and ISO 19.138 (Ariza-López et al., 2007).

The assessment of positional accuracy of geospatial data corresponds to a statistical analysis of the feature discrepancies identified on the map in relation to a given reference (DoD, 1990). Depending on the characteristics of the reference, positional accuracy has the following classification (Santos et al., 2017):

• Absolute (or external): statistics based upon the discrepancy of the observed coordinate in relation to a reference coordinate accepted as true. This commonly happens when the reference coordinate is measured with instruments that ensure their values faithfully translates the terrain reality. For example, with reference points collected by relative static positioning with dual frequency GNSS receivers.

• Relative (or internal): statistics based upon the discrepancy of observed coordinate in relation to coordinate of other dataset which respective position has better accuracy. For instance, the evaluation of features mapped to the scale 1: 25,000 having as a reference features mapped on the scale 1: 1,000.

For the relative positional accuracy assessment, Celestino & Philips (2014) suggest that the reference data should be three times better than the declared data for the tested data. In other words, Camargo et. al. (2009) confirm that the reference data must present positional error equal to or less than one-third of the standard error of the evaluated scale.

Although the assesment of absolute positional accuracy seems to be the most appropriate for mostly cases, the evaluation of DEM based on a small set of points may not be the most appropriate method (Józsa et al., 2014).

2.6. Brazilian Cartographic Accuracy Standard for Altimetry

All paper topographic charts produced officially in Brazil for systematic mapping must follow the positional accuracy accordingly to the Cartographic Accuracy Standard (Padrão de Exatidão Catográfica - PEC) which parameter values was established in Decree No. 89.817 of June 20, 1984 (Brasil, 1984), transcribed though the Table 1.

Table 1
Mean Error (Erro Médio - EM) and Standard Error (Erro-Padrão - EP) values for PEC classification of printed products (topographic charts).

Type	PEC	1:25,000		1:50,000		1:100,000		1:250,000
		EM	EP	EM	EP	EM	EP	EM	EP
Altimetric measurements	A	5,0	3,33	10	6,67	25	16,67	50	33,33
	B	6,0	4,0	12	8,0	30	20	60	40
	C	7,5	5,0	15	10	37,5	25	75	50

• Source: DSG, 2016.

With the recent evolution of digital cartography and the geothecnology's resources, new products require other quality parameters, including a refinement of the assessment for positional accuracy assessment (Ariza-López et. al, 2007) and computational programs to perform it (Nero et. al, 2017).

The Geographic Service Bureau (Diretoria de Serviço Geográfico - DSG, 2016) established the quality assessment parameters described in the Technical Specifications for Geospatial Set Products (ET-PCDG) and Geospatial Data Quality Control (ET-CQDG).

The new positional accuracy standards, known as the Cartographic Accuracy Standard for Digital Cartographic Products (PEC-PCD), consider small and large scales (Table 2 and Table 3). It is possible to be noted that the "B", "C" and "D" from PEC-PCD correspond respectively to "A", "B" and "C" classes of the Decree No. 89.817 (DSG, 2015) (França & Ferreira da Silva, 2018).

Table 2
Mean Error (Erro Médio - EM) and Standard Error (Erro-Padrão - EP) values for the classification of PEC-PCD for DEM on small scales.

Type	PEC-PCD	1:25,000		1:50,000		1:100,000		1:250,000
		EM	EP	EM	EP	EM	EP	EM	EP
Altimetric measurements	A	2,7	1,67	5,5	3,33	13,7	8,33	27	16,67
	B	5,0	3,33	10	6,67	25	16,67	50	33,33
	C	6,0	4,0	12	8,0	30	20	60	40
	D	7,5	5,0	15	10	37,5	25	75	50

• Source: DSG, 2016.

Table 3
Mean Error (Erro Médio - EM) and Standard Error (Erro-Padrão - EP) values for the classification of PEC-PCD for DEM on large scales.

Type	PEC-PCD	1:1,000		1:2,000		1:5,000		1:10,000
		EM	EP	EM	EP	EM	EP	EM	EP
Altimetric measurements	A	0,27	0,17	0,27	0,17	0,54	0,34	1,35	0,84
	B	0,5	0,33	0,5	0,33	1,0	0,67	2,5	1,67
	C	0,6	0,4	0,6	0,4	1,2	0,8	3,0	2,0
	D	0,75	0,5	0,75	0,5	1,5	1,00	3,75	2,5

• Source: DSG, 2016.

3. Study Region

The study region covers a part of the west of the Brazilian State of Bahia, corresponding to an extension of 3º of longitude by 2º of latitude. The analyzed models correspond to the DEM ASTER v.2 and SRTM v.3, both with resolution of 1 arc-second, distributed in the 1° x 1° joint by the United States Geological Survey (USGS, 2014; USGS, 2015), according to the frame shown in Figure 1.

Figure 1
Study region located in the west of the State of Bahia, presenting the the SRTM and ASTER models

Therefore, 12 (twelve) DEM were evaluated, 6 (six) corresponding to ASTER and 6 (six) related to SRTM.

4. Methodology

This chapter details the procedures for the evaluation of the absolute and relative positional accuracy of the DEM, showing how the data used as reference were obtained, as well as the method for comparing the results.

It is important to emphasize that all steps were performed with the General Public License (GNU), open-source software, being QGIS 2.18 and Python scripts based on the GDAL module. The developed scripts are available in the following Github repository: http://www.github.com/LEOXINGU/DEM_assessment.

4.1. Data Sources

The DEM ASTER and SRTM evaluated in this work and presented in figure 1 were obtained from the USGS EarthExplorer website: https://earthexplorer.usgs.gov.

The Ground Control Points (GCP) and DEM adopted respectively for absolute and relative reference are available in the Geographic Database of the Brazilian Army (Banco de Dados Geográfico do Exército - BDGEx) through the following website: http://www.geoportal.eb.mil.br/mediador/.

The DEM used as reference were obtained by digital photogrammetric flight performed by the company Engemap, through the airborne system of acquisition and post-processing of digital images (Sistema Aerotransportado de Aquisição e Pós-processamento de Imagens Digitais - SAAPI), during the years of 2009 and 2010 (Penha et al., 2012).

4.2. Absolute Positional Accuracy Assessment

For this evaluation were used GCP from GNSS orbital survey, through relative static method, executed by the Brazilian Army. It was collected more than 35,000 points in order to evaluate and approval the positional accuracy of the DEM for the Mapping Project of the State of Bahia (Penha et al., 2012). Of this total, 6,568 points were used in this research. Figure 2 shows the control points on the study region and Table 4 shows the number of GCP for each region.

Figure 2
Spatialization of the GCP used in the research.

Table 4
Number of control points for evaluation of absolute positional accuracy.

Region	Number of GCP
S12W044	807
S12W045	1,168
S12W046	1,434
S13W044	1,029
S13W045	1,015
S13W046	1,115
Sum	6,568

The orthometric altitude of the GCP has as altimetric reference the Datum of Imbituba, different of the vertical Datum of the evaluated DEM which is the EGM 96 WGS 84. However, as the difference between the altitudes in these references is in the order of 0.10 to 0.54 meters (Moura et al., 2014), the transformation between these vertical references was not performed, considering its smallness in relation to the values of EM and EP from PEC-PCD (Table 2).

Figure 3 illustrates, through a flowchart, the evaluation process of each DEM, being this process executed by the Python script "DEM Absolute Accuracy", where the output corresponds to an HTML file containing the PEC-PCD classification result and related statistics.

Figure 3
Flowchart of the Absolute Positional Accuracy Assessment.

The reference DEM also had their absolute positional accuracy assessed. For that, It was created mosaics with the reference DEM in order to verify if these models have better positional accuracy than the evaluated models. Those mosaics also was used for relative positional accuracy assessment.

4.3. Relative Positional Accuracy Assessment

In this evaluation, it was used the DEM generated by photogrammetry for the State of Bahia Mapping Project (Neto et al., 2014; Penha et al., 2012) as reference dataset. They have better resolution and positional accuracy than the ASTER and SRTM DEM.

The reference DEM has resolution of 5 meters in frames of 7.5' x 7.5', corresponding to the articulation of the scale 1: 25,000 for the Brazilian systematic mapping and referenced in the Vertical Datum of Imbituba and Horizontal Datum SIRGAS 2000.

Therefore, for each of the six ASTER and SRTM models in the study area a mosaic composed of 64 reference DEM was used, accordingly Figure 4.

Figure 4
Mosaics of reference DEM.

Figure 5 shows the flowchart of the evaluation process of the ASTER and SRTM DEM, executed by the Python script "DEM Relative Accuracy", which generates, in a similar way to the evaluation of absolute accuracy, an HTML file with the results.

Figure 5
Flowchart of the Relative Positional Accuracy Assessment.

4.4. Classification of PEC-PCD

The Standard of Cartographic Exactness (Padrão de Exatidão Cartográfica - PEC) for Digital Cartographic Products (Produtos Cartográficos Digitais - PCD) in the ET-CQDG (DSG, 2016), classifies geospatial data according to its accuracy in classes "A", "B", "C" or "D", based on the maximum permissible error (EM) and the standard error (EP ) to the altimetry.

Accordingly to Equation 2, the error of the altitude component in a reference position (x,y) is calculated by the difference of z_o (the altitude calculated by interpolation of the pixels’ values near the reference position) in relation to z_r which is the orthometric altitude of the reference point.

For the determination of the altitude z_o , the bilinear interpolation was adopted, similar to Reuter et. al. (2009), aiming to calculate the linear influence of the altitude values f (x,y) of the 4 pixels closest to the evaluated DEM in relation to the reference point, as shown in Figure 6 and Equation (6).

Figure 6
Bilinear Interpolation.

$$z_t=\left(1-dx\right)\left(1-dy\right)f\left(x,y\right)+dx\left(1-dy\right)f\left(x+1,y\right)+\left(1-dx\right)dyf\left(x,y+1\right)+dxdyf\left(x+1,y+1\right)$$ (6)

In the case of the relative accuracy assessment, the reference points correspond to the center of the pixels of the reference DEM, z_t is also calculated by bilinear interpolation.

According to ET-CQDG (DSG, 2016), the classification of the PEC-PCD is given by observing the following criteria:

• Each error value 𝜀 is compared with the maximum permissible error (EM) of the PEC-PCD table (according to class and scale), being classified where at least 90% of the absolute value of the errors is lower than EM for a specific class and scale.

• The result of the RMSE is compared with the EP from PEC-PCD table. If it is smaller, then the dataset is classified to that class, otherwise the RMSE is verified following each PEC-PCD class in a specific scale. If the result is not sorted in any of the classes, then it is said that the result is "not compliant", also called "R" class.

It is important to highlight that the Python scripts, developed by the authors, were written taking all care that the values of the null pixels, present both in the SRTM DEM and in the reference DEM, did not interfere with the calculations of the statistics.

5. Results and discussions

The following tables 5 and 6 present the results of the altimetric positional accuracy assessment for absolute and relative methods, respectively. It is shown statistical results of the calculated errors in meters (m) and the PEC-PCD category in which each ASTER and SRTM DEM have been classified for the following scales: 1:25,000 (25K), 1:50,000 (50k), 1:100,000 (100k) and 1:250,000 (250K). When the tables present the "R" for the PEC-PCD classification, it means that the data does not fit even the minimum standard (class "D") of the scale evaluated and therefore must be rejected.

5.1. Absolute positional accuracy Results

By calculating the average values of the results of Table 5, it is verified that the SRTM models have a tendency to be 3.69 meters displaced above the reference on the ground, however its precision of 2.01 meters is 3.22 times higher than the precision of 6.48 meters of ASTER DEM, that is, the altimetric errors are less dispersed (scattered) in the SRTM than in the ASTER, coming up very close to the precision of 1.1 meters of the photogrammetric DEM used as reference.

Table 5
Results of Absolute Positional Accuracy Assessment.

Absolute Positional Accuracy of the Reference DEM in meters (m)						PEC - PCD
File	RMSE	Tendency	Precision	𝜀 𝑚??𝑛	𝜀 𝑚𝑎𝑥	25K	50K	100K	250K
mosaico_S12W44	1,17	0,21	1,15	-7,76	3,74	A	A	A	A
mosaico_S12W45	1,33	0,56	1,20	-8,84	8,13	A	A	A	A
mosaico_S12W46	1,35	0,68	1,16	-4,86	16,05	A	A	A	A
mosaico_S13W44	1,25	0,56	1,11	-8,02	8,01	A	A	A	A
mosaico_S13W45	1,28	0,43	1,20	-10,62	8,40	A	A	A	A
mosaico_S13W46	1,11	0,60	0,93	-3,40	8,85	A	A	A	A
Absolute Positional Accuracy of SRTM DEM in meters (m)						PEC - PCD
File	RMSE	Tendency	Precision	𝜀 𝑚𝑖𝑛	𝜀 𝑚𝑎𝑥	25K	50K	100K	250K
s12_w044_1arc_v3	4,25	3,76	1,99	-3,48	10,45	D	B	A	A
s12_w045_1arc_v3	4,28	3,80	1,96	-2,91	12,69	D	B	A	A
s12_w046_1arc_v3	3,71	3,00	2,18	-3,72	14,72	C	B	A	A
s13_w044_1arc_v3	4,50	3,95	2,16	-5,90	14,54	D	B	A	A
s13_w045_1arc_v3	4,77	4,26	2,15	-5,29	13,72	D	B	A	A
s13_w046_1arc_v3	3,72	3,35	1,61	-1,52	12,81	C	B	A	A
ASTGTM2_S12W044_dem	7,93	-5,31	5,89	-54,31	19,05	R	D	A	A
ASTGTM2_S12W045_dem	7,79	3,00	7,19	-18,66	30,49	R	C	A	A
ASTGTM2_S12W046_dem	7,98	4,83	6,35	-48,99	30,28	R	C	A	A
ASTGTM2_S13W044_dem	5,24	0,49	5,22	-40,26	19,24	R	B	A	A
ASTGTM2_S13W045_dem	9,14	2,59	8,76	-43,45	42,42	R	D	B	A
ASTGTM2_S13W046_dem	7,41	5,04	5,44	-36,03	38,80	R	C	A	A

Figure 7 presents the histograms of the altimetric discrepancies of the absolute accuracy assessment for the ASTER and SRTM DEM, as well as for the Reference DEM.

Figure 7
Histograms of discrepancies for ASTER, SRTM and Reference DEM.

It is observed, through the histograms and Table 5, the superiority in terms of accuracy (tendency and precision) of the Reference DEM in relation to the ASTER and SRTM models. Thus, the Reference DEM should be used for the relative positional accuracy assessment, in view that, on average, the accuracy of these models are 3.4 times better than SRTM and 6.1 times better than ASTER DEM.

5.2. Relative Positional Accuracy Results

Table 6 presents the statistical results (in meters) and the PEC-PCD for the SRTM and ASTER DEM, having as reference the photogrammetric DEM generated for Bahia Mapping Project.

Table 6
Results of the relative positional accuracy assessment.

Relative Positional Accuracy of SRTM DEM in meters (m)						PEC - PCD
Arquivo	RMSE	Tendency	Precision	𝜀 𝑚𝑖𝑛	𝜀 𝑚𝑎𝑥	25K	50K	100K	250K
s12_w044_1arc_v3	3,68	2,63	2,57	-64,22	606,83	C	B	A	A
s12_w045_1arc_v3	3,61	2,54	2,57	-38,52	463,93	C	B	A	A
s12_w046_1arc_v3	2,98	1,79	2,39	-56,04	721,44	B	A	A	A
s13_w044_1arc_v3	4,66	3,43	3,15	-55,26	131,80	D	B	A	A
s13_w045_1arc_v3	3,97	3,02	2,57	-29,69	90,29	D	B	A	A
s13_w046_1arc_v3	3,14	2,61	1,75	-25,80	44,26	B	A	A	A
Relative Positional Accuracy of ASTER DEM in meters (m)						PEC - PCD
Arquivo	RMSE	Tendency	Precision	𝜀 𝑚𝑖𝑛	𝜀 𝑚𝑎𝑥	25K	50K	100K	250K
ASTGTM2_S12W044_dem	7,15	-4,61	5,47	-77,41	603,49	R	C	A	A
ASTGTM2_S12W045_dem	6,28	2,65	5,69	-87,85	454,43	R	B	A	A
ASTGTM2_S12W046_dem	7,36	4,38	5,91	-96,67	732,93	R	C	A	A
ASTGTM2_S13W044_dem	6,19	0,09	6,19	-106,23	138,08	R	B	A	A
ASTGTM2_S13W045_dem	8,06	1,52	7,92	-99,60	82,19	R	D	A	A
ASTGTM2_S13W046_dem	7,33	5,59	4,74	-65,29	82,72	R	C	A	A

5.3. Discussions

From Tables 5 and 6, it is verified that, in the relative method, the RMSE and the tendency are, on average, slightly smaller for both evaluated models. Differently, only, the precision values of the SRTM DEM proved to be larger (worse) in the relative method.

Table 6 presents the greatest discrepancies, in absolute terms, of the evaluated models for the study region, allowing to identify more clearly the "true" outliers, which is improbable to detect applying the absolute method.

About the classification of the PEC-PCD, the relative method led to better or equal results to the absolute assessment, with no case where the relative method was less than the absolute method.

Figures 8, 9 and 10 display images with altitude difference between the evaluated models and the reference model, as well as the difference between the two evaluated models, with the SRTM model as a reference to the ASTER. The location of the areas with tendency to be above the reference is represented in shades of green as well as the areas below the reference is in shades of purple.

Figure 8
Difference between the ASTER models and the Reference DEM.

Figure 9
Difference between the SRTM models and the reference DEM.

Figure 10
Difference between ASTER and SRTM models

It is verified from figures 8 and 10 that there is a certain pattern of distribution of errors. These areas are supposed to correspond to the sweep lines covered by the ASTER orbital sensor and/or by processing bands of that data, incurring anomalous "steps" in those data.

Figure 10 proves that the systematic error of the scan lines checked in Figure 8 is caused by the processing of the ASTER data, when it was used the SRTM models as reference.

Also, in Figure 8, it is possible to observe that, in the two DEM in the east, the values of the altitudes in the crest lines and peak of the elevations tend to be smaller than the reference, unlike the funds of valleys and drainage lines, where the altitude values are higher than the reference.

Figure 9 confirms the tendency of the SRTM models to be above the reference, occurring the biggest differences in areas of natural and cultivated vegetable cover.

Therefore, from the data in Table 5 and 6 and the information contained in Figure 8, 9 and 10, some advantages of the relative assessment may be cited in relation to absolute:

• Use of a much higher population of reference observations, corresponding to the center of the pixels of the Reference DEM, making the role of the GCP.

• The reference DEM covers 100% of the study area and, therefore, all the pixels of the models, except for the null pixels, were evaluated, allowing to check the largest and smallest outliers.

• The image of the difference between the DEM makes it possible to clearly identify the areas of major discrepancies, enabling to identify objectively the potential causes.

However, it is important to emphasize that to the relative positional accuracy assessment reaches more reliable results and not distort the results, the positional accuracy of the reference model should be at least three times better than the evaluated model (DSG, 2016; Celestino & Philips, 2014; Camargo et al., 2009).

6. Conclusion

The absolute positional accuracy assessment was performed from GCP collected by GNSS survey. The evaluation result is translated by means of the RMSE (Monico et al., 2009). Thus, briefly, it can be stated that, on average, the absolute accuracy of the Reference DEM are better more than three times than both evaluated models; and the SRTM model is better than ASTER, as indicated below (7).

$${RMSE}_{DEMRef.}=\mathrm{1,2}m<{RMSE}_{SRTM}=\mathrm{4,2}m<{RMSE}_{ASTER}=\mathrm{7,6}m$$ (7)

The component that interferes most in the RMSE of the SRTM is the tendency (approximately 3.7 meters). If this systematic error is eliminated, the RMSE of the SRTM will be approximately equal to the accuracy (Monico et al., 2009), that is, 2.0 meters, which would imply PEC-PCD "A" and "B" in scales 1:50,000 and 1:25,000, respectively.

In the relative assessment, the results of positional accuracy were slightly better than the absolute evaluation, being the averages of the RMSE of the models given by (8).

$${RMSE}_{SRTM}=\mathrm{3,7}m<{RMSE}_{ASTER}=\mathrm{7,1}m$$ (8)

The great accuracy of the DEM SRTM in relation to ASTER is due to the techniques used for the generation of the altimetric model, being the first carried out by radar and the second by photogrammetry, which was subject to interference of clouds and other variants (Goméz et al., 2012).

Although the ASTER models are inferior in positional accuracy than the SRTM, according to the results of the PEC-PCD (Tables 5 and 6), these models can be used for mapping in scales 1:100,000 and 1:250,000.

One advantage of ASTER DEM in relation to SRTM is the greater completeness, that is, do not present null pixels. Unlike the SRTM models of 1 arcseconds, which in some cases can contain many null pixels.

In this research, it was mainly aimed to define and present a practical example between the absolute and relative assessment in a case where the reference data has accuracy three times better than the evaluated models, being this is a fundamental verification in the evaluation of the positional accuracy.

Currently, the Brazilian norm for positional accuracy assessment of geospatial data (ET-CQDG) only adopts the absolute method and there are no measures which defines the relative method.

However, as the reference DEM accuracy is at least 3 times better than the other evaluated models, it was possible to prove that the results of the relative evaluation are similar or better statistically, being translated by PEC-PCD results.

The relative assessment gives the advantage of checking the entire pixel population of the models, allowing a global analysis and, thus, identifying a much larger amount of outliers, distortions and displacements, including being possible to delimit regions of outliers, which is less likely with a limited set of control points.

REFERENCES

• Ariza-López, F. J., Atkinson-Gordo, A. D. J., Nero, M. A. & Cintra, J. P. (2007) ‘La Componente Posicional de los Datos Geográficos: análisis crítico de los Métodos de Evaluación y Reporte en Hispanoamérica’, Revista Cartográfica, 83, pp. 95-152.
• BRASIL. (2008) “Decreto nº 6.666, de 27 de novembro de 2008: Institui, no âmbito do Poder Executivo federal, a Infraestrutura Nacional de Dados Espaciais - INDE, e dá outras providências.
• BRASIL. (1984). Decreto-Lei nº 89.817 de 20 de junho de 1984: Normas Técnicas da Cartografia Nacional.
• BRASIL. “Decreto-Lei nº 243, de 28 de fevereiro de 1967: Fixa as diretrizes e bases da cartografia brasileira e dá outras providências.
• Camargo, F. F., Florenzano, T. G., Almeida, C. M., & Oliveira, C. G. (2009) ‘Acurácia posicional de modelos digitais de elevação relativo e absoluto, gerados a partir de imagens ASTER/Terra’, Anais XIV Simpósio Brasileiro de Sensoriamento Remoto. Natal, RN, 25 - 30 April. Natal: INPE, pp. 1747-1754.
• Celestino, V. S. & Philips, J. W. (2016) ‘Avaliação altimétrica SRTM e ASTER GDEM em áreas de relevos diferenciados’, Revista Brasileira de Cartografia, 68, pp. 15-42.
• CONCAR- Comissão Nacional de Cartografia. “Especificação Técnica para a Estruturação de Dados Geoespaciais Vetoriais (ET-EDGV).” Brasília, DF, 2008.
• DoD (1990), MIL STD 60001: Mapping, Charting and Geodesy Accuracy, USA, Department of Defense, Washington.
• DSG- Diretoria do Serviço Geográfico do Exército. “Especificação Técnica para Produtos de Conjuntos de Dados Geoespacias (ET-PCDG).” 1. Ed., 2. Rev. Brasília, DF, 2015.
• DSG- Diretoria do Serviço Geográfico do Exército. “Especificação Técnica de Controle de Qualidade de Dados Geoespaciais (ET-CQDG).” EB80-N-72.004, 1. Ed. Brasília, DF, 2016.
• DSG- Diretoria do Serviço Geográfico do Exército. “Especificações Técnicas para a Aquisição de Dados Geoespaciais Vetoriais (ET-ADGV).” 1ª versão. Brasília, DF, 2008.
• Elkhrachy, I. (2017). Vertical accuracy assessment for SRTM and ASTER Digital Elevation Models: A case study of Najran city, Saudi Arabia. Ain Shams Engineering Journal.
• Forkuor, G., & Maathuis, B. (2012). Comparison of SRTM and ASTER derived digital elevation models over two regions in Ghana-Implications for hydrological and environmental modeling. In Studies on environmental and applied geomorphology. InTech.
• França, L., & Ferreira da Silva, L. (2018). Comparison between the Double Buffer Method and the Equivalent Rectangle Method for the quantification of discrepancies between linear features. Boletim de Ciências Geodésicas, 24(3), 300-317.
• França, L. L. S., Almeida, A. D. O., Penha, A. D. L. T. (2018). Controle de Qualidade dos Modelos Digitais de Elevação ASTER e SRTM segundo a ET-CQDG para o estado da BAHIA. Revista Brasileira de Cartografia, 69(9). 1797-1806
• Gemael, C. (1994). Introdução ao ajustamento de observações: aplicações geodésicas. Editora UFPR.
• Gómez, M. F., Lencinas, J. D., Siebert, A., & Díaz, G. M. (2012). Accuracy assessment of ASTER and SRTM DEMs: a case study in Andean Patagonia. GIScience & remote sensing, 49(1), 71-91.
• ISO. (2013). 19.157: 2013. Geographic information - Services. Quality management systems-Requirements (ISO 19.157: 2013).
• Józsa, E., Fábián, S. Á., e Kovács, M. (2014). An evaluation of EU-DEM in comparison with ASTER GDEM, SRTM and contour-based DEMs over the Eastern Mecsek Mountains. Hung. Geog. Bull, 63, 401-423.
• LP DAAC. NASA (2014) ‘Shuttle Radar Topography Mission (SRTM) Version 3.0 Global 1 arc second Data Released over Europe, North America, and South America.’ Available in: Available in: https://lpdaac.usgs.gov/about/nasa_shuttle_radar_topography_mission_srtm_version_30_global_1_arc_second_data_released_over (Acessed: 27 April 2018).
  » https://lpdaac.usgs.gov/about/nasa_shuttle_radar_topography_mission_srtm_version_30_global_1_arc_second_data_released_over
• Monico, João Francisco Galera, Dal Poz, Aluir Porfírio, Galo, Maurício, Dos Santos, Marcelo Carvalho, & Oliveira, Leonardo Castro de. (2009) ‘Acurácia e precisão: revendo os conceitos de forma acurada’, Boletim de Ciências Geodésicas, 15 (3).
• Moura, L. Z., de Souza Bias, E., & Brites, R. (2014). Avaliação da acurácia vertical de modelos digitais de elevação (MDEs) nas bacias do Paranoá e São Bartolomeu. Revista Brasileira de Cartografia, 1(66/1).
• Nero, M. A., Cintra, J. P., Ferreira, G. D. F., Pereira, T. Á. J., & Faria, T. S. (2017). A computational tool to evaluate the sample size in map positional accuracy. Boletim de Ciências Geodésicas, 23(3), 445-460.
• Neto, A. P. J.; Cerqueira, R. W.; Carvalho, J. A. B.; Greco, R. L. F. (2014). Projeto de Mapeamento da Bahia. XXVICongresso Brasileiro de Cartografia. Gramado - RS, 26, p.1-10.
• Penha, A. D. L. T., Morita, C. Y., & De Cerqueira, R. W. (2012). Geração de base cartográfica digital a partir de produtos fotogramétricos para a geração de ortofotocarta, carta topográfica e banco de dados geográficos-o caso do projeto de mapeamento do Estado da Bahia. Simpósio Brasileiro de Ciências Geodésicas e Tecnologias da Geoinformação, 4, 1-9.
• Reuter, H. I., Neison, A., Strobl, P., Mehl, W., & Jarvis, A. (2009, July). A first assessment of ASTER GDEM tiles for absolute accuracy, relative accuracy and terrain parameters. In Geoscience and Remote Sensing Symposium, 2009 IEEE International, IGARSS 2009(Vol. 5, pp. V-240). IEEE.
• Santos, A. D. P., das Graças Medeiros, N., dos Santos, G. R., & Rodrigues, D. D. (2016). Avaliação da acurácia posicional planimétrica em modelos digitais de superfície com o uso de feições lineares. Boletim de Ciências Geodésicas, 22(1), 157.
• Santos, A. D. S., Medeiros, N. D. G., Santos, G. R. D., & Lisboa Filho, J. (2017). se of geostatistics on absolute positional accuracy assesment of geospatial data. Boletim de Ciências Geodésicas, 23(3), 405-418.
• Tachikawa, T., Kaku, M., Iwasaki, A., Gesch, D. B., Oimoen, M. J., Zhang, Z., ... & Abrams, M. (2011). ASTER global digital elevation model version 2-summary of validation results. NASA.
• USGS- United States Geological Survey (2014) ‘ASTGTM: ASTER Global Digital Elevation Model V002.’ U.S./Japan ASTER Science Team. Available in: Available in: https://lpdaac.usgs.gov/node/1079 (Acessed: 27 April 2018).
  » https://lpdaac.usgs.gov/node/1079
• USGS- United States Geological Survey (2015) ‘Shuttle Radar Topography Mission (SRTM) 1 Arc-Second Global.’ Available in: Available in: https://lta.cr.usgs.gov/SRTM1Arc (Acessed: 27 April 2018).
  » https://lta.cr.usgs.gov/SRTM1Arc

Publication Dates

• Publication in this collection
  18 Apr 2019
• Date of issue
  2019

History

• Received
  01 May 2018
• Accepted
  24 Oct 2018