COMPARISON OF DIFFERENT FUSION ALGORITHMS IN URBAN AND AGRICULTURAL AREAS USING SAR (PALSAR AND RADARSAT) AND OPTICAL (SPOT) IMAGES

Image fusion techniques of remote sensing data are formal frameworks for merging and using images originating from different sources. This research investigates the quality assessment of Synthetic Aperture Radar (SAR) data fusion with optical imagery. Two different SAR data from different sensors namely RADARSAT-1 and PALSAR were fused with SPOT-2 data. Both SAR data have the same resolutions and polarisations; however images were gathered in different frequencies as C band and L band respectively. This paper contributes to the comparative evaluation of fused data for understanding the performance of implemented image fusion algorithms such as Ehlers, IHS (Intensity-Hue-Saturation), HPF (High Pass Frequency), two dimensional DWT (Discrete Wavelet Transformation), and PCA (Principal Component Analysis) techniques. Quality assessments of fused images were performed both qualitatively and quantitatively. For the statistical analysis; bias, correlation coefficient (CC), difference in variance (DIV), standard deviation difference (SDD), universal image quality index (UIQI) methods were applied on the fused images. The evaluations were performed by categorizing the test area into two as “urban” and “agricultural”. It has been observed that some of the methods have enhanced either the spatial quality or preserved spectral quality of the original SPOT XS image to various degrees while some approaches have introduced distortions. In general we noted that Ehlers’ spectral quality is far better than those Comparison of diferent fusion algorithms in urban and agricultural... Bol. Ciênc. Geod., sec. Artigos, Curitiba, v. 18, n 4, p. 509-531, out-dez, 2012. 5 1 0 of the other methods. HPF performs almost best in agricultural areas for both SAR images.


INTRODUCTION
Image fusion is an application dependent framework for making use of two or more complementary images/spectral bands of the same or different sensors for the same area to get more information which is not achieved using each image/band alone in order to enhance the quality of image interpretation.In the literature, Hall (1992), Genderen and Pohl (1994), Pohl and Genderen (1998), Wald (1998) and Simone et al. (2003) describe image fusion and its techniques in detail.Due to complementary information of different characterized spectral or spatial multi sensor data, image fusion can facilitate image interpretation (ZHOU et al., 1998).Images with similar acquisition dates and images with different dates are both used for image fusion.While the expectation is to obtain a fused image that retains the spatial resolution from the higher resolution images and colour content from the multispectral image with the first type of fusion processes, it is to detect the changes over a period of time with the latter one (ZENG et al., 2010).In the literature, terms such as merging, combination, synergy, integration, and several others that express more or less the same concept have been used related to fusion (HELMY et al., 2010).However the following definition, which explains clearly data fusion framework, has been adopted in remote sensing community: "Data fusion is a formal framework in which are expressed means and tools for the alliance of data originating from different sources.It aims at obtaining information of greater quality; the exact definition of 'greater quality' will depend upon the application" (WALD, 1999).
Image fusion technique can be done with several algorithms and integration of multi source data is of the essence for many applications (EHLERS, 1991;WANG et al., 2005;ZENG et al. 2010;HELMY et al., 2010).
In recent years, the launches of new SAR satellites such as ENVISAT, ALOS, TERRASAR and RADARSAT-2 have opened a new era for remote sensing applications.Previous studies proved that the combination of optical and SAR data provide more accurate identification when compared to the results obtained with the individual sensors (ASCHBACHER & LICHTENEGGER, 1990).Due to the fact that the geometry and structure of the earth surface are more effective in microwave backscatter than in the surface reflection occurring in optical images (POHL & GENDEREN, 1998), using these multiple types of sensors for image fusion increases the quality of images.However fusion of microwave data and optical data is still a challenge.There are many image fusion methods that can be used to produce high-resolution multispectral images from microwave data and multispectral images.In this paper, the terms 'microwave data' and 'multispectral data' are representing synthetic aperture radar (SAR) and optical data respectively.At present, IHS (LI and WANG, 2001;TSAI, 2004), Brovey Transformation (BINH et al., 2006), PCA (AMARSAIKHAN & DOUGLAS, 2004), HPF (BETHUNE et al., 1998;AIAZZI et al., 2006a), DWT (ZHANG & HONG 2005;JIN et al., 2006), Gram Schmidt Transformation (AIAZZI et al., 2006b), Smoothing Filter Based Intensity Modulation (LIU, 2000), Synthetic Variable Ratio (ZHANG, 1999) are among the pixel based image fusion algorithms generally used.In the literature, there are several studies comparing these techniques analyzed by Zhou et al. (1998), Zhang and Hong (2005), Colditz et al. (2006), Teggi et al. (2003), Shi et al. (2005), Wang and Bovik (2002).Fusion methods are used for gathering information from SAR and Optical data for various purposes such as qualifying the fused images (ROKHMATULOH et al., 2003;SHI et al., 2005) for enhancement of the geological information gathered (Pal et al., 2007), and assessing the contribution of fusion techniques to rural and urban land use classification (AMARSAIKHAN et al., 2012;KURUCU et al., 2009;KUPLICH et al., 2000;SUN et al., 2003).Moreover the contribution of image fusion algorithms to the fused images were investigated statistically and visually for various data sets such as; Ehlers et al. (2010) evaluated fusion results of TerraSARX with SPOT (2,4,5) and Formosat 2 images, where they combined X band SAR data with optical data.Palubinskas et al. (2010) proposed a fusion approach for very high-resolution optical and radar satellite imagery.
This article extends the previous work of Abdikan et al. 2008 as outlined in the following lines.Here image fusion results of HH polarized L-band PALSAR, and HH polarized C-band RADARSAT-1 image with three band SPOT-2 XS image are presented.Note that both radar data used in this study have same polarisation with the same resolution but have different frequencies for the same area.Particularly in SAR images the operating frequency is an important factor in the penetration depth affecting the backscatter value.Thus it is also investigated visually how different SAR features are preserved in the fused images as well as making the quality assessments of the resulting images visually and statistically.In addition to IHS, PCA, HPF, DCW techniques used previously, in this study Ehlers' method was also included in the comparisons as the first extension (i.e.here the results of five fusion methods were compared).The quality assessment analyses were applied to resulting multi sensor data.To evaluate the used fusion methods, all fused results were compared visually and statistically to SPOT-XS as the reference image.Secondly, the test area is separated into two as "urban" and "agricultural" for getting more precise results related to the homogenous areas under consideration.Finally, among the quality indices Bias, CC, DIV, SDD and UIQI statistical analyses were derived from the fused images for the image comparison.In forming the biases and SDDs, differing from Abdikan et al. 2008, we preferred simply taking the differences instead of applying normalizations.The fused images were also downsampled to the spatial resolution of the reference image (i.e SPOT XS) (see section 3.1).

STUDY AREA
This study has been performed on a test site located in the Menemen (Izmir) Plain to the west of Gediz Basin in the Aegean Region of Turkey.The Aegean Sea lies in the west of the study area, and Manisa Province lies in the North (Figure 1).The study area covers about 400 square km including both residential and agricultural areas.Due to the fact that texture and slope are important characteristics for SAR backscattering, the area has been selected with a micro relief where the slope was in general 1% (i.e.flat).In the SAR images, the surface roughness affects geometry and backscatter values.Depending on the SAR acquisition orbits (ascending or descending) SAR data has distortions such as shadow, layover and foreshortening.Especially in shadow areas there is no backscatter values related to earth surface.To avoid the distortions caused by the relief, an area with micro relief is preferred like in many SAR applications.
Study area covers residential areas and agricultural fields, thus we were able to choose test sites from both urban and rural areas.This agricultural area was chosen to investigate spectral preservation because the colour changes appear more in agricultural fields depending on the crop types and harvesting dates.In general, the study area was unplanted.However, some of the fields were covered with the winter crops such as wheat and pasture.The soil in the area was compacted to prevent moisture loss.The surface roughness of the study area is homogeneous.In this season, since the area had not received enough rain until the beginning of May, the soil moisture levels varied usually according to its water holding capacity.Hence most of the colour difference appears due to the planted and unplanted fields with different moisture content in the area.The view of the study area is derived from a false colour composite of SPOT-2 bands (NIR-Red-Green as RGB composite) in Figure 1.Here red, green and reddish black colours imply land cover of wheat and pasture, ploughed soil surfaces with different moisture contents, and swampy areas with nature plantation respectively.On the other hand the colour changes were less severe in the urban areas than agricultural fields.However urban areas were chosen as the other test areas to investigate the spatial improvement.

DATA SETS AND METHODOLOGY
Present paper contributes to the comparative evaluation of fused data for understanding the performance of the implemented image fusion algorithms for SAR and Optical data.Quality assessments of fused images were performed both qualitatively and quantitatively.For the statistical analysis bias, CC, DIV, SDD, UIQI methods were applied on the fused images.Two different test areas were defined to make the quality assessments as urban and agricultural areas.Each of the selected areas covers 11 km 2 approximately.It has been observed that some of the methods have enhanced either spatial quality or preserved spectral quality of the original SPOT XS image to various degrees while some approaches have introduced distortions.We will discuss these in detail in the upcoming sections.

1 Data and Pre-processing
The assessments of data fusion algorithms have been performed on two different SAR data from different sensors namely RADARSAT-1 and PALSAR, both images were fused with SPOT-2 data (Figure 2).Although the PALSAR and the RADARSAT-1 images have the same resolutions (6.25 m x 6.25 m) and same polarisations, the images were gathered in different frequencies as L-band and Cband respectively.To avoid the effects of opposite passes, both SAR images were chosen in ascending orbits.A SPOT HRV-2 XS MS imagery having three spectral bands with a 20 m resolution were used as the optical data.A fine beam single polarized (FBS) mode of PALSAR (Phased Array type L-band Synthetic Aperture Radar) data of ALOS satellite (Advanced Land Observing Satellite) imagery and a Fine Beam 1 mode of RADARSAT-1 imagery were used as SAR data having observation capability of high spatial resolution (Table 1).Before the image fusion process SAR images were pre-processed by the commonly used speckle reducing filter techniques.For the filtering of SAR images, among the different sized kernel windows, Gamma filtering of 3x3 kernel size was chosen to suppress the speckle noise.In this study image fusion was conducted at the pixel level.In order to avoid the combination of unrelated data, spatial registration accuracies should be at the sub pixel level.Particularly in SAR data terrain distortions are mainly the causes of the combination of unrelated pixels during the fusion processes.To remove the possible terrain distortions (see Section 2), it is essential to register images perfectly.These geometric distortions are different from optical distortions, and they may be severe in rough topographic areas.If the creation of co-registered datasets is not accurate, the quality of the fused image will decrease significantly.Therefore in fusion applications geometric correction is very important for the registration of the images.Thus SAR images were registered to the SPOT image by using image to image rectification method with a root mean square error of less than 1 pixel, and registered SAR images were resampled to 8 m (see ERSDAC -Palsar User Guide, 2006).Cadastral maps on a scale of 1/5000 and topographic maps on a scale of 1/25000 were used for the rectification of SPOT images with a first degree polynomial function using the nearest neighbour resampling technique.

Image Fusion
In this study, among the presently used pixel based image fusion algorithm, five different fusion algorithms, namely High Pass Filter (HPF), Principle Component Analysis (PCA), Intensity Hue and Saturation (IHS), Discrete Wavelet Transformation (DWT) and Ehlers, have been examined.Both SAR (RADARSAT-1 and PALSAR) images were fused with the 3 bands of a multispectral SPOT-2 image resulting in a data set of 10 fused images.Resulting fused images were resampled to the higher resolution of SAR images as 8m x 8m.Short explanations of the approaches used for fusion are given below: Highpass Filtering uses a band addition approach to fuse both spectral and spatial information of the images.For this purpose, a high resolution image is filtered with a high pass filter to compute the high frequency component.High frequency component, which is concerned to spatial information, is added pixel by pixel basis to each low resolution multispectral images (WANG et al., 2005).In conclusion, by adding a filter to a low resolution band, spatial information content of the high resolution image is replaced and seen in the fused image (BETHUNE et al., 1998).
Principal Component Analysis converts a multivariate data set of intercorrelated variables into new uncorrelated linear combinations of the original values.The principal component domain of the multispectral image is created by principle component transformation.It reduces the dimensionality of the data set due to having high correlation between multispectral bands.The first PC contains more information since it has a large percentage of all the variance.Generally, %95 of the total variance is contained in the first three principal components (POHL & GENDEREN, 1998;TEGGI et al., 2003).The difference between IHS and PCA is that while IHS is used for 3 bands, PCA method can be used for more than three bands.
Intensity Hue Saturation method transforms a low resolution 3-band image as red (R), green (G), blue (B) to intensity (I), hue (H), and (S) saturation components where I refers to the total brightness of the image, H to the dominant or average wavelength of the light contributing to the colour, and S to the purity of the colour (EHLERS et al., 2010).Next the intensity component, which is the spatial information of the image, is replaced with a high resolution image to enhance the spatial resolution.After a reverse transformation from IHS to RGB, a high spatial resolution multispectral image is produced.The disadvantage of IHS is that, it can only process three bands of a multispectral image.For more than 3 bands the IHS transform has to be used more than once depending on the number of bands to be fused (EHLERS, 1991;POHL & GENDEREN, 1998).
In Discrete Wavelet Transformation, a high resolution image is separated in to its low and high frequency components.The aim is to select the base of a waveform to be used.Once the basis waveform is mathematically defined, a family of multiples can be created with increasing frequency as retaining the high-pass images for later image reconstruction.In practice, three or four recursions are sufficient (i.e. the related wavelets of twice the frequency, three times the frequency, four times the frequency, etc. can be created).Here the high resolution image is decomposed to its low and high frequencies.Multi spectral image is replaced with the low pass image which has same resolution.As a last step wavelet decomposition process is reversed to keep the original high resolution level with the spectral information (ERDAS USER GUIDE v9, 2005;SHI et al., 2005).
In Ehlers Fusion, first three low resolution multispectral band images are transformed to an IHS image.Later a two dimensional Fast Fourier Transformation (FFT) is used to transform the intensity (I) component of the image and a high resolution image into the frequency domain.Than a low pass filter is applied on the intensity spectrum, and for the spectrum of high resolution image an inverse high pass filter is used.An inverse FFT is performed on these filtered images, and a new fused intensity image component is formed by adding these filtered images together.New intensity is composed with the high and low frequency information that are extracted from high and low resolution images respectively.This algorithm was developed specifically for a spectral characteristics preserving image fusion (EHLERS, 2004;EHLERS, 2006;EHLERS et al., 2010).

Quality Assessment
In image fusion processes quality refers to both spectral and spatial quality of fused images.In general, quality assessment of fused images can be often performed both qualitatively based on visual inspection and quantitatively based on spectral and/or spatial comparisons (JAIN, 1988).The aim of image fusion techniques is to inject the spatial detail into the multispectral (MS) imagery while keeping the original spectral values.Today although used standard image fusion methods are often successful, the spectral truth remains in the merged images to be checked quantitatively in order to evaluate the performance of each applied fusion algorithm precisely (EHLERS et al., 2010;EHLERS, 2004;KLONUS & EHLERS, 2007;KLONUS, 2008).In general, reference MS images at higher spatial resolution with the same spectral intervals of input MS images are not available for assessing the quality of the fused images.The lack of availability of these reference images makes quality assessment particularly difficult (WALD et.al.1997).The only available reference images are the original MS images at the "low" spatial resolution.To overcome this problem there are two ways.It is either degrading the fused image back to the original image resolution prior to assessment or degradation of both pan and multispectral imagery by the same factor prior to fusion.In this way a number of statistical criteria can be calculated to verify the accuracy of fused images.
In our application, the factors computed to qualify the fused images are bias (BETHUNE et al., 1998;TAHA et. al., 2010), DIV (CAKIR et.al., 2008), SDD (CAKIR et al., 2008;TAHA et. al., 2010), CC (SHI et al., 2005) and UIQI (WANG & BOVIK, 2002;WALD, 2002).As a result fused images were visually and statistically evaluated for colour preservation, spatial enhancement and for spectral fidelity respectively.Assessment analyses for each of the PALSAR-SPOT and RADARSAT-SPOT fused images were determined.To evaluate the preserved spectral quality of the each image fusion techniques used, the original SPOT XS image was compared with 10 resulting fused images in terms of information improvement and fidelity of spectral characteristics.The comparison was performed by statistical and graphical interpretation (Table 2, Table 3, and Figure 5, Figure 6).Prior to the statistical comparisons, the fused images were downsampled to the spatial resolution of the reference image (i.e SPOT XS).Visual evaluations (Figure 3, Figure 4) and statistical analysis were performed considering the local characteristics as urban and agricultural areas rather than performing globally on the entire image scenes of the fused data.For the graphical analyses, a transect (Figure 7) was defined on the same section of the images including urban and agricultural areas (Figure 8 and Figure 9).

Qualitative Visual Comparisons
Visual interpretation was done to compare the fused images with the original SPOT XS image.Figure 3 and Figure 4 show the different fusion performances of RADARSAT-SPOT and PALSAR-SPOT images for urban and agricultural areas respectively.Quality of the spatial resolution was analysed comparing the features like field borders, roads and buildings visually.It is depicted that all methods enhance spatially SPOT-XS image to various degrees but some methods also introduce spectral distortions.The visual comparison of colour information between the fused images and the SPOT XS shows that HPF and PCA give very similar results in both urban and agricultural areas.On the other hand, comparing the spatial quality of all the fused images (Figure 3 and 4) visually, it is obvious that the spatial characteristics inherited from SAR images are more apparent in PCA and HPF given in this order.Especially in urban areas, among the five methods HPF shows features more detailed than PCA.DWT follows them in the third order.Although IHS shows borders clearly especially with PALSAR fused images in agricultural areas, it has the worst colour quality in both PALSAR and RADARSAT fused images.It produces significant colour distortion with respect to the original SPOT XS image, and among all it is the worse in preserving colours.Meanwhile, as seen from the output images, Ehlers method keeps spectral consistency better than other methods.Particularly the colours of the resulting Ehlers images of agricultural and urban areas are almost the same as that of the original SPOT XS image.Besides, DWT method follows Ehlers method as being close to the original SPOT XS image on RADARSAT fusion result.But it produces a noticeable colour distortion in agricultural areas especially for the PALSAR results.From the comparison of the spatial effects it can be seen that the results of the five methods using SAR images contribute to the SPOT XS image in agricultural areas.However in urban areas, which include smaller objects, only HPF and PCA contribute to the results.For both urban and rural test sides; IHS method shows border lines and roads much sharper in agricultural areas whereas it is the worse in residential areas with its noisy texture.IHS method performs better with PALSAR images.
Comparison of the resulting RADARSAT and PALSAR fused images (i.e C band versus L band) was done according to the brightness and sharpness of the futures inherited from SAR data.In radar images, ground penetration depends on the wavelength.It increases with longer wavelengths.Ground penetration is inversely depends on the complex di-electric constant which means that the higher the water content on the ground surface, the higher the reflectivity of radar waves.This means radar backscatter is stronger on soil or vegetation which has moisture content (KUNTZ et. al. 1999).If we look at closer to the image portion of the study area in Figure 4, bare lands and vegetated areas (greenish and reddish colors respectively in SPOT images) are resulting with different brightness in PALSAR and RADARSAT images depending on the frequency differences.Due to the dense features of urban areas, it easier to see the properties of the bands in the agricultural areas than those of in the urban areas.Especially in bare lands the backscatter values are differing according to the penetration in C band and L band SAR data.In agricultural areas better results were obtained with L band images especially for different reflectivity content in the fields and for extracting border lines of the fields and roads.Among the five fusion techniques applied on L band PALSAR; the methods can be put in the order of IHS, PCA and HPF which is the order indicating better preservation of SAR effects.Both spatial characteristics from SAR images and spectral characteristics from optical images are preserved better in PCA whereas IHS is the best for carrying SAR backscatter characteristics.On the other hand colour distortion occurred with DWT is more evident in L band than in C band.In urban areas, the results of PALSAR and RADARSAT have similar visual quality.All the L band-PALSAR fused results are slightly superior in sharpness and brightness.

Statistical Inspections
Here we provide assessments specifically for urban and agricultural areas shown in Figures 3 and 4. First of all, the down sampling of fused images, made the difference in DIV and SSD smaller as expected (Table 2 and Table 3).Both for the fusion of PALSAR-SPOT and RADARSAT-SPOT, HPF and DWT methods in general showed smaller biases and higher correlations (i.e.CC and UIQI, Figure 5 and Figure 6) compared to PCA and IHS.However, best performance on the values of correlations was obtained from Ehlers' method in this study.This study also revealed that HPF method performs better in the agricultural areas.This conclusion is also supported by the visual inspections in the previous section and by the spectral analyses over a transaction that can be found in the upcoming lines.IHS produced the poorest results almost in all categories.We performed an additional spectral analysis for a test site including both agricultural and residential areas using a transect (Figure 7).We compare the spectral profile for SPOT XS in Figure 8 with the spectral profiles of the fused images from different methods in Figure 9.The upper 3 plots of Figure 9 indicate that the correlation of the spectral profiles from HPF, PCA, and IHS with the SPOT image is disturbed in the urban area falling into the ranges of 1500 m and 2500 m.This is relatively better for DWT through which only a little section is disturbed at some 2400 meters.The best method that produces spectral correlations is the Ehlers method.Note that all the original gray values between 60 and 160 are preserved.

CONCLUSIONS
In this article, the results from the five different fusion methods as HPF, IHS, PCA, DWT, and Ehlers were compared visually and statistically, and the results are interpreted.For this purpose, an optical image (SPOT-2) and two pairs of high resolution SAR images (RADARSAT-1 and PALSAR) have been acquired over a flat area including both rural and urban areas-Menemen Plane in Turkey.In order to see the impacts of different penetrations, SAR images were chosen with the same polarisation but with the different bands as C-band and L-band respectively.Quality assessment was performed to select the best merged images, and the comparisons were made between the SAR components of the fusion process.As well as the visual comparisons, five different statistics were applied to evaluate the resulting ten fused images.
Our first major conclusion for this study is that separating the study area as urban and agriculture revealed a fact that HPF method relatively does not perform well in urban areas.Secondly, Ehlers' method performs best among all other methods as far as qualitative analysis is concerned.Quantitative analysis also supports this result (i.e UIQI and CC assessments indicate large correlation values).Third, although the linear features became visible in the merged images of IHS, the statistical results were the poorest for this method.In addition, colour and brightness were distorted in IHS fused images.
Considering the colours of the merged images, Ehlers' fused images are almost the same as those of the original SPOT XS image.Ehlers method shows the most coherent colours which can be useful to perform visual analysis for land use/cover.Statistical and graphical results of Ehlers method also confirmed those visual results.Considering all indicators together for the fusion results of RADARSAT-SPOT and PALSAR-SPOT images, quantitative analysis illustrates that the Ehlers' fusion method performs satisfactorily in preserving spectral fidelity.Qualitative analysis show that, the HPF approach performs satisfactorily in enhancing spatial information for multi-sensor remote sensing images.As a result, with the rapid fusion of optical and radar data, specific imaging may become advantageous for monitoring the earth for different purposes using Ehlers and HPF image fusion techniques.
From our overall evaluations, especially based on the statistical comparisons, we provide the following conclusions for our study area: As far as the use of RADARSAT images for urban areas is concerned Ehlers' method can be preferred over all other 4 methods.Only the biases are slightly worse compared to the HPF and DWT methods however in other statistical measures Ehlers' is superior.
DWT can be considered to be an optimal method in urban areas if PALSAR images are used.It is because Ehlers has large biases for this comparison and HPF correlations are significantly lower compared to DWT and Ehlers ones.Slightly lower DWT correlation values compared to Ehlers solutions would not degrade the homogeneity of the statistical results from different categories.
The user can confidently prefer HPF for agricultural areas for both RADARSAT and PALSAR images over all other methods.Only the CC and UIQI values are slightly lower than the Ehlers ones however HPF is superior in all other indicators.ZHANG Y.A new merging method and its spectral and spatial effects.

Figure 3 -
Figure 3 -Comparison of the spectral and spatial effects of the fused SAR-SPOT results for urban areas.

Figure 4 -
Figure 4 -Comparison of the spectral and spatial effects of the fused SAR-SPOT results for agricultural fields.

Figure 5 -
Figure 5 -Statistical graphics of CC and UIQI of fusion methods for urban areas.

Figure 6 -
Figure 6 -Statistical graphics CC and UIQI of fusion methods for agricultural areas.

Table -
Data information.