The weighted gradient: a color image gradient applied to morphological segmentation

^IDepartment of Informatics State University of Maringá, Av. Colombo, 5790, Zona 7 Phone: [phone] (FAX) {fcflores - ampolido }@din.uem.br

^IIDepartment of Computer Engineering and Industrial Automation, School of Electrical and Computing Engineering, State University of Campinas, P.O.Box 6101, Zip 13083-852 - Campinas - SP - BRAZIL. {fcflores - lotufo }@dca.fee.unicamp.br

ABSTRACT

This paper proposes a method for color gradient computation applied to morphological segmentation of color images. The weighted gradient (with weights estimated automatically), proposed in this paper, applied in conjunction with the watershed from markers technique, provides excelent segmentation results, according to a subjective visual criterion. The weighted gradient is computed by linear combination of the gradients from each band of an image under the IHS color space model. The weights to each gradient are estimated by a systematic method that computes the similarity between the image to compute the gradient and an "ideal image", whose histogram has an uniform distribution. Several experiments were done in order to compare the segmentation results provided by the weighted gradient to the results provided by other color space metrics, also according to a subjective criterion, and such comparison is present in this paper.

Keywords: Weighted Gradient, Color Gradient, Morphological Segmentation, Watershed from Markers, Bray-Curtis Distance Function, Weight Estimation.

1 Introduction

The edge enhancement of an image by gradient computation is an important step in morphological image segmentation via watershed from markers [1, 10]. For grayscale images, the morphological gradient [14, 7] is a very good option and its computation is simple: for each point in the image, a structuring element is centered to it and the difference between the maximum and the minimum graylevels inside the structuring element is computed. Here, the similarity information exploited is the intensity differences among pixels inside the structuring element.

Such concept does not extends naturally to color images. Despite the similarity information of color images is richer than the similarity of grayscale ones, the design of methods to edge enhancement in color images is complex. Note that the metric that measures the natural similarity information of color images is unknown. Also note that if one considers the color space as a complet lattice [14, 2], the order relation is not total and even if a total order is imposed, it will be not natural for the human eye. It does not compare two colors and decides which one is higher. It makes no sense to say: "yellow is higher than blue".

This paper proposes a metric to compute a gradient for color images by exploiting the intuitive notion of similarity and not by imposing any total order relation in the color space model. The weighted gradient operator [3, 8] consists in, given a color image in the IHS color space model, the linear combination of the weighted gradients from the three bands of image. The role of weights is to enhance or obscure the importance of a gradient in the linear combination: the greater the importance of information contained in a band, the greater the weight to be assigned to its gradient. For instance, if the color image presents a great variation of hues, the hue gradient should receive a major weight.

One way to compute automatically the weights to each band is via application of a similarity function [3]. The idea is to consider an "ideal image" (whose histogram has a uniform distribution) and to compute the similarity between the input image and the ideal one. Several ways to compute the similarity between pairs of objects (in our case, images) were found in the literature [12, 5, 9]. In this paper, the similarity is computed by a distance function called Bray-Curtis distance [12] which gives a value from 0 (similarity) to 1 (not similarity). The weight to a gradient is given by the application of this distance function.

This paper is organized as follows: section 2 presents preliminary definitions. Section 3 gives the definitions of weighted gradient. Section 4 states the Bray-Curtis distance function, used to estimate the weights. Section 5 introduces the weight estimator to the weighted gradient operator. Section 6 explains the morphological image segmentation via watershed form markers. Section 7 shows the comparison among several color image gradients and the segmentation results provided by them. And, finally, we conclude this paper in section 8.

2. Preliminary Definitions

This section presents some definitions used along the text.

2.1. Color Images

A partial ordered relation is a relation whose properties is reflexive, transitive and anti-symmetric. A set whose elements have partial order relation is called a partial ordered set. Given a ordered set L, with a relation order "<". L is a totally ordered set if, ∀x,y ∈ L, × < y or y < x. The relation order "<" is said total. A partial ordered set L is a complete lattice (also called chain) if any subset of L have an infimum and a supremum.

Let L₁,L₂,...,L_m be totally ordered complete lattices. For instance: a subset of , or a closed subset of are chains. Let V denote the supremum, or maximum, and Λ denote the infimum, or minimum, operations in chains. Let L be the Cartesian product of L₁,L₂,...,L_m, i.e., l ∈ L ⇔ l = {l₁,l₂,...,l_m},l_i ∈ L_i,i = {1,...,m}.

Let E be a non empty set that is an Abelian group with respect to a binary operation denoted by +. A mapping f from E to L, f(z) = (f₁(z),f₂(z),...,f_m (z)),f_i : E → L_i and z ∈ E is called multivalued or multispectral image. The mappings f_i are called bands of the image. A color image is an example of a multivalued function where L = L₁ × L₂ × L₃ is the representation of the colors under a certain color space model (a coordinate system where each point represents an unique color) [6, 10]. Let Fun[E,L] denote the set of all mappings from E to L, i.e., all possible color images.

2.2. Color Space Models

In this paper, we used two color space models: the RGB (Red, Green and Blue) and the IHS (Intensity, Hue and Saturation) color coordinate systems.

The RGB color space model is a cube defined in a Cartesian system. This cube is defined by three subspaces related to one of the three primary colors, red, green and blue, and the pure representation of these colors are located at three corners of the cube. Generally, one color is represented by a point inside this cube.

The IHS color space model is defined by a coordinate transformation of the RGB system [10], and it is composed by three attributes: Intensity (holds the luminosity information), Hue (describes uniquely a color in its pure form); for example, the red color without any information from other attributes) and Saturation (measures the amount of white light mixed with pure colors).

3. Weighted Gradient

In this section, we present the definition of the weighted gradient. This operator is a transformation from a color image under the IHS color space to a grayscale one, by the linear combination of the gradients from each band.

Let K = [0,1,...,k - 1] and = [0,359]. The set K is a closed interval and has a total order relation. The set , however, does not have such relation, since it can be considered as a "circular interval", because its elements are angles [6]. Let f ∈ Fun[E,L] be a color image under the IHS color space model, where E ⊂ × and L = K × × K.

Since intensity and saturation are represented by functions g ∈ Fun[E,K], their gradients can be computed by classical morphological gradient operators [14, 7]. However, it is not the case for hue: since the information of the hue band consists of angles [6], the band is given by a function h ∈ Fun[E,K], and its gradient can not be computed by classical morphological gradient, because does not have a total order relation. In order to compute a gradient for the hue band, it is necessary to define a metric function for it.

Definition 3.1Let. The hue distance between θ₁ and θ₂is given by

The distance introduced above returns an integer value between 0 and 180. In order to compute the weighted gradient, these values will be normalized to K. Let m_K : → K be the normalization function.

Let x ∈ E. The translation of B ⊂ E by x, denoted by B_x, is given by B_x = {y ∈ E : (y - x) ∈ B}.

Definition 3.2Given an image h ∈ Fun[E, ], the angular gradient : Fun[E, ] → Fun[E,K], is given by, for all x ∈ E

where B ⊂ E is a structuring element centered at the origin of E.

Let a,b,c ∈ ₊ . The following gradient is defined as a linear combination of the morphological gradients for the intensity and saturation bands with the angular gradient for the hue band. The three gradients are weighted by three weights a, b and c in order to enhance or to obscure the peaks of a gradient image.

Definition 3.3 Given f ∈ Fun[E,L] in the IHS color space. Given three weight values a,b,c ∈ ₊. The weighted gradient : Fun[E,L] → Fun[E, +] is defined by

where f₁, f₂ and f₃ represent respectively the intensity, hue and saturation color bands, B ⊂ E is a structuring element centered at the origin of E and ∇_B is the classical morphological gradient [14, 7]. The floor operation in the equation means that all , were rounded down, since the linear combination of the weighted gradients is real.

In the weighted gradient, each gradient is scaled in order to increase or decrease their influence in the final result. For example, on images where the hue information is the most important, the hue gradient could give a good result by itself. However, there are cases that it is not sufficient to enhance the border between objects with similar hues; it could be necessary to weight, for example, the saturation gradient in order to distinguish the borders.

Figure 1 (b) shows the result of the weighted gradient applied to Fig. 1 (a). The weights assigned to the hue, saturation and intensity gradients are, respectively, a = 0.1055, b = 0.1016 and c = 0.1133.

The first version of the weighted gradient operator was introduced in [8]. Despite this operator shows itself as a good solution to compute color gradients, it was not defined, in that occasion, a systematic method to find weights to each gradient: the weights assigned to each gradient in the linear combination step had been imposed manually. It was necessary a priori to analyze subjectively the color image to be segmented in order to choose the weights to be applied. Sometimes, it was necessary to readjust several times the weights to achieve a satisfactory result. It motivated the design of the weight estimator presented in section 4 and the automatic weighted gradient introduced in section 5.

4. Bray-Curtis Distance Function

A similarity function quantifies the association degree between a pair of objects. We have found that similarity functions could be suitable for application in order to estimate the weights to the gradients [13, 5, 9]. In the method introduced below, it was used a well known method to numerical ecology researchers, the Bray-Curtis distance function [13]. The distance measured by this function gives a degree of similarity, which is a value from 0 to 1. The closer the degree is to 0 (1), the greater (lower) the similarity. The Bray-Curtis distance function is defined below.

Definition 4.1Let A and B be two n-dimensional vectors. The Bray-Curtis distance function D : A × B → [0,1], is given by,

where A_i and B_i are, respectively, the i-th element of A and the i-th element of B.

5. Weight Estimation

As stated above, the drawback of the manual weighted gradient technique was the lacking of a systematic method to find the weights, which have been imposed manually. The main criterion applied to choose the weight to a gradient was the diversity of information contained in the original image. For instance:

1. if there are many different hues appearing in the image, it is imposed a high weight to the hue gradient original;

2. if there are just blue tones in the image, the hue gradient is not so important and its weight could be low (however, this gradient does not need to be discarded, since it can still help to separate, for example, objects with similar saturation);

3. if the image presents a great variation in luminance, the intensity gradient should receive a higher weight.

Despite the drawback of manual imposition of weights, the weighted gradient gives excellent results when the criterion cited above is applied. This criterion also should be applied to the automatic method to weight estimation in order to still achieve good results.

If a band presents all possible values in same quantities (i.e., its histogram has an uniform distribution), its gradient should receive the maximum weight. To do this, we consider that there is an "ideal image" where all possible values appear in same quantities (i.e., with a histogram whose distribution is an uniform one). The distance between the band and the ideal image is computed; if the distance is closer to 0, it means that the information in the band is rich and the weight to be assigned should be high.

The distance between the band and the ideal image can be obtained by computing the distance between their histograms. Note that there is no real "ideal image" but just only its histogram, which has an uniform distribution. Such histogram is easy to be created: just consider it as a function with constant value equals to the mean of histogram of the band.

Definition 5.1 Let us consider the histogram of an image f ∈ Fun[E,K] as a function h_f : K → ₊. Let f_i : E → L_i be one of the image bands and let g_i : E → L_i be the ideal image. The weight w_i assigned to the gradient of f_i is given by,

where D is the Bray-Curtis distance function and h_gi is the histogram of the ideal image, computed as explained above.

Bray-Curtis distance function is a simple and suitable way to compute the weight to a gradient. Its computation is fast and it gives the similarity measure as a normalized value contained in the interval [0,1] (whose boundaries denotes, in our approach, respectively, similarity and disparity), and thus, providing a simple way to estimate the weight by the criterion pointed above.

Note that the Bray-Curtis distance function gives small results when the histograms are close to. So, it is necessary to subtract the function result from 1 (the maximum weight possible) in order to obtain the weight to the gradient.

Given the definition above, the weighted gradient can be redefined as follows:

Definition 5.2 Given f ∈ Fun[E,L] in the IHS color space. The weighted gradient : Fun[E,L] → Fun[E, +] is defined by

where f₁, f₂ and f₃ represent respectively the intensity, hue and saturation color bands, w₁,w₂,w₃ ∈ ₊are their respective weights estimated by the distance function, B ⊂ E is a structuring element centered at the origin of E, ∇_Bis the classical morphological gradient [14, 7] and is the angular gradient.

In the following, it is presented some experiments done in order to demonstrate the estimation of weights by Bray-Curtis distance function. Each experiment consisted in a creation of a synthetic pure blue image under a IHS color space model and changing one of the bands by the classical cameraman image, in order to enrich the information of the band and, thus, to demonstrate how the method assigns a higher weight to this band. The synthetical image created to the hue, saturation and intensity experiments are shown, respectively in Fig. 2 (a), 3 (a) and 4 (a).

In the first experiment (Fig. 2), we demonstrate the estimation to the hue band. Figure 2 (a) shows an image obtained by taking a constant blue image and changing its hue band by the classical cameraman normalized image. We want to show that, since the hue band is the most important in this image (saturation and intensity bands in this image are constant), the weight to be assigned to the hue gradient should be high and the weights assigned to the saturation and intensity bands should be low.

The proposed distance function computed the following weights to Fig. 2 (a): hue w₁ = 0.8945 , saturation w₂ = 0.0039 and intensity w₃ = 0.0039. Figure 2 (b) shows the weighted gradient of Fig. 2 (a).

The other two experiments shown in this section are similar to the first one. Figure 3 (a) was obtained by taking a constant blue image and changing its saturation band by the cameraman image. The saturation band of Fig. 3 (a) is the most important band, since the other two bands are constant, so it is expected that the weight assigned to the saturation gradient should be high and the other two weights should be low. The distance function provided to Fig. 3 (a) the weights: hue w₁ = 0.0078 , saturation w₂ = 0.8555 and intensity w₃ = 0.0039. Figure 3 (b) shows the weighted gradient of Fig. 3 (a).

The image used in the third experiment (Fig. 4 (a)) was computed by taking a constant blue image and changing its intensity band by the cameraman image, in order to emphasize the intensity information and lower the importance of the hue and saturation bands. It were computed the weights: hue w₁ = 0.0078 , saturation w₂ = 0.0078 and intensity w₃ = 0.9844. The weighted gradient of Fig. 4 (a) is shown in Fig. 4 (b).

6. Watershed from Markers

Morphological segmentation via watershed from markers [1] consists in a sequential application of morphological operators [7] to segment objects in the image. Basically, it consists in application of morphological gradient operator [7, 4], marker selection and watershed from markers technique [1, 16].

The watershed from markers can be described by flooding a topographical relief model of the gray-scale image. The markers are holes in the image relief where colored water can enter as the relief is flooded. There is one color associated to each set of markers. As the relief is uniformly flooded, different colored water may meet but cannot be mixed. When all the relief is flooded, each colored water region defines the catchment basin [12] associated to the marker. The classical watershed transform is when the markers are the regional minima of the image.

Watershed from markers is an efficient method to image segmentation, since it reduces the problem of objects segmentation to the problem of finding markers to these objects.

7. Experimental Results

This section presents the segmentation results provided by application of several color image gradients. All of them, excepting the automatic weighted gradient, proposed in this paper, were compared previously in [8] and the manual weighted gradient provided the best segmentation results. The experiments described below aim to compare the automatic and the manual weighted gradients to other gradients introduced in [8]. It is expected segmentation results provided by the automatic weighted gradient, at least, as good as the results provided by the manual method.

Preceding the experiment descriptions, it is necessary to review the gradients that will be compared. Note that the weighted gradient is applied to color images under the IHS color space model. The following operators (Gradients I to IV ) are applied to color images under the RGB color space model.

7.1 Gradient I

Let f ∈ Fun[E,L], the Gradient I of f, (f) is given by

where ∇_B(f) is the morphological gradient. The resulting gradient is an image containing the supremum among the maxima differences em each band of f.

7.2 Gradient II

Let f ∈ Fun[E,L], the Gradient II of f, is given by

where is the internal morphological gradient [7]. The resulting gradient is similar to Gradient I, however only internal border of the objects are enhanced.

7.3 Gradient III

Let d_III be the norm of a color l ∈ L, given by

where is the value of p rounded down.

Let a_x,B,b_x,B ∈ E are two points such that

Let f ∈ Fun[E,L], the Gradient III of f, (f) is given by

where u_f,x,B = (f₁(a_x,B) -f₁(b_x,B),f₂(a_x,B) -f₂(b_x,B), f₃(a_x,B) - f₃(b_x,B)).

7.4 Gradient IV

Let d_IV be the color distance between l and l' , l,l' ∈ L, given by,

Let f ∈ Fun[E,L], the Gradient IV of f, (f) is given by

7.5 Segmentation Results

The first experiment consists in to segment the three flowers (Fig. 5 (a)) given the set of markers imposed manually to each flower and to the background (Fig. 6 (a)). Note the difficult in to distinguish the borders among the flowers, mainly between the middle and right ones. Here, the segmentation results were compared.

Figures 5 (b-e) show, respectively the results of Gradient I, II, III, IV . Figure 5 (f) shows the manual weighted gradient (with weights {a = 1,b = 1,c = 1}) and Fig. 5 (g) shows the automatic weighted gradient (whose estimated weights were {w₁ = 0.9570,w₂ = 0.9219,w₃ = 0.9336}). Figure 6 (b-g) show their respective segmentation results. Again, in order to have a better visualization, the gradient images were negated and the watershed lines were dilated and overlaid to the original image.

All gradients provided the correct segmentation of the left flower, but the separation of the middle flower from the rightest one showed to be critical. Gradients I to IV segmented the middle flower with a piece of the rightest one. Both weighted gradients almost segmented correctly the flowers. Note that the contrast between the middle and the right flowers is very low.

The goal in the second experiment was to segment a set of billiard balls from the table (Fig. 7 (a)), given the set of markers shown in Fig. 8 (a), by application of the gradients presented above and to compare subjectively their segmentation results.

Figures 7 (b-e) show, respectively the results of Gradient I, II, III, IV . Figure 7 (f) shows the manual weighted gradient (with weights {a = 1,b = 1,c = 1}) and Fig. 7 (g) shows the automatic one (whose estimated weights were {w₁ = 0.5977,w₂ = 0.7109,w₃ = 0.1055}). Figure 8 (b-g) show their respective segmentation results. The gradient images were negated and the watershed lines were dilated and overlaid to the original image for a better visualization.

Gradient I provided a good segmentation of the balls, except to the black one; this ball and its shadow were segmented together. The black ball segmentation showed itself a critical problem since it was not also achieved by the Gradients II, III and IV . Gradient II provides an unpleasant segmentation of the ball and Gradients III and IV did not achieved a good segment of the red ball.

Both manual and automatic weighted gradients provided good segmentation results, excepting by a minor distortion in the segmentation of the pink ball, the segmentation results provided by the automatic method is very close to results provided by the manual one.

In order to make the third experiment, we created a color image (Fig. 9 (a)) where the hue band is a transition from red (left )to magenta (right) passing by yellow, green, cyan and blue colors. The saturation band is constant with the maximum allowed value. The intensity band is given by classical foreman image in grayscale. The goal in this experiment is to segment the foreman, given the set of markers in Fig. 10 (a). Again, the segmentation results are compared subjectively.

The results of Gradient I, II, III, IV are shown, respectively, in Fig. 9 (b-e). Figure 9 (f) shows the manual weighted gradient (with weights {a = 1,b = 1,c = 1}) and Fig. 9 (g) shows the automatic one (whose estimated weights were {w₁ = 0.7305,w₂ = 0.0078,w₃ = 0.9844}). Figure 10 (b-g) show their respective segmentation results. Once more, the gradient images were negated and the watershed lines were dilated and overlaid to the original image.

Note that the helmet region showed itself critical in the segmentation process. Gradients I, II, III and IV missed in several degrees the helmet border and the background invaded the helmet. Gradient IV also provided a wrong segmentation in the eye of the foreman.

The trivial set of weights imposed to the manual version of the weighted gradient ({a = 1,b = 1,c = 1}) did not provide a correct segmentation result. The top of the helmet joined to the upper background. The automatic weighted gradient estimated a better set of weights {w₁ = 0.7305,w₂ = 0.0078,w₃ = 0.9844}) and provided a correct segmentation of the foreman.

8. Conclusion

This paper proposes a gradient for color images by linear combination of weighted gradients from the three bands of an image under the IHS color space model: the weighted gradient operator. The function of the weights is to raise or lower the contribution of each gradient in the linear combination, and they are estimated by an automatic method that computes the similarity between the band histogram and an histogram whose distribution is uniform. Such similarity is computed by a Bray-Curtis distance function, whose gives a value from 0 (similarity) to 1 (not similarity).

Several experiments were done in order to compare the segmentation results provided by various color gradients following a subjective visual criterion. The weighted gradient, with weights estimated by Bray-Curtis distance function, provided the best segmentation results.

One of the advantages of the proposed method is that it not requires any parameters other than the color image itself and a connectivity parameter. The evaluation criterion of the diversity of information in an image, given by the distance between the image band and an "ideal image" whose histogram has an uniform distribution, is a very good criterion to give weights to each band gradients.

Future works include the application of other techniques, such as other distance functions and information theory methods (as first-order entropy), in order to do weight estimation in the proposed framework. It will be also studied the exploitation of other color space models in order to compute color gradients.

Acknowledgements

The second author is on leave from UEM to UNESP at Presidente Prudente-SP-Brazil for doctorate purposes.

• [1] S. Beucher and F. Meyer. Mathematical Morphology in Image Processing, chapter 12. The Morphological Approach to Segmentation: The Watershed Transformation, pages 433-481. Marcel Dekker, 1992.
• [2] J. Chanussot and P. Lambert. Total Ordering Based on Space Filling Curves for Multivalued Morphology. In Henk J.A.M. Heijmans and Jos B.T.M. Roerdink, editors, Mathematical Morphology and its Applications to Image and Signal Processing, volume 12 of Computational Imaging and Vision, pages 51-58. Kluwer Academic Publishers, Dordrecht, May 1998.
• [3] F. C. Flores, A. M. Polidório and R. A. Lotufo. Color Image Gradients for Morphological Segmentation: The Weighted Gradient Improved by Automatic Imposition of Weights. In IEEE Proceedings of SIBGRAPI'2004, pages 146-153, Curitiba, Brazil, October 2004.
• [4] F. C. Flores, R. Hirata Jr., J. Barrera, R. A. Lotufo, and F. Meyer. Morphological Operators for Segmentation of Color Sequences. In IEEE Proceedings of SIBGRAPI'2000, pages 300-307, Gramado, Brazil, October 2000.
• [5] G. F. Estabrook, D. J. Rogers. A general method of taxonomic description for a computed similarity measure. Bioscience, 16:789-793, 1966.
• [6] R. C. Gonzalez and R. E. Woods. Digital Image Processing Addison-Wesley Publishing Company, 1992.
• [7] H. J. A. M. Heijmans. Morphological Image Operators Academic Press, Boston, 1994.
• [8] R. Hirata Jr., F. C. Flores, J. Barrera, R. A. Lotufo, and F. Meyer. Color Image Gradients for Morphological Segmentation. In IEEE Proceedings of SIBGRAPI'2000, pages 316-322, Gramado, Brazil, October 2000.
• [9] J. C. Gower. A general coeficient of similarity and some of its properties. Biometrics, 27:857-871, 1971.
• [10] F. Meyer and S. Beucher. Morphological Segmentation. Journal of Visual Comunication and Image Representation, 1(1):21-46, September 1990.
• [11] Pratt. Digital Image Processing, chapter 18. Image Segmentation, pages 597-627. Wiley, 1991.
• [12] R. A. Lotufo and A. X. Falcão. The Ordered Queue and the Optimality of the Watershed Approaches. In J. Goutsias, L. Vincent, and D. S. Bloomberg, editors, Mathematical Morphology and its Applications to Image and Signal Processing, volume 18, pages 341-350. Kluwer Academic Publishers, 2000. Fifth ISMM.
• [13] R. J. Bray , J. T. Curtis. An ordination of the upland forests communities of southern Wiscounsin. Ecological Monography, 27:325-349, 1957.
• [14] J. Serra. Image Analysis and Mathematical Morphology Academic Press, 1982.
• [15] H. Talbot, C. Evans, and R. Jones. Complete Ordering and Multivariate Mathematical Morphology: Algorithms and Applications. In Henk J.A.M. Heijmans and Jos B.T.M. Roerdink, editors, Mathematical Morphology and its Applications to Image and Signal Processing, volume 12 of Computational Imaging and Vision, pages 27-34. Kluwer Academic Publishers, Dordrecht, May 1998.
• [16] L. Vincent and P. Soille. Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(6):583-598, June 1991.

The weighted gradient: a color image gradient applied to morphological segmentation

Franklin César Flores^I,II; Airton Marco Polidório^I; Roberto de Alencar Lotufo^II

Publication Dates

• Publication in this collection
  26 Oct 2010
• Date of issue
  Apr 2006