Optimal image quantization, perception and the median cut algorithm

Optimal Image Quantization, Perception and the Median Cut Algorithm

CICERO MOTA¹, JONAS GOMES² and MARIA I. A. CAVALCANTE¹

¹Instituto de Ciências Exatas, Universidade do Amazonas

Av. Gel. R. O. J. Ramos 3000, 69077-000 Manaus, AM, Brazil

²Visgraf Laboratory, IMPA

Estrada Dona Castorina 110, 22460-320 Rio de Janeiro, RJ, Brazil

Manuscript received on March 12, 2001; accepted for publication on March 19, 2001;

contributed by JONAS GOMES^*

ABSTRACT

We study the perceptual problem related to image quantization from an optimization point of view, using different metrics on the color space. A consequence of the results presented is that quantization using histogram equalization provides optimal perceptual results. This fact is well known and widely used but, to our knowledge, a proof has never appeared on the literature of image processing.

Key words: Image quantization, color quantization, optimization, median cut, information theory.

1. INTRODUCTION

To represent an image by digital means it is necessary to substitute its continuous range of colors by a finite subset of them. This process is called quantization. If the function f : U

C represents an image with color range in C, the quantization process consists of the choice of a function q : C

Q = {q₁,..., q_n}, defined on the color space C and taking values in a finite subset Q C. The function q is called a quantizer for f, while Q is the reproducing alphabet. The colors q₁,..., q_n are called quantization levels. The subset Q, with known size, can be given a priori or may be part of the quantization problem.

The image quantization problem appears very early and at different stages of image processing. It first appears at the acquisition level and it is a basic process at the coding level. Quantization always implies a perceptual loss of quality of the image, and most of the well known quantization techniques exploit some biological limitations of the human visual system, e.g., spatial acuity, nevertheless they hardly touch some psychophysics aspects of that (Lloyd 1957, Heckbert 1982, Linde et al. 1980). In this work, we use known psychophysics results on the human visual system (Fechner 1858, Stevens 1961, von Helmholtz 1891) to study quantization algorithms that are adapted to the way humans perceive colors. The resulting algorithms will depend on the choice of a distortion measure for the color space. Different choices will lead to different strategies for color quantization. We have designed distortion measures based on the so called psychophysics response function for the visual system and some of them were implemented for comparison. Also, as a consequence of the results presented, we derive the well known result that quantization by histogram equalization has, from the viewpoint of the information theory, optimal perceptual qualities for the resulting quantized image.

2. BACKGROUND

The problem of image quantization can be easily posed as optimization problem, once we define a metric on the color space and correlate the quantization error to the distribution of colors present in the image. We then look for a quantizer that minimizes the expect error introduced during the process. Before we proceed to the n-dimensional problem, we will present the scalar case, which was introduced by Lloyd (1957).

2.1 OPTIMAL SCALAR QUANTIZATION

When an image f

U

C is quantized an error is introduced between a color x C and its corresponding quantized value q(x). This punctual error can be measured by the choice of distortion measure in the color space C of the image, see subsection 2.3. Denote this measure by d, the expect value for d (x, q(x))

(1)

will provide an measure of the overall impact of the error introduced by the quantization process, where p(x) is the probability density function (pdf) for the colors in the image.

The quantizer q is called optimal, when E(d, q) is minimum. A common choice is d (x₁, x₂) = | x₁ - x₂|², which enables an elegant solution for the case of grayscale images (Lloyd 1957): if x_k and x_{k + 1} are the extremes of the quantization cell, then one can obtain, by simple computing derivatives, that the quantization intervals boundaries are given by

(2)

and the quantization levels are computed from the expression

(3)

The solution presented above was simplified by two fairly strong hypothesis: the assumption that the colors present in the image have a probability distribution function, and the particular choice for the distortion measure d.

Nevertheless, both hypothesis are not common in practice. Most natural images present large areas with constant colors, which annihilates the hypothesis of the existence of a probability density function. Besides that, the choice for d had no direct relation with the ultimate end of the quantization process, that is, the image will be sought by a human being. Thus, general methods that allow for direct experimentation with humans and that could avoid the use of derivatives are desired. In the following, we present a general class of such methods as well a description of the quantization cells for color images.

2.2 WEBER'S LAW

A fundamental aspect of the human visual system is that it does not perceive light intensity continually but in steps. If

x_k = x_{k + 1} - x_k is the lower quantity such that is possible to perceive difference between gray light intensities x_k and x_{k + 1}, the Weber's law (Weber 1834, Fechner 1858) states that

=

(4)

In consequence, if we start with an intensity value x₀, the k-th value will be x_k = x₀(1 + )^k = x₀e^klog(1 + ), which shows that the human visual system uses a non-uniform quantization of the color space.

Intuitevely, the Weber's law states that the intensities values x_k and x_{k + 1} differs from each other by k units. Therefore, one can model the psychophysics response function by (x) = clog(x). The constant value c incorporates a possibly change in the basis of the logarithm.

The psychophysics response function can be derived from the axioms of information theory (Resnikoff 1987). In this context, the only possibilities for are:

(

(5)

or

(

(6)

The model (x) = clog(x) was proposed by Fechner (1858), while the model (x) = cx^r was proposed by Stevens (1961).

2.3 DISTORTION MEASURES

From a general viewpoint, we can use as a distortion measure on a color space C any function d : C x C

, which satisfies

1) d (x, y) > 0 if x ¹ y and d (x, x) = 0;

2) d (x, y) = d (y, x). When, in addition, we have

3) d (x, z) d (x, y) + d (y, z), the measure d is called a metric.

Property 3) is handy since it permits to estimate the overall error in a multi step quantization process from the errors introduced at each step.

Another way to interpret the Weber's law is to think that the unity used to measure distances in the color space changes according with the color, this change being proportional to the inverse of the color intensity. The mathematical concept that abstracts this notion of adaptive change of unity for each point in a surface is the Riemannian metric.

In the context of Riemannian metrics, the Weber's law for color intensities suggests the metric for the one-dimensional color space. For the three-dimensional color space, this metric generalizes to

(

(7)

which is known as metric of von Helmholtz (1891). In such a metric, the distance between the colors x = (x₁, x₂, x₃) and y = (y₁, y₂, y₃) is

log

(8)

In consequence, if two color differ only by intensity, i.e., y = r x, one have

| log(

(9)

which is the psychophysics response function proposed by Fechner.

Let's say that the psychophysics function is . We can easily construct distortion measures that take in account. One such choice is

(

(10)

Observe that the particular choice = (,,), where (x) = log(x_i) and r = 2 is equivalent to the choice of the von Helmholtz's metric for the three-dimensional color space.

2.4 VORONOI DIAGRAM

Given a set of points {x₁,..., x_n} C, the subsets

C_j = {x Î C | d (x, x_j) £ d (x, x_k), fork = 1,..., n} (11)

form together a partition of C called Voronoi diagram, which has an ubiquitous presence in the computational geometry literature due to its good computational properties and many applications. In particular, the are many known efficient algorithms to compute the Voronoi diagram (de Figueiredo and Carvalho 1991, Fortune 1987). As we will see, it also plays a role in the quantization problem.

3. OPTIMAL QUANTIZATION

In the following we will consider the problem of optimal quantization for a measurable function f : U

C, U

². For this, transport the measure of U to C. That is, for a subset C₁ of C, define m(C₁) = (f^-1(C₁)), where is the standard Lebesgue measure for U, see (Fernandez 1976) for details on measure theory. For simplicity, we will suppose (U) = 1, which means that m will be a measure of probability for C.

The quantization problem, thus, consists of given a distortion measure d, find a quantizer q : C

Q = {q₁,..., q_n} that minimizes the expected error

(12)

It should be remarked that the computation of the quantization levels q_j , j = 1,..., n, is part of the problem.

For each quantization level q_j we define the corresponding quantization cell C_j by C_j = q^-1(q_j), where q is the quantization function. In other words, C_j is the set of all colors in C which are quantized to the level q_j.

The problem of computing the quantization levels q_j, j = 1,..., n and the corresponding quantization cells C_j are closely related. If we have the quantization levels in Q, we can compute the quantization cells. In fact, let C_i be the cells in the Voronoi diagram for Q, we have

E(d, q) = d (x, q(x)) dm =

d (x, q(x)) dm

d (x, q_j) dm.

Therefore, the Voronoi diagram is the best partition for this choice of the quantization levels. Reciprocally, suppose that we have the quantization cells C_j. Thus the best value for q_j is certainly

=

(13)

From the above properties of the quantization levels and quantization cells, one can design a descendent algorithm to compute the optimal quantizer. The basics for such an algorithm is presented below as Algorithm 1. We note that at each step of the algorithm, the value for E(d, q) decreases. It is possible to show that this algorithm belongs to a class of convergent algorithms, under certain conditions, see (Gray et al. 1979).

The followings propositions will be useful for the construction of Table I to be used for implementation purpose.

PROPOSITION 1. Let q : C

Q = {q₁,..., q_n} be an optimal quantizer for an image f : U

C and B the common border of the quantization cells C_i and C_j. Then

1) B is halfway between q_i and q_j, that is, if x B, then d (x, q_i) = d (x, q_j).

2) In particular for scalar quantization, if d (x, y) = |(x) - (y)|^r and is one-to-one, we have

(

(14)

PROOF. 1) Since C_k are cells of the Voronoi diagram for Q, we have that x C_kif and only if d (x, q_k) d (x, q_s) for any s = 1,..., n. Therefore, if x B, then d (x, q_i) d (x, q_j) and d (x, q_j) d (x, q_i), which shows that d (x, q_i) = d (x, q_j).

2) We may suppose, without loss of generality, that q_i ¹ q_j , otherwise, we join the cells C_i and C_j in a single one. Thus, for d (x, y) = |(x) - (y)|^r, it follows that

(x) - (q_i) = (q_j) - (x)

or

(x) - (q_i) = (x) - (q_j)

but the second equation implies (q_i) = (q_j). From the injectivity of , we have q_i = q_j, which has been excluded.

PROPOSITION 2. Let q : C{q₁,..., q_n} be an optimal quantizer for an image f : U

C. Denote the quantization cells by C_k , then:

1) If d (x, y) = |(x) - (y)|², (q_k) is the mean of (x) over C_k, that is

(

(15)

2)If d (x, y) = |(x) - (y)|, then (q_k) is the median of (x) over C_k.

PROOF. To proof the proposition, just observe that (James 1981)

mean =

E(X - c)²

and

median =

E| X - c|.

For the particular case of scalar quantization, maps the median of x on the median of (x). Therefore, Proposition 2 presents a noticeable fact: If the distortion measure is given by d (x, y) = |(x) - (y)|, the quantization level is the median of the quantization interval and therefore does not depend on the function .

Table I presents the discrete formulas. They will be used for the implementation to be discussed in Section 4. In this table, z_jdenotes the color in the image range, p_j the total number of the occurrences of the color z_j in the cell C_k and n_k the number of the color occurrences in the cell C_k.

3.1 QUANTIZATION AND INFORMATION GAIN

In this section, we will show that color quantization by histogram equalization provides optimal results in the sense that it maximizes the information retained by the quantization process. In the literature of computer graphics this quantization technique was introduced by P. Heckbert (1982) and is known as the median cut algorithm. Also, it is well known in the area of image processing that a simple histogram equalization improves the perceptual properties of an image. Although the good perceptual qualities of histogram equalization are well known and widely used, to our knowledge, a proof has never appeared on the literature.

Quantization by histogram equalization consists in choosing the quantization cells in such way that each of then contains the same number of colors. This is equivalent to a constant color histogram in the quantized image. For scalar quantization, this means that we should choose the borders of the quantization cells as x_k = F^-1(k/n), where F(x) = dm is the accumulated probability distribution of the colors in the image.

Given a measure m on C and two measurable subsets C₂

C₁ of C, the information gain from C₂ with respect to C₁ is defined by (Resnikoff 1987)

(16)

In particular, when m is a probability measure, the information gain of a subset C₁ related to the set C is

(17)

From now on, we will be concerned only with probabilities measures. Let = {C₁,..., C_n} be a partition of C by measurable subsets, that is, C = C_k and m(C_i C_j) = 0, if i ¹j. Denote by p_k = m(C_k), since m(C) = 1, we have p_k = 1. Therefore p_k is discrete probability and the expected gain of information or entropy related to this subdivision of C is defined by

) = -

(18)

Intuitively, I is a measure of the information present in a given quantization of an image, with cells C_k. Let's refine the partition by introducing a new cell, that is, we split C_k = C_k1

C_k2 with m(C_k1

C_k2) = 0. Let's call this new partition and write p_k = p_k1 + p_k2, where p_ki = m(C_ki). We have

- p_klog(p_k) = - (p_k1 + p_k2)log(p_k1 + p_k2) £ - p_k1log(p_k1) - p_k2log(p_k2)

from where,

I() £ I().

The above equation corresponds to the intuitive notion that the perceptual quality of a quantized image improves if we use more quantization cells. On the other side, each term of the sum in I is a function of the type g(x) = - xlog(x), x > 0 and therefore it satisfies g(x) = g(x) = 0. This in turn implies that cells with very high or very small probability will produce a negligible contribution for the information gain. The former possess little information, while the latter have small probability. Therefore, a natural question arises: what are the best partitions for C in order to maximize the information gain?

PROPOSITION 3. Let f : U

C be an image and = {C₁,..., C_n} a partition of C. Then the information gain is maximum for p_k = m(C_k) = 1/n.

PROOF. We want to maximize I = - p_klog(p_k) subject to the restriction p_k = 1. Therefore, we can apply the Lagrange method searching for the singular points of I. Let L be defined by

Differentiating , we find

= - log(

Therefore p_k = e - 1 and using the restriction, we have p_k = 1/n.

Since I is a concave function, its restriction to the subspace {(p₁,..., p_n) | p_k = 1} is also concave, therefore the only critical point is actually the place where I reaches its global maximum.

4. IMPLEMENTATION

From the formulas in Table I, we see that the expressions for x_k and q_k are defined in a recursive way. Once we know the values for x₁,..., x_n, we can compute the values for q₁,..., q_nand vice-verse. This suggests Algorithm 2.

The stop criteria can be the number of iterations or one can halt the algorithm when E(d, q) stops decreasing, in this case some kind of threshold has to be used. The algorithm was implemented by Romildo Silva according to Table I and applied to the well known image of Lena. Originally with 256 colors, the image was quantized to 16, 8, 4 and 2 colors for comparison between different possibilities for d. The algorithm of the median cut was also implemented for comparison. The results are presented in Figures 1 to 5 at the end of the paper.

Fig. 1
- Lena image quantized to 16, 8, 4 and 2 bits/pixel and distortion measure d (x, y) = | x - y|.

Fig. 2 - Lena image quantized to 16, 8, 4 and 2 bits/pixel and distortion measure d (x, y) = | x - y|².

Fig. 3 - Lena image quantized to 16, 8, 4 and 2 bits/pixel and distortion measure d (x, y) = | log(x) - log(y)|.

Fig. 4 - Lena image quantized to 16, 8, 4 and 2 bits/pixel and distortion measure d (x, y) = | log(x) - log(y)|².

Fig. 5 - Lena image quantized to 16, 8, 4 and 2 bits/pixel and using the median cut algorithm.

5. ACKNOWLEDGMENTS

This research has been developed in the VISGRAF laboratory at IMPA. The laboratory is sponsored by CNPq, FAPERJ, FINEP, and IBM Brasil. C. M. thanks the Institute for Signal Processing, University of Luebeck, where this work has been finished.

RESUMO

O problema perceptual relacionado a imagens quantizadas é estudado do ponto de vista da otimização, usando diferentes métricas no espaço de cores. Como conseqüência dos resultados apresentados, mostra-se que quantização por equalização de histograma fornece resultados perceptuais ótimos. Esse resultado é conhecido e amplamente usado mas, ao que saibamos, sua prova nunca apareceu na literatura de processamento de imagens.

Palavras-chave: Quantização de imagens, quantização de cores, otimização, algoritmo do corte mediano, teoria da informação.

REFERENCES

DE FIGUEIREDO LH AND CARVALHO PCP. 1991. Introdução à Geometria Computacional, 18° Colóquio Brasileiro de Matemática, IMPA.

FECHNER GT. 1858. Über ein wichtiges psychophysiches Grundgesetz und dessen Beziehung zur Schäzung der Sterngrössen. Abk. k. Ges. Wissensch., Math.-Phys. K1, 4.

FERNANDEZ PJ. 1976. Medida e integração. Instituto de Matemática Pura e Aplicada, CNPq. Projeto Euclides.

FORTUNE SJ. 1987. A sweepline algorithm for Voronoi diagrams. Algorithmica, 2: 153-174.

GRAY RM, KIEFFER JC AND LINDE Y. 1979. Locally optimal block quantization for sources without a statistical model. Technical Report L-904-1, Stanford University Information Systems Lab.

HECKBERT P. 1982. Color image quantization for frame buffer display. Computer Graphics 16: 297-307.

JAMES BR. 1981. Probabilidade: Um Curso em Nível Intermediário. Instituto de Matemática Pura e Aplicada. Projeto Euclides.

LINDE Y, BUZO A. AND GRAY RM. 1980. An algorithm for vector quantizer design. IEEE Transactions on Communications, COM-28(1): 84-95.

LLOYD SP. 1957. Least squares quantization in PCM's. Bell Telephone Laboratories Paper, Murray Hill, NJ.

RESNIKOFF HL. 1987. The Illusion of Reality. Springer Verlag: 22-86, 184-208. STEVENS SS. 1961. To honor Fechner and repeal his law. Science 133: 80-86.

VON HELMHOLTZ H. 1891. Versuch einer erweiterten Anwendung des Fechnerschen Gesetzes im Fabensystem. Z Psychol Physiol Sinnesorg, 2: 1-30.

WEBER EH. 1834. De pulsu, resorptione, audita et tactu. Annotationes anatomicae et physiologicae, Leipzig: Koehler.

• DE FIGUEIREDO LH AND CARVALHO PCP. 1991. Introduçăo ŕ Geometria Computacional, 18° Colóquio Brasileiro de Matemática, IMPA.
• FECHNER GT. 1858. Über ein wichtiges psychophysiches Grundgesetz und dessen Beziehung zur Schäzung der Sterngrössen. Abk. k. Ges. Wissensch., Math.-Phys. K1, 4.
• FERNANDEZ PJ. 1976. Medida e integraçăo. Instituto de Matemática Pura e Aplicada, CNPq. Projeto Euclides.
• FORTUNE SJ. 1987. A sweepline algorithm for Voronoi diagrams. Algorithmica, 2: 153-174.
• GRAY RM, KIEFFER JC AND LINDE Y. 1979. Locally optimal block quantization for sources without a statistical model. Technical Report L-904-1, Stanford University Information Systems Lab.
• HECKBERT P. 1982. Color image quantization for frame buffer display. Computer Graphics 16: 297-307.
• JAMES BR. 1981. Probabilidade: Um Curso em Nível Intermediário. Instituto de Matemática Pura e Aplicada. Projeto Euclides.
• LINDE Y, BUZO A. AND GRAY RM. 1980. An algorithm for vector quantizer design. IEEE Transactions on Communications, COM-28(1): 84-95.
• LLOYD SP. 1957. Least squares quantization in PCM's. Bell Telephone Laboratories Paper, Murray Hill, NJ.
• RESNIKOFF HL. 1987. The Illusion of Reality. Springer Verlag: 22-86, 184-208.
• STEVENS SS. 1961. To honor Fechner and repeal his law. Science 133: 80-86.
• VON HELMHOLTZ H. 1891. Versuch einer erweiterten Anwendung des Fechnerschen Gesetzes im Fabensystem. Z Psychol Physiol Sinnesorg, 2: 1-30.
• WEBER EH. 1834. De pulsu, resorptione, audita et tactu. Annotationes anatomicae et physiologicae, Leipzig: Koehler.

Publication Dates

• Publication in this collection
  05 Oct 2001
• Date of issue
  Sept 2001

History

• Accepted
  19 Mar 2001
• Received
  12 Mar 2001