Generalized line criterion for Gauss-Seidel method

Generalized line criterion for Gauss-Seidel method

M.V.P. Garcia^I; C. Humes Jr.^II; J.M. Stern^II

^IDepartment of Applied Mathematics, University of São Paulo, Brasil

^IIDepartment of Computer Science, University of São Paulo, Brasil

E-mail: [e-mail]/ [e-mail]/ [e-mail]

ABSTRACT

We present a module based criterion, i.e. a sufficient condition based on the absolute value of the matrix coefficients, for the convergence of Gauss-Seidel method (GSM) for a square system of linear algebraic equations, the Generalized Line Criterion (GLC).

We prove GLC to be the ''most general'' module based criterion and derive, as GLC corollaries, some previously know and also some new criteria for GSM convergence. Although far more general than the previously known results, the proof of GLC is simpler. The results used here are related to recent research in stability of dynamical systems and control of manufacturing systems.

Mathematical subject classification: 65F10, 65F35, 15A09.

Key words: Gauss-Seidel method, line criterion, stability and control of manufacturing systems.

1 Introduction

We present a module based criterion, i.e. a sufficient condition based on the absolute value of the matrix coefficients, for the convergence of Gauss-Seidel method (GSM) for a square system of linear algebraic equations, the Generalized Line Criterion (GLC).

We prove GLC to be the ''most general'' module based criterion and derive, as GLC corollaries, some previously know and also some new criteria for GSM convergence. Although far more general than the previously known results, the proof of GLC is simpler. GLC is shown to be a necessary and sufficient condition of the convergence for a class of matrices defined by its sign structure.

The results here presented are analogous to important results in the stability of manufacturing systems, where matrices with the sign structure of S(A) are common. These matrices appear in the dynamics of systems operating under generalized Round-Robin policies, either explicitly imposed, as in [5], or induced by polyhedral cones [8], [3], [4].

It is quite natural that convergence and stability results are closely related, as extremely well exploited in [6], [7]. In many instances the GLC, and the associated linear algebra operations, like scaling and permutation, have natural interpretations for system stabilization and control. Therefore, the major motivation for the GLC is to present these results in a self contained form, that is completely in the context of simple computational linear algebra. The simple linear form of the GLC also facilitates the use of several related techniques like optimization, sensitivity, and post-optimal analysis for manufacturing systems.

2 Problem setting

In this paper, the following notation is used, 1 is the vector [1, 1, ¼1]^t, the diagonal operator D = diag(d) defines the diagonal matrix D whose diagonal elements are given by vector d, whereas a = diag(A) defines vector a as the main diagonal of matrix A, so D = diag(diag(A)) gives D_i,i = A_i,i, D_i,j = 0, i ¹ j.

The matrices |A| and S(A) associated to a given matrix, A, are defined next. Matrices with the sign structure of S(A) appear naturally in the control theory of manufacturing systems [5], [3].

|A_i,j| = |A_i,j| ; S(A)_i,i = |A_i,i| and S(A)_i,j = -|A_i,j| , i ¹ j

The problem in question is the solution of the linear system of equations Ax = b, where A Î ^n×n, b Î ⁿ and the coefficient matrix has no null diagonal elements, A_i,i¹ 0, as usual in GSM. GSM, and several other iterative methods can be written as stationary iterative methods:

x^k+1 = Bx^k + c

where B is called the method's iteration matrix. Many of those methods are obtained splitting the coefficient matrix A = M – N, M non-singular, and then writing

Mx^k+1 = Nx^k + b , or x^k+1 = M^–1 Nx^k + M^–1b .

Splitting the coefficient matrix in its lower, upper triangular and diagonal parts: A = L + D + U, i > j L_i,j = 0, j > i U_i,j = 0 and D = diag(diag(A)), GSM is the stationary iterative method with M = (D + L) and N = –U, or B = (D + L)^–1U and c = (D + L)^–1b.

If the above iteration is performed ''in parallel'', i.e., using at iteration k+1 only x^k and not the already available elements of x^k+1, we have the Jacobi method, JM, where M = D and N = -(L + U), or B = -D^–1(L + U) and c = D^–1b.

The method converges, for any initial point x⁰, iff r(B) º max|eig(B)| < 1, i.e. iff all eigenvalues of the iteration matrix lie inside the unit circle.

This general necessary and sufficient criterion is usually hard to use when asserting convergence, because finding the largest eingenvalue of B is a hard non-linear problem. However since, for any consistent matrix norm, r(B) < B, the condition B < 1 is a sufficient convergence criterion. In particular, the infinite norm B

_¥º max_x = 1Bx = max_iå_j |B_i,j| provides strict diagonally dominance as an easily verified sufficient criterion for JM and GSM convergence [1]. This is known as the

Line Criterion (LC). A obbeys LC if S(A)1 > 0.

The following lemma will be needed:

Lemma 1. The convergence of the iterative method Mx^k+1 = Nx^k + b implies that A = M – N is non-singular.

Proof. Assume by contradiction that for some h ¹ 0, Ah = 0, so h = M^–1 Nh. Then B = M^–1N has 1 as an eingenvalue, and the method can not be convergent [2].

3 Generalized Line Criterion (GLC)

Scaling the equations and variables of a liner system Ax = b is the operation of multiplying the coefficient matrix by left and right non-singular diagonal matrices obtaining the equivalent system EAWy = Eb. Scaling is widely used in computational linear algebra for pre-conditioning, control floating point error accumulation, etc. The GSM convergence has the interesting property of not being affected by scaling the linear system:

Lemma 2. Let E and W be non-singular diagonal matrices. Then the GSM converges on Ax = b iff it converges on EAWy = Eb.

Proof. Since the pre-multiplication of A by E does not affect the GSM formulas and considering the bijection Wy = x, the result follows immediately.

Let us now define the GLC convergence criterion for the GSM on Ax = b.

Generalized Line Criterion (GLC). A obeys GLC iff there is a vector w > 0 such that S(A)w > 0.

It is trivial that if A obeys LC then A obeys GLC with w = 1. Moreover, using W = diag(w), if A obeys GLC then AW obeys the line criterion. This observation and lemma 2 lead to:

Theorem 1. If A obeys the GLC then the GSM iteration applied to Ax = b converges.

Although GLC is a sufficient condition for GSM convergence, it is not a necessary condition as it can be seen by the example:

where the eigenvalues of B = (D + L)^–1U all lie inside the unit circle although the system does not satisfy GLC, because det(S(A)W) = det(S(A)) = 0. However, if A has a special sign structure, then GLC becomes also a necessary condition:

Theorem 2. If A has a sign structure such that, in each line, the diagonal element sign is opposite to the sign of all other elements, then GLC turns out to be a necessary and sufficient condition for GSM convergence.

Proof. Using the left scaling E = diag(diag(A))^–1, we have S(A) = A, and it suffices to prove the theorem for the case diag(A) = 1 and A_i,j< 0 , i ¹ j. Since the sufficiency of GLC has already been established in Theorem 1, we only have to show that if GSM converges for an A such that diag(A) = 1 and A_i,j< 0 , i ¹ j, then A obeys GLC.

Splitting A = L + U, diag(L) = 1, and we can write L and T = L^–1 in product form, L = L¹L²¼L^n–1 and T = T^n–1T^n–2¼T¹, where L^k and T^k = (L^k)^–1, have the structure:

The factors in the product form perform elementary row operations, i.e. if y = L^kx then y_k+h = x_k+h + . Let us now consider b = 1, x⁰> 0, and the corresponding GSM iteration sequence

x^k+1 = –L^–1Ux^k + L^–11 = L^–1(–U)x^k + L^–11

From the product form of T = L^–1 and remembering that L_i,j< 0 and U_i,j< 0, it is easy to see that L^–11 > 0 and –L^–1U > 0. From the GSM iteration it follows that

"x⁰> 0 , x^k+1> L^–11 > 0.

Assuming GSM convergence and x⁰> 0,

so A obeys GLC.

From theorem 2 we see that GLC is the most general ''module based'' criterion of convergence of GSM. Some small variations, including a minor generalization of theorem 1 can be stated, for example:

Corollary 1. If there is a w > 0 such that S(A) > 0 with no more than one coordinate of S(A)w equal to zero, then the GSM converges for A.

Proof. If S(A)w > 0 we are in the situation of theorem 1. Otherwise let h be the unique null coordinate of S(A)w. Using S(A)_h,h > 0 and the continuity of S(A): ⁿ

ⁿ, there is a sufficiently small e > 0 such that S(A)w¢ > 0 for w¢ = [w₁, ¼ w_h+ e¼ w_n] > 0.

Corollary 2. If det(S(A)) ¹ 0 and if there is a w > 0 such that S(A)w > 0 then the GSM converges to A.

Proof. Since S(A) in non singular let y = S(A)^–11. By continuity of S(A): ⁿ

ⁿ, there is a sufficiently small e > 0 such that w¢ = w + ey > 0 and S(A)w¢ = S(A)w + e1 > 0.

From Theorem 2, any module based convergence criterion for GSM must also assert GLC. The simplest such module based criterion is LC, that is GLC for w = 1. Finally we consider the effect of permutation operations:

Lemma 3. (Permutations). Let p and q be permutations of [1, ¼n], with the corresponding row and column permutation matrices, P_i,j = I_p(i),j and Q_i,j = I_i,q(j), so that the permuted matrix P A Q has no null diagonal elements. GSM converges for A iff GSM converges for P A Q.

Proof. We only have to remember that any permutation matrix P is orthogonal, i.e. P P^t = I, so the 2-norm of any vector v is not changed by the rotation w = Pv, and neither is the spectral radius of the permuted matrix P A Q.

For symmetric permutations, P A P^t, we have a simple permutation of diagonal elements, i.e. diag(P A P^t) = P diag(A). Again, this special case is of particular interest in manufacturing systems as in [4] and [5].

Received: 8/XI/01.

#507/00.

• [1] D.P. Bertsekas and J.N. Tsitsiklis, Parallel and Distributed Computation, Numerical Methods. Prentice-Hall, (1989).
• [2] G.H. Golub, C.F. van Loan, Matrix Computations, The Johns Hopkins Univ. Press, (1989).
• [3] C. Humes Jr., L.O. Brandao and M.P. Garcia, A mixed dynamics approach for linear corridor policies (a revisitation of dynamic setup scheduling and flow control in manufacturing systems), Discrete Event Dynamical Systems, 5, 1, 59-82, January (1995).
• [4] C. Humes Jr. and T.I. Seidman, Some kanban-controlled manufacturing systems: a first stability analysis", IEEE Transactions on Automatic Control, 41, 7, 1013-1018, July (1996).
• [5] J.R. Perkins, C.Humes Jr. and P.R. Kumar, Distributed control of flexible manufacturing systems: stability and performance, IEEE Transactions on Robotics and Automation, 10, 2, 133-141, April (1994).
• [6] E.Polak and E. Polak, Optimization: Algorithms and Consistent Approximations, Springer (1997).
• [7] J. Renegar, A Polynomial-Time Algorithm based on Newton's Method for Linear Programming, Mathematical Programming, 40 (1988), pp. 59-93.
• [8] A. Sharifnia, M. Caramanis and S. Gershwin, Dynamical setup scheduling and Flow control in Manufacturing Systems, Discret Event Dynamical Systems: Theory and Applications, 1 (1991), pp. 149-175.

Publication Dates

• Publication in this collection
  19 July 2004
• Date of issue
  2003

History

• Received
  08 Nov 2001