Abstracts
This paper is concerned with an algorithm proposed by Alizadeh for linear semidefinite programming. The proof of convergence given by Alizadeh relies on a wrong inequality, we correct the proof. At each step, the algorithm uses a line search. To be efficient, such a line search needs the value of the derivative, we provide this value. Finally, a few numerical examples are treated.
semidefinite programming; interior point methods
Este artigo considera um algoritmo proposto por Alizadeh para programação semidefinida linear. A prova de convergência apresentada por Alizadeh baseia-se numa inequação errada, corrigimos a demonstração. Em cada passo, o algoritmo utiliza uma busca linear. Para ser eficiente, esta busca linear precisa do valor da derivada, apresentamos este valor. Finalmente, alguns exemplos numéricos são tratados.
programação semidefinida; métodos de pontos interiores
A numerical implementation of an interior point method for semidefinite programming
Djamel BenterkiI; Jean-Pierre CrouzeixII, * * Corresponding author / autor para quem as correspondências devem ser encaminhadas ; Bachir MerikhiIII
IDépartement de Mathématiques Faculté des Sciences Université Ferhat Abbas Sétif Algérie
IILIMOS Université Blaise Pascal Aubière Cedex France. crouzeix@math.univ-bpclermont.fr
IIIDépartement de Mathématiques Faculté des Sciences Université Ferhat Abbas Sétif Algérie
ABSTRACT
This paper is concerned with an algorithm proposed by Alizadeh for linear semidefinite programming. The proof of convergence given by Alizadeh relies on a wrong inequality, we correct the proof. At each step, the algorithm uses a line search. To be efficient, such a line search needs the value of the derivative, we provide this value. Finally, a few numerical examples are treated.
Keywords: semidefinite programming; interior point methods.
RESUMO
Este artigo considera um algoritmo proposto por Alizadeh para programação semidefinida linear. A prova de convergência apresentada por Alizadeh baseia-se numa inequação errada, corrigimos a demonstração. Em cada passo, o algoritmo utiliza uma busca linear. Para ser eficiente, esta busca linear precisa do valor da derivada, apresentamos este valor. Finalmente, alguns exemplos numéricos são tratados.
Palavras-chave: programação semidefinida, métodos de pontos interiores.
1. Introduction
The object of this paper is to present an implementation of an interior point method. Alizadeh (1995) has proposed an attractive extension of an interior point algorithm by Ye (1991) for linear programming to linear semidefinite programming. Actually, the proof of convergence given by Alizadeh relies on a wrong inequality, we provide a counterexample. We amend the proof to make it correct. At step k, the algorithm makes use of a line search, Goldstein-Price's methods are efficient to perform such a line search, but they need the knowledge of the derivative. We show how to obtain it, allowing an efficient implementation of the algorithm. Finally, we treat some numerical examples for solving the so-called linear semidefinite program:
where b Î Rm, C and Ai, belong to E, the linear space of n x n symmetric matrices. The inner product on E of two matrices A and B is the trace of their product, i.e., áA, Bñ = tr(AB) = SijAij Bij Finally, K denotes the cone of positive semidefinite matrices of E and the interior of K denoted by int(K) is the cone of positive definite matrices of E.
Max-cut problems in a graph, graph bisection problems, the problem of finding a largest clique in a graph, min-max eigenvalue problems can be formulated as linear semidefinite programs, see for instance Helmberg et al. (1996).
Linear semidefinite programming presents a great similitarity with linear programming. The objective function is linear, the constraint functions are affine, the difference consists of the positivity cone. Testing the positive semidefinitess of a n x n matrix X requires to check áXh, hñ > 0 for all h with norm 1, an infinity of linear constraints, while testing that a vector belongs to the nonnegative orthant in an euclidean space involves, by definition, a finite number of linear constraints. The duality scheme in linear semidefinite programming is about the same than the one in linear programming; there are also complementarity slackness conditions. It turns out that interior point methods used in linear programming can be easily extended to linear semidefinite programming.
Next, we precise the notation used in the paper. We have already defined the sets E, K,int(K) and the scalar product á , ñ in E. The identity matrix of E is denoted by I. Given A Î E its Frobenius norm is
where li = i = 1,2,...,n are the eigenvalues of A. Besides this norm, we will also use the spectral norm on E:||A||s = maxi(|li|). Given a rectangular m x n matrix B, vecB is the vector of Âmn defined by (vecB) i+(j-1)m= Bijfor i = 1,2...,m, j = 1,2,...,n. For instance the set of relations áAi , Xñ = bi for i = 1,..,m may be written as vec X = b where is a m x n2 matrix, that is row i of is vectAi. F Ä G is the Kronecker product of matrices: if F is a m X n matrix and G a p x q matrix then F Ä G is the m x n block matrix whose the (i,j) block is the p x q matrix FijG.
Let u, v Î Âr we denote by utv their inner product and by ||u||2, ||u||¥ the euclidean and the max-norm of u, respectively. Given u Î Âr,Diag(u) is the r x r diagonal matrix with the components of u as the diagonal entries. Conversely, given X an r x r matrix (not necessarily diagonal), diag(X) is the vector in  r obtained from the diagonal elements of X.
2. Duality in SDP
Let us consider the problems:
and
X is said to be a strictly feasible solution of (SDP) if X Î int(K)and áAi, X ñ = bifor i = 1...m. y Î Â m is said to be a strictly feasible solution of (DSDP) if (C - yiAi) Î int(K). A weak duality result says that md< mp. A strong duality result says that if both (SDP) and (DSDP) have strictly feasible solutions, then md = mp and both problems have optimal solutions. Furthermore, for such optimal solutions X and y, the following complementarity slackness condition holds
More information on semidefinite programming can be found in Alizadeh (1995), Helmberg et al. (1996) and Vanderbeghe & Boyd (1996).
3. A potential reduction method for (SDP)
The method proposed by Alizadeh (1995) is a direct extension of a method given by Ye (1991) for linear programming. It is based on the reduction of a potential function and a projective transform. Let us consider the (SDP) program and his dual (DSDP). Given z < mp and a fixed integer r > 0, the algorithm uses the two following functions:
and
It is clear that f is well defined for all X strictly feasible solution of (SDP), while y is well defined for all X positive definite and y strictly feasible solution of (DSDP).
The function f (the function y) is called the primal (the primal-dual) potential function. The convergence of the algorithm is obtained from the decrease of the sequence {y (Xk, yk)}
This method needs the knowledge of initial strictly feasible solutions X0 and y0.
3.1 The projective transform
At step k, the current Xk Î int(K). The Cholesky factorization of Xk gives a lower triangular matrix Lk with positive diagonal entries such that LkLtk = Xk. Then, we consider the projective transform
where
Here eris the vector of ones in Âr Notice that X = (, ). Via this projective transform, (SDP) is equivalent to the problem
where
Remark: The potential function associated to (TSDP) is
It is easily proved (see Alizadeh, 1995) that
4. The algorithm iteration
In a similar way than in the Karmarkar's method for classical linear programming, we relax the problem (TSDP) in the problem
It is easily seen that, for b Î ]0,1[, the constraint || - I||2 + || - er||< b2 implies that Î int(k)and > 0. Using a matricial formulation, we have
where
is a (m + 1)x (n2 + r) matrix.
It results, from a classical optimality condition, that the optimal solution (, ) of (R) is given by
where p(zk) is the projection of the vector on the affine subspace corresponding to the equality, i.e.,
where Pn(zk) is an n x n matrix and pris a vector in Âr.
Let
Then y(z) is a solution for the linear system
This system can be efficiently solved thanks to a Cholesky factorization.
Remark: b may stay fixed during the different iterations, but it is better to adjust b at each iteration in order to accelerate the speed of convergence. This can be done thanks to a Goldstein-Price line search. For that, it is necessary to know the derivative of the potential function at 0. This is the object of the following lemma.
Lemma 1. Let P Î E and
Then
Proof. We have
Set
H(b) = det(X - b LPLt) = det(LLt - b LPLt) = det(X) x det(I - bP).
P being symmetric, there exist an orthogonal matrix U and a diagonal matrix D such that P = Ut DU. Hence,
Then
Finally,
n
4.1 Convergence of the algorithm
The proof of convergence given by Alizadeh for his algorithm is based on the following inequality.
Actually, this inequality is false as shown by the following counter-example:
In fact, we have:
Lemma 2. Let Î K and Î Âr.Then > 0
Proof. Reporting to Alizadeh (1995), the following inequality holds
But
and
Hence
Finally (1) gives
If we replace in the proof of Alizadeh d(b) = ab by d(b) = ab , then his proof becomes valid. In particular, we have
and
where
Furthermore, Alizadeh proves the polynomial convergence:
after a number of iteration of order 0 ( |log e|) for e > 0.
n
5. Implementation of the Alizadeh algorithm
In the algorithm, a Î [0,1] and e > 0 are fixed.
The Alizadeh algorithm
a) Initialization: Start with X0,y0, a Î [0,1], z0 = bty0 and k=0,
b) If (áC, Xkñ - btyk) < e STOP: (Xk, yk) is an e approximate solution
If not:
Compute p (zk).
If ||p(zk)||2> a then
* Find b*= Argmin 0 <b<1 (Y (Xk - bLkPn(zk) Ltk,y (zk))).
* Take = - b*
* Take Xk+1 = (, )and go to c).
If not:
* Find z* = Argminz<zk (Y(Xk,y(z))).
* Take yk+1 = y(z*), zk+1 = b ty(z*)and go to c).
c) Do k = k + 1 and go to back b).
6. Numerical examples
In the treated examples, the initial primal-dual feasible solutions have been given .
6.1 Example 1
where:
and b = (0,1)t.
After 4 iterations we obtain the optimal dual solution y* = (1,0)tand the optimal primal solution
Example 2
In this example n=2m.
where C = - I. For k = 1,...,m, bk = 2 and the n x n matrices Ak are defined by
with k+m+1 < n.
The following array gives the number of iterations needed to obtain the primal and the dual solutions for different sizes.
7. Conclusion
The present paper corrects a small point in the proof of the convergence given by Alizadeh (1995) and gives the value of the derivative of the potential function at 0 needed to perform an efficient line search.
Recebido em 02/2002, aceito em 11/2002 após 1 revisão
- (1) Alizadeh, F. (1995). Interior point methods in semidefinite programming with applications to combinatorial optimization. SIAM Journal on Optimization, 5, 13-51.
- (2) Helmberg, C.; Rendl, F.; Vanderbei, R.J. & Wolkowicz, H. (1996). An interior point method for semidefinite programming. SIAM Journal on Optimization, 6, 342-361.
- (3) Jansen, B.; Roos, C. & Terlaky, T. (1997). A family of polynomial affine scaling algorithms for semidefinite linear complementarity problems. SIAM Journal on Optimization, 7, 126-140.
- (4) Keraghel, A. (1989). Etude adaptative et comparative des principales variantes dans l'algorithme de Karmarkar. Dissertation thesis, Universitè Joseph Fourier, Grenoble, France.
- (5) Kojima, M.; Shindoh, S. & Hara, S. (1997). Interior point methods for the monotone semidefinite linear complementarity problem in symmetric matrices. SIAM Journal on Optimization, 7, 86-125.
- (6) Marcus, M. & Ming, H. (1964). A survey of matrix theory and matrixinequalities University of California, Santa Barbara, Allyn and Bacon, ING, Boston.
- (7) Monteiro, R.D.C. (1997). Primal-dual path following algorithms for semidefinite programming. SIAM Journal on Optimization, 7, 663-678.
- (8) Overton, M. & Wolkowicz, H. (1997). Semidefinite programming. Mathematical Programming, Series B, 77, 105-109.
- (9) Rama, M.V.; Tuncel, L. & Wolkowicz, H. (1997). Strong duality for semidefinite programming. SIAM Journal on Optimization, 7, 641-662.
- (10) Vanderbeghe, L. & Boyd, S. (1996). Positive definite programming. SIAM Review, 38, 49-95.
- (11) Wright, S.J. (1997). Primal-dual interior point methods SIAM, Philadelphia, PA.
- (12) Ye, Y. (1991). An O(n3L)- potential reduction algorithm for linear programming. Mathematical Programming, 50, 239-258.
Publication Dates
-
Publication in this collection
26 May 2003 -
Date of issue
Jan 2003
History
-
Accepted
Nov 2002 -
Received
Feb 2002