A COMPUTATIONAL STUDY ON THE GLOBAL SOLUTION OF NONCONVEX QUADRATIC CONSTRAINTS AND QUADRATIC PROGRAMMING PROBLEMS

ABSTRACT

Quadratically constrained quadratic programming (QCQP) problems appear in a wide range of engineering fields, including computer science, communication engineering, and finance. A key difficulty in solving these problems lies in efficiently finding global solutions, especially for large-scale and nonconvex instances. To address this, Wen & Yin (2013) proposed a method that reformulates semidefinite programming (SDP) relaxations of QCQPs into a nonlinear and nonconvex low-rank problem. This reformulated problem can be efficiently solved using a curvilinear search method combined with Barzilai-Borwein (BB) steps, known as the CSBB algorithm. In this study, we compare two approaches for solving QCQPs: the conventional convex SDP relaxation and the nonconvex low-rank reformulation introduced by Wen and Yin. We propose a set of general QCQP models that are compatible with the low-rank framework and conduct a series of numerical experiments to evaluate their performance. This study evaluates performance using several classes of NP-hard problems, including Binary Integer Quadratic (BIQ) problems, Max-Cut problems, Boolean Least Squares (BLS) problems, and 0-1 Quadratic Knapsack Problems (QKP). The results demonstrate that the low-rank approach offers competitive performance and shows strong potential for solving large-scale QCQPs more efficiently than traditional convex methods.

Keywords:
0-1 QCQP; semidefinite programming; low-rank factorization; nonconvex QCQP

1 INTRODUCTION

In a convex optimization problem, both the objective and the constraints are convex functions, whereas in a nonconvex optimization problem, either the objective or at least one of the constraints is nonconvex. The convex optimization problem guarantee a globally optimal solution provided it has a feasible solution exists. In contrast, nonconvex optimization problems do not offer such guarantees. Furthermore, a feasible convex optimization problem typically has a unique globally optimal solution, while a feasible nonconvex problem may have multiple locally optimal solutions. So, if the global solution exists for a nonconvex optimization problem, its identification is a tedious task.

Extensive research has focused on the convexification of nonconvex problems. However, obtaining a globally optimal solution for a nonconvex problem without convexification remains a significant challenge. Convexifying large-scale nonconvex problems can introduce various drawbacks. One notable issue is the increased computational time required by even high-performance computers to approach a global solution. This is often due to the looseness of the convex envelope used to approximate the original nonconvex problem.

The prime challenge in solving many engineering real-world and NP-hard problems is how fast and accurately we reach the global solution. Even a faster algorithm is sometimes not the best choice. On the other hand, one must seek an alternative, like obtaining a global solution by solving the nonconvex one, if at all possible.

In the recent past, semidefinite relaxation (SDR) has attracted many researchers as it is a powerful, computationally efficient approximation technique and moreover it is a convex reformulation for many real world optimization problems that evolve from many engineering applications. As these problems are generally NP-hard, they require relaxation, which results in suboptimal solutions. Since the class of nonconvex quadratic programming problems, especially quadratically constrained quadratic programs (QCQPs) captures many problems from Boolean least square problems (BLS), Max-cut problems, image processing problems, signal processing problems, communication problems, etc., we are attracted to solve the nonconvex QCQPs in this research.

The worst-case complexity of solving a generic SDP problem is about 𝒪(n ^6.5), using the interior point method (IPM) with a matrix variable of size n×n and 𝒪(n) linear constraints. Hence, this method is not suitable for large scale problems. An SDP cut formulation for BQPs proposed by Wang et al. (2013) presented similar relaxation bounds to the conventional SDP formulations, which have the same degree of complexity as spectral methods. The method has a complexity of 𝒪(kn ³), where k denotes the number of gradient-descent steps in L-BFGS-B. But the method is impractical for large scale optimization problems, as it calculates the gradient of the dual objective function at each gradient-descent step. Note that both the methods discussed are SDP formulations, which means convex reformulation of the given optimization problem. Also, in recent years, many researchers have tried to find a better relaxed solution by adding valid linear inequality constraints, which strengthened the convex relaxation.

Sherali (2007) and Sherali & Adams (2013) first introduced the concept of reformulation linearization technique (RLT) to formulate linear programming relaxation for nonconvex problems. The RLT linearizes the product of any pairs of linear constraints, and a tight relaxation can be obtained via enhanced SDP relaxation with the RLT constraint. Anstreicher (2009) proposed a theoretical analysis of the SDP+RLT relaxation for QCQP with box constraints, showing that RLT constraints remove a large portion of the feasible region, and suggested that a combination of SDP and RLT constraints leads to a tighter bound.

The D.C. decomposition scheme by Zheng et al. (2011), and the αBB underestimators scheme by Anstreicher (2012) are some of the alternative methodologies for convexifying the quadratic form over the feasible region.

In their seminal survey, Burer & Saxena (2012) discussed a variety of methods for generating valid inequalities aimed at tightening the semidefinite programming (SDP) relaxations of quadratically constrained quadratic programs (QCQPs). Alongside these approaches, several studies have focused on developing approximation algorithms for QCQPs, particularly those involving ellipsoidal constraints. For instance, Ye (1999) extended the randomized rounding technique introduced by Goemans & Williamson (1995) to construct feasible solutions from SDP relaxations. Fu et al. (1998) presented approximation algorithms designed to produce feasible solutions that adhere to provable quality guarantees. Additionally, Tseng (2003) conducted a comprehensive analysis of the approximation bounds associated with SDP relaxations for QCQPs with more general quadratic constraints.

Jiang & Li (2019) reviewed SDP based convex relaxation for QCQP problems. Burer & Yang (2015) demonstrated that the SDP+RLT+SOC (Second-Order Cone) relaxation has no gap in an extended trust region problem of minimizing a quadratic function subject to a unit ball and multiple linear constraints, where the linear constraints do not intersect with each other in the interior of the ball. Yamada & Takeda (2018) proposed a new convex relaxation method that is computationally faster but weaker than the SDP relaxation method. Their method reformulates the QCQP as a Lagrangian dual optimization problem and successively solve subproblems while updating the Lagrangian multipliers.

Departing from the above traditional convexification techniques, Low-rank decomposition methods have been widely investigated for their applications in optimization, machine learning, and signal processing. Kaushal et al. (2023) introduced LORD, a low-rank decomposition technique aimed at compressing monolingual code language models, demonstrating its effectiveness in reducing model size while maintaining performance. Bertsimas et al. (2023) investigated sparse plus low-rank matrix decomposition through a discrete optimization approach, emphasizing its utility in high-dimensional data analysis. Wang et al. (2022) applied low-rank decomposition to time-frequency representation for diagnosing bearing faults under variable speed conditions, showcasing its robustness in industrial applications. Hu & Ye (2023) explored the linear convergence properties of an alternating polar decomposition method for low-rank orthogonal tensor approximations, providing theoretical insights into its efficiency. In the field of deep learning, Chen et al. (2021) proposed DRONE, a data-aware low-rank compression strategy designed to optimize large NLP models by balancing storage efficiency and accuracy. Furthermore Liu et al. (2022) developed a randomized quaternion singular value decomposition method to enhance low-rank matrix approximation, contributing to advancements in numerical linear algebra. These studies collectively highlight the significance of low-rank decomposition in improving computational efficiency, reducing data redundancy, and enhancing model interpretability.

The low-rank matrix decomposition method proposed by Burer & Monteiro (2003) involves factorizing the optimization variable in the SDP formulation, X, into RR ^⊤. The rank of the factorization, determined by the number of columns in matrix R, is carefully chosen to enhance computational speed while maintaining equivalence with the optimal solution of the SDP.

Furthermore, Wen & Yin (2013) introduced the curvilinear search with BB-step (CSBB) algorithm, which effectively solves nonconvex nonlinear optimization problems and ensures a globally optimal solution for semidefinite programming (SDP).

1.1 Notation

In this paper, X⪰0 denotes that the symmetric matrix X is positive semidefinite, and X⪰Y means that X-Y is positive semidefinite. For two n×n matrices X and Y, the matrix inner product is written as X·Y=Trace(X ^⊤ Y). For an n×n matrix X, diag(X) refers to the vector x whose elements are the diagonal entries of X, i.e., x _i =X _ii for i=1, ..., n. The notation x≤y is used to indicate element-wise inequality between two vectors x and y. Additionally, rank(·) represents the rank of a matrix, and e is the vector of ones.

1.2 Global optimality

In this subsection, the global optimality results for the p largest eigenvalue problem and problems corresponding to SDPs with constraints on the diagonal entries only are presented. We consider the following semidefinite programming (SDP) problem:

$\begin{array}{c}\underset{X⪰0}{min}C·X\\ s.t.X_{ii}=1,i=1,...,n,\end{array}$ (1)

where C is a given n×n real symmetric matrix, and X is an n×n symmetric matrix that is required to be positive semidefinite. The primary challenge in solving this problem arises from the positive semidefiniteness constraint X⪰0, as the objective function is linear in X, which makes the semidefinite constraint the most difficult aspect to handle.

If the solution $\hat{X}$ has rank p, then following the decomposition method by Burer & Monteiro (2003), X can be decomposed into V ^⊤ V with V=[V ₁,V ₂, ...,V _n ]∈ℝ^p×n , we obtain an equivalent problem as

$\begin{array}{c}minC·V^{\top }V\\ s.t.||V_i||=1,i=1,...,n\end{array}$ (2)

The significant advantages and disadvantages of problem (2) compared to (1) are as follows:

• Problem (2) involves fewer variables than problem (1).

• The objective function in (2) is no longer linear; it is quadratic and, in general, nonconvex.

Although problem (2) is nonconvex, it has been shown that its local minimizer can also be a global minimizer (Wen & Yin, 2013).

Motivated by the low-rank factorization in Burer & Monteiro (2003) and the success of the CSBB algorithm (Wen & Yin, 2013), we present several QCQP formulations that fit into the framework of (2) and can be efficiently solved using the CSBB algorithm. Specifically, we consider general binary QCQPs and convert them into the SDP formulation given in (1). By utilizing the decomposition approach in (2), we model the problem as a low-rank nonconvex model and apply the CSBB algorithm to solve it. The main objective of this research is to provide a comparative study between the solutions of convex and nonconvex formulations of QCQPs.

The outline of this paper is as follows: In Section 2, we introduce the QCQPs and discuss the SDP relaxation. Section 3, presents the low-rank factorized formulations of the QCQPs described in Section 2 along with a discussion on the existence of a global optimal solution. In Section 4, we provide numerical experiments on several problems and validate our claims by comparing the results obtained through the SDP method. Finally, Section 5 concludes the paper with remarks.

2 THE PROBLEM

We consider the binary QCQP as follows:

$\begin{array}{c}QCQP:min{f}_0\left(y\right)=y^{\top }{P}_{0}y+q_0^{\top }y\\ s.t.f_i\left(y\right)=y^{\top }{P}_{i}y+q_i^{\top }y+r_i\le 0,i=1,2,...,m\\ yisbinary\end{array}$ (3)

where y∈ℝⁿ is the optimization binary vector, P _i ∈ℝ^n×n , q _i ∈ℝⁿ are given problem data, for i=0, 1, ..., m, r _i ∈ℝ for i=1, ..., m. Here the inequality “≤” in f _i (y) is element-wise. Note that y is binary means y∈{0, 1}ⁿ or y∈{-1, 1}ⁿ . Since the binary variable domain {0, 1}ⁿ can be changed to the domain {-1, 1}ⁿ by a suitable substitution y=2x-e, then in the rest of the article we consider the binary domain as {-1, 1}ⁿ . We assume that all the matrices P _i ∈S ⁿ and

$\mathcal{F}=\left\{y\in {\left\{-1,1\right\}}^n:f_i\left(y\right)\le 0,i=1,2,...,m\right\}$

denotes the feasible set of problem QCQP (3). If P _i ⪰0 for each i∈{0}∪{1, 2, ..., m}, then QCQP (3) is a convex programming problem and can be solved in polynomial time. But, in general the problem is NP-hard. If $P_i\underline{\nsucc }0$ for some i∈{0}∪{1, 2, ..., m}, then in order to obtain a lower bound for the nonconvex QCQP (3), we have to convexify it. Since the current research is focused on solving the nonconvex QCQPs (3), we are least focused on the several approaches available for convexification with some pros and cons.

Letting $C_i=\left(\begin{array}{cc}{P}_i& \frac{1}{2}{q}_i\\ \frac{1}{2}{q}_i^{\top }& 0\end{array}\right)$, QCQP (3) can be presented as

$\begin{array}{c}QCQP:min{f}_0\left(x\right)=x^{\top }{C}_{0}x\\ s.t.f_i\left(x\right)=x^{\top }{C}_{i}x+r_i\le 0,i=1,2,...,m\\ x\in {\left\{-1,1\right\}}^{n+1}.\end{array}$ (4)

2.1 The Semidefinite relaxation

Although SDR (in both primal and dual forms) makes general QCQPs convex, we investigate its failure in convexifying the binary QCQP stated in (4). Before applying SDR, we first reformulate QCQP (4) into an equivalent problem with a positive semidefinite variable X.

Proposition 1. The optimization problem QCQP (4) is equivalent to the following QCQP (5) , where X=xx ^⊤ , and x and X represent the optimal solutions to the problems QCQP (4) and QCQP (5) , respectively.

$\begin{array}{c}QCQ{P}_{SDP}:min{f}_0\left(x\right)=C_0·X\\ s.t.f_i\left(x\right)=C_i·X+r_i\le 0,i=1,2,...,m\\ Diag\left(X\right)=e\\ Rank\left(X\right)=1\\ X⪰0\end{array}$ (5)

Proof. Letting X=xx ^⊤, we lift the QCQP (4) into the space of rank one matrices. Thus, the vector x∈{-1, 1}ⁿ⁺¹ leads to X _ii =1 and Rank(X)=1. Finally, for i=0, 1, ..., m, x ^⊤ C _i x=Tr(C _i X)=C _i ·X. □

In QCQP_SDP(5), the constraint Rank(X)=1 is non-convex and poses a significant challenge. This constraint can either be omitted though this may compromise optimality or relaxed. Since the optimal solution of QCQP_SDP(5) coincides with that of QCQP (4) when Rank(X)=1, we choose to relax the rank-one constraint rather than discard it. To this end, we introduce a convex inequality constraint, ||X||_F ≤n+1, as suggested in Nayak & Mohanty (2019), where ||·||_F denotes the Frobenius norm.

Based on this relaxation, we propose the following versions of the relaxed formulations of QCQP_SDP(5).

Proposition 2.

$\begin{array}{c}SD{P}_{v_1}:\underset{X⪰0}{min}{f}_0\left(x\right)=C_0·X+\mu \left({||X||}_F^2-{\left(n+1\right)}^2\right)\\ s.t.f_i\left(x\right)=C_i·X+r_i\le 0,i=1,2,..,m\\ Diag\left(X\right)=e.\end{array}$ (6)

Since ||X||_F ≤n+1, the objective value in QCQP (6) is not larger than that of QCQP (5). When µ→0, then QCQP (6) is equivalent to QCQP (5) and for a small µ, the problem QCQP (6) approximates QCQP (5).

The second version of the relaxed problem formulation is

Proposition 3.

$\begin{array}{c}SD{P}_{v_2}:\underset{X⪰0}{min}{f}_0\left(x\right)=C_0·X+\mu \left({||X||}_F^2-{\left(n+1\right)}^2\right)+\sum _{i=1}^m{\lambda }_i\left[{\left(C_i·X\right)}^2-r_i^2\right]\\ s.t.Diag\left(X\right)=e.\end{array}$ (7)

Since ||X||_F ≤n+1 and C _i ·X+r _i ≤0, for i=1, 2, ..., m, the objective value in QCQP (7) is not larger than that of QCQP (5). Also, when µ→0 and λ→0, m constraints are satisfied as second and third penalty terms are zero and QCQP (7) is equivalent to QCQP (5).

The third version of the relaxed problem formulation is

Proposition 4.

$\begin{array}{c}SD{P}_{v_3}:\underset{X⪰0}{min}{f}_0\left(x\right)=C_0·X+\mu \left({||X||}_F^2-{\left(n+1\right)}^2\right)+\sum _{i=1}^m{\lambda }_i{\left(max\left[0,\left(C_i·X+r_i\right)\right]\right)}^2\\ s.t.Diag\left(X\right)=e.\end{array}$ (8)

Here all the m constraints are satisfied as if f _i (x)≤0, then the second penalty term leads to zero. It is also clear that the second penalty term is greater than zero when all the m constraints are positive, which leads to a contradiction.

2.2. SDP relaxation with cutting planes

In this subsection, we describe SDP relaxations based on the cutting plane method such as the SDP+RLT cut for given QCQP problems. As we discussed above, any binary QCQP (3) can be presented as a particular form of QCQP (4).

Relaxation based on SDP and RLT both utilised variable X _ij that replaced the product terms x _i x _j of the original problem. The SDP relaxation is based on the fact that since X=xx ^⊤, in the actual solution of QCQP (4), one can obtain a relaxation of QCQP by imposing a convex constraint X-xx ^⊤⪰0 instead of X=xx ^⊤. Also, using the Schur complement, X-xx ^⊤⪰0 can be replaced by

$\left(\begin{array}{cc}1& x^{\top }\\ x& X\end{array}\right)⪰0.$

Thus, the relaxed QCQP is:

$\begin{array}{c}QCQ{P}_{SDP}:min{C}_0·X\\ s.t.C_i·X+r_i\le 0,i=1,2,...,m\\ diag\left(X\right)=e\\ \left(\begin{array}{cc}1& x^{\top }\\ x& X\end{array}\right)⪰0.\end{array}$ (9)

Since the vector x∈{-1, 1}ⁿ⁺¹ is equivalent to $x_i^2=1$, we replace $x_i^2-1=0$ by the constraint diag(X)=e. When the QCQP (3) is a convex problem then QCQP_SDP(9) is equivalent to QCQP (3). However, When QCQP (3) is nonconvex, the SDP may be unbounded, even though all of the original variables have finite upper and lower bounds. The remedy suggested by Anstreicher (2009) adding the upper bounds to the diagonal components of X is also incorporated in our formulation. Specifically, we assume that $X_{ii}=x_i^2\le 1$ in our QCQP formulation.

2.3 SDP with cutting planes

The SDP relaxation can further be strengthened by requiring X to satisfy additional inequalities, known as the Reformulation Linearization Technique (RLT).

2.3.1 RLT cut

The RLT relaxation of QCQP (4) is based on LP relaxation proposed by Sherali (2007); Sherali & Adams (2013). For two variables x _i and x _j we replace the product term x _i x _j with the new variable X _ij . Since x∈{-1, 1}ⁿ⁺¹ , first we relax it with a box constraint x∈[-1, 1]ⁿ⁺¹ . As RLT relaxation utilizes products of upper bound and lower bound constraints on the original variables to generate valid linear inequality constraints on the new variable X _ij , the resulting set of RLT constraints are obtained after multiplying and replacing with the new variable:

$\begin{array}{c}{X}_{ij}\ge -x_i-x_j-1\\ X_{ij}\ge x_i+x_j-1\\ X_{ij}\le x_i-x_j+1\\ X_{ij}\le -x_i+x_j+1.\end{array}$

By adding the RLT constraints to QCQP (4), the resulting relaxed problem, denoted as QCQP_RLT , becomes a standard linear programming (LP) problem with $\frac{n\left(n+3\right)}{2}$ variables and a total of m+n(2n+3) constraints. Although this formulation is computationally efficient and applicable to large-scale LP problems, it has two primary drawbacks: increased dimensionality and relatively weak lower bounds.

To overcome these bottlenecks, many researchers (Sherali & Fraticelli, 2002; Anstreicher, 2009, 2012; Sherali & Adams, 2013) have proposed the combined SDP+RLT relaxation method, which has been proven to be much more effective than either the SDP or RLT relaxations alone. Therefore, we present an SDP+RLT relaxation by adding the RLT condition to the semidefinite relaxation of QCQP. Let

$X_L=\left(\begin{array}{cccc}1& X_{12}& \cdots & X_{1n}\\ X_{12}& 1& \cdots & X_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ X_{1n}& X_{2n}& \cdots & 1\end{array}\right).$

The matrix X _L is obtained by replacing the quadratic term x _i x _j by a linear term X _ij and implementing diag(X)=e in the matrix X. The resulting SDP+RLT relaxation can be written as follows:

$\begin{array}{c}QCQ{P}_{SDP+RLT}:min{C}_0·X_L\\ s.t.C_i·X_L+r_i\le 0,i=1,2,...,m\\ X_{ij}\ge -x_i-x_j-1\\ X_{ij}\ge x_i+x_j-1\\ X_{ij}\le -x_i+x_j+1\\ X_{ij}\le -x_i+x_j+1\\ -1\le x\le 1\\ \left(\begin{array}{cc}1& x^{\top }\\ x& X_L\end{array}\right)⪰0.\end{array}$ (10)

Although the SDP+RLT relaxation produces a tighter lower bound, it has the drawback of requiring a significant amount of CPU time to reach a near-optimal solution, due to the increase in the number of constraints and variables. As a result, it is not suitable for large-scale problems.

3 LOW RANK DECOMPOSITION OF THE PROPOSED FORMULATIONS

The low rank decomposition method is a restriction of the semidefinite programming problem (SDP) in which a bound r is imposed on the rank of X, and it is well known that low rank semidefinite programming problem (LRSDP_r ) (Burer & Monteiro, 2003) is equivalent to SDP if r is not too small. The local minima and optimal convergence of LRSDP_r proved via the algorithm’s distinguishing feature is a nonconvex change of variables that replaces the symmetric, positive semidefinite variable X of the SDP with a rectangular variable V according to the factorization X=VV ^⊤. The rank of the factorization, i.e., the number of columns of V, is chosen minimally so as to enhance computational speed while maintaining equivalence with the SDP. In this process the original problem can be transformed into one over V subject to spherical constraints ||V _i ||₂=1, i=1, ..., n. The problem with spherical constraints is that they are not only nonconvex but numerically expensive to preserve during iterations. Wen & Yin (2013) present a curvilinear search with BB step (CSBB) algorithm that solves the nonconvex nonlinear optimization problem and provides a global optimal solution to a SDP. We implemented CSBB algorithm to our proposed model in (6), (7), and (8) to get a global optimal solution. For this, we express these models into the form of models in (2).

The equivalent low rank formulation of (6) is

Proposition 5.

$\begin{array}{c}\underset{V}{min}{C}_0·\left(V^{\top }V\right)+\mu \left({||V^{\top }V||}_F^2-{\left(n+1\right)}^2\right)\\ s.t.C_i·\left(V^{\top }V\right)+r_i\le 0,i=1,2,...,m\\ ||V_i||=1,2,...,n.\end{array}$ (11)

Similarly, the equivalent low rank formulation of (7) is

Proposition 6.

$\begin{array}{c}\underset{V}{min}{C}_0·\left(V^{\top }V\right)+\mu \left({||V^{\top }V||}_F^2-{\left(n+1\right)}^2\right)+\sum _{i=1}^m{\lambda }_i\left[{\left(C_i·\left(V^{\top }V\right)\right)}^2-r_i^2\right]\\ s.t.||V_i||=1,i=1,2,...,n.\end{array}$ (12)

The equivalent low rank formulation of (8) is

Proposition 7.

$\begin{array}{c}\underset{V}{min}{C}_0·\left(V^{\top }V\right)+\mu \left({||V^{\top }V||}_F^2-{\left(n+1\right)}^2\right)+\sum _{i=1}^m{\lambda }_{i}max{\left[0,\left(C_i·\left(V^{\top }V\right)+r_i\right)\right]}^2\\ s.t.||V_i||=1,i=1,2,...,n\end{array}$ (13)

We now establish the result stated in Theorem 1 for the formulations given in (11), (12), and (13). To do so, we first require the following well-known result from nonlinear optimization theory:

Lemma 1. Let x* is a solution to a general constrained optimization problem, and the linear independence constraint qualification (LICQ) is satisfied. If λ* is a Lagrange multiplier vector such that the Karush-Kuhn-Tucker (KKT) conditions hold, then the following second-order necessary condition is satisfied:

$w^{\top }{\nabla }_{xx}^2ℒ\left(x*,\lambda *\right)w\ge 0,forallw\in \mathcal{C}\left(x*,\lambda *\right),$

where ${\nabla }_{xx}^2ℒ\left(x*,\lambda *\right)$ is the Hessian of the Lagrangian with respect to x, and ℭ (x*, λ*) denotes the critical cone at x* associated with λ*.

Proof. Refer Theorem 12.5 of Nocedal & Wright (2006). □

Theorem 1. There exists $\overline{p}\le n$ such that, if $p\ge \overline{p}$ , any local minimizer $\overline{V}$ of (11) (or (12) , or (13) ) is globally optimal and ${\overline{V}}^{\top }\overline{V}$ solves (11) (or (12) , or (13) ). In particular, $\overline{p}$ can be taken as (n+1)-inf{rank(C+D)):D is diagonal} or n, whichever is smaller.

Proof. As problem (11) (or (12), or (13)) satisfies the LICQ condition. Therefore, by Lemma 1, the problem also satisfies the first- and second-order necessary optimality conditions.

Therefore, the local minimizer $\overline{V}$ satisfies both the first- and second-order conditions. As a result, $\overline{V}$ is a global optimum of problem (11) (or (12), or (13)), even when $rank\left(\overline{V}\right)\le n$ (see Theorem 3 in Wen & Yin (2013)). □

4 NUMERICAL EXPERIMENT

We demonstrate the efficiency and effectiveness of solving the aforementioned problems using the SDP algorithm by comparing it with the nonconvex low-rank relaxation of the QCQPs. The comparison is based on the quality of lower bounds and CPU time. Since the SDP+RLT algorithm requires significantly more computation time than the SDP algorithm alone, and the focus of this research is to compare convex and nonconvex formulations, we concentrate our experiments on the SDP (Dual) formulation. The results are then compared with those obtained using the low-rank formulation.

The test problems include binary integer quadratic (BIQ) problems, max-cut problems, Boolean least squares (BLS) problems, and 0-1 quadratic knapsack problems (QKPs). All experiments were conducted using Matlab on an HP laptop with an 11th Gen Intel i5 processor (3 GHz) and 8 GB of memory.

4.1 Binary integer quadratic problems

The binary integer quadratic problem (BIQ) is described as

$\begin{array}{c}BIQ:min{y}^{\top }Ay\\ subjecttoy\in {\left\{0,1\right\}}^n,\end{array}$ (14)

where A is a n×n symmetric real matrix. We can replace y∈{0, 1}ⁿ as $y_i^2=y_i$, to get the quadratic relaxation.

Since we are interested in keeping our decision variable y in {-1, 1}, we set x=2y-e with e=(1, 1, ..., 1)^⊤. Thus, BIQ can be formulated as:

$\begin{array}{c}\underset{\left\{-1,1\right\}}{BIQ-QP}:min{x}^{\top }Ax+2{\left(Ae\right)}^{\top }x+e^{\top }Ae\\ subjecttox\in {\left\{-1,1\right\}}^n.\end{array}$ (15)

The SDP relaxation is

$\begin{array}{c}BIQ-SDP:minB·Y\\ subjectto{Y}_{ii}=1,i=1,2,...,n+1,\\ Y⪰0\end{array}$ (16)

where, $B=\left(\begin{array}{cc}\frac{1}{4}A& \frac{1}{4}Ae\\ \frac{1}{4}{\left(Ae\right)}^{\top }& \frac{1}{4}{e}^{\top }Ae\end{array}\right),Y=\left(\begin{array}{cc}X& x\\ x^{\top }& 1\end{array}\right)$.

Thus, the low rank decomposed formulation is

$\begin{array}{c}LR-BIQ:minB·V^{\top }V\\ subjectto||V_i||=1,i=1,2,...,n+1.\end{array}$ (17)

Table 1 presents numerical results for solving (16) and (17) and direct DNN (Kim et al., 2016). The 40 test problems are taken from the BIQMAC library (Wiegele, 2007). We present the optimal value and CPU time of each method. We also experimented with SDP+RLT (Sherali & Fraticelli, 2002) on instances with dimensions 50 and 100. We observed that SDP+RLT, although producing a tight lower bound, takes much longer time in comparison with our proposed LR-BIQ.

Table 1
Optimal value and CPU time (in seconds) of SDP+RLT, Direct DNN, SDP relaxations, and LR-BIQ.

BQP	SDP+RLT		DNN		SDP{-1,1}		LR-BIQ
	Optimal	Time	Optimal	Time	Optimal	Time	Optimal	Time
bqp50-1	-2098.00	11.00	-2345.47	1.45	-2345.47	0.31	-2345.47	0.03
bqp50-2	-3702.00	9.89	-3796.34	0.63	-3796.34	0.33	-3796.34	0.01
bqp50-3	-4626.00	9.93	-4659.10	0.65	-4659.10	0.20	-4659.10	0.01
bqp50-4	-3544.00	10.57	-3645.14	0.57	-3645.14	0.24	-3645.14	0.01
bqp50-5	-4012.00	11.51	-4151.82	0.54	-4151.82	0.22	-4151.82	0.01
bqp50-6	-3693.00	10.94	-3733.79	0.51	-3733.79	0.20	-3733.79	0.01
bqp50-7	-4520.00	10.69	-4686.00	0.44	-4686.00	0.21	-4686.00	0.01
bqp50-8	-4216.00	11.43	-4373.94	0.44	-4373.94	0.21	-4373.94	0.01
bqp50-9	-3780.00	10.81	-4054.33	0.47	-4054.33	0.20	-4054.33	0.01
bqp50-10	-3507.00	11.23	-3725.81	0.45	-3725.81	0.24	-3725.80	0.01
bqp100-1	-8036.66	104.36	-8721.10	1.26	-8721.11	0.33	-8721.10	0.01
bqp100-2	-11036.00	109.08	-11704.18	1.07	-11704.18	0.29	-11704.18	0.01
bqp100-3	-12723.00	109.69	-13336.70	1.28	-13336.70	0.26	-13336.69	0.01
bqp100-4	-10368.00	111.32	-10927.92	1.21	-10927.93	0.27	-10927.93	0.01
bqp100-5	-9083.00	110.07	-9736.93	1.25	-9736.93	0.25	-9736.93	0.01
bqp100-6	-10341.53	109.08	-11073.07	1.13	-11073.07	0.30	-11073.07	0.01
bqp100-7	-10159.42	107.89	-10906.86	1.20	-10906.86	0.27	-10906.85	0.01
bqp100-8	-11435.00	108.90	-12078.47	1.27	-12078.48	0.29	-12078.48	0.01
bqp100-9	-11455.00	106.99	-11926.96	1.10	-11926.97	0.28	-11926.97	0.01
bqp100-10	-12565.00	105.87	-13151.24	1.27	-13151.28	0.28	-13151.28	0.01
bqp250-1	x	x	-48726.91	7.62	-48732.37	0.79	-48732.29	0.02
bqp250-2	x	x	-48091.31	6.34	-48093.50	0.81	-48093.50	0.02
bqp250-3	x	x	-51745.19	8.47	-51745.40	0.81	-51745.37	0.02
bqp250-4	x	x	-44388.52	7.07	-44391.58	0.80	-44391.52	0.01
bqp250-5	x	x	-50796.99	6.04	-50803.63	0.81	-50803.56	0.02
bqp250-6	x	x	-44545.47	6.88	-44547.52	0.87	-44547.48	0.02
bqp250-7	x	x	-49708.60	5.78	-49709.76	0.92	-49709.70	0.02
bqp250-8	x	x	-40005.39	8.79	-40005.60	0.77	-40005.60	0.02
bqp250-9	x	x	-52328.89	6.33	-52330.23	0.76	-52330.18	0.02
bqp250-10	x	x	-44025.80	9.30	-44026.14	0.86	-44026.14	0.01
bqp500-1	x	x	-553939.12	45.23	-128402.72	4.42	-128402.31	0.04
bqp500-2	x	x	65535.00	41.98	-138237.20	4.51	-138236.30	0.04
bqp500-3	x	x	-570914.69	24.17	-140738.05	4.83	-140737.57	0.04
bqp500-4	x	x	-572248.97	56.88	-141602.11	4.58	-141601.50	0.04
bqp500-5	x	x	-565091.50	24.29	-136578.72	4.56	-136577.91	0.05
bqp500-6	x	x	-558167.20	23.11	-132960.20	4.37	-132959.74	0.04
bqp500-7	x	x	-565744.09	22.85	-134273.56	4.43	-134273.22	0.04
bqp500-8	x	x	-568072.95	27.41	-135438.79	4.86	-135438.32	0.04
bqp500-9	x	x	-560556.91	29.49	-132615.73	4.41	-132614.79	0.05
bqp500-10	x	x	-566953.10	23.36	-141076.28	4.70	-141075.76	0.04

• ‘NA’: Not available; ‘x’: computation not done due to heavy CPU time.

Table 2 presents a comparison of optimal value and CPU time among the actual optimal solutions: SDP+RLT, Direct DNN, Lagrangian-DNN (Kim et al., 2016), and SDP (IPM method), and Low-Rank (LR) method. The instances are taken from the BIQMAC library. For dimensions like 250 and higher, SDP+RLT is costly due to CPU time. It is observed that CSBB is the fastest among all and has the same optimal value as direct DNN and SDP. However, LAG-DNN is slightly better with respect to the optimal solution than LR-BIQ and inferior with respect to CPU time.

Table 2
Comparison of optimal value and CPU time (in seconds) among Actual optimal value, SDP+RLT, Direct DNN, LAG-DNN, SDP relaxations, and LR-BIQ for selected problems.

BQP	Optimal value	SDP+RLT		DNN		LAG-DNN		SDP{-1,1}		LR-BIQ
		Optimal	Time	Optimal	Time	Optimal	Time	Optimal	Time	Optimal	Time
bqp100-1	-7970.00	-8036.66	104.36	-8721.10	1.26	-8046.875	29.00	-8721.11	0.33	-8721.10	0.01
bqp100-2	-11036.00	-11036.00	109.08	-11704.18	1.07	-11044.92	26.00	-11704.18	0.29	-11704.18	0.01
bqp100-3	-12723.00	-12723.00	109.69	-13336.70	1.28	-12724.61	29.00	-13336.70	0.26	-13336.69	0.01
bqp100-4	-10368.00	-10368.00	111.32	-10927.92	1.21	-10371.09	28.00	-10927.93	0.27	-10927.93	0.01
bqp100-5	-9083.00	-9083.00	110.07	-9736.93	1.25	-9089.36	30.00	-9736.93	0.25	-9736.93	0.01
bqp250-1	-45607.00	x	x	-48726.91	7.62	-46269.53	272.00	-48732.37	0.79	-48732.29	0.02
bqp250-2	-44810.00	x	x	-48091.31	6.34	-45605.47	377.00	-48093.50	0.81	-48093.50	0.02
bqp250-3	-49037.00	x	x	-51745.19	8.47	-49492.19	420.00	-51745.40	0.81	-51745.37	0.02
bqp250-4	-41274.00	x	x	-44388.52	7.07	-42050.78	303.00	-44391.58	0.80	-44391.52	0.01
bqp250-5	-47961.00	x	x	-50796.99	6.04	-48457.03	398.00	-50803.63	0.81	-50803.56	0.02

4.2 Max-Cut Problem

The maximum cut (Max-Cut) problem is a very well known problem in various real-world fields, such as network design, VLSI design, statistical physics, etc. If G=(V, E) be a given undirected and connected graph, with V=1, 2, ..., n and E⊂{(i, j):1≤i<j≤n}, the max-cut problem is to find a bipartition (V ₁, V ₂) of V so that the sum of the weights of the edges between V ₁ and V ₂ is maximized. Let the edge weights w _ij =w _ji be given such that w _ij =0 for (i, j)∉E, and in particular, let w _ii =0. The max-cut problem can be formulated as a BQP as

$\begin{array}{c}max\frac{1}{4}{x}^{\top }\left(Diag\left(We\right)-W\right)x\\ s.t.x\in {\left\{-1,1\right\}}^n,\end{array}$ (18)

where W=[w _ij ]∈S ⁿ is the weighted adjacency matrix of the graph G. The problem (18) has the same solution as that of the following BQP

$\begin{array}{c}MC:min{x}^{\top }Wx\\ s.t.x\in {\left\{-1,1\right\}}^n.\end{array}$ (19)

Taking X=xx ^⊤ and dropping the rank(X)=1 constraint, the SDP relaxation of it is

$\begin{array}{c}MC-SDP:minW·X\\ s.t.diag\left(X\right)=1.\end{array}$ (20)

As discussed in Section 3, the low rank formulation of it is as follows:

$\begin{array}{c}LR-MC:minW·V^{\top }V\\ s.t.||V_i||=1,i=1,2,...,n.\end{array}$ (21)

To test the efficiency of the proposed model, we will now execute the method on the dataset Gset, which is available in http://web.stanford.edu/~yyye/yyye/Gset/.

We consider only 35 instances of the Gset problems with dimensions ranging from 800 to 20, 000 vertices. There are graphs without weighted edges (all weights are 1) as well as graphs with weighted edges where the weights are either+1 or -1.

Table 3 presents a comparison of optimal value and CPU time among benchmark optimal solutions computed by SBM (Matsuda, 2019), SDP (20), and the proposed Low-rank decomposition model LR (21). The column dimension describes the node and edges of the graphs. The columns SDP and LR describe the optimal solutions of SDP (20) and LR (21). Similarly, the columns T-SDP and T-LR describe the CPU time taken by SDP and LR respectively. The column Diff-SDD describes the difference of SDP with SBM and the column Diff-LR describes the difference between the optimal value LR-MC and SBM. By comparing the CPU time and difference columns, it is observed that LR is not only the fastest but also provides the optimal solution close to the SBM in most of the instances.

Table 3
Optimal value and CPU time (in seconds) of SBM, SDP(dual), and LR-MC.

Problem	Dimension	SBM	SDP	T-SDP	LR	T-LR	Diff-SDP	Diff-LR
G1	(800,19176)	11624	11336	8.02	11414	0.13	288	210
G3	(800,19176)	11622	11396	8.82	11392	0.15	226	230
G5	(800,19176)	11631	11381	8.90	11432	0.14	250	199
G7	(800,19176)	2006	1729	9.65	1736	0.11	277	270
G9	(800,19176)	2054	1782	15.39	1800	0.39	272	254
G10	(800,19176)	2000	1785	17.99	1766	0.48	215	234
G12	(800,1600)	556	512	10.27	514	1.36	44	42
G14	(800,4694)	3064	2944	8.14	2966	0.28	120	98
G16	(800,4672)	3052	2961	7.58	2956	0.24	91	96
G18	(800,4694)	992	892	7.57	894	0.21	100	98
G20	(800,4672)	941	848	7.53	837	0.21	93	104
G21	(800,4667)	931	811	7.82	809	0.22	120	122
G23	(2000,19990)	13344	11791	73.26	12984	1.16	1553	360
G24	(2000,19990)	13337	12022	70.09	12973	1.56	1315	364
G27	(2000,19990)	3341	2073	70.79	2905	2.08	1268	436
G30	(2000,19990)	3413	2160	73.88	3007	0.80	1253	406
G32	(2000,4000)	1410	1002	46.72	1272	2.00	408	138
G33	(2000,4000)	1382	944	46.50	1218	2.03	438	164
G35	(2000,11778)	7687	6983	61.92	7424	2.01	704	263
G36	(2000,11766)	7680	7010	64.56	7403	2.02	670	277
G38	(2000,11779)	7688	6972	65.03	7385	1.93	716	303
G41	(2000,11785)	2405	1338	62.00	2114	1.48	1067	291
G42	(2000,11779)	2481	1486	162.89	2160	3.79	995	321
G47	(1000,9990)	6657	5977	39.94	6454	0.84	680	203
G50	(3000,6000)	5880	5880	101.22	5880	3.09	0	0
G51	(1000,5909)	3848	3530	12.84	3739	0.42	318	109
G54	(1000,5916)	3852	3539	12.18	3711	0.37	313	141
G56	(5000,12498)	4017	372	515.43	3600	11.30	3645	417
G58	(5000,10000)	19293	1998	397.15	3146	11.77	17295	16147
G59	(5000,29570)	6086	3747	709.16	5341	12.82	2339	745
G61	(7000,17148)	5796	494	2420.24	5250	15.49	5302	546
G62	(7000,14000)	4870	2936	1422.93	4352	21.84	1934	518
G64	(7000,41459)	8751	5455	2859.64	7748	22.05	3296	1003
G65	(7000,16000)	5562	2744	1947.89	5036	27.83	2818	526
G67	(7000,20000)	6950	3624	3597.01	6314	45.84	3326	636

4.3 Boolean least squares

The basic problem in digital communications, especially maximum likelihood estimation for digital signals, can be presented as an optimization problem as

$min\left\{{||Ax-b||}^2:x\in {\left\{-1,1\right\}}^n\right\}$

and can be expressed as a nonconvex QCQP as

$min\left\{x^{\top }{A}^{\top }Ax-2b^{\top }Ax+b^{\top }b:x_i^2=1,i=1,2,...,n\right\}.$

A brute force solution is to check all 2ⁿ possible values of x, which is usually impractical and leads to some relaxation methods. The SDP relaxation of it is

$\begin{array}{c}min\left[\left(A^{\top }A\right)·X-2b^{\top }Ax+b^{\top }b\right]\\ subjectto{X}_{ii}=1,i=1,2,...,n,\\ \left(\begin{array}{cc}1& x\\ x^{\top }& X\end{array}\right)⪰0,\end{array}$ (22)

where, x∈ℝⁿ and $X\in S_{+}^n$. By using the cyclicity of the property of Trace of the matrix, we obtain

$\begin{array}{c}Trace\left(x^{\top }{A}^{\top }Ax\right)-2b^{\top }Ax+b^{\top }b=Trace\left[\left(x^{\top }{A}^{\top }-b^{\top }\right)\left(Ax-b\right)\right]\\ =Trace\left[\left(x^{\top }1\right)\left(\begin{array}{c}{A}^{\top }\\ -b^{\top }\end{array}\right)\left(A-b\right)\left(\begin{array}{c}x\\ 1\end{array}\right)\right]\\ =Trace\left[\left(\begin{array}{c}{A}^{\top }\\ -b^{\top }\end{array}\right)\left(A-b\right)\left(\begin{array}{c}x\\ 1\end{array}\right)\left(x^{\top }1\right)\right]\\ =Trace\left[\left(\begin{array}{cc}{A}^{\top }A& -A^{\top }b\\ -b^{\top }A& b^{\top }b\end{array}\right)\left(\begin{array}{cc}X& x\\ x^{\top }& 1\end{array}\right)\right]\end{array}$

where, X=xx ^⊤. Thus, we can rewrite the SDP formulation of (22) as

$\begin{array}{c}SDP-BLS:minB·Y\\ subjectto{Y}_{ii}=1,i=1,2,...,n+1,\\ Y⪰0\end{array}$ (23)

where, $B=\left(\begin{array}{cc}{A}^{\top }A& -A^{\top }b\\ -b^{\top }A& b^{\top }b\end{array}\right)$ and $Y=\left(\begin{array}{cc}X& x\\ x^{\top }& 1\end{array}\right)$. Thus, the low rank decomposed formulation is

$\begin{array}{c}LR-BLS:minB·V^{\top }V\\ subjectto||V_i||=1,i=1,2,...,n+1,\end{array}$ (24)

4.3.1 Experiment on BLS

In this subsection, we compare the efficiency of the low rank nonconvex formulation (24) strategy with SDP relaxation (23) with respect to lower bounds and CPU time.

The optimal value of LR and SDP-BLS is the lower bound for BLS. We have solved four instances with different m and n. The Matlab command for A and b is mentioned in column 1. Note that for a particular pair (m, n), we have run the algorithm for 50 iterations. The average of running times in seconds and optimal values are reported in Table 4.

Table 4
Boolean Least Square Problem.

size	m	n	SDP-BLS	LR	Time-SDP	Time-LR
Example-1
A=rand(m,n); xc=floor(100*randn(n,1)); b=A*xc+normrnd(0,0.05,m,1);	100	80	1286991386.01	1286991386.00	0.36	0.0095
	150	130	61311054.03	61311054.00	0.35	0.0044
	200	180	152067976.39	152067975.99	0.44	0.0042
	250	230	301678181.01	301678180.99	0.60	0.0076
	300	280	531274475.03	531274474.97	0.75	0.0137
Example-2
A=floor(100*rand(m,n)-5); b=floor(100*rand(m,1)-5);	200	150	9096907353.07	9096907353.00	0.25	0.0031
	250	200	20014552148.26	20014552148.00	0.31	0.0041
	300	250	Dual Infeas	37170944499.00	0.43	0.0084
	350	300	Dual Infeas	62717185547.00	0.53	0.0095
	500	450	Dual Infeas	201350589919.00	1.27	0.0292
Example-3
A=floor(10*sprand(m,n,0.1)-5); b=floor(100*rand(m,1));	100	80	17612749.01	17612749.00	0.18	0.0019
	150	130	61790525.03	61790525.00	0.23	0.0028
	200	180	152994659.38	152994659.00	0.32	0.0039
	250	230	305781074.17	305781073.98	0.40	0.0063
	300	280	530295425.06	530295424.98	0.54	0.0077
Example-4
A=floor(100*rand(m,n)-5); b=floor(100*rand(m,1)-5);	300	250	Dual Infeas	37473371398.00	0.42	0.0072
	400	350	Dual Infeas	98041617285.00	0.70	0.0149
	500	450	Dual Infeas	201245418480.00	1.31	0.0292
	750	700	Dual Infeas	731357190310.00	3.34	0.0911
	1000	900	Dual Infeas	1611202943603.00	8.66	0.2017

It is observed that SDP dual shows infeasibility for large m and n, while LR-BLS is the best with respect to CPU time and lower bound.

4.4 0-1 Quadratic Knapsack Problem

The Quadratic Knapsack Problem (QKP) introduced by Gallo et al. (1980) has a wide spectrum of applications in the areas of the telecommunication industry Witzgall (1975), location selection problem with a budget constraints (Rhys, 1970) and some applications in weighted maximum b-clique problems (Dijkhuizen & Faigle, 1993; Park et al., 1996; Pisinger, 2007).

Since QKP is NP-hard (Pardalos & Vavasis, 1991), it is difficult to find a polynomial time algorithm. Therefore, finding an approximation algorithms for QKP is always the primary choice. the semidefinite relaxation method (Helmberg et al., 2000), the Lagrangian relaxation method (Michelon & Veilleux, 1996; Caprara et al., 1999; Billionnet & Soutif, 2004a; Létocart et al., 2012), the conic approximation method (Zhou et al., 2013) are some of the approximation methods that best suits for QKP.

The general 0-1 quadratic knapsack problem is given as

$\begin{array}{c}\underset{\left\{0,1\right\}}{QKP}:max{y}^{\top }Py+p^{\top }y\\ subjectto{d}^{\top }y\le c\\ y\in {\left\{0,1\right\}}^n\end{array}$ (25)

where, y is a vector of decision variables, P is an n×n real symmetric matrix; p, d∈ℝⁿ and c∈ℝ. Since y∈{0, 1}ⁿ implies y ²=y _i , modelling the linear constraint d ^⊤ y≤c by restricting the diagonal elements of Y where, Y=yy ^⊤, yielding the diagonal representation as D·Y≤c. Thus, the semidefinite relaxation is,

$\begin{array}{c}maxQ·Y\\ s.t.D·Y\le c\\ Diag\left(Y\right)=y\\ \left(\begin{array}{cc}Y& y\\ y^{\top }& 1\end{array}\right)⪰0,\end{array}$ (26)

where, Q=P+Diag(p ^⊤) and D=Diag(d ^⊤). Noted that Diag(y) is a diagonal matrix with diagonal entries as vector y.

Since we need to express problem (25) to our chosen binary domain {-1, 1}, we set x=2y-e, where, e=(1, 1, ..., 1)^⊤. With this, the QKP is

$\begin{array}{c}\underset{\left\{-1,1\right\}}{QKP}:max\frac{1}{4}{x}^{\top }Px+\frac{1}{2}{\left(Pe+p\right)}^{\top }x+\left(\frac{1}{4}{e}^{\top }Pe+\frac{1}{2}{p}^{\top }e\right)\\ subjectto{d}^{\top }x\le \left(2c-d^{\top }e\right),\\ x\in {\left\{-1,1\right\}}^n.\end{array}$ (27)

The Lagrangian relaxation of (27) yields as follows:

$\begin{array}{c}L\left(x,\lambda \right)=\frac{1}{4}{x}^{\top }Px+\frac{1}{2}{\left(Pe+p\right)}^{\top }x+\left(\frac{1}{4}{e}^{\top }Pe+\frac{1}{2}{p}^{\top }e\right)+\lambda \left(d^{\top }x-2c+d^{\top }e\right)\\ =\frac{1}{4}{x}^{\top }Px+\frac{1}{2}{\left(Pe+p+2\lambda d\right)}^{\top }x+\left(\frac{1}{4}{e}^{\top }Pe+\frac{1}{2}{p}^{\top }e-2\lambda c+\lambda d^{\top }e\right),\end{array}$

Letting z=[x·x _n+1 ]^⊤, with x _n+1 =±1 and taking Z=zz ^⊤, the problem can be rewritten as follows

$\begin{array}{c}SDP-QKP:maxB·Z\\ subjecttoDiag\left(Z\right)=e,\left(\begin{array}{cc}Z& z\\ z^{{}^{\top }}& 1\end{array}\right)⪰0.\end{array}$ (28)

where, $B=\left(\begin{array}{cc}\frac{1}{4}P& \frac{1}{4}\left(Pe+p+2\lambda d\right)\\ \frac{1}{4}{\left(Pe+p+2\lambda d\right)}^{\top }& \left(\frac{1}{4}{e}^{\top }Pe+\frac{1}{2}{p}^{\top }e-2\lambda c+\lambda d^{\top }e\right)\end{array}\right)$.

Therefore, the low-rank nonconvex model of it is

$\begin{array}{c}LR-QKP:min-B·\left(V^{\top }V\right)\\ subjectto{||V_i||}^2=1,i=1,2,...,n.\end{array}$ (29)

4.4.1 Experiment on QKP

In this section, we compare the efficiency of the low rank nonconvex formulation (29) strategy over the SDP-QKP (28) with respect to lower bounds and CPU time.

We solved 29 QKP instances from Billionnet and Soutif (BS family) (Billionnet & Soutif, 2004a, b) available at http://cedric.cnam.fr/~soutif/QKP/QKP.html. The BS family problems have a density ranging from 25% to 100% with dimensions 100 to 300, and the results of some specific problems are available at their sites. We compare the objective and CPU time among the BS family results, SDP result, and low rank solution. The objective value and running times (in seconds) are reported in Table 5. We observed that the low rank method is the fastest among all with better solution quality.

Table 5
Comparison of Objective value and CPU time (in seconds) among Lagrangian decomposition of BS family (Billionnet & Soutif, 2004a), SDP relaxations, and LR-QKP.

BS Family				SDP		LR
Problem	Den	Obj	time	Obj	time	Obj	time
r_100_25_1	25%	18558	429.24	18903.21	1.30	18138.30	0.04
r_100_25_2	25%	56525	14.25	56598.58	0.70	56665.40	0.01
r_100_25_3	25%	3752	60.18	3984.90	0.74	3581.20	0.01
r_100_25_4	25%	50382	37.31	50526.75	0.65	51475.20	0.01
r_100_25_5	25%	61494	15.92	61630.66	0.58	61620.00	0.02
r_100_25_10	25%	24930	228.62	25216.15	0.52	24312.10	0.01
r_100_50_1	50%	NA	NA	83977.05	0.58	83918.00	0.01
r_100_50_5	50%	NA	NA	56717.69	0.58	56429.48	0.00
r_100_75_2	75%	NA	NA	95515.80	0.63	100411.60	0.00
r_100_75_8	75%	NA	NA	19718.04	0.51	192038.98	0.00
r_100_100_1	100%	NA	NA	82627.22	0.58	82557.30	0.01
r_100_100_10	100%	NA	NA	195077.50	0.70	195029.00	0.00
r_200_25_1	25%	NA	NA	204724.02	1.79	204637.60	0.01
r_200_25_9	25%	NA	NA	49786.45	1.49	47159.20	0.01
r_200_50_2	50%	NA	NA	211659.98	1.62	224500.00	0.01
r_200_75_1	75%	NA	NA	443325.05	1.86	442622.80	0.01
r_200_75_4	75%	NA	NA	128477.46	1.70	165066.00	0.01
r_200_75_9	75%	NA	NA	517859.46	1.86	516396.00	0.01
r_200_100_1	100%	937149	556714.87	942498.10	1.99	947430.00	0.00
r_200_100_3	100%	29596	342.63	32092.07	1.67	32094.30	0.01
r_200_100_7	100%	701094	1013.55	702039.43	1.98	701804.40	0.01
r_200_100_9	100%	629587	1245.45	629630.99	1.82	643385.60	0.01
r_300_25_1	25%	29140	5783.5	30268.58	3.97	300224.00	0.01
r_300_25_2	25%	281990	27535.97	282452.67	3.46	283094.00	0.01
r_300_25_6	25%	269782	3876.28	270049.92	3.46	261950.60	0.02
r_300_50_1	50%	513379	2007.62	513797.26	3.67	514875.00	0.01
r_300_50_4	50%	307124	2102.16	307476.26	3.64	307019.20	0.01
r_300_50_5	50%	727820	2051.13	728180.58	3.87	726976.00	0.01
r_300_50_8	50%	767977	3555.53	768562.95	3.82	767870.00	0.01

• NA: Not available on their site (Billionnet & Soutif, 2004a, b).

5 CONCLUSION

In this paper, we have presented an experimental framework for low-rank nonconvex relaxation to improve its effectiveness and efficiency. We have tested the method over BIQ, Max-cut, BLS, and 0-1 QKPs. Beside the above four problems discussed, we can apply our LR modeling to other BQP such as graph bisection problems, image segmentation with partial grouping constraints, image segmentation with histogram constraints, Image co-segmentation problems, etc. The computational efficiency of low-rank modeling on different QCQP problems suggests that the method achieves a near global solution with faster CPU time, which attracts us to use it on some engineering problems, for which we may conclude that this method may be more tested on other large scale binary quadratic programming problems.

Acknowledgements

The author(s) are grateful to the referees and editor for their helpful comments and valuable suggestions which have contributed to the final presentation of the paper.

References

• ANSTREICHER KM. 2009. Semidefinite programming versus the reformulation-linearization technique for nonconvex quadratically constrained quadratic programming. Journal of Global Optimization, 43(2): 471-484.
• ANSTREICHER KM. 2012. On convex relaxations for quadratically constrained quadratic programming. Mathematical programming, 136(2): 233-251.
• BERTSIMAS D, CORY-WRIGHT R & JOHNSON NA. 2023. Sparse plus low rank matrix decomposition: A discrete optimization approach. Journal of Machine Learning Research, 24(267): 1-51.
• BILLIONNET A & SOUTIF É. 2004a. An exact method based on Lagrangian decomposition for the 0-1 quadratic knapsack problem. European Journal of operational research, 157(3): 565-575.
• BILLIONNET A & SOUTIF E. 2004b. Using a Mixed Integer Programming Tool for Solving the 0-1 Quadratic Knapsack Problem. INFORMS J. on Computing, 16(2): 188-197.
• BURER S & MONTEIRO RDC. 2003. A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization. Math. Program, 95: 329-357.
• BURER S & SAXENA A. 2012. The MILP road to MIQCP. Mixed integer nonlinear programming, pp. 373-405.
• BURER S & YANG B. 2015. The trust region subproblem with non-intersecting linear constraints. Mathematical Programming, 149(1): 253-264.
• CAPRARA A, PISINGER D & TOTH P. 1999. Exact solution of the quadratic knapsack problem. INFORMS Journal on Computing, 11(2): 125-137.
• CHEN P, YU HF, DHILLON I & HSIEH CJ. 2021. Drone: Data-aware low-rank compression for large nlp models. Advances in neural information processing systems, 34: 29321-29334.
• DIJKHUIZEN G & FAIGLE U. 1993. A cutting-plane approach to the edge-weighted maximal clique problem. European Journal of operational research, 69(1): 121-130.
• FU M, LUO ZQ & YE Y. 1998. Approximation algorithms for quadratic programming. Journal of combinatorial optimization, 2(1): 29-50.
• GALLO G, HAMMER PL & SIMEONE B. 1980. Quadratic knapsack problems. In: Combinatorial optimization. pp. 132-149. Springer.
• GOEMANS MX & WILLIAMSON DP. 1995. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM (JACM), 42(6): 1115-1145.
• HELMBERG C, RENDL F & WEISMANTEL R. 2000. A semidefinite programming approach to the quadratic knapsack problem. Journal of combinatorial optimization, 4(2): 197-215.
• HU S & YE K. 2023. Linear convergence of an alternating polar decomposition method for low rank orthogonal tensor approximations. Mathematical Programming, 199(1): 1305-1364.
• JIANG R & LI D. 2019. Semidefinite Programming Based Convex Relaxation for Nonconvex Quadratically Constrained Quadratic Programming. In: World Congress on Global Optimization. pp. 213-220. Springer.
• KAUSHAL A, VAIDHYA T & RISH I. 2023. Lord: Low rank decomposition of monolingual code llms for one-shot compression. arXiv preprint arXiv: 2309.14021.
• KIM S, KOJIMA M & TOH KC. 2016. A Lagrangian-DNN relaxation: a fast method for computing tight lower bounds for a class of quadratic optimization problems. Mathematical Programming, 156(1-2): 161-187.
• LÉTOCART L, NAGIH A & PLATEAU G. 2012. Reoptimization in Lagrangian methods for the 0-1 quadratic knapsack problem. Computers & operations research, 39(1): 12-18.
• LIU Q, LING S & JIA Z. 2022. Randomized quaternion singular value decomposition for low-rank matrix approximation. SIAM Journal on Scientific Computing, 44(2): A870-A900.
• MATSUDA Y. 2019. Benchmarking the MAX-CUT problem on the Simulated Bifurcation Machine.
• MICHELON P & VEILLEUX L. 1996. Lagrangean methods for the 0-1 quadratic knapsack problem. European Journal of operational research, 92(2): 326-341.
• NAYAK RK & MOHANTY NK. 2019. Improved row-by-row method for binary quadratic optimization problems. Annals of Operations Research, 275: 587-605.
• NOCEDAL J & WRIGHT S. 2006. Numerical optimization. 2. ed. Springer series in operations research and financial engineering. New York, NY: Springer. Available at: http://gso.gbv.de/DB=2.1/CMD?ACT=SRCHA&SRT=YOP&IKT=1016&TRM=ppn+502988711&sourceid=fbw_bibsonomy
  » http://gso.gbv.de/DB=2.1/CMD?ACT=SRCHA&SRT=YOP&IKT=1016&TRM=ppn+502988711&sourceid=fbw_bibsonomy
• PARDALOS PM & VAVASIS SA. 1991. Quadratic programming with one negative eigenvalue is NP-hard. Journal of Global Optimization, 1(1): 15-22.
• PARK K, LEE K & PARK S. 1996. An extended formulation approach to the edge-weighted maximal clique problem. European Journal of operational research, 95(3): 671-682.
• PISINGER D. 2007. The quadratic knapsack problem - a survey. Discrete applied mathematics, 155(5): 623-648.
• RHYS JM. 1970. A selection problem of shared fixed costs and network flows. Management Science, 17(3): 200-207.
• SHERALI HD. 2007. RLT: A unified approach for discrete and continuous nonconvex optimization. Annals of Operations Research, 149(1): 185.
• SHERALI HD & ADAMS WP. 2013. A reformulation-linearization technique for solving discrete and continuous nonconvex problems. vol. 31. Springer Science & Business Media.
• SHERALI HD & FRATICELLI BM. 2002. Enhancing RLT relaxations via a new class of semidefinite cuts. Journal of Global Optimization, 22(1): 233-261.
• TSENG P. 2003. Further results on approximating nonconvex quadratic optimization by semidefinite programming relaxation. SIAM Journal on Optimization, 14(1): 268-283.
• WANG P, SHEN C & HENGEL AVD. 2013. A Fast Semidefinite Approach to Solving Binary Quadratic Problems. Proc. IEEE Conf. Computer Vision and Pattern Recognition.
• WANG R, FANG H, YU L, YU L & CHEN J. 2022. Sparse and low-rank decomposition of the time-frequency representation for bearing fault diagnosis under variable speed conditions. ISA transactions, 128: 579-598.
• WEN Z & YIN W. 2013. A feasible method for optimization with orthogonality constraints. Math. Program, 142: 397-434.
• WIEGELE A. 2007. Biq Mac Library - A collection of Max-Cut and quadratic 0-1 programming instances of medium size. Preprint, 51: 112-127.
• WITZGALL C. 1975. Mathematical methods of site selection for Electronic Message Systems (EMS). NASA STI/Recon Technical Report N, 76: 18321.
• YAMADA S & TAKEDA A. 2018. Successive Lagrangian relaxation algorithm for nonconvex quadratic optimization. Journal of Global Optimization, 71(2): 313-339.
• YE Y. 1999. Approximating global quadratic optimization with convex quadratic constraints. Journal of Global Optimization, 15(1): 1-17.
• ZHENG X, SUN X & LI D. 2011. Nonconvex quadratically constrained quadratic programming: best DC decompositions and their SDP representations. Journal of Global Optimization, 50(4): 695-712.
• ZHOU J, CHEN D, WANG Z & XING W. 2013. A conic approximation method for the 0-1 quadratic knapsack problem. Journal of Industrial and Management Optimization, 9(3): 531-547.

• Data Availability
  The datasets analyzed in this study are publicly available and referenced in the Numerical Experiment Section along with their respective URLs.
• Funding
  The authors received no financial support for the research and the publication of this article.

Edited by

• Editor responsible for the review
  Editor-in-Chief: Annibal Parracho Sant’Anna.

Data availability

The datasets analyzed in this study are publicly available and referenced in the Numerical Experiment Section along with their respective URLs.

Publication Dates

• Publication in this collection
  08 Aug 2025
• Date of issue
  2025

History

• Received
  21 Feb 2025
• Accepted
  02 June 2025