www.elsevier.com/locate/dsw
On improved Choi–Goldfarb solution-containing ellipsoids
in linear programming
Igor S. Litvinchev
1Computing Center, Russian Academy of Sciences, Vavilov 40, 117967 Moscow, Russia
Received 1 May 1998; received in revised form 1 February 2000
Abstract
Ellipsoids that contain all optimal primal solutions, those that contain all optimal dual slack solutions, and primal– dual ellipsoids are derived. They are independent of the algorithm used and have a smaller size than the Choi–Goldfarb ellipsoids [J. Optim. Theory Appl. 80 (1994) 161–173]. c 2000 Elsevier Science B.V. All rights reserved.
Keywords:Linear programming; Interior point methods; Containing ellipsoids
1. Introduction
Unlike the simplex method for linear programming, which moves from one basic feasible solution to another basic feasible solution, interior algorithms create a sequence of interior points converging to an optimal feasible solution. To identify optimal basic and non-basic variables during the course of the algorithm several criteria have been proposed, associated with the interior point techniques used. These criteria were obtained by constructing ellipsoids which contain all optimal primal solutions and=or ellipsoids which contain all optimal dual slack solutions, and are all algorithm dependent (see e.g. [2,3,5,7–9]). Choi and Goldfarb [1] have further developed this approach, deriving solution containing ellipsoids which are independent of the interior point algorithm used, but larger than algorithm-dependent ellipsoids. Concerning solution-containing ellipsoids we also note that the rst two-sided ellipsoidal approximation of polyhedra based on primal barrier functions is due to Sonnevend [4].
In this paper we propose the approach to diminish the size of the solution-containing ellipsoids derived in [1]. To get the tighter estimation of the ellipsoid size a certain localization of the optimal solution set is
E-mail address:[email protected] (I.S. Litvinchev).
1The research supported in part by the grants from FAPESP, Brazil (No. 1997=7153-2) and RFBR, Russia (No. 99-01-01071).
used. It is shown that the proper choice of the localization results in a smaller containing ellipsoid. A similar approach is used to derive the primal–dual containing ellipsoids.
The paper is organized as follows. In Section 2, some basic notations and results, associated with Choi– Goldfarb ellipsoids are stated. In Section 3, the improved primal ellipsoids are constructed. In Section 4, the primal–dual ellipsoids are considered, and a new criterion for identifying the optimal basis is stated and compared with the previous results.
2. Denitions and notation
Consider the primal–dual pair of linear programs:
min{ctx|Ax=b; x¿0}; (1)
max{bty|Aty+z=c; z¿0}; (2)
where A ∈ Rm×n and b; c; x; y and z are suitably dimensioned. Let x∗ and ( z∗
; y∗) be optimal solutions to
(1) and (2), respectively. It is assumed that the optimal solution sets for both (1) and (2) are bounded, or equivalently, strictly feasible primal and dual solutions exist, and that rank(A) =m: The upper case letters X
andZ are used to denote diagonal matrices whose elements are the elements of the vectors xandz, andk · k denotes the Euclidean norm.
It was established in [1] that if x and ( z;y) are primal and dual feasible solutions, then the ellipsoid
Ep={x∈Rn| kZ(x−x)k6};
where
2=kZxk2+2; = xtz
contains all optimal primal solutions x∗
: Here the “ellipsoid” is understood in a general sense allowing also unbounded sets.
Using this ellipsoid the lower bound x−
i () forx ∗
i was derived by the solution of the problem
min{xi| kZ(x−x)k6; Ax=b; ctx6ctx}:
If z ¿0 (which will be assumed valid throughout the paper) then the optimal solutionx−
i () of this problem
takes the following form. Let ei denote the ith unit vector and
R( Z) = Z−1[I−Z−1At(AZ−2At)−1AZ−1 ] Z−1:
Then
x−
i () = xi−
q et
iR( Z)ei−(ctR( Z)ei)2=ctR( Z)c for ctR( Z)ei¡0
and
x−
i () = xi−
q et
iR( Z)ei for ctR( Z)ei¿0:
Respectively, the following criterion for identifying optimal basic variables was obtained: ifx−
i ()¿0; then
the ith primal variable is an optimal basic variable, i.e., x∗
i ¿0 and z ∗ i = 0:
We see that the smaller the size of the ellipsoidEp; the larger is the value of x
−
i (): Below we consider
3. Improved primal solution-containing ellipsoid
Denote by P∗
the optimal solution set of (1) and let W be a closed bounded subset of Rn such that
W⊇P∗
: We will refer to such a W as a primal localization. Let S ∈ Rn×n be a diagonal matrix having
positive diagonal entries si:
For any x∈P∗
we have
kSZ(x−x)k2 =kSZxk2+kSZx k2−2 xt(SZ)2x
6kSZxk2+ max
w∈W{kS
Zwk2−2 xt(SZ)2w}
≡2(S; W); (3) where the inequality follows from the condition x∈P∗
⊆W:
Then the ellipsoid
Ep(S; W) ={x∈Rn| kSZ(x−x)k6(S; W)}
contains all optimal primal solutions.
Using the ellipsoid Ep(S; W) the lower bound x
−
i can be derived similarly as in Section 2. It is not hard
to verify that we need only to substitute R( Z) for
R(SZ) = (SZ)−1[I
−(SZ)−1At(A(SZ)−2At)−1A(SZ)−1](SZ)−1: and use (S; W) instead of :
The reason for using the scaled ellipsoid is as follows. Suppose for example that the pair ( x;z) was produced by a primal–dual method [6]. Let S= (XZ)−1=2: For this case R(SZ) takes the form
R(D−1) =D[I−DAt(AD2At)−1AD]D;
where D= X1=2Z−1=2: Note that the matrix AD2At is computed and decomposed into its Cholesky factor in most of the primal–dual methods.
To dene a localization W we can use some constraints of the primal problem or a linear combination of them. Below we consider the localization which results in a closed form expression for (S; W):
Proposition 1. Let the primal localization be dened by W1={w∈Rn|ztw6; w¿0}: Then
2(S; W1) =kSZxk2+max
i {s
2
i(−2 zixi)}:
Proof. For any x∈P∗
we have
ztx= (c−Aty)tx=ctx−ytb6ctx−ytb= ztx=
and henceP∗
⊆W1:Moreover since z ¿0 thenW1 is bounded, i.e.,W1 is a localization. To estimate(S; W1) in (3) we should solve the problem
f∗
= max
w∈W1{k
SZw k2−2 xt(SZ)2w}:
The objective function of this problem is strictly convex and hence the optimal solution is attained at a vertex of the non-degenerate simplex W1: The non-zero component of the ith vertex is=zi and thus
f∗
=max
i {s
2
i(−2 zixi)};
Note that ifS=I and x ¿0; then
2(I; W1) =kZxk2+2−2min
i zixi¡
2
and hence Ep(I; W1)⊂Ep:
The other choice of the scaling matrix S associated with the primal–dual algorithms is S= (XZ)−1=2: For this case, we have
2((XZ)−1=2; W 1) =
2 minizixi −
:
A similar approach can be used to derive a dual ellipsoid that contains all optimal dual slack solutions, i.e.
Ed(S; V) ={z∈Rn| kSX(z−z)k6(S; V)};
where
2(S; V) =kSXzk2+ max
∈V{kS
X k2−2 zt(SX)2}:
A dual localization V is a closed bounded subset of Rn such thatz∗
∈V for any optimal dual slack solution
z∗
: It is not hard to verify that if x ¿0; then V1={ ∈ Rn|xt6; ¿0} is the dual localization and
(S; V1) =(S; W1): The ellipsoid Ed(I; V1) is smaller than the dual-containing ellipsoid Ed={z∈Rn| kX(z−
z)k6} considered in [1] and it can be used similarly to identify optimal non-basic primal variables.
4. Primal–dual solution-containing ellipsoid
Denote by F∗
⊆R2n the set of all optimal primal–dual pairs (x∗; z∗) and let be a closed bounded subset
of R2n such that ⊇F∗: We will refer to such an as a primal–dual localization. In this section we assume
( x;z)¿0:
For any (x; z)∈F∗
we have similar to (3):
kSZ(x−x)k2+kSX(z−z)k262kSZxk2+ max (w;)∈{kS
Zwk2+kSX k2
−2 xt(SZ)2w−2 zt(SX)2} ≡2(S; ) and hence the ellipsoid
Epd(S; ) ={(x; z)∈R2n| kSZ(x−x)k2+kSX(z−z)k262(S; )}
contains all optimal primal–dual pairs (x∗ ; z∗
):
Proposition 2. Let the primal–dual localization be dened by1={(w; )∈R2n|ztw+ xt=; (w; )¿0}:
Then
2(S; 1) = 2kSZxk2+max
i {s
2
i(−2 zixi)}:
Proof. For any (x; z)∈F∗ we have ztx+ xtz= since (x
−x)t(z−z) = 0 and xtz= 0 by the complementarity condition. Hence F∗
⊆1 and 1 is a localization. Then the required expression for (S; 1) is obtained similar to the proof of Proposition 1.
Now we use Epd to identify, if possible, the optimal basis. Consider the problem
Ax=b; Aty+z=c; (5)
Here q is such that the objective of this problem coincides with xi up to a positive scale factor. In [9] the
problem similar to (4) – (6) was discussed for q=Dei:
Proposition 3. The optimal solution x−
of problem(4)–(9) is such that
(a) if etqv¿0 and kqvk¡ ( etqv); then
Proof. The KKT optimality conditions for (4) – (9) give
q+Atu+ 2(SZ)2(x−x)−(−) z= 0; (10)
v+ 2(SX)2(z−z)−(−) x= 0; (11)
Av= 0; ¿0; ¿0; ¿0;
where; ; and are the Lagrange multipliers for the restrictions (7) – (9). Multiplying (10) by A(SZ)−2 and utilizing the condition A(x−x) = 0 we get
u= [A(SZ)−2At]−1A(SZ)−1((
−) e−q):
Multiplying (10) by (SZ)−2 and substituting the expression for u we obtain
and hence
z−z=−At(y−y) =− 2 (SX)
−1( e−e
u):
By (7) we have xt(z−z) + zt(x−x) =−: Substituting x−xand z−zin this expression we obtain the equation for which gives
=d2[ etqv+kevk2+(kek2− keuk2)−2]:
If ¿0;then by the complementarity condition for (8) we have xtz=:Hence by (7) it follows 0= ztx ¡
and= 0 by the complementarity condition for (9). Similarly if ¿0, then= 0. We can write both these conditions in the form = 0.
Remembering that= xtzthe complementarity conditions for (8) and (9) take the formxt(z−z) = 0 and
zt(x−x) = 0. Substituting the expressions for x−xand z−zwe obtain
(−) = 0;
[(−)kevk2−etqv] = 0:
Utilizing the expression for and the condition = 0 we get
d2( etqv−kevk2−2) = 0; (12)
−etqvkek 2− ke
uk2
kevk2 −
d2(kek2− ke
uk2)−2d2
= 0: (13)
If etqv60 then the term in the brackets in (12) is strictly negative and hence = 0. By (13) we have that if etqv¿0 then = 0.
Since the quadratic constraint (6) must be active, then
2=
1 2
2
k(−) ev−qvk
2+
−
2 2
ke−euk2:
Substituting the expression for we obtain after the algebraic manipulations:
2=2d2+
1 2
2
(kqvk2−d2( etqv)2)−
1 2
2
d2{( etqv)(−)(kek2− keuk2)
+ 2[kevk2+(kek2− keuk2)]}: (14)
Now we are in a position to derive the main results of the proposition.
Suppose that etqv¿0 and kqvk¡ ( etqv). By (13) it follows that = 0. Let ¿0. Then by (12) we have
=e tq
v−2
kevk2
: (15)
Substituting this in (14) with = 0 we obtain
1 2
2
=
2ke
vk2−2
kqvk2ke
vk2−( etqv)2
:
x−x; and we have
which nally proves part (a) of the proposition.
Note that if etqv¿0 and kqvk¿( etqv) then the case ¿0 is impossible and we have == 0.
Substituting this in (14) with = 0 we obtain
Note that under our assumption the numerator of this expression is strictly positive. Moreover, if we substitute this in (16), then ¿0.
which proves part (b) of the proposition. Note that if etqv60 and 2¿B[2(kek2− ke
required in part (c) of the proposition.
Since (x∗; z∗; y∗) is feasible to (4) – (9) for=(S;
1), we have the following criterion for identifying the optimal basis: if qtx−¿0, then x∗
i ¿0 and z∗i = 0.
Note that for part (c) of Proposition 3 we have == 0 and hence constraints (8) and (9) can be relaxed without changing the optimal value. For parts (a) and (b), either (8) or (9) is active and hence
qtx−
In [9] the primal–dual containing ellipsoid centered at the origin was considered for S= (XZ)−1=2 and ( x;z;y)∈N2(), where ∈[0;1),
and F0 is the set of all strictly feasible primal–dual solutions.
For ( x;z;y)∈N2() we have zixi¿(1−)=n; i= 1; n and by Proposition 2
The lower bound qtx−
YT obtained in [9, Theorem 3] by the solution of the problem similar to (4) – (6) for
q=D−1e
i is as follows:
qtx−
YT=etiPe−
q
( 2−)(1−pii);
where P=DtAt(AD2At)−1AD and p
ii is the ith diagonal entry of P.
Since u=v= (I−P) andd2=−1 for S= (XZ)−1=2, part (c) of Proposition 3 gives
qtx−
c =etiPe−
s
( 2−)
1−pii− 1
[e
t
i(I−P) e]2
¿ qtx−
YT:
Hence Proposition 3 results in the stronger criterion for identifying the optimal basis. It is interesting to compare the lower bound obtained in Proposition 3 with the bound x−
i () in Section 2.
We have
2(I; 1) = 2kZxk2+2−2min
i zixi=
2+
kZxk2−2min
i zixi:
If ( x;z;y)∈N2(), then (1−)=n6zixi6(1 +)=n and
kZxk2−2min
i zixi6
2(2+ 4−1)=n:
For ∈[0;√5−2) the right-hand side of the latter inequality is strictly negative and (I; 1)¡ . Since R( Z)= Z−1( Z−1)v for any ∈Rn and ( Z
−1
c)v=ev by Aty+ z=c, it is not hard to verify that
x−
i () coincides with the optimal solution of the following problem:
x−
i () = min{xi| kZ(x−x)k6; Ax=b; ztx6}: (17)
Comparing (17) with the problem (4) – (9) forq=ei,S=I; ( x;z;y)∈N2(),∈[0; √
5−2),=(I; 1)¡ , we obtain x−
i ()¡ etix −
that results in improved criterion for identifying the optimal basis.
The ellipsoids considered in this paper are algorithm-independent, that is they do not depend on how the primal–dual pair ( x;z) was obtained. Meanwhile the size of the ellipsoid depends on the proximity of ( x;z) to the central path: the closer to the central path, the smaller the size. This is one more reason for maintaining ( x;z) in the neighborhood of the central path during the course of the interior algorithm.
Acknowledgements
The author gratefully acknowledges two anonymous referees for their helpful comments and remarks.
Appendix
Since our expression for x−
i () is slightly dierent from the one used in [1, Criterion (C4), p. 170], we
give here the complete derivation for x− i ().
Proposition A1. The optimal solutionx−
i () of the problem
min{etix|(x−x)tZ
2
(x−x)62; Ax=b; ctx6ctx} (A.1)
is as follows:
x−
i () = xi−
q et
iR( Z)ei−(ctR( Z)ei)2=ctPAZ−1c forc tR( Z)e
and
x−
i () = xi−
q et
iR( Z)ei forctR( Z)ei¿0:
Proof. The Lagrangian, associated with (A.1) is
L=etix+[(x−x)tZ2(x−x)−2] +ut(Ax−b) +(ctx−ctx):
The KKT condition @L=@x= 0 gives
ei+ 2Z
2
(x−x) +c+Atu= 0: (A.2)
Multiplying (A.2) by (x−x) and utilizing conditions [(x−x)tZ2(x−x)−2] = 0, (ctx−ctx) = 0,A(x−x) = 0 we obtain
= ( xi−xi)=22:
Multiplying (A.2) by AZ−2 we get
u=−(AZ−2At)−1AZ−2
(ei+c):
From (A.2) we have
2( x−x) = Z−2(Atu+ei+c) =R( Z)(ei+c): (A.3)
Multiplying (A.3) by ct we obtain
062ct( x−x) =ctR( Z)ei+ctR( Z)c: (A.4)
If ctR( Z)e
i¡0, then it follows from (A.4) that ¿0 and hence by the complementarity condition
ct( x−x) = 0. From (A.4) we have =−ctR( Z)e
i=ctR( Z)c. Substituting and in (A.3) and
multiply-ing by ei we obtain the rst part of Proposition A1.
LetctR( Z)e
i¿0. Suppose that we have relaxed the restrictionctx6ctxin (A.1) or, just the same, put = 0
in the Lagrangian. Then similar to (A.3) we obtain for the solution of the relaxed problem
2( x−x) =R( Z)ei
and since ¿0 andctR( Z)e
i¿0 we have ctx6ctx. Since the relaxed solution is feasible to (A.1), then it is
optimal to (A.1). This immediately gives the required expression for x− i ().
Note that in [1, Criterion (C4), p. 170] there is a little aw. The rst expression forx−
i () in Proposition A1
was associated there with the case ctR( Z)e
i¿0.
References
[1] I.C. Choi, D. Goldfarb, On solution-containing ellipsoids in linear programming, J. Optim. Theory Appl. 80 (1994) 161–173. [2] J.L. Gon, J.P. Vial, On the computation of weighted analytic centers and dual ellipsoids with the projective algorithm, Math.
Programming 60 (1993) 81–92.
[3] M. Kojima, Determining basic variables of optimal solutions in Karmarkar’s new LP algorithm, Algorithmica 1 (1986) 499–515. [4] G. Sonnevend, An ‘Analytical Centre’ for Polyhedrons and New Classes of Global Algorithms for Linear (Smooth, Convex)
Programming, in: Lecture Notes in Control and Information Sciences, Vol. 84, A. Prekopa, J. Szelezsan and B. Strazicky (Eds.), Springer, Berlin, 1986, pp. 866 –875.
[5] M.J. Todd, Improved bounds and containing ellipsoids in Karmarkar’s linear programming algorithm, Maths. Oper. Res. 13 (1988) 650–659.
[6] S.J. Wright, Primal–Dual Interior-Point Methods, SIAM Publications, Philadelphia, PA, 1997.
[7] Y. Ye, Recovering the optimal basis in Karmarkar’s polynomial algorithm for linear programming, Math. Oper. Res. 15 (1990) 564–572.
[8] Y. Ye, A “build-down” scheme for linear programming, Math. Programming 46 (1990) 61–72.
