www.elsevier.com/locate/dsw
Two simple proofs for analyticity of the central path
in linear programming
Margareta Halicka
Department of Applied Mathematics, Faculty of Mathematics and Physics, Comenius University, Mlynska dolina, 842 15 Bratislava, Slovakia
Received 2 December 1998; received in revised form 1 January 2000
Abstract
Several papers have appeared recently establishing the analyticity of the central path at the boundary point for both linear programming (LP) and linear complementarity problems (LCP). While the proofs for LP are long, proceeding from limiting properties of the corresponding derivatives, the proofs for LCP are very simple, consisting of an application of the implicit function theorem to a certain system of equations. Inspired by the approach for LCP, this paper gives two simple ways of proving the analyticity of the central path for LP. One follows the idea for LCP, the other is based on a proper partition of the system dening the central path. c2001 Elsevier Science B.V. All rights reserved.
MSC:primary: 90C05; secondary: 90C33
Keywords:Linear programming; Interior point methods; (weighted) Central path; Limiting behavior; Analyticity
1. Introduction
Consider the following pair of dual linear programming (LP) problems:
(P) min{cTx|Ax=b; x¿0};
(D) max{bTy|ATy+s=c; s¿0};
where A is an m×n matrix, rank(A) =m ¡ n; c; x; s∈Rn; b; y∈Rm. We restrict our attention to problems satisfying the assumption
(AS) there exist x ¿0; s ¿0 such that Ax=b; ATy+s=c:
E-mail address: halicka@fmph.uniba.sk (M. Halicka).
In the context of interior point methods for solving (P) and (D), it is important to study the proper-ties of certain interior point paths. We recall here the denition and some properproper-ties of these paths, called
!-weighted central paths.
For any ! ¿0; !∈Rn, dene the following parameterized system
Ax=b; x¿0;
ATy+s=c; s¿0;
Xs=!; (1)
whereX:= diag(x) and∈R; ¿0. Note that for= 0 system (1) gives necessary and sucient conditions for optimality of both (P) and (D). Under assumption (AS), for any ¿0 there exists the unique solution (x(); y(); s()) of (1) such that x()¿0; s()¿0. The set of points {x(); y(); s()}¿0 is called an
(!-weighted) central path. Due to the rank condition for A, we have a one-to-one correspondence betweeny
and s in (1). This enables us to omit y() from the denition of an (!-weighted) central path.
The usual central path is the !-weighted central path corresponding to the weight vector != (1; : : : ;1)T.
In this paper we consider the (!-weighted) central path corresponding to the arbitrarily chosen weight vector
! ¿0. We study the properties of this central path under the natural parameterization given in (1). That is, we are interested in the properties of (x(); s()) as a function of ¿0.
It is easy to see that the central path has nice analytical properties. Actually, the function
G(x; y; s;) =
Ax−b ATy+s−c
Xs−!
is real analytic and its Jacobian with respect to (x; y; s) is nonsingular at those points where all components of both x and s are non-zero. Moreover, the central path satises
G(x(); y(); s();) = 0 (2)
for each ¿0. Thus, by the implicit function theorem, the central path is analytic in for ¿0. That is, it is innitely dierentiable and the Taylor series of (x(); s()) for any 0¿0 converges to (x(); s()) at
a neighborhood of 0. (More analytical properties of the central path for ¿0 were studied in [17,15].)
The limiting property of the !-weighted central path is that there exists a nite limit of (x(); s()) as
↓ 0, and this limit value forms a strictly complementary solution of (P) and (D). For the proof of this assertion see [9,6,8]. Therefore, we can extend the domain of the central path (as a function of ) to the closed interval [0;∞) by
(x(0); s(0)) := lim
↓0 (x(); s()): (3)
Now, Eq. (2) holds even for = 0, but the study of the analytical properties of the central path becomes more complicated since some information contained in system (1) vanishes at = 0. In fact, system (1) can have many solutions at = 0, and the Jacobian, G′(x; y; s; 0), at these solutions is singular. Consequently, the implicit function theorem does not apply to Eq. (2) at = 0. Thus eort was concentrated on the analysis of limiting properties of the kth derivatives, (x(k)(); s(k)()), as ↓ 0. The existence of such limits was
established in [1,16] for k= 1 and, in [4] for any k¿1.
Although these proofs for LP are constructive and insightful, they are rather technical and quite long. On the contrary, the proof by Stoer and Wechs [12] (see also [13]) for the monotone LCP is, surprisingly, very simple. It consists of an application of the implicit function theorem to a system of equations, the existence of which was deduced from the system of equations dening the central path. Let us note that the construction of that system is described in the follow-up paper by Stoer et al. [14].
Inspired by the approach for LCP, this paper gives two simple ways of proving the analyticity of the central path for LP. The main idea of both proof techniques is common: we rewrite the system of equations describing the central path, so that the corresponding Jacobian remains nonsingular as ↓ 0. One proof technique (Section 3) follows the ideas for LCP from [12–14] and allows geometric interpretation. The other (Section 4) uses a partition of A, rst described in [7]. Here, the system is rewritten in a dierent form and the resulting Jacobian exhibits some (skew-)symmetric properties. A common conclusion of both these proof techniques is given in Section 5.
2. Preliminaries
The main tool in developing limiting properties of the central path is the concept of an optimal partition (B; N) of (P) and (D). Recall that the optimal partition is a partition of the index set {1; : : : ; n} such that every strictly complementary optimal solution (x; s) of (P) and (D) has the following property: xi¿0; si= 0 for i∈B andxi= 0; si¿0 for i∈N. For further details of the optimal partition concept see [2], or [11].
We use the notationxBandxN to refer to the restriction of any vectorx∈Rnto the coordinate setsBandN, respectively. Correspondingly, the submatrices AB and AN are formed by taking theB and N columns of A, respectively. Thus, we can write A= (AB; AN) by reordering the columns in A, if necessary. Finally, by |B| and |N| we denote the cardinality of the index sets B and N, respectively. In this notation, we can rewrite Eqs. (1) as
ABxB+ANxN=b;
ATBy+sB=cB;
ATNy+sN=cN;
XBsB=!B;
XNsN=!N: (4)
Since (x(0); s(0)) is strictly a complementary optimal solution of (P), (D) and (B; N) is the optimal partition, we have
xB(0)¿0; xN(0) = 0;
sB(0) = 0; sN(0)¿0: (5)
We now use a technique, as used by Stoer and Wechs [12,13] in the context of LCP’s, and dene the new “tilde” variables
˜
sB:=XB−1!B ⇒sB=s˜B;
˜
xN:=SN−1!N ⇒xN=x˜N: (6) Then, instead of (xB(); xN(); sB(); sN()), we study the corresponding “tilde” path (xB();x˜N();s˜B();
sN()) for ¿0. It is easy to see that for each ¿0 the “tilde” path is the solution to the system
ATBy+s˜B=cB; (7b)
ATNy+sN=cN; (7c)
XBs˜B=!B; (7d)
˜
XNsN =!N: (7e)
More precisely, (xB();x˜N();s˜B(); sN()) is even the unique solution to (7) such thatxB()¿0; sN()¿0. Moreover, by (5) and (6), we have ˜sB(0)¿0 and ˜xN(0)¿0.
The Jacobian of (7) is
J(xB; y; sN;x˜N;s˜B;) =
AB 0 0 AN 0
0 ATB 0 0 IN
0 ATN IN 0 0
˜
SB 0 0 0 XB
0 0 X˜N SN 0
;
and its values along the “tilde” path are denoted by J(), i.e. J() :=J(xB(); y(); sN();x˜N();s˜B();) for ¿0. It is easy to see that J() is nonsingular for ¿0, but, at = 0, its nonsingularity depends on the
m× |B| matrix AB. In fact, J(0) is nonsingular if and only if
AB 0 0 ATB
is nonsingular, which is equivalent
to the two conditions |B|=m; rank(AB) =m. Note that these conditions are also equivalent to the uniqueness of the optimal solutions of (P) and (D) (see [5,10]). The case of |B| 6=m or rank(AB)6=m is treated in the next two sections.
3. The rst approach
As mentioned above, Stoer and Wechs proved the analyticity of the central path for the monotone LCP in [12] and for the sucient LCP in [13]. The main idea of their proof resides in adding some equations to the original system of equations describing the central path. Then, the enlarged (overdetermined) system of equations has a Jacobian with full column rank, and the existence of a subsystem with a nonsingular Jacobian is ensured. Thus, the analyticity follows by an application of the implicit function theorem. The study of this problem was completed in [14], where the authors describe explicitly how to select a subsystem from the enlarged system with a nonsingular Jacobian at = 0.
Denote
Then (7a) and (7b) can be rewritten as
P xB
Multiplying (8) by W on the left we obtain
W1P
is a property of the Jacobian J() of original system (7). The next theorem summarizes these results and establishes the nonsingularity of JI(0).
ATNdy+ dsN = 0; (12c) ˜
SB(0) dxB+XB(0) d ˜sB= 0; (12d)
˜
XN(0) dsN+SN(0) d ˜xN= 0: (12e)
First, we consider (12a). Since W2P= 0, we have WP
dxB dy
!
= 0 and due to the nonsingularity of W we
obtain
P dxB
dy
!
= 0: (13a)
Further, we consider (12b). According to (9) applied to (12b), there exist u1 andu2 such that
P u1 u2
!
+Q d ˜xN
d ˜sB !
= 0: (13b)
Moreover, setting u3=−ATNu2, we have
ATNu2+u3= 0: (14)
From (13a) and (13b) we obtain
P u1+dxB u2+dy
!
+Q d ˜xN
d ˜sB !
= 0 for all∈R: (15)
Thus, from (15), (12c) and (14) it follows that
AB(u1+dxB) +AN d ˜xN = 0;
ATB(u2+dy) + d ˜sB= 0;
ATN(u2+dy) + (u3+dsN) = 0 for all∈R: (16)
Since the vectors
u1+dxB d ˜xN
!
and d ˜sB
u3+dsN !
are orthogonal we have
uT1d ˜sB+ d ˜xTNu3+(dxTB d ˜sB+ d ˜xTN dsN) = 0 for all∈R which implies
dxTB d ˜sB+ d ˜xTN dsN= 0: (17)
From (12d) and (12e) we now obtain
d ˜sB=−XB(0)−1S˜B(0) dxB (18a)
and
dsN=−X˜N(0)−1SN(0) d ˜xN: (18b)
Substituting (18a) and (18b) into (17) yields
Finally, by (12c) and dsN = 0, we getATNdy= 0 and from (13a) we obtain ATBdy= 0. Now, the full row rank of A yields dy= 0. Thus dh= 0 and the theorem is proved.
The results of this section are closely related to the geometric interpretation of the central path as described by Adler and Monteiro [1] (see also [3]). By this interpretation the (!-weighted) central path is, at any¿0, the (!-weighted) analytic center of a certain set P(), where P(0) is the set of optimal solutions for (P) and (D). In our case, we dene P(); ¿0, as the set of variables xB¿0; y; sN¿0 satisfying (10a) and (7c) where ˜xN = ˜xN() and ˜sB= ˜sB() are xed. Then it is straightforward to verify that Eqs. (10a), (11) and (7cde) represent necessary and sucient conditions describing the (!-weighted) analytic center of P() for all ¿0.
Since the analytic center ofP(); ¿0, is dened uniquely, the above interpretation implies the uniqueness of the (positive) solutions of (10a), (11) and (7cde) for all ¿0.
4. The second approach
We now present an alternative procedure that yields a dierent algebraic description for the central path with non-vanishing Jacobian. First we note that, prior to this section, the symbol := was used to dene the object on its left-hand side. In this section we also use the symbol =: to dene the object on the right.
Let rank(AB) =:r. From the rank conditions on A and AB one has that A can be partitioned in the form (by reordering the columns in AB or in AN, if necessary)
A= [AB ...AN] =:
is a nonsingular m×mmatrix. Multiplying A by R−1 on the left, we obtain
Substituting (20) into (21) – (23) yields
xB1+ AB12xB2+AN12x˜N2= b1; (28)
Eqs. (30) and (32) can be written in the form
y1=cB1−s˜B1; (30′)
y2=cN1−sN1 (32′)
Now (28), (31′′), (29′), (33′) together with (24) – (27) form the full system of 2n equations for the 2n
The values of the above Jacobian along the “tilde” path are denoted by JII(). Similarly as in the rst
approach, the nonsingularity of JII() is evident for ¿0.
Theorem 2. For any¿0the“tilde”path(xB();x˜N();s˜B(); sN())is a solution to the system(28); (29′); (31′′); (33′); (24)–(27) of 2n equations; and the nonsingularity of the Jacobian J
II() extends to = 0.
Proof. To prove the nonsingularity of JII() at = 0 we use the following formula for the determinant of
where the last relation holds because XB1; XB2; XB−21S˜B2 are positive denite and A T
B12XB−11S˜B1AB12 is positive
semidenite. Analogously, det(M4(0))6= 0 and so det(JII(0))6= 0.
5. Conclusion
The results of Sections 3 and 4 imply the analyticity of the central path. In fact, the variables xB;x˜N; y;s˜B; sN and enter into both systems (described in Theorems 1 and 2) analytically and thus the application of the analytic version of the implicit function theorem yields the analyticity of the “tilde” path as the function of
at any ¿0. Since xN() =x˜N(), sB() =s˜B(), the analyticity ofxN() and sB() is evident. For some applications it is often important to consider the central path (i.e. the solution to (1)) not only as a function of the parameter ¿0 but also as a function of the weight vector ! ¿0. Since !enters into the systems described in Theorems 1 and 2 analytically, we immediately obtain the analyticity of (x(; !); s(; !)) at any ¿0 and ! ¿0. Obviously, the analyticity of the central path implies some weaker results on the limiting properties of its derivatives and these limiting properties (together with the analyticity of the central path at !) have applications to the analysis and construction of polynomial-time algorithms which are also locally superlinearly or quadratically convergent. (For more details see e.g. [14].)
The proofs of analyticity of the central path for LP presented in this paper are simpler, not only than the proofs of analyticity from [7,15], but even than the proofs of the weaker results on the limiting properties of derivatives from [1,16,4]. For this reason, the proofs presented here can be useful both as a teaching and as a research tool.
Acknowledgements
The author wishes to thank Milan Hamala for many stimulating discussions on the subject of this paper. Thanks also to Pavol Brunovsky, Joseph Gruendler and two anonymous referees for their comments and suggestions which resulted in the improvement of the readability of this paper. This work was supported in part by VEGA grants 1/4302/97 and 1/7675/20.
References
[1] I. Adler, R.D.C. Monteiro, Limiting behavior of the ane scaling continuous trajectories for linear programming problems, Math. Programming 50 (1991) 29–51.
[2] A. Goldman, A. Tucker, Theory of linear programming, in: H. Kuhn, A. Tucker (Eds.), Linear Inequalities and Related Systems, Vol. 38, Princeton University Press, Princeton, NJ, 1956, pp. 53–97.
[3] C. Gonzaga, Path-following methods for linear programming, SIAM Rev. 34 (2) (1992) 167–224.
[4] O. Guler, Limiting behavior of weighted central paths in linear programming, Math. Programming 65 (1994) 347–363.
[5] O. Guler, D. den Hertog, C. Roos, T. Terlaky, T. Tsuchiya, Degeneracy in interior point methods for linear programming: A survey, Ann. Oper. Res. 46 (1993) 107–138.
[6] O. Guler, C. Roos, T. Terlaky, J.-Ph. Vial, A survey of the implications of the behavior of the central path for the duality theory of linear programming, Management Sci. 41 (1995) 1922–1934.
[7] M. Halicka, Analytical properties of the central path at boundary point in linear programming, Math. Programming 84 (1999) 335–355.
[8] B. Jansen, C. Roos, T. Terlaky, J.P. Vial, Interior-point methodology for linear programming: duality, sensitivity analysis and computational aspects, K. Frauendorfer, H. Glavitsch, R. Bacher (Eds.), Optimization in Planning and Operation of Electric Power Systems, Lecture Notes of the SVOR/ASRO Tutorial, Thun, October 14 –16, 1992, Physica–Verlag Heidelberg, 1993, pp. 57–123. [9] L. McLinden, An analogue of Moreau’s proximation theorem, with applications to the nonlinear complementarity problem, Pacic
Journal of Mathematics 88 (1) (1980) 101–161.
[11] C. Roos, J.-Ph. Vial, Interior point methods, J.E. Beasley (Ed.), Advances in Linear and Integer Programming, Oxford Lecture Series in Mathematics and its Applications, Vol. 4, Oxford University Press, Oxford, Great Britain, 1996, pp. 47–102.
[12] J. Stoer, M. Wechs, On the analyticity properties of infeasible-interior point paths for monotone linear complementarity problems, Numer. Math. 81 (1999) 631–645.
[13] J. Stoer, M. Wechs, Infeasible-interior-point paths for sucient linear complementarity problems and their analyticity, Math. Programming 83 (1998) 403–423.
[14] J. Stoer, M. Wechs, S. Mizuno, High order infeasible-interior-point methods for solving sucient linear complementarity problems, Math. Oper. Res. 23 (1998) 832–862.
[15] M. Wechs, The analyticity of interior-point-paths at strictly complementary solutions of linear programs, Optimization, Methods and Software 9 (1998) 209–243.
[16] C. Witzgall, P.T. Boggs, P.D. Domich, On the convergence behavior of trajectories for linear programming, Contemporary Mathematics 114 (1990) 161–187.
