www.elsevier.com/locate/orms

## On the convergence of the block nonlinear Gauss–Seidel method

## under convex constraints

### (

### L. Grippo

a_{, M. Sciandrone}

b;∗
a_{Dipartimento di Informatica e Sistemistica, Universita di Roma “La Sapienza”, Via Buonarroti 12-00185 Roma, Italy}
b_{Istituto di Analisi dei Sistemi ed Informatica del CNR, Viale Manzoni 30-00185 Roma, Italy}

Received 1 August 1998; received in revised form 1 September 1999

Abstract

We give new convergence results for the block Gauss–Seidel method for problems where the feasible set is the Cartesian product ofmclosed convex sets, under the assumption that the sequence generated by the method has limit points. We show that the method is globally convergent form= 2 and that form ¿2 convergence can be established both when the objective functionfis componentwise strictly quasiconvex with respect tom−2 components and whenfis pseudoconvex. Finally, we consider a proximal point modication of the method and we state convergence results without any convexity assumption on the objective function. c2000 Elsevier Science B.V. All rights reserved.

Keywords:Nonlinear programming; Algorithms; Decomposition methods; Gauss–Seidel method

1. Introduction

Consider the problem

minimize f(x)

subject to x∈X=X1×X2× · · · ×Xm⊆Rn;

(1)

where f : Rn _{→} _{R} _{is a continuously dierentiable}

function and the feasible setX is the Cartesian prod-uct of closed, nonempty and convex subsetsXi⊆Rni,

for i= 1; : : : ; m, withPm_{i}_{=1}ni=n. If the vectorx ∈

Rn _{is partitioned into}_{m}_{component vectors}_{x}
i ∈Rni,

(_{This research was partially supported by Agenzia Spaziale}
Italiana, Roma, Italy.

∗_{Corresponding author. Fax: +39-06 7716461.}
E-mail address:sciandro@iasi.rm.cnr.it (M. Sciandrone)

then the minimization version of theblock-nonlinear

Gauss–Seidel(GS)method for the solution of (1) is

dened by the iteration:

x_{i}k+1= arg min

yi∈Xi

f(x_{1}k+1; : : : ; x_{i}k−+11; yi; xik+1; : : : ; xmk);

which updates in turn the components ofx, starting
from a given initial point x0 ∈ X and generates a
sequence{xk_{}}_{with}_{x}k_{= (}_{x}k

1; : : : ; xmk).

It is known that, in general, the GS method may not converge, in the sense that it may produce a sequence with limit points that are not critical points of the problem.

Some well-known examples of this behavior have
been described by Powell [12], with reference to the
coordinate method for unconstrained problems, that is
to the casem=nandX =Rn_{.}

Convergence results for the block GS method have been given under suitable convexity assumptions, both in the unconstrained and in the constrained case, in a number of works (see e.g. [1,3,5,6,9 –11,13,15]).

In the present paper, by extending some of the
re-sults established in the unconstrained case, we prove
new convergence results of the GS method when
ap-plied to constrained problems, under the assumptions
that the GS method is well dened (in the sense that
every subproblem has an optimal solution) and that
the sequence{xk_{}}_{admits limit points.}

More specically, rst we derive some general properties of the limit points of the partial updates generated by the GS method and we show that each of these points is a critical point at least with re-spect to two consecutive components in the given ordering. This is shown by proving that the global minimum value in a component subspace is lower than the function value obtainable through a conver-gent Armijo-type line search along a suitably dened feasible direction. As a consequence of these re-sults, we get a simple proof of the fact that in case of a two block decomposition every limit point of

{xk} is a critical point of problem (1), even in the

absence of any convexity assumption on f. As an example, we illustrate an application of the two-block GS method to the computation of critical points of nonconvex quadratic programming problems via the solution of a sequence of convex programming subproblems.

Then we consider the convergence properties of the GS method for the general case of am-block decom-position under generalized convexity assumptions on the objective function. We show that the limit points of the sequence generated by the GS method are crit-ical points of the constrained problem both when (i)

fis componentwise strictly quasiconvex with respect tom−2 blocks and when (ii)fis pseudoconvex and has bounded level sets in the feasible region.

In case (i) we get a generalization of well known convergence results [10,5]; in case (ii) we extend to the constrained case the results given in [14] for the cyclic coordinate method and in [6] for the uncon-strained block GS method. Using a conuncon-strained version of a Powell’s counterexample, we show also that non-convergence of the GS method can be demonstrated for nonconvex functions, whenm¿3 and the preced-ing assumptions are not satised.

Finally, in the general case of arbitrary decompo-sition, we extend a result of [1], by showing that the limit points of the sequence generated by a proximal point modication of the GS method are critical points of the constrained problem, without any convexity as-sumption on the objective function.

Notation. We suppose that the vector x ∈ Rn _{is}

partitioned into component vectors xi ∈ Rni, as

x= (x1; x2; : : : ; xm). In correspondence to this

par-tition, the function value f(x) is also indicated by

f(x1; x2; : : : ; xm) and, for i = 1;2; : : : ; m the partial

both xandyare partitioned into component vectors, it is easily seen that x ∈ X is a critical point for Problem (1) if and only if for alli= 1; : : : ; mwe have:

3_{i}f( x)T_{(}_{y}

i−xi)¿0 for everyyi∈Xi.

We denote by L0

X the level set of f relative to

X, corresponding to a given point x0 ∈ X, that is

L0

X:={x∈X:f(x)6f(x0)}. Finally, we indicate by

k · kthe Euclidean norm (on the appropriate space).

2. A line search algorithm

In this section we recall some well-known proper-ties of an Armijo-type line search algorithm along a feasible direction, which will be used in the sequel in our convergence proofs.

Let{zk_{}}_{be a given sequence in}_{X}_{, and suppose that}

zk is partitioned aszk= (z_{1}k; : : : ; zmk), withzik ∈Xifor

i= 1; : : : ; m. Let us choose an indexi∈ {1; : : : ; m}and

assume that for allkwe can compute search directions

dik=wik−zik withwik∈Xi; (2)

such that the following assumption holds.

Assumption 1. Let {dk

i} be the sequence of search

directions dened by (2). Then:

(i) there exists a numberM ¿0 such thatkdk ik6M

for allk;

(ii) we have3_{i}f(zk_{)}T_{d}k

An Armijo-type line search algorithm can be

de-In the next proposition we state some well-known
properties of Algorithm LS. It is important to observe
that, in what follows, we assume that{zk_{}}_{is a given}

sequence that may not depend on Algorithm LS, in the
sense thatz_{i}k+1 may not be the result of a line search
along dk

i. However, this has no substantial eect in

the convergence proof, which can be deduced easily from the known results (see e.g. [3]).

Proposition 1. Let {zk_{}} _{be a sequence of points in}

X and let{dk

i}be a sequence of directions such that

Assumption 1 is satised. Let k

i be computed by

means of AlgorithmLS.Then:

(i) there exists a nite integer j such thatk i = (i)j

satises the acceptability condition(3);
(ii) if{zk_{}}_{converges to}_{z}_{}_{and}_{:}

In this section we derive some properties of the GS method that are at the basis of some of our conver-gence results. First, we state them-block GS method in the following form:

Unless otherwise specied, we assume in the sequel that the updating rule (6) is well dened, and hence that every subproblem has solutions.

We consider, for allk, the partial updates introduced by the GS method by dening the following vectors belonging toX:

w(k;0) =xk;

w(k; i) = (x_{1}k+1; : : : ; x_{i}k−+11; xik+1; xik+1; : : : ; xmk)

i= 1; : : : ; m−1;

w(k; m) =xk+1:

For convenience we set also

w(k; m+ 1) =w(k+ 1;1):

By construction, for each i ∈ {1; : : : ; m}, it follows from (6) that w(k; i) is the constrained global mini-mizer offin theith component subspace, and there-fore it satises the necessary optimality condition:

3_{i}_{f}_{(}_{w}_{(}_{k; i}_{))}T_{(}_{y}

i−xik+1)¿0 for everyyi∈Xi:

(7)

We can state the following propositions.

Proposition 2. Suppose that for somei∈ {0; : : : ; m}

instructions of the algorithm we have

f(w(k+ 1; i))6f(w(k; i)): (8)

Then, the continuity of f and the convergence of

{w(k; i)}Kimply that the sequence{f(w(k; i))}has a

subsequence converging tof( w). As {f(w(k; i))} is nonincreasing, this, in turn, implies that{f(w(k; i))}

is bounded from below and converges to f( w). Then, the assertion follows immediately from the fact that

f(w(k+ 1; i))6f(w(k+ 1; j))6f(w(k; i))

and

f(w(k+ 2; i))6f(w(k+ 1; j))6f(w(k+ 1; i))

for i ¡ j6m:

Proposition 3. Suppose that for somei∈ {1; : : : ; m} the sequence{w(k; i)} admits a limit pointw.Then we have

3_{i}_{f}_{( }_{w}_{)}T_{(}_{y}

i−wi)¿0 for everyyi∈Xi (9)

and moreover

3_{i}∗f( w)T(y_{i}∗−w_{i}∗)¿0 for everyy_{i}∗ ∈X_{i}∗; (10)

wherei∗_{=}

i(modm) + 1.

Proof. Let {w(k; i)}K be a subsequence converging

to w. From (7), taking into account the continuity
assumption on3_{i}f, we get immediately (9).

In order to prove (10), suppose rsti∈ {1; : : : ; m−

1}, so thati∗

=i+ 1. Reasoning by contradiction, let
us assume that there exists a vector ˜y_{i}_{+1}∈Xi+1such

that

3_{i}+1f( w)T( ˜y_{i}_{+1}−wi+1)¡0: (11)

Then, letting

d_{i}k_{+1}= ˜y_{i}_{+1}−w(k; i)i+1= ˜yi+1−xik+1

as{w(k; i)}Kis convergent, we have that the sequence

{dk

i+1}K is bounded. Recalling (11) and taking into

account the continuity assumption on3_{i}_{f} _{it follows}

that there exists a subsetK1⊆K such that

3_{i}_{+1f}_{(}_{w}_{(}_{k; i}_{))}T_{d}k

i+1¡0 for allk∈K1;

and therefore the sequences{w(k; i)}K1 and{d k i+1}K1

are such that Assumption 1 holds, provided that we
identify{zk_{}}_{with}_{{w}_{(}_{k; i}_{)}_{}}

K1.

Now, for allk∈K1suppose that we computeik+1

by means of Algorithm LS; then we have

f(x_{1}k+1; : : : ; x_{i}k+1; x_{i}k_{+1}+_{i}k_{+1}d_{i}k_{+1}; : : : ; x_{m}k)6f(w(k; i)):

Moreover, asxk

i+1 ∈Xi+1,xik+1+dik+1 ∈Xi+1,ik+1 ∈

(0;1], andXi+1is convex, it follows that

x_{i}k_{+1}+_{i}k_{+1}d_{i}k_{+1}∈Xi+1:

Therefore, recalling that

f(w(k; i+ 1)) = min

yi+1∈Xi+1

f(x_{1}k+1; : : : ; x_{i}k+1; yi+1; : : : ; xmk);

we can write

f(w(k; i+ 1))

6f(x_{1}k+1; : : : ; x_{i}k+1; x_{i}k_{+1}+_{i}k_{+1}d_{i}k_{+1}; : : : ; x_{m}k)

6f(w(k; i)): (12)

By Proposition 2 we have that the sequences

{f(w(k; j))}are convergent to a unique limit for all

j∈ {0: : : : ; m}, and hence we obtain

lim

k→∞; k∈K1

f(w(k; i))−f(x_{1}k+1; : : : ; x_{i}k+1; x_{i}k_{+1}

+_{i}k_{+1}d_{i}k_{+1}; : : : ; x_{m}k) = 0:

Then, invoking Proposition 1, where we identify{zk_{}}

with{w(k; i)}K1, it follows that

3_{i}_{+1f}_{( }_{w}_{)}T_{( ˜}_{y}

i+1−wi+1) = 0;

which contradicts (11), so that we have proved that (10) holds wheni ∈ {1; : : : ; m−1}. Wheni=m, so thati∗

= 1, we can repeat the same reasonings noting thatw(k; m+ 1) =w(k+ 1;1).

The preceding result implies, in particular, that
ev-ery limit point of the sequence{xk_{}}_{generated by the}

GS method is a critical point with respect to the com-ponents x1 andxm in the prexed ordering. This is

formally stated below.

Corollary 1. Let{xk_{}}_{be the sequence generated by}

theGSmethod and suppose that there exists a limit

pointx.Then we have

31f( x)T(y1−x1 )¿0 for every y1∈X1 (13)

and

3_{m}f( x)T(ym−xm)¿0: for every ym∈Xm: (14)

4. The two-block GS method

Let us consider the problem:

minimize f(x) =f(x1; x2);

x∈X1×X2 (15)

given a function of the form

if we decompose the problem variables into the two blocksx1 and (x2; : : : ; xN), then oncex1 is xed, the

objective function can be minimized in parallel with respect to the componentsxifori= 2; : : : ; N.

Whenm=2, a convergence proof for the GS method (2Block GS method) in the unconstrained case was given in [6]. Here the extension to the constrained case is an immediate consequence of Corollary 1.

Corollary 2. Suppose that the sequence{xk_{}}_{}

gener-ated by the2Block GSmethod has limit points. Then;
every limit pointxof{xk_{}}_{is a critical point of }

Prob-lem(15).

As an application of the preceding result we con-sider the problem of determining a critical point of a nonlinear programming problem where the objec-tive function is a nonconvex quadratic function and we have disjoint constraints on two dierent blocks of variables. In some of these cases the use of the two-block GS method may allow us to determine a critical point via the solution of a sequence of convex programming problems of a special structure and this may constitute a basic step in the context of cutting plane or branch and bound techniques for the compu-tation of a global optimum.

As a rst example, we consider a bilinear program-ming problem with disjoint constraints and we reob-tain a slightly improved version of a result already established in [7] using dierent reasonings.

Consider a bilinear programming problem of the form:

lems of this form can be obtained, for instance, as an equivalent reformulation on an extended space of concave quadratic programming problems.

Suppose that the following assumptions are satis-ed:

(i) the setsX1={x1 ∈Rn1:A1x1=b1; x1¿0}and

X2={x2∈Rn2_{:}_{A2x2}_{=}_{b2; x2}_{¿}_{0}_{}}_{are non empty;}

(ii) f(x1; x2) is bounded below onX=X1×X2.

Note that we do not assume, as in [7] thatX1andX2are

bounded. Starting from a given point (x0

1; x02)∈X, the

two-block GS method consists in solving a sequence of two linear programming subproblems. In fact, given (xk

Under the assumption stated, it is easily seen that prob-lems (17) and (18) have optimal solutions and hence that the two block GS method is well dened. In fact, reasoning by contradiction, assume that one subprob-lem, say (17), does not admit an optimal solution. As the feasible setX1 is nonempty, this implies that the

objective function is unbounded from below on X1.

Thus there exists a sequence of pointszj _{∈} _{X1} _{such}

that

But this would contradict assumption (ii), since
(zj; x_{2}k) is feasible for allj.

We can also assume thatx_{1}kandx_{2}kare vertex
solu-tions, so that the sequence{(x_{1}k; x_{2}k)}remains in a
-nite set. Then, it follows from Corollary 2 that the two
block-GS method must determine in a nite number
of steps a critical point of problem (16).

As a second example, let us consider a (possibly nonconvex) problem of the form

minimize f(x1; x2)= 1_{2}xT_{1}Ax1+1_{2}xT_{2}Bx2+xT_{1}Qx2

Suppose that one of the following assumptions is veried:

(i) X1andX2 are bounded;

(ii) Ais positive denite andX2is bounded. Under either one of these assumptions, it is easily seen that the level setL0

X is compact for every (x01; x02)∈
X1×X2. This implies that the two-block GS method
is well dened and that the sequence{xk_{}} _{has limit}

points. Then, again by Corollary 2, we have that ev-ery limit point of this sequence is a critical point of problem (19).

5. The block GS method under generalized convexity assumptions

In this section we analyze the convergence proper-ties of the block nonlinear Gauss–Seidel method in the case of arbitrary decomposition. In particular, we show that in this case the global convergence of the method can be ensured assuming the strict componen-twise quasiconvexity of the objective function with respect to m−2 components, or assuming that the objective function is pseudoconvex and has bounded level sets.

We state formally the notion of strict componen-twise quasiconvexity that follows immediately from a known denition of strict quasiconvexity [10], which sometimes is called also strong quasicon-vexity [2].

Denition 1. Let i ∈ {1; : : : ; m}; we say that f is strictly quasiconvex with respect to xi ∈ Xi on X if

for everyx∈X andyi∈Xiwithyi6=xiwe have

f(x1; : : : ; txi+ (1−t)yi; : : : ; xm)

¡max{f(x); f(x1; : : : ; yi; : : : ; xm)}

for allt∈(0;1):

We can establish the following proposition, whose proof requires only minor adaptations of the arguments used, for instance, in [10,5].

Proposition 4. Suppose that f is a strictly quasicon-vex function with respect toxi∈Xion X in the sense

of Denition1.Let{yk_{}}_{be a sequence of points in}

X converging to somey ∈ X and let {vk_{}} _{be a }

se-quence of vectors whose components are dened as

follows:

v_{j}k=

y_{j}k if j6=i;

arg min_{}∈Xif(y
k

1; : : : ; ; : : : ; ymk) if j=i:

Then; if limk→∞f(yk) − f(vk) = 0; we have
limk→∞kv_{i}k−y_{i}kk= 0.

Then, we can state the following proposition.

Proposition 5. Suppose that the function f is strictly

quasiconvex with respect to xi on X; for each i=

1; : : : ; m−2in the sense of Denition1and that the

sequence{xk_{}}_{generated by the}_{GS}_{method has limit}

points. Then;every limit pointxof{xk_{}}_{is a critical}

point of Problem(1).

Proof. Let us assume that there exists a subsequence

{xk_{}}

K converging to a point x∈X. From Corollary

1 we get

3_{m}f( x)T(ym−xm)¿0 for every ym∈Xm: (20)

Recalling Proposition 2 we can write

limk→∞f(w(k; i))−f(xk) = 0 fori= 1; : : : ; m:

Using the strict quasiconvexity assumption on f

and invoking Proposition 4, where we identify{yk_{}}

with {xk_{}}

K and {vk} with {w(k;1)}K, we obtain

limk→∞; k∈Kw(k;1) = x. By repeated application of

Proposition 4 to the sequences {w(k; i−1)}K and

{w(k; i)}K, fori= 1; : : : ; m−2, we obtain

lim

k→∞; k∈K w(k; i) = x for i= 1; : : : ; m−2:

Then, Proposition 3 implies

3_{i}_{f}_{( }_{x}_{)}T_{(}_{y}

i−xi)¿0 for every yi∈Xi;

i= 1; : : : ; m−1: (21)

Hence, the assertion follows from (20) and (21).

In the next proposition we consider the case of a pseudoconvex objective function.

Proposition 6. Suppose that f is pseudoconvex on X

and that L0

X is compact. Then; the sequence {xk}

generated by the GS method has limit points and

every limit pointxof{xk_{}}_{is a global minimizer of f}_{.}

Proof. Consider the partial updatesw(k; i), with i=

we havef(xk+1_{)}_{6}_{f}_{(}_{w}_{(}_{k; i}_{))}_{6}_{f}_{(}_{x}k_{) for}_{i}_{=0}_{; : : : ; m}_{.}

By Proposition 2 we have

f( x) =f( wi) for i= 1; : : : ; m: (28)

Obviously, (32) holds if‘=j(see (30)). Therefore, let us assume‘6=j. By (25) – (27) we have

let us consider the feasible point

z() = wj−1_{+}_{d}_{(}_{}_{)}_{;}

wheredh() = 0 forh6=‘andd‘() =∈Rn‘. Then,

from (29) and (31), observing that (33) and the fact that‘6=jimply

It follows by the pseudoconvexity offthat

f(z())¿f( wj) for all∈Rn‘ _{such that}

which, recalling the denition ofz(), implies (32). Finally, taking into account (30), and using the fact that (31) implies (32), by induction we obtain

3_{j}_{f}( w0)T_{(}_{y}

j−w0j) =3jf( x)T(yj−xj)¿0

for allyj ∈Xj:

Since this is true for everyj∈ {1; : : : ; m}, the thesis is proved.

As an example, let us consider a quadratic program-ming problem with disjoint constraints, of the form

minimize f(x) =

(ii) the matrix

is symmetric and semidenite positive.

In this case the function f is convex and the level setL0

X is compact for every feasible x0, but the

ob-jective function may be not componentwise strictly quasiconvex. In spite of this, them-block GS method is well dened and, as a result of Proposition 6, we can assert that it converges to an optimal solution of the constrained problem, through the solution of a se-quence of convex quadratic programming subprob-lems of smaller dimension.

6. A counterexample

In this section we consider a constrained version of a well-known counterexample due to Powell [12] which indicates that the results given in the preceding sections are tight in some sense. In fact, this exam-ple shows that the GS method may cycle indenitely without converging to a critical point if the numberm

of blocks is equal to 3 and the objectivefis a noncon-vex function, which is componentwise connoncon-vex but not strictly quasiconvex with respect to each component. The original counterexample of Powell consists in the unconstrained minimization of the function

f:R3_{→}_{R}_{, dened by}

Powell showed that, if the starting pointx0is the point (−1−;1 +1

2;−1− 1

4) the steps of the GS method

“tend to cycle round six edges of the cube whose vertices are (±1;±1;±1)”, which are not stationary points of f. It can be easily veried that the level sets of the objective function (35) are not compact; in fact, settingx2=x3=x1 we have thatf(x)→ −∞

askxk → ∞. However, the same behavior evidenced by Powell is obtained if we consider a constrained

problem with the same objective function (35) and a compact feasible set, dened by the box constraints

−M6xi6M i= 1; : : : ;3

withM ¿0 suciently large.

In accordance with the results of Section 3, we may note that the limit points of the partial updates gen-erated by the GS method are such that two gradient components are zero.

Nonconvergence is due to the fact that the limit
points associated to consecutive partial updates are
distinct because of the fact that the function is not
com-ponentwise strictly quasiconvex; on the other hand, as
the function is not pseudoconvex, the limit points of
the sequence{xk_{}}_{are not critical points.}

Note that in the particular case ofm= 3, by Propo-sition 5 we can ensure convergence by requiring only the strict quasiconvexity offwith respect to one com-ponent.

7. A proximal point modication of the GS method

In the preceding sections we have shown that the global convergence of the GS method can be ensured either under suitable convexity assumptions on the ob-jective function or in the special case of a two-block decomposition.

Now, for the general case of nonconvex objective function and arbitrary decomposition, we consider a proximal point modication of the Gauss–Seidel method. Proximal point versions of the GS method have been already considered in the literature (see e.g. [1,3,4]), but only under convexity assumptions onf. Here we show that these assumptions are not required if we are interested only in critical points. The algorithm can be described as follows.

The convergence properties of the method can be established by employing essentially the same argu-ments used in [1] in the convex cases and we can state the following proposition, whose proof is included here for completeness.

Proposition 7. Suppose that thePGSmethod is well
dened and that the sequence{xk_{}} _{has limit points.}

Then every limit pointxof{xk_{}}_{is a critical point of}

Problem(1).

Proof. Let us assume that there exists a subsequence

{xk_{}}

Kconverging to a point x∈X. Dene the vectors

˜

w(k;0) =xk;

˜

w(k; i) = (x_{1}k+1; : : : ; xik+1; xik+1; : : : ; xmk)

for i= 1; : : : ; m:

Then we have

f( ˜w(k; i))

6f( ˜w(k; i−1))−1

2ikw˜(k; i)−w˜(k; i−1)k 2

;

(37)

from which it follows

f(xk+1)

6f( ˜w(k; i))6f( ˜w(k; i−1))6f(xk)

for i= 1; : : : ; m: (38)

Reasoning as in Proposition 2 we obtain

limk→∞f(xk+1)−f(xk) = 0;

and hence, taking limits in (37) fork→ ∞we have

lim

k→∞k ˜

w(k; i)−w˜(k; i−1)k= 0; i= 1; : : : ; m;(39)

which implies

lim

k→∞; k∈K w˜(k; i) = x; i= 0; : : : ; m: (40)

Now, for every j∈ {1; : : : ; m}, asx_{j}k+1 is generated
according to rule (36), the point

˜

w(k; j) = (x_{1}k+1; : : : ; x_{j}k+1; : : : ; x_{m}k)

satises the optimality condition

[3_{j}_{f}_{( ˜}_{w}_{(}_{k; j}_{)) +}_{}_{j}_{( ˜}_{w}_{j}_{(}_{k; j}_{)}

−w˜j(k; j−1))]T(yj−w˜j(k; j))¿0

for allyj∈Xj:

Then, taking the limit fork → ∞; k ∈ K, recalling (39), (40) and the continuity assumption on3f, for everyj∈ {1; : : : ; m}we obtain

3_{j}_{f}_{( }_{x}_{)}T_{(}_{y}

j−xj)¿0 for allyj∈Xj;

which proves our assertion.

Taking into account the results of Section 5, it fol-lows that if the objective functionfis strictly quasi-convex with respect to some componentxi, with i∈

{1; : : : ; m}, then we can seti= 0. Moreover,

reason-ing as in the proof of Proposition 5, we can obtain the same convergence results if we setm−1=m= 0.

As an application of Proposition 7, let us consider the quadratic problem (34) of Section 5. Suppose again that the setsXi are nonempty and bounded, but

now assume thatQis an arbitrary symmetric matrix. Then the objective function will be not pseudoconvex, in general, and possibly not componentwise strictly quasiconvex. In this case, the block-GS method may not converge. However, the PGS method is well de-ned and, moreover, if we seti¿−2min(Qii), then

the subproblems are strictly convex and the sequence generated by the PGS method has limit points that are critical points of the original problem.

Acknowledgements

The authors are indebted to the anonymous reviewer for the useful suggestions and for having drawn their attention to some relevant references.

References

[1] A. Auslender, Asymptotic properties of the Fenchel dual functional and applications to decomposition problems, J. Optim. Theory Appl. 73 (3) (1992) 427–449.

[2] M.S. Bazaraa, H.D. Sherali, C.M. Shetty, in: Nonlinear Programming, Wiley, New York, 1993.

[3] D.P. Bertsekas, in: Nonlinear Programming, Athena Scientic, Belmont, MA, 1995.

[4] D.P. Bertsekas, P. Tseng, Partial proximal minimization algorithms for convex programming, SIAM J. Optim. 4 (3) (1994) 551–572.

[5] D.P. Bertsekas, J.N. Tsitsiklis, in: Parallel and Distributed Computation, Prentice-Hall, Englewood Clis, NJ, 1989. [6] L. Grippo, M. Sciandrone, Globally convergent

[7] H. Konno, A cutting plane algorithm for solving bilinear programs, Math. Programming 11 (1976) 14–27.

[8] H. Konno, Maximization of a convex quadratic function under linear constraints, Math. Programming 11 (1976) 117–127.

[9] Z.Q. Luo, P. Tseng, On the convergence of the coordinate descent method for convex dierentiable minimization, J. Optim. Theory Appl. 72 (1992) 7–35.

[10] J.M. Ortega, W.C. Rheinboldt, in: Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970.

[11] M. Patriksson, Decomposition methods for dierentiable optimization problems over Cartesian product sets, Comput. Optim. Appl. 9 (1) (1998) 5–42.

[12] M.J.D. Powell, On search directions for minimization algorithms, Math. Programming 4 (1973) 193–201. [13] P. Tseng, Decomposition algorithm for convex dierentiable

minimization, J. Optim. Theory Appl. 70 (1991) 109–135. [14] N. Zadeh, A note on the cyclic coordinate ascent method,

Management Sci. 16 (1970) 642–644.