www.elsevier.com/locate/orms
A variable target value method for nondierentiable optimization
Hanif D. Sherali
∗, Gyunghyun Choi, Cihan H. Tuncbilek
Department of Industrial and Systems Engineering, Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA
Received 1 March 1998; received in revised form 1 July 1999
Abstract
This paper presents a newVariable target value method(VTVM) that can be used in conjunction with pure or deected subgradient strategies. The proposed procedure assumes no a priori knowledge regarding bounds on the optimal value. The target values are updated iteratively whenever necessary, depending on the information obtained in the process of the algorithm. Moreover, convergence of the sequence of incumbent solution values to a near-optimum is proved using popular, practically desirable step-length rules. In addition, the method also allows a wide exibility in designing subgradient deection strategies by imposing only mild conditions on the deection parameter. Some preliminary computational results are reported on a set of standard test problems in order to demonstrate the viability of this approach. c2000 Elsevier Science B.V. All rights reserved.
Keywords:Nondierentiable optimization; Subgradient algorithm; Conjugate subgradient methods; Deected subgradient directions; Variable target value method
1. Introduction
Consider the nondierentiable optimization prob-lem
NDO: Minimize{f(x):x∈X} (1)
wheref is a convex function that is not necessarily
dierentiable, and X is a nonempty, closed, convex
subset of Rn. We assume that (1) has an optimum
x∗
and that the set of subgradients off overX is a
bounded set.
One approach to solve such problems is to use subgradient optimization methods. Here, given an
∗Corresponding author. Fax:+1-540-231-3322. E-mail address:[email protected] (H.D. Sherali)
iteratexk∈X; k¿1, a direction of motiondk is
gen-erated based on the set of subgradients offat or about
xk, and a step-lengthk is taken along this direction.
The new iteratexk+1 is then computed according to
xk+1=PX[xk+kdk], wherePX[·] denotes the
pro-jection operation onto the setX. The eectiveness of
this scheme is strongly dependent on the choice ofdk
andk. In this paper, we will focus on the more
pop-ularpureordeectedsubgradient strategies that have been spurred by Lagrangian relaxation applications in
whichdkis selected as either−gk or−gk+ kdk−1,
respectively, where gk is a subgradient of f at xk,
and where k¿0 is a suitable deection parameter
associated with the previous direction of motion, with
d0 ≡0. (See [3,6,10,15,17–19] for various
subgradi-ent deection strategies.) Note that in our analysis,
we permit the use of any deection strategy so long as
the deection parameters k¿0 are chosen such that
kdkk¡ M for allk; for some suciently
large number M: (2)
This should be easy to ensure, given our assumption
on the boundedness of the set of subgradients of f
overX. In particular, both themodied gradient
tech-nique(MGT) of Camerini et al. [3] and theaverage direction strategy(ADS) of Sherali and Ulular [18] satisfy condition (2).
As far as related step-length rules are concerned, these play an important role not only in ensuring con-vergence, but also in governing the practical rate of convergence to optimality (see [4,7,17] for example). The more eective step-length rules, popularized by Held and Karp [6] and Held et al. [7], are of the type
k=k
f(xk)−w
kdkk2
; (3)
where 0¡6k6 ¡ˆ 2, and where w is a target
value.
Often, w is taken as a xed lower bound on the
problem. On the other hand, Bazaraa and Sherali [1] present some rules to choose the target value as a convex combination of a xed lower bound and the current best objective value. A similar idea is used by Kim et al. [9] in their variable target value method for minimizing strongly convex functions. However, both these papers assume that some initial lower bound estimate is available, and moreover, Kim et al. [9] additionally assume that an upper bound on
kx1−x∗kis known, wherex∗ is an optimal solution
to NDO.
Our motivation in this paper is to present a new variable target value method (VTVM) for general nondierentiable optimization problems that adopts an eective subgradient step-length rule of type (3) and that assumes no a priori knowledge whatsoever re-garding the optimal objective value. Brannlund [2] de-scribes an alternate scheme in his dissertation for such problems, Gon and Kiwiel [5] present a convergence analysis for a variant of Brannlund’s method, and Kulikov and Fazylov [14] propose a xed, short-step variant of our algorithm. For a more extensive dis-cussion on related subgradient projection methods, we refer the reader to Kiwiel [12,13].
The remainder of this paper is organized as fol-lows. In Section 2, we present the proposed algorithm VTVM, and we analyze its convergence properties in Section 3. Section 4, provides some preliminary com-putational results using a variety of standard test prob-lems from the literature.
2. The algorithm VTVM
The proposed algorithm operates in aninner loop
and anouter loop. Theinner loopinvolves the main
process of generating a sequence of iterates.
Depend-ing on the progress durDepend-ing suchinner loopiterations,
the target value and other related parameters are
peri-odically updated in anouter loopadjustment step. The
algorithm is designed to theoretically converge in ob-jective value to within any a priori specied tolerance
¿0 of the optimal value, while preserving a
reason-able degree of computational eectiveness in practice. It accomplishes the latter by permitting the use of a variety of subgradient deection direction strategies, and by employing the practically eective step-length rule (3).
Below, we highlight our notation and then present a statement of the algorithm. The principal algorithmic parameters are described in the Initialization Step, and recommended values of these parameters are provided later in Remark 1.
Notation:
• counters:k≡total iteration counter,‘≡outer loop
iteration counter,≡current inner loop’s iteration
counter, and ≡ counter of ongoing consecutive
nonimprovements.
• For any iteration k:xk = iterate; fk = f(xk);
gk= subgradient of f atxk; dk= direction; k=
step-length; k = step-length parameter; andzk =
incumbent solution value. Also,k= accumulated
improvements within the current set of inner loop
iterations until the beginning of iterationk.
• For any outer loop‘:w‘= target value, and‘=
acceptance tolerancefor declaring that the current incumbent value is close enough to the target value
w‘.
• Optimum:x∗
∈ argmin{f(x): x∈X}, andf∗
≡
f(x∗
). Also, ˆx= incumbent solution and ˆg=
• Algorithmic parameters: = acceptance interval
parameter,= fraction of cumulative inner loop
im-provement that is used to decrease the target value,
= maximum allowable iterations in the inner loop
without coming within the acceptance tolerance of
the target value, and = maximum consecutive
non-improvements permitted within the inner loop.
Algorithm (VTVM)
Initialization. Select the step-length parameter
toler-ance 0¡61, and termination parameters 0¿0 for
the tolerance on subgradient norms, ¿0 for the
over-all convergence tolerance, andkmax6∞for the limit
on the maximum number of iterations. Select
val-ues for the algorithmic parameters ∈ (0;13]; ∈
(0;1];;, and let1∈[ ;1] (see Remark 1 below for
recommended values).
Select a starting solution x1 ∈ X, compute f1 ≡
f(x1) and letd1=−g1. If kg1k60, then stop with
x1 as the prescribed solution. Otherwise, set ˆx=x1
and ˆg=g1, and record z1=f1 as the best known
objective function value. Initialize the target value
w1= max{LB; f1− kg1k2=2}and the acceptance
tol-erance1=(f1−w1), whereLBis any known lower
bound on f∗
, being taken as−∞ if no such lower
bound is available. (Note that any reasonable value
¡ f1would suce for the second term in the
maxi-mand forw1. The stated value corresponds to the
min-imum value of a second-order approximation offat
x1with assumed gradientg1and an identity Hessian.)
Initializek=‘== 1; = 0, and1= 0.
Step 1 (Inner loop main iteration). If k ¿ kmax,
stop. Else, determinedk=−gk+ kdk−1, where k¿0
is selected via any suitable strategy so long as (2)
holds true, and whered0 ≡0. Ifkdkk60, then set
dk=−gk. Also, compute the step-length
k=k
fk−w‘
kdkk2
: (4)
Find the new iteratexk+1=PX[xk+kdk], and
deter-minefk+1 andgk+1. If kgk+1k60, terminate the
al-gorithm withxk+1 as the prescribed solution. Update
k+1=k+ max{0; zk−fk+1}. Iffk+1¡ zk, go to
Step 2(a), and otherwise, go to Step 2(b).
Step 2(a) (Improvement in the inner loop). Put
= 0; zk+1=fk+1, and update ˆx=xk+1and ˆg=gk+1.
Ifzk+16w‘+‘, then go to Step 3(a). Otherwise, if
¿, go to Step 3(b); else, setk+1=k, increment
kandby one, and return to Step 1.
Step2(b) (Nonimprovement in the inner loop). Put
zk+1=zk, and incrementby one. If¿or¿, go
to Step 3(b). Otherwise, setk+1=k, increment k
andby one, and return to Step 1.
Step3(a) (Outer loop success iteration:zk+16w‘+
‘). Compute
w‘+1=zk+1−‘−k+1
and (5)
‘+1= max{(zk+1−w‘+1); }:
(See Remark 1 below.) Put= 1; k+1= 0; k+1=k,
increment‘andkby one, and return to Step 1.
Step3(b) (Outer loop failure iteration:zk+1¿ w‘
+‘). Compute
w‘+1=
(zk+1−‘) +w‘
2
and (6)
‘+1= max{(zk+1−w‘+1); }:
(Optionally, if¿, adjust as recommended in Re-mark 1 and restart with the incumbent solution as sug-gested in Remark 2 (no restarts are to be performed after some nite number of iterations); also, adjust the
step-length parameterktok+1∈[ ;1] if necessary,
as described in Remark 2.) Put= 0; = 1; k+1= 0,
increment‘andkby one, and return to Step 1.
Remark 1. From a practical eciency point of view, to ensure adequate step-lengths during improving phases of the algorithm, we can replace the target
value update in (5) by w‘+1 =zk+1 − max{‘ +
k+1; r|zk+1|}, where 0¡ r ¡1, and wherer is
di-vided by some r ¿1 wheneverw‘+1is determined by
the second term in this maximand. (We usedr= 0:08
and r = 1:08 in our computations and this
strat-egy gave an improved computational performance.) The convergence analysis of Section 3 continues to hold with this modication. Other recommended
val-ues of the parameters are = 10−6;
0 = 10−6; ∈
[10−6;10−1] (we used= 0:1 in our computations),
kmax= 2000; 1= 0:95; = 0:15; ∈[0:75;0:95] (we
used = 0:75);= 75, and being initialized at 20
Remark 2 (Some practical considerations). Note that a restarting technique is often an important com-putational ingredient of subgradient procedures (see [1,7,18]). In the same spirit, for VTVM, whenever the target needs to be increased at Step 3(b) due
to consecutive failures, we restart the algorithm by
setting xk = ˆx; gk = ˆg, and fk =zk at the end of
Step 3(b), and then at the next visit to Step 1, we
adoptdk=−gk. (For convergence purposes, no restarts
are performed after some nite number of iterations.)
Also, if w‘+1−w‘60·1 max{1;|zk+1|}, we replace
k byk+1= max{k=2;}.
3. Convergence analysis
In this section, we establish the convergence of
Al-gorithm VTVM underkmax=∞and0= 0.
Lemma 1. Consider Algorithm VTVM; and for convenience; let us denote the target value at any
(inner) iteration k by ˆwk. Suppose that there exist
valueswandw such that
f∗
6w ¡wˆk¡w6fk for all k: (7)
Then we must have both{wˆk} →w and{fk} →w.
Proof. Let us rst show that
dtk−1(x
∗
−xk)¿0 for allk: (8)
Note that this is trivially true fork= 1, sinced0= 0.
By induction, consider any k¿2, and assume that
dt k−2(x
∗
−xk−1)¿0. Using the denition ofdk−1, the
induction hypothesis, the convexity off, Eq. (7), the
fact that k−1 ∈ [ ;1], and the Cauchy–Schwarz
in-equality, we obtain the following string of relations:
dtk−1(x
nonexpansiveness ofPX that
dtk−1(x
Hence, assertion (8) holds true. Using the denition
ofdk, we get
The last inequality holds from (8) and the denition of
a subgradient. Now, from (4) and (7), we getk(f∗−
fk)6k( ˆwk−fk) =−k(fk−wˆk)2=kdkk2. Hence,
using this in (9) along with (4) and the assumption
onk we have
ing sequence and is therefore convergent.
Consequently, we have, limk→∞(fk−wˆk)2=kdkk2=
0. From (2) and (7), this can happen only if both
{fk} → w and {wˆk} → w, and this completes the
proof.
It is interesting to note that this analysis requires
with the step-length (4) permits 1¡ k6ˆ ¡2 as
well. Now, observe that {zk} is a monotone
nonin-creasing sequence that is bounded below byf∗
, and
is hence a convergent sequence. Let zbe the limit of
this sequence, and consider the following results.
Lemma 2. For Algorithm VTVM; there exists an outer iterationLsuch that‘≡for all‘¿L.
Proof. Consider Algorithm VTVM for iterations
k¿K such that zk¡z+ for all k¿K, where
0¡ ¡ . Let us examine any outer loop‘for which
k¿K. Given w‘ and ‘, note that w‘+1 and ‘+1
for the subsequent outer loop iteration are given by either (5) or (6). In the case of (5), having
com-Similarly, in the case of (6), having computedw‘+1,
we have that
at the previous outer loop for some iterationk′
¡ k,
from (5) or (6) at the previous outer loop that for some
k′
{‘}decreases at least at a geometric rate, ultimately
becoming equal toat some nite outer loop iteration
L, after which it remains at from Case (i) above.
This completes the proof.
Lemma 3. Suppose that for Algorithm VTVM; we have z−−f∗
Proof. Following the proof of Lemma 2, let us
con-sider the algorithmic process once we havek¿Kthat
is suciently large so that zk¡z+ for all k¿K,
and that the outer loop index‘¿L, whereL is large
enough so that‘≡, for all‘¿L.
Suppose thatw‘6z−−for some outer loop‘.
Then,zk+1¿ zk+1−¿z−¿w‘+for all
corre-sponding inner loop iterationsk. Hence, the algorithm
continues to increase, and ultimately, increases the
target value at Step 3(b). By successively increasing the target value according to (6) in this fashion, we
will reach an outer iteration ˜‘such thatw‘˜¿z−−
. Moreover, since any such increase via (6) while
w‘6z−−andzk+1¡z+yields
when an improvement in objective value at Step 2(a)
causeszk+1 to fall beloww‘+‘. Therefore in this
case, we must have had ‘ = and w‘¿z−+
when for the rst time afterk+1¿K, an improvement
causedzk+1to fall below z+, and hence beloww‘+
, thereby triggering a transfer to Step 3(a). Using
(5) with‘=, the consequent decrease in the target
value yields w‘+1=zk+1−−k+1¡( z+)−
−k+1¡z−+. Incrementing ‘ by 1, either
this revisedw‘ then satises the lower bound in (14)
case, as above, we will again obtain (14) holding for
some ˜‘ after successive increases in the target value.
Therefore, we have shown thus far that for some inner
iteration ˜k¿K during an outer iteration ˜‘, we have
z−− ¡ w‘˜¡z−+: (15)
Now, it remains to show that (15) continues to hold
for all‘¿‘˜. Letj(1) be the rst iteration after ˜k at
which the target value is changed at the next outer loop, so that either the target value is decreased via (5) or it is increased via (6) to yield, respectively,
w‘˜+1= (zj(1)−)−j(1)
or (16)
w‘˜+1=
(zj(1)−) +w‘˜
2 :
For the rst case in (16), we havej(1)6j(1)¡
because fork¿Kthe objective value improves by less
than. Hence, z−+ ¿ w‘˜¿ w‘˜+1= (zj(1)−)−
j(1)¿z−−. The second case in (16) yields,
using (15) that
z−− ¡ w‘˜¡ w‘˜+1=
(zj(1)−) +w‘˜
2
¡( z+−) + ( z−+)
2 = z−+:
Hence, in either case,w‘˜+1 continues to satisfy (15).
By induction, this completes the proof.
Theorem 1. Algorithm VTVM generates a sequence
{zk} →z; wherez−6f∗.
Proof. Consider the algorithm after the nal restart has been performed. Assume on the contrary that
z − ¿ f∗
. We can therefore choose a
satisfy-ing ¿ ¿0 such that z − − f∗
¿ ¿0. By
Lemma 3, we can nd an outer iteration ˜‘ such that
w ≡ z−− ¡ w‘¡z−+ ≡ w for all‘¿‘˜.
Sincef∗
6w ¡ w‘¡w ¡ z6fkfor allk and‘
suf-ciently large, we get by Lemma 1 that{fk} → w.
But this is a contradiction because w ¡zsince ¿ ,
and so the proof is complete.
Remark 3. As evident from the proofs of Lemma 3 and Theorem 1, Algorithm VTVM can be operated
with ‘ held xed at ∀‘, and we would still have
{zk} →zwhere z−6f∗. However, from the
view-point of computational eciency (as veried by our
experiments), it is important to permit variable
accep-tance tolerances ‘, as prescribed by the stated
pro-cedure. This is so because a small, xed acceptance
tolerance of‘=can possibly lead to a sequence of
increases in the target value until the gap (fk−w‘)
becomes relatively small, thereby resulting in small step-lengths via (4), and inducing a slow progress at iterates that are as yet remote from optimality.
4. Computational experience
We now present some preliminary computational results using 15 convex test problems from the litera-ture. Table 1 gives the sizes and the sources of these problems, along with the standard, prescribed, starting solutions.
Table 2 gives the results obtained. All the algo-rithms tested were coded in C and run on an IBM
RS=6000 computer. Run 1 corresponds to
Algo-rithm VTVM with the ADS deection strategy (see
Section 1) and with a maximum limit ofkmax= 2000
iterations. Note that we have used a xed set of parameter values as prescribed in Remark 1, along with the strategies of Remarks 1 and 2, and with an
initial lower bound LB=−∞ for all the runs. Run
2 corresponds to Kim et al.’s [9] algorithm run for 2000 iterations using their recommended parameters, including a strong convexity constant of 1 as used in
their runs, and with an initial lower bound of −106
for all the problems. Run 3 is the same as Run 1, but with an additional improvement-based stopping crite-rion. Note that as stated, the algorithm is terminated
whenever the iteration count exceedskmaxor when the
norm of the current subgradient becomes suciently small. Additionally, we can terminate the algorithm based on its progress in improving the incumbent value. Hence, in Run 3, we terminate the algorithm when each of the following conditions holds. (i)
k ¿500, (ii) the algorithm has executed Step 3(a)
(outer loop success iteration) at least once, and (iii) Step 3(b) is visited four consecutive times via Step 2(b), with the average of the relative improvements
k+1=(zk+1 −w‘) over these four visits being less
than or equal to 0.05.
Table 1 Test problems
Problem n Starting solutionx1 Source(s)
1 5 (0;0;0;0;1) Shor’s Problem [11]: Test 1
2 10 (1; : : : ;1) Lemarechal and Miin [16, p. 151] (MAXQUAD)
3 50 (i−(n+ 1)=2; i= 1; : : : ; n) Gon’s polyhedral problem [11]: Test 3
4 48 (0; : : : ;0) Lemarechal and Miin [16, p. 161] (TR48)
5 48 (0; : : : ;0) Lemarechal and Miin [16, p. 165] (A48)
6 50 (0; : : : ;0) Kiwiel [11]: Test 5
7 30 (0; : : : ;0) Kiwiel [11]: Test 6
8 4 (0;0;0;0) Streit’s Problem no. 1, Kiwiel [11]: Test 8 9 4 (0;0;0;0) Streit’s Problem no. 2, Kiwiel [11]: Test 9 10 6 (0;0;0;0;0;0) Streit’s Problem no. 3, Kiwiel [11]: Test 10 11 5 (0;0;0;0;1) Hock and Schittkowski [8, p. 105]
12 4 (0;0;0;0) Hock and Schittkowski [8, p. 66]
13 4 (15;22;26;11) Chatelon et al.’s Minimax Location Problem [11]: Test 13 14 6 (0;0;0;0;0;0) Chatelon et al.’s Minimax Location Problem [11]: Test 14
15 10 (0; : : : ;0) Kiwiel [11]: Test 7
Table 2
Computational results for runs 1, 2, and 3
Problem f(x∗) Run 1: VTVM+ADS Run 2: Kim et al. (1991) Run 3: Run 1+Improvement-based stopping criterion
f(xbest) (cpu s) f(xbest) (cpu s) f(xbest) Iters
1 22.60016 22.600162 0.05 22.685788 0.04 22.600162 1995
2 −0:841408 −0:801792 0.10 −0:786540 0.09 −0:801792 1971
3 0.0 0.000002 0.06 481.154344 0.05 0.000002 1999
4 −638565 −626175:71 1.15 −560439:57 1.16 −626175:71 1071
5 −9870 −9870:00 1.12 −9546:11 1.13 −9870:00 1541
6 0.0 0.025849 0.67 1.055688 0.67 0.025849 1982
7 0.0 0.398131 0.18 0.058670 0.16 0.407191 1994
8 0.707107 0.707107 0.05 0.794324 0.04 0.707107 1992
9 1.014214 1.014214 0.04 1.014360 0.04 1.014214 1959
10 0.014706 0.128475 0.67 0.119763 0.68 0.128475 1966
11 −32:348679 −32:3254 0.04 −28:144825 0.03 −32:317630 1967
12 −44:0 −43:950579 0.04 −43:971590 0.01 −43:74894 1990
13 23.886767 24.818542 0.42 26.708094 0.39 24.818605 276
14 68.82856 68.831859 0.60 68.836753 0.63 68.832059 796
15 −0:368166 −0:339827 0.13 −0:299131 0.12 −0:339827 1980 Legend:f(x∗): optimal objective value.
f(xbest): best objective value obtained by the corresponding algorithm. cpu s: total execution time (in s) on an IBM RS=6000 computer. Iters: total number of iterations until termination occurred for Run 3.
might be acceptable in Lagrangian relaxation appli-cations, for example. Moreover, it is simple to im-plement. In contrast, the bundle method implemented in Kiwiel [11] typically yields more accurate solu-tions in signicantly fewer iterasolu-tions, although each
iteration is more complex in that it requires the solu-tion of a quadratic program.
test problems. The results for Run 3 indicate that the improvement-based stopping criterion prescribed above oers a reasonable alternative to the criterion based simply on the maximum number of iterations. Note that for most test problems, progress that is ac-ceptable to this criterion continues until close to the limit of 2000 iterations.
Finally, we comment that for Run 1, we also attempted the MGT and the pure subgradient strate-gies (see Section 1). The ADS strategy performed the same or better than the MGT (respectively, the pure subgradient) strategy on 12 (respectively, 13) out of the 15 test problems. Also, Algorithm VTVM performed the same or better on 13 out of the 15 test
problems than its variant in which‘ is held xed at
the value(see Remark 3). In addition, we attempted
to solve some larger sized, randomly generated, dual assignment test problems. For example, using test
cases of sizes 200×200; 250×250, and 300×300
having optimal values −2157;−2697;−3246,
re-spectively, Algorithm VTVM terminated within 500 iterations in each case, nding solutions of objective
values−2156:46;−2696:78, and−3245:88,
consum-ing a total of 26.8, 42.4 and 61.3 cpu s, respectively.
Acknowledgements
This material is based upon work supported by the National Science Foundation under Grant No. DMI-9521398 and DMI-9812047 and the Air Force Oce of Scientic Research under Grant No. F-49620-96-1-0274.
References
[1] M.S. Bazaraa, H.D. Sherali, On the choice of step sizes in subgradient optimization, Eur. J. Oper. Res. 7 (1981) 380–388.
[2] U.G. Brannlund, On relaxation methods for nonsmooth convex optimization, Ph.D. Thesis, Kungliga Tekniska Hogskolan, S-100 44 Stockholm, Sweden, 1993.
[3] P.M. Camerini, L. Fratta, F. Maoli, On improving relaxation methods by modied gradient techniques, Math. Programm. Study 3 (1975) 26–34.
[4] J.L. Gon, On the convergence rate of subgradient methods, Math. Programm. 13 (1977) 329–347.
[5] J.L. Gon, K.C. Kiwiel, Convergence of a simple subgradient level method, GERAD Report G-96-56, 1998, Math. Programm., to appear.
[6] M. Held, R.M. Karp, The traveling salesman problem and minimum spanning trees: part II, Math. Programm. 1 (1971) 6–25.
[7] M. Held, P. Wolfe, H.D. Crowder, Validation of subgradient optimization, Math. Programm. 6 (1974) 62–88.
[8] W. Hock, K. Schittkowski, Test examples for nonlinear programming codes, Lecture Notes in Economics and Mathematical Systems, Vol. 187, Springer, Berlin, 1981. [9] S. Kim, H. Ahn, S. Cho, Variable target value subgradient
method, Math. Programm. 49 (1991) 359–369.
[10] K.C. Kiwiel, Methods of Descent of Nondierentiable Optimization, Lecture Notes in Mathematics, Vol. 1133, Springer, Berlin, 1985.
[11] K.C. Kiwiel, Proximity control in bundle methods for convex nondierentiable minimization, Math. Programm. 46 (1990) 105–122.
[12] K.C. Kiwiel, The eciency of subgradient projection methods for convex optimization, part I: general level methods; part II: implementations and extensions, SIAM J. Control Optim. 34 (2) (1996) 660–697.
[13] K.C. Kiwiel, Subgradient method with entropic projections for convex nondierentiable minimization, J. Optim. Theory Appl. 96 (1) (1998) 159–173.
[14] A.N. Kulikov, V.R. Fazylov, Convex optimization with prescribed accuracy, USSR Comput. Maths. Math. Phys. 30(3) (1990) 16 –22. (Zh. Vychisl Mat. i Fiz. 30(5) (1990) 663– 671.)
[15] C. Lemarechal, Numerical experiments in nonsmooth optimization, in: E.A. Nurminski (Ed.), Progress in Nondierentiable Optimization, Pergamon Press, Oxford, 1982, pp. 61–84.
[16] C. Lemarechal, R. Miin, Nonsmooth optimization, Proceedings of an IIASA Workshop, March 28–April 8, 1977, Pergamon Press, Oxford, 1978.
[17] B.T. Polyak, A general method for solving extremal problems, Sov. Math. 8 (1967) 593–597.
[18] H.D. Sherali, O. Ulular, A primal-dual conjugate subgradient algorithm for specially structured linear and convex programming problems, Appl. Math. Optim. 20 (1989) 193– 221.
