One-Dimensional Unconstrained Minimization
2.2 Theory Related to Single Variable (Univariate) Minimization 47
L =
x
V (πx2/4)=πx4V2 Figure E2.1. Refrigerated tank.
We denote the total cost asf. We have f =(10) (2)
πx2 4
+(6) (πxL)+80
2.πx2
4 +πxL
=45πx2+86πxL, Substituting
L= (50) (4) πx2 = 200
πx2, we get
f =45πx2+ 17200 x
The function f(x) is the objective function to be minimized. One variable plot of the function over a sufficiently large range ofxshows the distinct characteristics.
The one variable problem may be stated as
minimizef(x) for all realx (2.1)
The pointx∗ is aweaklocal minimum if there exists aδ >0, such that f(x∗)≤ f(x) for allxsuch that|x–x∗|< δ, that is, f(x∗)≤ f(x) for allxin aδ-neighborhood ofx∗. The pointx∗ is astrong(orstrict) local minimum if f(x∗)≤ f(x) is replaced by f(x∗) < f(x) in the preceding statement. Further, x∗ is a global minimum if f(x∗)< f(x) for allx. These cases are illustrated inFig. 2.1. If a minimum does not exist, the function is not bounded below.
f(x)
x Point of
Inflection
Weak Local Minimum
Global Minimum
Strong Local Minimum
Figure 2.1. One-variable plot.
Optimality Conditions
If the functionfin Eq. (2.1)has continuous second-order derivatives everywhere (i.e., C2 continuous), the necessary conditions for x∗ to be a local minimum are given by
f(x∗)=0 (2.2)
f(x∗)≥0 (2.3)
where f(x∗) represents the derivative offevaluated atx =x∗. In deriving these conditions, we make use of Taylor’s expansion in the neighborhood of x∗. For a small real numberh>0, we use the first-order expansion atx∗+handx∗–h,
f(x∗+h)= f(x∗)+hf(x∗)+O(h2) (2.4a) f(x∗−h)= f(x∗)−hf(x∗)+O(h2) (2.4b) where the term O(h2) can be understood from the definition: O(hn)/hr → 0 as h→0 for 0≤r<n. For sufficiently smallh, the remainder term can be dropped in comparison to the linear term. Thus, for f(x∗) to be minimum, we need
f(x∗+h)− f(x∗)≈ hf(x∗)≥0 f(x∗−h)− f(x∗)≈ −hf(x∗)≥0
2.2 Theory Related to Single Variable (Univariate) Minimization 49 Since h is not zero, these inequalities are satisfied when f(x∗)=0, which is Eq. (2.2). Condition given in Eq. (2.3) can be obtained using the second-order expansion
f(x∗+h)= f(x∗)+hf(x∗)+h2
2 f(x∗)+O(h3)
At the minimum, f(x∗) is zero. Also the term h22f(x∗)dominates the remainder termO(h3). Thus,
f(x∗+h)− f(x∗)≈ h2
2 f(x∗)≥0 Sinceh2is always positive, we need f(x∗)≥0.
Points of inflection or flat regions shown in Fig. 2.1 satisfy the necessary conditions.
Thesufficient conditionsforx∗to be a strong local minimum are
f(x∗)=0 (2.5)
f(x∗)> 0 (2.6)
These follow from the definition of a strong local minimum.
Example 2.2
Find the dimensions of the minimum cost refrigeration tank of Example 2.1 using optimality conditions.
Solution
We derived the objective function
min f(x)=45πx2+17200 x f(x)=90πx−17200
x2 =0 x3 = 17200
90π =60.833 Diameter x=3.93 m
Length L= 200
πx2 =4.12 m Cost f =45πx2+17200
x =$6560 Also f(x)=90π+(3) (17200)
x3
which is strictly positive at x = 3.93 m. Thus, the solution is a strict or strong minimum.
f(x1)
f(x2) f(x)
f(x)
x1 x=αx1 +(1 − α)x2
αf(x1) +(1 − α)f(x2)
x2 Figure 2.2. Convex function.
Convexity ideas can be used in defining optimality. A setSis called aconvex set if for any two points in the set, every point on the line joining the two points is in the set. Alternatively, the setSisconvexif for every pair of pointsx1 andx2inS, and everyαsuch that 0≤α≤1, the pointαx1+(1−α)x2is inS. As an example, the set of all real numbersR1is a convex set. Any closed interval ofR1is also a convex set.
A function f(x) is called aconvex functiondefined on the convex setS if for every pair of pointsx1andx2inS, and 0≤α≤1, the following condition is satisfied f(αx1+(1−α)x2)≤αf(x1)+(1−α)f(x2) (2.7) Geometrically, this means that the graph of the function between the two points lies below the line segment joining the two points on the graph as shown in Fig. 2.2. We observe that the function defined in Example 2.1 is convex forx>0.
The functionf isstrictly convexif f(αx1+(1−α)x2)< αf(x1)+(1−α)f(x2) for 0< α <1. A functionfis said to beconcaveif−fisconvex.
Example 2.3
Prove thatf= |x|,x∈R1, is a convex function.
Using the triangular inequality|x+y| ≤ |x| + |y|, we have, for any two real num- bersx1andx2and 0< α <1,
f(αx1+(1−α)x2)=αx1+(1−α)x2≤α|x1| +(1−α)|x2|
= αf(x1)+(1−α)f(x2) which is the inequality defining a convex function in(2.7).
2.2 Theory Related to Single Variable (Univariate) Minimization 51 f(x)
f(x)
x x
y
f(x) + f ′(x)(y − x) f(y)
Figure 2.3. Convex function property.
Properties of Convex Functions
1. Iffhas continuous first derivatives thenfis convex over a convex setSif and only if for everyxandyinS,
f(y)≥ f(x)+ f(x)(y−x) (2.8) This means that the graph of the function lies above the tangent line drawn at point as shown inFig. 2.3.
2. Iffhas continuous second derivatives, then fis convex over a convex setS if and only if for everyxinS,
f(x)≥0 (2.9)
3. If f(x∗) is a local minimum for a convex functionfon a convex setS, then it is also a global minimum.
4. Iffhas continuous first derivatives on a convex setS and for a pointx∗ in S, f(x∗)(y−x∗)≥0 for everyyinS, thenx∗is a global minimum point offover S. This follows from property(2.8) and the definition of global minimum.
Thus, we have used convexity ideas in characterizing the global minimum of a func- tion. Further, the sufficiency condition (2.6) is equivalent to f being locally strictly convex.
Maximum of a Function
The problem of finding the maximum of a functionφ is the same as determining the minimum off= −φas seen inFig. 2.4. While the minimum locationx∗ is the same, the function valueφ(x∗) is obtained as −f(x∗). Whenfis positive, another transformation may be used to maximize f, namely, minimize 1/(1+ f). In this case, care must be taken as convexity aspects of the original function are entirely changed.
Example 2.4
Determine the dimensions of an open box of maximum volume that can be constructed from an A4 sheet (210 mm×297 mm) by cutting four squares of sidexfrom the corners and folding and gluing the edges as shown inFig. E2.4.
x x
x x
x
x
x x
x
297 mm
210 mm
Volume V = (297 – 2x)(210 – 2x)x
210 – 2x
297 – 2 x
Figure E2.4. Open box problem.
Solution
The problem is to
maximize V=(297−2x)(210−2x)x=62370x−1014x2+4x3 We set f = −V= −62370x+1014x2−4x3
Setting f(x)=0, we get
f(x)= −62370+2028x2−12x2=0
2.2 Theory Related to Single Variable (Univariate) Minimization 53
x∗ x
f
f = − φ φ
Figure 2.4. Minimumfat maximum ofφ.
Using the solution for the quadratic equation, x= −2028±
20282−4 (12) (62370) (2) (−12)
The two roots are
x1=40.423 mm x1=128.577 mm
A physically reasonable cut is the first rootx∗=40.423 mm. The second deriva- tivef(x∗)=2028−24x∗=1057.848.f(x∗)>0 implies thatx∗is a strict minimum off, or maximum ofV.
The dimensions of the box are Heightx∗=40.423 mm
Length=297−2x∗=216.154 mm Width=210−2x∗=129.154 mm
Maximum volume=40.423×216.154×129.154=1128495.1 mm3=1128.5 cm3. We now proceed to discuss various methods of finding the interval that brackets the minimum, called theinterval of uncertainty, and the techniques of locating the minimum.
x f
x∗
Figure 2.5. A unimodal function.