Chapter 16 Calculus Methods of Optimization

(1)

Min Soo KIM

Department of Mechanical and Aerospace Engineering Seoul National University

Optimal Design of Energy Systems

(2)

Chapter 16. Calculus Methods of Optimization

16.1 Continued exploration of calculus methods

- A major portion of this chapter deals with principles of calculus method - Substantiation for the Largrange multiplier equations will be provided

- Physical interpretation of λ, and test for maxima-minima are also provided

(3)

16.1 Continued exploration of calculus methods

- Lagrange multipliers (from Chap.8)

1 2

( , , ,

_n

) y  y x x x

1 1 2

1 2

( , , , ) 0

n

m n

x x x





1

0

m

i i

i

y  



    

1 1 2

1 2

( , , , ) 0

n

m n

x x x





n+m scalar equations

(4)

16.2 The nature of the gradient vector ( )

1. normal to the surface of constant y

1 1

ˆ

2 2

ˆ

3 3

ˆ dx i  dx i  dx i

1 2 3

y y y 0

dy dx dx dx

x x x

  

   

  

2 2 3 3

1

( / ) ( / )

/

y x dx y x dx

dx y x

    

  

 

2 2 3 3

1 2 2 3 3

1

( / ) ( / ) ˆ ˆ ˆ

/

y x dx y x dx

T i dx i dx i

y x

      

      

0 T   y

1 2 3

ˆ ˆ ˆ

y y y

y i i i

x x x

  

   

  

For arbitrary vector :

To be tangent to the surface :

Tangent vector :

Gradient vector :

→ gradient vector is normal to all tangent vectors

→ gradient vector normal to the surface

 y

(5)

Chapter 16. Calculus Methods of Optimization

2. indicate direction of maximum rate of change of y with respect to x

1 2

n n

y y y

dy dx dx dx

x x x

  

   

  

2 2 2 2

1 2

( dx )  ( dx )   ( dx

_n

)  r

constraints :

to find maximum dy :

 y

The constraint indicates a circle of radious for 2 dimensions, and a sphere for 3 dimensions

(6)

2 0 1

i i

2

i i

y y

dx dx

x  x



     

 

using Lagrange multipliers :

1 1 2 2 1 2

1 2

1 1

ˆ ˆ ˆ ˆ ˆ ˆ

2 2

n n n

n

y y y

dx i dx i dx i i i i y

x x x

 

    

            

 y

In vector form :

→ indicates the direction of maximum change for a given distance in the space

 y

(7)

Chapter 16. Calculus Methods of Optimization

3. points in the direction of increasing y

 y

small move in the x₁-x₂ space : ₁ ₂

1 2

y y

dy dx dx

x x

 

 

 

2 2

1 2

y y 0

dy c

x x

       

 

                

If the move is made in the direction of :y

1

( /

1

),

2

( /

2

)

/

i i

dx c dx c y x dx c y x

y x        

 

→ dy is equal or greater than zero

→ y always increase in the direction of

 y

(8)

Optimize subject to

16.5 Two variables and one constraint

(prove Lagrange multiplier method)

1 2

y y

y x x

x x

     

              

2

1 2 1 2

1 2 1

0 /

/

d x x x x x

x x x



 

       

                     

2

1 1 2

/ /

x

y y

y x

x x x



     

            

1 2

( , )

y x x  ( x x

₁

,

₂

)  0

Taylor expansion :

substituting the result for ∆𝑥₁

(9)

In order for no improvements of Δy,

Chapter 16. Calculus Methods of Optimization

16.5 Two variables and one constraint

(prove Lagrange multiplier method)

2 2

y 0

x x

 

   

  y

₁ ₁

0

x x

 

   

 

∆𝑦 = − 𝜕𝑦

𝜕𝑥

₁

𝜕𝜙 𝜕𝑥

₂

𝜕𝜙 𝜕𝑥

₁

+ 𝜕𝑦

𝜕𝑥

₂

Δ𝑥

₂

= 0 − 𝜕𝑦

𝜕𝑥

₁

𝜕𝜙 𝜕𝑥

₂

𝜕𝜙 𝜕𝑥

₁

+ 𝜕𝑦

𝜕𝑥

₂

= 0

If λ is defined as

𝜕𝑦

𝜕𝑥

₁

𝜕𝜙

𝜕𝑥

₁

(10)

16.6 Three variables and one constraint

As a same manner with 2 variables, 𝑦 𝑥₁, 𝑥₂, 𝑥₃ is need to be optimized subject to the constraint, 𝜙 𝑥₁, 𝑥₂, 𝑥₃

The first degree terms in the Taylor seriers are

𝜕𝑦

𝜕𝑥

₁

Δ𝑥

₁

+ 𝜕𝑦

𝜕𝑥

₂

Δ𝑥

₂

+ ( 𝜕𝑦

𝜕𝑥

₃

)Δ𝑥

₃

𝜕𝜙

𝜕𝑥

₁

Δ𝑥

₁

+ 𝜕𝜙

𝜕𝑥

₂

Δ𝑥

₂

+ ( 𝜕𝜙

𝜕𝑥

₃

)Δ𝑥

₃

(11)

Chapter 16. Calculus Methods of Optimization

Any two of the three variables can be moved independently, but the motion of the third variables must abide by the constraint. Arbitraily choosing 𝑥₂ as a dependant one,

∆𝑦 = 𝜕𝑦

𝜕𝑥

₁

Δ𝑥

₁

− 𝜕𝑦

𝜕𝑥

₂

𝜕𝜙 𝜕𝑥

₁

𝜕𝜙 𝜕𝑥

₂

Δ𝑥

₁

+ 𝜕𝜙 𝜕𝑥

₃

𝜕𝜙 𝜕𝑥

₂

Δ𝑥

₃

+ 𝜕𝑦

𝜕𝑥

₃

Δ𝑥

₃

= 0

In order for no improvement to be possible

𝜕𝑦

𝜕𝑥

₁

− 𝜕𝑦

𝜕𝑥

₂

𝜕𝜙

𝜕𝑥

₁

𝜕𝜙

𝜕𝑥

₂

= 0 𝜕𝑦

𝜕𝑥

₃

− 𝜕𝑦

𝜕𝑥

₂

𝜕𝜙

𝜕𝑥

₃

𝜕𝜙

𝜕𝑥

₂

= 0

(12)

16.6 Three variables and one constraint

Define 𝜆 as

𝜕𝑦

𝜕𝑥

₁

𝜕𝜙

𝜕𝑥

₁

𝜕𝑦

𝜕𝑥

₁

− 𝜆 𝜕𝜙

𝜕𝑥

₁

= 0

Then

𝜕𝑦

𝜕𝑥

₂

− 𝜆 𝜕𝜙

𝜕𝑥

₂

= 0 𝜕𝑦

𝜕𝑥

₃

− 𝜆 𝜕𝜙

𝜕𝑥

₃

= 0

(13)

Define 𝜆 as

𝜕𝑦

𝜕𝑥

₁

𝜕𝜙

𝜕𝑥

₁

𝜕𝑦

𝜕𝑥

₁

− 𝜆 𝜕𝜙

𝜕𝑥

₁

= 0

Then

𝜕𝑦

𝜕𝑥

₂

− 𝜆 𝜕𝜙

𝜕𝑥

₂

= 0 𝜕𝑦

𝜕𝑥

₃

− 𝜆 𝜕𝜙

𝜕𝑥

₃

= 0

(14)

16.8 Alternate expression of constrained optimization problem

optimize y x x (

₁

,

₂

)

1 2

( x x , ) b

 

subject to

1 2 1 2 1 2

( , ) ( , ) [ ( , ) ]

L x x  y x x    x x  b unconstrained function

optimum occurs where   L 0

1 1 1

2 2 2

1 2

0

( , ) 0

L y

x x x

L y

x x x

x x b

 



  

  

  

  

  

 

find optimum point

(15)

Chapter 16. Calculus Methods of Optimization

16.9 Interpretation λ of as the sensitivity coefficient

* * *

1 2

( , ) y y x y x (1)

y x x SC

b x b x b

 

  

   

    

1 2

( , ) x x 1 0 (2)

x x b

b x b x b

  

           

    

* * * *

1 2

* *

1 2

( / ) ( / )

y x y x

x x

  

   

 

   

1 2

(2) y x y x 0

x b x b

     

    

    sensitivity coefficient (SC) :

(1)  SC  