Advanced Topics in Optimization

(1)

Piecewise Linear

Approximation of a

Nonlinear Function

(2)

Introduction and Objectives

Introduction

¾ There exists no general algorithm for nonlinear programming due to its irregular behavior

¾ Nonlinear problems can be solved by first representing the nonlinear function (both objective function and constraints) by a set of linear functions and then apply simplex method to solve this using some restrictions

Objectives

¾ To discuss the various methods to approximate a nonlinear function using linear functions

¾ To demonstrate this using a numerical example

(3)

Piecewise Linearization

¾ A nonlinear single variable function f(x) can be

approximated by a piecewise linear function

¾ Geometrically, f(x) can be shown as a curve being represented as a set of connected line segments

(4)

Piecewise Linearization: Method 1

¾

Consider an optimization function having only one nonlinear term f(x)

¾

Let the x-axis of the nonlinear function f(x) be divided by ‘p’

breaking points t

₁, t₂, t₂, …, t_p

¾

Corresponding function values be f(t

₁), f(t₂),…, f(t_p)

¾

If ‘x’ can take values in the interval , then the breaking points can be shown as

X x ≤

≤ 0

X t

t

t < < <

_p

≡

≡ ...

0

₁ ₂

(5)

Piecewise Linearization: Method 1 …contd.

¾

Express ‘x’ as a weighted average of these breaking points

¾

Function f(x) can be expressed as

where

p p

t w t

w t

w

x =

₁ ₁

+

₂ ₂

+ ... +

∑

=

^p

i

i i

t w x

e i

1

., .

( ) ( ) ( ) ( ) ∑ ( )

=

= +

+ +

=

^p

i

i i p

p

f t w f t

w t

f w t

f w x

f

1 2

2 1

1

...

∑

= p

=

i

w

i 1

1

(6)

¾

Finally the model can be expressed as

subject to the additional constraints

( ) ∑ ( )

=

^p

i

i i

f t w x

f Min or

Max

1

∑

=

p

i

i p

i

i i

w

x t

w

1 1

1

(7)

¾ This linearly approximated model can be solved using simplex method with some restrictions

¾ Restricted condition:

¾ There should not be more than two ‘w_i’ in the basis and

¾ Two ‘w_i’ can take positive values only if they are adjacent. i.e., if ‘x’ takes the value between t_i and t_i+1, then only w_i and w_i+1(contributing weights to the value of ‘x’) will be positive, rest all weights be zero

¾ In general, for an objective function consisting of ‘n’ variables (‘n’ terms) represented as

( ) x f ( ) x f ( ) x f

ⁿ

( ) x

ⁿ

f Min or

Max =

₁ ₁

+

₂ ₂

+ .... +

(8)

¾ subjected to ‘m’ constraints

¾ The linear approximation of this problem is

( ) x g ( ) x g ( ) x b for j m

g

₁_j ₁

+

₂_j ₂

+ .... +

_nj _n

≤

_j

= 1 , 2 ,...,

n k

for w

m j

for b

t g w to

subjected

t f w Min

or Max

p

i

ki

j n

k

p

i

ki kj ki n

k

p

i

ki k ki

,..., 2 , 1 1

,..., 2 , 1 )

( ) (

1

1 1 1 1

=

≤

∑

∑∑

=

= =

(9)

Piecewise Linearization: Method 2

¾

‘x’ is expressed as a sum, instead of expressing as the weighted sum of the breaking points as in the previous method

where u_i is the increment of the variable ‘x’ in the interval i.e., the bound of u_i is

¾

The function f(x) can be expressed as

¾ where represents the slope of the linear approximation between the points and

∑

⁻

−

= +

=

+ +

=

¹

1 1

1 2

1

....

p

i i

p

t u

u u

u t

x

) ,

( t

_i

t

_i₊₁

i i

i

t t

u ≤ −

≤

₊1

0

( ) ( )

^p ⁱ

i

u t

f x

f ∑

⁻

=

+

=

¹

1

α

αi

ti +1

ti

( ) ( )

ⁱ ⁱ

i

t f t

f

−

= ⁺¹ −

α

(10)

¾ Finally the model can be expressed as

subjected to additional constraints

( ) ( )

^p ⁱ

i

u t

f x

f Min or

Max ∑

⁻

=

+

=

¹

1

α

1 ,....,

2 , 1 ,

0

₁

1

1 1

−

=

−

≤

= +

+

−

∑

=

p i

t t

u

x u

t

i i

i p

i i

(11)

Piecewise Linearization: Numerical Example

¾ The example below illustrates the application of method 1

¾ Consider the objective function

¾ subject to

¾ The problem is already in separable form (i.e., each term consists of only one variable).

2 3

1

x

x f

Maximize = +

0 4 0

15 2

2

2 1

2 2

1

≥

≤

≤ +

x

(12)

…contd.

¾ Split up the objective function and constraint into two parts

where

¾ and are treated as linear variables as they are in linear form

( ) ( ) ( )

¹ ¹²

( )

²

11 1

2 2 1

1

x g x

g g

x f x

f f

+

=

+

=

( ) ( )

( )

¹ ¹² ¹²

( )

² ²

11

2 2

2 3 1 1

1

2

; 2

;

x x

g x x

g

x x

f x x

f

=

( )

2 2 x

f

g

¹²

( ) x

²

(13)

…contd.

¾ Consider five breaking points for x₁

¾ ^f¹

( )

^x¹ can be written as,

( ) ( )

64 27

8 1

0 ₁₂ ₁₃ ₁₄ ₁₅

11 5

1

1 1 1 1

1

× +

×

=

∑

=

w w

w

t f w x

f

i

i i

(14)

Piecewise Linearization: Numerical Example

…contd.

¾ can be written as,

¾ Thus, the linear approximation of the above problem becomes

subject to

( )

¹

11 x g

( ) ( )

32 18

8 2

0 ₁₂ ₁₃ ₁₄ ₁₅

11 5

1

1 1 1 1

11

× +

×

=

∑

=

w w

w

t g w x

g

i

i i i

2 15

14 13

12

8 w 27 w 64 w x

w f

Maximize = + + + +

5 ,..., 2 , 1 0

1

15 2

32 18

8 2

15 14

13 12

11

1 2 15

14 13

12

=

≥

= +

+ +

+

= +

+ +

i for w

w w

w

s x

w w

(15)

…contd.

¾ This can be solved using simplex method in a restricted basis condition

¾ The simplex tableau is shown below

(16)

Piecewise Linearization: Numerical Example

…contd.

¾ From the table, it is clear that should be the entering variable

¾ should be the exiting variable

¾ But according to restricted basis condition and cannot occur together in basis as they are not adjacent

¾ Therefore, consider the next best entering variable

¾ This also is not possible, since should be exited and and cannot occur together

¾ The next best variable , it is clear that should be the exiting variable

w

15

s

1

w

15

w

₁₁

w

14

s

1

w

¹⁴

w

11

w

13

w

₁₁

(17)

Piecewise Linearization: Numerical Example

…contd.

¾ The entering variable is . Then the variable to be exited is and this is not acceptable since is not an adjacent point to

¾ Next variable can be admitted by dropping .

w

15

s

1

w

15

w

₁₃

w

14

s

1

(18)

…contd.

¾ Now, cannot be admitted since cannot be dropped

¾ Similarly and cannot be entered as cannot be dropped

w

15

w

14

w

11

w

₁₂

w

₁₃

(19)

Piecewise Linearization: Numerical Example

…contd.

¾ Since there is no more variable to be entered, the process ends

¾ Therefore, the best solution is

¾ Now,

¾ The optimum value is

¾ This may be an approximate solution to the original nonlinear problem

¾ However, the solution can be improved by taking finer breaking points

7 . 0

; 3 .

0

₁₄

13

= w =

w

7 . 2 3 2 ₁₄

13 5

1

1 1

1 =

∑

= × + × =

=

w w

t w x

i

i i

2

= 0 x and

3 .

= 21

f

(20)

Advanced Topics in Optimization - Nptel