SOS and SDP Relaxation of Polynomial Optimization Problems

(1)

SOS and SDP Relaxation

of Polynomial Optimization Problems

Masakazu Kojima Tokyo Institute of Technology

November 2010

At National Cheng-Kung University, Tainan

. – p.1/58

Purpose of this talk — Introduction to

Sum Of Squares (SOS)and Semidefinite Programming (SDP) Relaxation of

Polynomial Optimization Problems (POPs) Contents

1. Global optimization of nonconvex problems 2. Polynomial Optimization Problems (POPs) 3. Basic idea ofSOSandSDPrelaxations 4. SOS relaxation ofPOPs

5. SDP relaxation ofPOPs

6. Exploiting sparsity inSOSandSDPrelaxations ofPOPs 7. Numerical results

. – p.2/58

Purpose of this talk — Introduction to

Sum Of Squares (SOS)and Semidefinite Programming (SDP) Relaxation of

Polynomial Optimization Problems (POPs) Contents

1. Global optimization of nonconvex problems 2. Polynomial Optimization Problems (POPs) 3. Basic idea of SOS and SDP relaxations 4. SOS relaxation of POPs

5. SDP relaxation of POPs

6. Exploiting sparsity in SOS and SDP relaxations of POPs 7. Numerical results

. – p.3/58

OP : Optimization problem in then-dim. Euclidean spaceRⁿ min. f(x)sub.tox∈S ⊆Rⁿ, wheref :Rⁿ→R. We want to approximatea global optimal solutionx^∗;

x^∗∈S andf(x^∗)≤f(x)for allx∈S if it exists. But, impossible without any assumption.

Various assumptions

continuity, differentiability, compactness,. . . convexity⇒local opt. sol. ≡global opt. sol.

⇒local improvement leads to a global opt. sol.

Powerful software for convex problems'LPs,SDPs,. . .

. – p.4/58

(2)

OP : Optimization problem in then-dim. Euclidean spaceRⁿ min. f(x)sub.tox∈S ⊆Rⁿ, wheref :Rⁿ →R. We want to approximate a global optimal solutionx^∗;

x^∗ ∈S and f(x^∗)≤f(x)for allx∈S if it exists. But, impossible without any assumption.

Various assumptions

continuity, differentiability, compactness, . . . convexity ⇒local opt. sol. ≡global opt. sol.

⇒local improvement leads to a global opt. sol.

Powerful software for convex problems'LPs, SDPs,. . . What can we do beyond convexity?

We still need proper models and assumptions

Polynomial Optimization Problems (POPs) — this talk Main tool is SDP relaxation — this talk

Powerful in theory but expensive in practice

Exploiting sparsity in large scale SDPs & POPs — this talk

. – p.5/58

Contents

. – p.6/58

POP: minf0(x)sub.tofk(x)≥0 or = 0 (k= 1, . . . , m) x= (x1, . . . , xn) ∈Rⁿ : a vector variable

fk(x): a real-valued polynomial inx1, . . . , xn(k= 0,1, . . . , m) Example. n= 3,x= (x1, x2, x3)∈R³: a vector variable

min f0(x)≡x³₁−2x1x²₂+x²₁x2x3−4x²₃ sub.to f1(x)≡ −x²₁+ 5x2x3+ 1≥0,

f2(x)≡x²₁−3x1x2x3+ 2x3+ 2≥0, f3(x)≡ −x²1−x²₂−x²₃+ 1≥0.

x1(x1−1) = 0 (0-1integer cond.), x2 ≥0, x3≥0, x2x3= 0(comp. cond.).

•Various problems (including0-1integer programs)⇒POP

•POP serves as a unified theoretical model for global optimization in nonlinear and combinatorial optimization problems.

. – p.7/58

POP: minf0(x)sub.tofk(x)≥0 or = 0 (k= 1, . . . , m) [1] Lasserre, “Global optimization with polynomials and the

problems of moments”,SIAM J. on Optim. (2001).

[2] Parrilo, “Semidefinite programming relaxations for semialgebraic problems”,Math. Prog. (2003).

primal approach⇒a sequence of SDP relaxations.

dual approach⇒a sequence of SOS relaxations.

Main features:

(a) Lower bounds for the optimal value.

(b) Convergence to global optimal solutions under assump.

(c) Each relaxed problem can be solved as an SDP;its size↑ rapidlyalong “the sequence” asthe size of POP↑, the deg.

of poly.↑, and/orwe require higher accuracy.

(d) Expensive to solve large scale POPs in practice.

⇒Exploiting Sparsity.

. – p.8/58

(3)

problems of moments”, SIAM J. on Optim. (2001).

[2] Parrilo, “Semidefinite programming relaxations for semialgebraic problems”,Math. Prog. (2003).

Main features:

of poly. ↑, and/orwe require higher accuracy.

. – p.8/58

problems of moments”,SIAM J. on Optim. (2001).

[2] Parrilo, “Semidefinite programming relaxations for semialgebraic problems”,Math. Prog.(2003).

Main features:

of poly. ↑, and/orwe require higher accuracy.

. – p.8/58

Contents

1. Global optimization of nonconvex problems 2. Polynomial Optimization Problems (POPs) 3. Basic idea of SOS and SDP relaxations

Three ways of describing the SDP relaxation by Lasserre:

(a) Probabilty measure and its moments (b) Linearization of polynomial SDPs (c) Sum of squares of polynomials

. – p.9/58

µ: a probability measure onRⁿ .We assumen= 2.For every r= 0,1,2, . . ., define

u_r(x) = (1, x1, x2, x²₁, x1x2, x²₂, x³₁, . . . , x^r₂)^T : a column vector (all monomials with degree≤r)

Mr(y) =

!

R²ur(x)ur(x)^Tdµ

"

moment matrix, symmetric, positive semidefinite

#

yαβ =

!

R²x^α₁x^β₂dµ =(α,β)-element depending onµ, y00= 1

Example withr = 2: y21=

!

R²x²₁x2dµ

M_r(y) =







y00 y10 y01 y20 y11 y02

y10 y20 y11 y30 y21 y12

y01 y11 y02 y21 y12 y03

y20 y30 y21 y40 y31 y22

y11 y21 y12 y31 y22 y13

y02 y12 y03 y22 y13 y04







, y00= 1

. – p.10/58

(4)

µ: a probability measure onRⁿ .We assumen= 2.For every r = 0,1,2, . . ., define

ur(x) = (1, x1, x2, x²₁, x1x2, x²₂, x³₁, . . . , x^r₂)^T : a column vector (all monomials with degree≤r)

Mr(y) =

!

R²ur(x)ur(x)^Tdµ

"

#

yαβ =

!

R²x^α₁x^β₂dµ =(α,β)-element depending onµ, y00 = 1

Example withr = 2: y21=

!

R²x²₁x2dµ

M_r(y) =







y00 y10 y01 y20 y11 y02

y10 y20 y11 y30 y21 y12

y01 y11 y02 y21 y12 y03

y20 y30 y21 y40 y31 y22

y11 y21 y12 y31 y22 y13

y02 y12 y03 y22 y13 y04







, y00 = 1

. – p.10/58

µ: a probability measure onRⁿ .We assumen= 2.For every r= 0,1,2, . . ., define

ur(x) = (1, x1, x2, x²₁, x1x2, x²₂, x³₁, . . . , x^r₂)^T : a column vector (all monomials with degree≤r)

Mr(y) =

!

R²ur(x)ur(x)^Tdµ

"

#

yαβ =

!

R²x^α₁x^β₂dµ =(α,β)-element depending onµ, y00= 1

µ: a probability measure onR²

⇓

y00= 1,M_r(y),O(positive semidefinite)(r = 1,2, . . .) We will use this necessary cond. with a finiter forµto be a probability measure in relaxation of a POP⇒next slide.

. – p.11/58

µ: a probability measure onRⁿ .We assumen= 2.For every r = 0,1,2, . . ., define

u_r(x) = (1, x1, x2, x²₁, x1x2, x²₂, x³₁, . . . , x^r₂)^T : a column vector (all monomials with degree≤r)

Mr(y) =

!

R²ur(x)ur(x)^Tdµ

"

#

yαβ =

!

R²x^α₁x^β₂dµ =(α,β)-element depending onµ, y00 = 1

µ: a probability measure onR²

⇓

y00= 1,Mr(y) ,O(positive semidefinite)(r = 1,2, . . .) We will use this necessary cond. with a finiter forµto be a probability measure in relaxation of a POP⇒next slide.

. – p.11/58

SDP relaxation (Lasserre ’01)of a POP — an example POP: min f0(x) =x⁴₁−2x1x2 opt. val. ζ^∗: unknown sub. to x∈S ≡

*

x∈R² : f1(x) =x1 ≥0

f2(x) = 1−x²₁−x²₂ ≥0 +

. -

min ,

(x⁴₁x⁰₂−2x1x2)dµ

sub. to µ: a prob. meas. onS.

0 -1 1

1

^x¹

x^*= (x₁^*,x^*₂) ? x₂

S

⇓yαβ =

!

R²x^α₁x^β₂dµ min y40−2y11 ⇒SDP relaxation, opt. val.ζr ≤ζ^∗

sub. to “a certain moment cond. with a parameterr

for µto be a probability measure onS”⇒next slide ζr ≤ζr+1 ≤ζ^∗, andζr →ζ^∗ asr → ∞under a moderate assumption that requiresS is bounded (Lasserre ’01).

. – p.12/58

(5)

SDP relaxation (Lasserre ’01)of a POP — an example POP: min f0(x) =x⁴₁−2x1x2 opt. val. ζ^∗ : unknown sub. to x∈S ≡

*

x∈R²: f1(x) =x1≥0

f2(x) = 1−x²₁−x²₂≥0 +

. -

min ,

(x⁴₁x⁰₂−2x1x2)dµ

sub. to µ: a prob. meas. onS.

0 -1 1

1

^x¹

x^*= (x₁^*,x₂^*) ? x₂

S

⇓ yαβ =

!

R²x^α₁x^β₂dµ

min y40−2y11 ⇒SDP relaxation, opt. val. ζr ≤ζ^∗ sub. to “a certain moment cond. with a parameterr

forµto be a probability measure onS”⇒next slide ζr ≤ζr+1 ≤ζ^∗, andζr →ζ^∗ asr → ∞under a moderate assumption that requiresS is bounded (Lasserre ’01).

. – p.12/58

SDP relaxation (Lasserre ’01)of a POP — an example r= 2

minµ y40−2y11 s.t. !

S





 1 x1

x²₁











 1 x1

x²₁







T

x1dµ,O, ⇐x1≥0

1−x²₁−x²₂ ≥0⇒

!

S





 1 x1

x2











 1 x1

x2







T

(1−x²₁−x²₂)dµ,O,

(moment matrix) !

S





 1 x1

x2

x²₁ x1x2

x²₂











 1 x1

x2

x²₁ x1x2

x²₂







T

dµ,O.

Hereµdenotes a probability measure onS.

⇓

. – p.13/58

SDP relaxation (Lasserre ’01)of a POP — an example µ: a probability measure

yαβ =

!

S

x^α₁x^β₂dµ f1(x) =x1 ≥0⇒

!

S





 1 x1

x²₁











 1 x1

x²₁







T

x1dµ,O

⇔

!

S







x1 x²₁ x³₁ x²₁ x³₁ x⁴₁ x³₁ x⁴₁ x⁵₁





dµ,O ⇔







y10 y20 y30

y20 y30 y40

y30 y40 y50





,O

f2(x) = 1−x²₁−x²₂ ≥0⇒

!

S





 1 x1

x2











 1 x1

x2







T

f2(x)dµ,O

⇒







1−y20−y02 y10−y30−y12 y01−y21−y03

y10−y30−y12 y20−y40−y22 y11−y31−y13

y01−y21−y03 y11−y31−y13 y02−y22−y04





,O

. – p.14/58

SDP relaxation (Lasserre ’01)of a POP — an example µ: a probability measure

yαβ =

!

S

x^α₁x^β₂dµ

!

S





 1 x1

x2

x²₁ x1x2

x²₂











 1 x1

x2

x²₁ x1x2

x²₂







T

dµ,O

⇒







1 y10 y01 y20 y11 y02

y10 y20 y11 y30 y21 y12

y01 y11 y02 y21 y12 y03

y20 y30 y21 y40 y31 y22

y11 y21 y12 y31 y22 y13

y02 y12 y03 y22 y13 y04





 ,O

Consequently, we obtain

. – p.15/58

(6)

SDP relaxation (Lasserre ’01)of a POP — an example min y40−2y11s.t.







y10 y20 y30

y20 y30 y40

y30 y40 y50





,O,







1−y20−y02 y10−y30−y12 y01−y21−y03

y10−y30−y12 y20−y40−y22 y11−y31−y13

y01−y21−y03 y11−y31−y13 y02−y22−y04





,O,

(moment matrix)







1 y10 y01 y20 y11 y02

y10 y20 y11 y30 y21 y12

y01 y11 y02 y21 y12 y03

y20 y30 y21 y40 y31 y22

y11 y21 y12 y31 y22 y13

y02 y12 y03 y22 y13 y04





 ,O.

We can apply SDP relaxationto general POPs inRⁿ. Poweful in theory but very expensive in computation

⇒Exploiting sparsityis crucial in practice. . – p.16/58

Contents

(a) Probabilty measure and its moments (b) Linearization of polynomial SDPs

— similar to (a), but easier to understand (c) Sum of squares of polynomials

. – p.17/58

SDP relaxation (Lasserre ’01)of a POP — an example POP: min f0(x) =x⁴₁−2x1x2 opt. val. ζ^∗ : unknown sub. to x∈S ≡

*

x∈R²: f2(x) =x1≥0

f1(x) = 1−x²₁−x²₂≥0 +

. -

Polynomial SDP minx⁴₁−2x1x2

s.t.





 1 x1

x²₁











 1 x1

x²₁







T

f1(x) ,O,





 1 x1

x2











 1 x1

x2







T

f2(x),O,





 1 x1

x2

x²₁ x1x2

x²₂











 1 x1

x2

x²₁ x1x2

x²₂







T

,O. Expand poly.mat.inequalities.

Linearize the problem by replacingx^α₁x^β₂ byyαβ ⇒

. – p.18/58

SDP relaxation (Lasserre ’01)of a POP — an example POP: min f0(x) =x⁴₁−2x1x2 opt. val. ζ^∗: unknown sub. to x∈S ≡

*

x∈R² : f2(x) =x1 ≥0

f1(x) = 1−x²₁−x²₂ ≥0 +

. Polynomial SDP -

minx⁴₁−2x1x2

s.t.





 1 x1

x²₁











 1 x1

x²₁







T

f1(x),O,





 1 x1

x2











 1 x1

x2







T

f2(x) ,O,





 1 x1

x2

x²₁ x1x2

x²₂











 1 x1

x2

x²₁ x1x2

x²₂







T

,O. Expand poly.mat.inequalities.

Linearize the problem by replacingx^α₁x^β₂ byyαβ ⇒

. – p.18/58

(7)

SDP relaxation (Lasserre ’01)of a POP — an example minx⁴₁−2x1x2 ⇒miny40−2y11

f1(x) =x1 ≥0⇔





 1 x1

x²₁











 1 x1

x²₁







T

x1 ,O

⇔







x1 x²₁ x³₁ x²₁ x³₁ x⁴₁ x³₁ x⁴₁ x⁵₁





,O⇒







y10 y20 y30

y20 y30 y40

y30 y40 y50





,O

. – p.19/58

SDP relaxation (Lasserre ’01)of a POP — an example

f2(x) = 1−x²₁−x²₂≥0⇔





 1 x1

x2











 1 x1

x2







T

f2(x),O

-







1−x²₁x⁰₂−x⁰₁x²₂ x¹₁x⁰₂−x³₁x⁰₂−x¹₁x²₂ x⁰₁x¹₂−x²₁x¹₂−x⁰₁x³₂ x¹₁x⁰₂−x³₁x⁰₂−x¹₁x²₂ x²₁x⁰₂−x⁴₁x⁰₂−x²₁x²₂ x¹₁x¹₂−x³₁x¹₂−x¹₁x³₂ x⁰₁x¹₂−x²₁x¹₂−x⁰₁x³₂ x¹₁x¹₂−x³₁x¹₂−x¹₁x³₂ x⁰₁x²₂−x²₁x²₂−x⁰₁x⁴₂







,O

⇓







1−y20−y02 y10−y30−y12 y01−y21−y03

y10−y30−y12 y20−y40−y22 y11−y31−y13

y01−y21−y03 y11−y31−y13 y02−y22−y04





,O

. – p.20/58

SDP relaxation (Lasserre ’01)of a POP — an example Similarly,





 1 x1

x2

x²₁ x1x2

x²₂











 1 x1

x2

x²₁ x1x2

x²₂







T

,O

⇒







1 y10 y01 y20 y11 y02

y10 y20 y11 y30 y21 y12

y01 y11 y02 y21 y12 y03

y20 y30 y21 y40 y31 y22

y11 y21 y12 y31 y22 y13

y02 y12 y03 y22 y13 y04





 ,O

Consequently, we obtain

. – p.21/58

SDP relaxation (Lasserre ’01)of a POP — an example miny40−2y11s.t.







y10 y20 y30

y20 y30 y40

y30 y40 y50





,O,







1−y20−y02 y10−y30−y12 y01−y21−y03

y10−y30−y12 y20−y40−y22 y11−y31−y13

y01−y21−y03 y11−y31−y13 y02−y22−y04





,O,

(moment matrix)







1 y10 y01 y20 y11 y02

y10 y20 y11 y30 y21 y12

y01 y11 y02 y21 y12 y03

y20 y30 y21 y40 y31 y22

y11 y21 y12 y31 y22 y13

y02 y12 y03 y22 y13 y04





 ,O.

Ifxis a feasible sol. of Poly. SDP (or POP), then

y= (yαβ =x^α₁x^β₂)is a feasible sol. ofSDPwith the same obj. val.;x⁴₁−2x1x2 =y40−2y11=⇒Relaxation . – p.22/58

(8)

Contents

(a) Probabilty measure and its moments (b) Linearization of polynomial SDPs (c) Sum of squares of polynomials

— dual approach

— easy to understand in unconstrained case

. – p.23/58

f(x) : an SOS (Sum of Squares) polynomial -

∃ polynomialsg1(x), . . . , gq(x);f(x) =

q

-

j=1

gj(x)². N : the set of nonnegative polynomials inx∈Rⁿ.

SOS∗: the set of SOS. Obviously,SOS∗ ⊂N.

SOSr ={f ∈SOS∗:degf ≤2r}: SOSs w. degree at most2r.

n= 2. f(x1, x2) = (x²₁−2x2+ 1)²+ (3x1x2+x2−4)²∈ SOS2. n= 2. f(x1, x2) = (x1x2−1)²+x²₁ ∈SOS2.

•In theory,SOS∗(SOS) ⊂N. SOS∗3=N in general.

•Ifn= 1,SOS∗=N. {f ∈N :degf ≤2}≡SOS1.

•In practice,f(x)∈N\SOS∗is rare.

•So we replaceN bySOS∗ =⇒SOS Relaxations.

. – p.24/58

Representation of

SOSr ≡

* _q -

j=1

gj(x)²:q ≥1, gj(x)is a poly. of deg ≤r +

⊂SOS∗.

∀poly. g(x) of deg≤r,∃a∈R^d(r);g(x) =a^Tu_r(x), where ur(x)=(1, x1, . . . , xn, x²₁, x1x2, x1x3, . . . , x²_n, . . . , x^r₁, . . . , x^r_n)^T (a column vector of a basis for polynomials of degree≤r), d(r) =

"

n+r r

#

: the dimension ofu_r(x).

⇓

SOSr = . /q

j=1

0a^T_ju_r(x)12

: q ≥1, a_j ∈R^d(r)2

= .

ur(x)^T3 /q

j=1aja^T_j4

ur(x) : q ≥1, aj ∈R^d(r)2

= 5

ur(x)^TVur(x) : V ,O6 .

. – p.25/58

Example.n= 1, SOS polynomials of degree≤3inx∈R.

SOS3 ≡

* _q -

j=1

gj(x)² : q ≥1, gj(x)is a poly. of degree≤3 +

=















 1 x x² x³







T

V





 1 x x² x³







: V is4×4psd matrix











. – p.26/58

(9)

Example. n= 2, SOS polynomials of degree≤2inx=(x1, x2).

SOS2 ≡

* _q -

j=1

gj(x)² : q ≥1, gj(x) is a poly. of degree≤2 +

=















 1 x1

x2

x²₁ x1x2

x²₂







T

V





 1 x1

x2

x²₁ x1x2

x²₂







: V is a 6×6psd matrix











. – p.27/58

minf(x) =−x1+ 2x2+ 3x²₁−5x²₁x²₂+ 7x⁴₂ ζ = 3.1: fixed maxζ; f(x)−ζ ≥0 (∀x) =⇒SOS relaxationSDP

maxζs.t. f(x)−ζ ∈SOS2 (SOS of poly. of degree≤2) -

maxζ ∃V ∈S⁶; s.t. f(x)−ζ = Sum of Squares





 1 x1

x2

x²₁ x1x2

x²₂







T





V11 V12 V13 V14 V15 V16

V12 V22 V23 V24 V25 V26

V13 V23 V33 V34 V35 V36

V14 V24 V34 V44 V45 V46

V15 V25 V35 V45 V55 V56

V16 V26 V36 V46 V56 V66











 1 x1

x2

x²₁ x1x2

x²₂





 (∀(x1, x2)^T ∈Rⁿ), V ,O

- Compare the coef. of1, x1, x2, x²₁, x1x2,x²₂ on both sides of=

. – p.28/58

LMI:∃V ∈S⁶?;

max ζ s.t. −ζ =V11, −1 = 2V12, 2 = 2V13, 3 = 2V14+V22,

−5 = 2V46+V55, 7 =V66, all others 0 =· · · , V ,O

In general, each equality constraint is a linear equation in ζ andV.

. – p.29/58

Unconstrained minimization

P: minf(x) =−x1+ 2x2+ 3x²₁−5x²₁x²₂+ 7x⁴₂ - (a special case of Lagragian dual) P’: max ζ s.t f(x)−ζ ≥0 (∀x∈Rⁿ)

-

f(x)−ζ ∈N (the nonnegative polynomials) Herexis a parameter (index) describing inequality constraints.

f(x)

ζ ζ

∗

x

. – p.30/58

(10)

P: minf(x) =−x1+ 2x2+ 3x²₁−5x²₁x²₂+ 7x⁴₂ -(a special case of Lagragian dual) P’: max ζ s.t f(x)−ζ ≥0 (∀x∈Rⁿ)

-

⇓a subproblem ofP’ (or a relaxation ofP)

P": max ζ sub.tof(x)−ζ ∈SOS2 (SOS of poly. of degree≤2)

⇓

. – p.31/58

P: minf(x) =−x1+ 2x2+ 3x²₁−5x²₁x²₂+ 7x⁴₂ - (a special case of Lagragian dual) P’: max ζ s.t f(x)−ζ ≥0 (∀x∈Rⁿ)

-

⇓a subproblem ofP’ (or a relaxation ofP)

P": max ζ sub.tof(x)−ζ ∈SOS2(SOS of poly. of degree≤2)

⇓

. – p.31/58

maxζ ∃V ∈S⁶; s.t. f(x)−ζ = Sum of Squares





 1 x1

x2

x²₁ x1x2

x²₂







T





V11 V12 V13 V14 V15 V16

V12 V22 V23 V24 V25 V26

V13 V23 V33 V34 V35 V36

V14 V24 V34 V44 V45 V46

V15 V25 V35 V45 V55 V56

V16 V26 V36 V46 V56 V66











 1 x1

x2

x²₁ x1x2

x²₂





 (∀(x1, x2)^T ∈Rⁿ), V ,O

- Compare the coef. of1, x1, x2, x²₁, x1x2,x²₂on both side of=

. – p.32/58

SDP (Semidefinite Program)

max ζ s.t. −ζ =V11, −1 = 2V12, 2 = 2V13, 3 = 2V14+V22,

−5 = 2V46+V55, 7 =V66, all others 0 =· · · , V ,O

SOS relaxation of general constrained POPs

=⇒Next section We will use a generalized Lagrangian dual.

. – p.33/58

(11)

SDP (Semidefinite Program)

max ζ s.t. −ζ =V11, −1 = 2V12, 2 = 2V13, 3 = 2V14+V22,

−5 = 2V46+V55, 7 =V66, all others 0 =· · · , V ,O

SOS relaxation of general constrained POPs

=⇒Next section We will use a generalized Lagrangian dual.

. – p.33/58

Contents

. – p.34/58

POP:ζ^∗ =min. f0(x)s.t. fk(x) ≥0 (k= 1, . . . , m)

⇓

A sequence of SOS relaxation problemsSOS^r depending on the relaxation order r= 1,2, . . ., which determines the degree of polynomials used;

(a) Under a moderate assumption,

opt. sol. ofSOS^r →opt sol. ofPOPasr → ∞ (Lasserre 2001).

(b) r =6“the max. deg. of poly. inPOP”/27+ 0∼3is usually large enough to attain opt sol. ofPOPin practice.

(c) Such anrisunknownin theory except∃special cases.

(d) SOS^r can be converted into anSDP^r and solved by the primal-dual interior-point method.

(e) The size ofSDP^r increases asr → ∞.

. – p.35/58

POP:ζ^∗ =min. f0(x)s.t. fk(x)≥0 (k= 1, . . . , m) Rough sketch ofSOS relaxation ofPOP

“Generalized LagrangianDual”

+

“SOS relaxation of unconstrained POPs”

⇓

SOS relaxation ofPOP

. – p.36/58

(12)

POP:ζ^∗ =min. f0(x)s.t. fk(x) ≥0 (k= 1, . . . , m) Rough sketch of SOS relaxation ofPOP

“Generalized LagrangianDual”

+

“SOS relaxation of unconstrained POPs”

⇓

SOS relaxation ofPOP

. – p.36/58

POP:ζ^∗ =min. f0(x)s.t. fk(x)≥0 (k= 1, . . . , m)

⇓

Generalized Lagrangianfunction L(x,λ1, . . . ,λm) = f0(x)−/m

k=1λk(x)fk(x) for∀x∈Rⁿ, ∀λk ∈SOS∗

IfR' λk ≥0thenLis the standard Lagrangian function.

Generalized LagrangianDual λ1 ∈SOS∗, . . . ,maxλm∈SOS∗

minx∈RⁿL(x,λ1, . . . ,λm) - X ⇔max ηs.t.L(x,λ1, . . . ,λm)−η≥0 (∀x∈Rⁿ) Generalized LagrangianDual

max η s.t. L(x,λ1, . . . ,λm)−η≥0 (∀x∈Rⁿ), λ1 ∈SOS∗, . . . ,λm∈SOS∗

xis not a variable but the index for infinite no. of inequalities.

. – p.37/58

⇓

IfR'λk ≥0thenLis the standard Lagrangian function.

Generalized LagrangianDual λ1 ∈SOS∗, . . . ,maxλm∈SOS∗

minx∈RⁿL(x,λ1, . . . ,λm) - X ⇔max ηs.t.L(x,λ1, . . . ,λm)−η≥0 (∀x∈Rⁿ) Generalized LagrangianDual

max η s.t. L(x,λ1, . . . ,λm)−η≥0 (∀x∈Rⁿ), λ1∈SOS∗, . . . ,λm∈SOS∗

xis not a variable but the index for infinite no. of inequalities.

. – p.37/58

POP:ζ^∗ =min. f0(x)s.t. fk(x)≥0 (k= 1, . . . , m)

⇓

Generalized LagrangianDual

max η s.t. L(x,λ1, . . . ,λm)−η≥0 (∀x∈Rⁿ), λ1 ∈SOS∗, . . . ,λm∈SOS∗

⇓SOS relaxation

SOS^r:η^r = max η s.t. L(x,λ1, . . . ,λm)−η ∈SOSr (∀x∈Rⁿ), λ1 ∈SOSr₁, . . . ,λm∈SOSr_m

SOSr_k : the set of sum of square polynomials with deg.≤rk

6max{deg(fk) :k= 0, . . . , m}/27 ≤r: the relaxation order rk =r− 6deg(fk)/27(k= 1, . . . , m); chosen to balance the degrees of all the termsλk(x)fk(x) (k= 1, . . . , m).

η^r ≤η^r+1,η^r →ζ^∗asr → ∞under a moderate assumption which requires the feasible region ofPOPis compact.

. – p.38/58

(13)

⇓

Generalized LagrangianDual

max η s.t. L(x,λ1, . . . ,λm)−η≥0 (∀x∈Rⁿ), λ1∈SOS∗, . . . ,λm∈SOS∗

⇓SOS relaxation

SOS^r:η^r = max η s.t. L(x,λ1, . . . ,λm)−η∈ SOSr (∀x∈Rⁿ), λ1 ∈SOSr₁, . . . ,λm ∈SOSr_m

SOSr_k : the set of sum of square polynomials with deg.≤rk

6max{deg(fk) :k= 0, . . . , m}/27 ≤r : the relaxation order rk =r− 6deg(fk)/27(k= 1, . . . , m); chosen to balance the degrees of all the termsλk(x)fk(x) (k= 1, . . . , m).

η^r ≤η^r+1,η^r →ζ^∗asr → ∞under a moderate assumption which requires the feasible region ofPOPis compact.

. – p.38/58

Contents

. – p.39/58

-

the relax. orderr ≥ 6max{deg(fk) :k= 0, . . . , m}/27 rk =r− 6deg(fk)/27(k= 1, . . . , m)

u_r(x) = (1, x1, x2, x²₁, x1x2, x²₂, x³₁, . . . , x^r₂)^T (all monomials with degree≤r) Poly.SDP: min f0(x) s.t. u_r_k(x)u_r_k(x)^Tfk(x),O(∀k)

u_r(x)u_r(x)^T ,O.

⇓

Linearize:

Expand all the polynomial inequalities

Replacex^α₁x^β₂· · ·x^γ_n by a single variableyαβ...γ

LinearSDP^r:ζ^r =min. a linear function inyαβ...γ

s.t. linear matrix inequalities inyαβ...γ

ζ^r ≤ζ^r+1,ζ^r →ζ^∗ asr → ∞under a moderate assumption which requires the feasible region ofPOPis compact.

. – p.40/58

Example

POP: min. f0(x) =x³₁−2x²₂s.t. f1(x) = 1−x²₁−x²₂≥0.

-

r = 2≥ 6max{deg(fk) :k= 0,1}/27=63/27= 2.

r1=r− 6deg(f1)/27= 2− 62/27= 1.

u₁(x) = (1, x1, x2)^T,u₂(x) = 1, x1, x2, x²₁, x1x2, x²₂)^T. Poly.SDP: min x³₁−2x²₂

s.t. u₁(x)u1(x)^Tf1(x),O u₂(x)u2(x)^T ,O.

⇓Expand matrix inequalities

. – p.41/58