max 6x1 + x2 − x3 − x4
x1 + 2x2 + x3 + x4 ≤ 5 3x1 + x2 − x3 ≤ 8 x2 + x3 + x4 = 1 x2,x3,x4 ≥ 0 x1unrestricted
We wish to check if one of the following assignments is an optimal solution.
a) x1=2,x2=1,x3=0,x4=0 b) x1=3,x2=0,x3=1,x4=0
To this end, we use Complementary Slackness. Let us discuss the theory first.
Theory
As usual, letxdenote the vector of variables, letcbe the vector of coefficients of variables of the objective function, letAbe the coefficient matrix of the left-hand side of our constraints, and letbbe the vector of the right-hand side of the constraints. Letybe the variables of the dual.
max cTx Ax ≤ b
x ≥ 0
PRIMAL
min bTy ATy ≥ c
y ≥ 0
DUAL We say that vectorsx= (x1, . . . ,xn)andy= (y1, . . . ,ym)are complementary if
yT(b−Ax
| {z } slack in
primal
) =0andxT(ATy−c
| {z } slack in
dual ) =0
In other words,
7.4. COMPLEMENTARY SLACKNESS 49
• wheneveryi>0, thenxsatisfies thei-th constraint with equality (“the constraint is tight”)
• wheneverxi>0, thenysatisfies thei-th constraint of the dual with equality
Exercise. Show that the shadow pricesπdefined by (a basic solution)xare always complementary tox. Recall that Strong Duality this says that ifxis an optimal solution to the primal andyis an optimal solution to the dual, thencTx=bTy. In fact, more is true.
Complementary Slackness (and some consequences): Assume thatxis an optimal solution to the primal.
• Ifyis an optimal solution to the dual, thenxandyare complementary.
• Ifyis a feasible solution in the dual and is complementary tox, thenyis optimal in the dual.
• There exists a feasible solutionyto the dual such thatxandyare complementary.
Notice that the last bullet follows from our observation about shadow prices. Another consequence of this is:
Ifxis a basic feasible primal solution andπare the corresponding shadow prices, then xis optimal if and only ifπis a feasible solution of the dual
If we have equalities,≥-inequalities, unrestricted or non-positive variables, everything works just the same.
Back to example
To check if the provided solutions are optimal, we need the dual.
max 6x1 + x2 − x3 − x4
x1 + 2x2 + x3 + x4 ≤ 5 3x1 + x2 − x3 ≤ 8 x2 + x3 + x4 = 1 x2,x3,x4 ≥ 0 x1unrestricted
min 5y1 + 8y2 + y3 y1 + 3y2 = 6 2y1 + y2 + y3 ≥ 1 y1 − y2 + y3 ≥ −1 y1 + y3 ≥ −1 y1,y2 ≥ 0 y3unrestricted
DUAL
a) x1=2,x2=1,x3=0,x4=0 → assumexis optimal, then
→there arey1,y2,y3such thaty= (y1,y2,y3)is feasible in the dual and complementary tox check1stprimal constraint:x1+2x2+x3+x4=2+2+0+0=4<5not tight
→thereforey1must be0becauseyis complementary tox
check2ndprimal constraint:3x1+x2−x3=6+1−0=7<8not tight
→thereforey2must be0becauseyis complementary tox
Knowing this, check the1stdual constraint:y1+3y2=0+0=06=6
→this shows that(y1,y2,y3)not feasible in the dual, but we assume that it is.
This means that our assumptions were wrong and so(x1,x2,x3,x4)is not optimal.
b) x1=3,x2=0,x3=1,x4=0 →again assume thatxis optimal, then
→there arey1,y2,y3such thaty= (y1,y2,y3)is feasible in the dual and complementary tox check1stprimal constraint:x1+2x2+x3+x4=3+0+1+0=4<5not tight → y1=0 check2ndprimal constraint:3x1+x2−x3=9+0−1=8tight
check3rdprimal constraint:x1+x2+x3=1tight
check sign constraints:x2,x3,x4≥0→we conclude that(x1,x2,x3,x4)is feasible in the primal Now we look at values inxwith respect to the dual constraints:
x1is unrestricted → 1stdual constrainty1+3y2=6is (always) tight
50 CHAPTER 7. DUALITY x3>0we deduce → 3rddual constraint must be tight:y1−y2+y3=−1
Together we have y1 = 0
y1 + 3y2 = 6 y1 − y2 + y3 = −1
This has a unique solutiony1=0,y2=2,y3=1. By construction, this solution is complementary tox.
The last step is to check ifyis also feasible in the dual. We already checked1stand3rddual constraint.
check2nddual constraint:2y1+y2+y3=0+2+1=3≥1→the constraint is satisfied check4thdual constraint:y1+y3=0+1≥ −1the constraint is satisfied
check sign restrictions:y1=0≥0andy2=2≥0→sign restrictions satisfied
→this shows that(y1,y2,y3)indeed a feasible solution to the dual.
From this we can conclude that(x1,x2,x3,x4)is indeed optimal.
Summary
• givenx, check ifxis feasible
• then find which variablesyishould be0
• then find which dual constraints should be tight
• this yields a system of equations
• solve the system
• verify that the solution is feasible in the dual
If all these steps succeed, then the givenxis indeed optimal; otherwise, it is not.
Question: what happens ifxis feasible but not a basic solution ?
Review
Max 3x1 + 2x2 x1 + x2 ≤ 80 2x1 + x2 ≤ 100
x1 ≤ 40
x1,x2 ≥ 0
x3 = 80 − x1 − x2
x4 = 100 − 2x1 − x2
x5 = 40 − x1
z = 0 + 3x1 + 2x2
x1 = 20 + x3 − x4
x2 = 60 − 2x3 + x4 x5 = 20 − x3 + x4 z = 180 − x3 − x4
Original problem Initial dictionary Optimal dictionary
Add a new activity: toy cars,12h carving,1h finishing,1unit towards demand limit,$1price→x6=#cars Max 3x1 + 2x2 + x6
x1 + x2 + 12x6 ≤ 80 2x1 + x2 + x6 ≤ 100 x1 + x6 ≤ 40 x1,x2,x6 ≥ 0
x3 = 80 − x1 − x2 − 12x6
x4 = 100 − 2x1 − x2 − x6
x5 = 40 − x1 − x6
z = 0 + 3x1 + 2x2 + x6
?
Original problem Initial dictionary Optimal dictionary
in the optimal dictionary→makex6non-basic (no production of cars)→feasible modified dictionary x3+12x6 = 80 − x1 − x2
x4+x6 = 100 − 2x1 − x2
x5+x6 = 40 − x1
z−x6 = 0 + 3x1 + 2x2
x3′ = 80 − x1 − x2
x4′ = 100 − 2x1 − x2
x5′ = 40 − x1
z′ = 0 + 3x1 + 2x2
→
x1 = 20 + x′3 − x′4 x2 = 60 − 2x′3 + x′4 x′5 = 20 − x′3 + x′4 z′ = 180 − x′3 − x′4 substitute:x3′ =x3+12x6, x′4=x4+x6, x′5=x5+x6, z′ =z−x6
7.4. COMPLEMENTARY SLACKNESS 51 x1 = 20 + (x3+12x6) − (x4+x6)
x2 = 60 − 2(x3+12x6) + (x4+x6) x5+x6 = 20 − (x3+12x6) + (x4+x6) z−x6 = 180 − (x3+12x6) − (x4+x6)
x1 = 20 + x3 − x4 − 12x6
x2 = 60 − 2x3 + x4
x5 = 20 − x3 + x4 − 12x6
z = 180 − x3 − x4 − 12x6
Add a new constraint: packaging, 250 units of cardboard, 3/soldier, 4/train, 1/car→x7=slack Max 3x1 + 2x2 + x6
x1 + x2 + 12x6 ≤ 80 2x1 + x2 + x6 ≤ 100 x1 + x6 ≤ 40 3x1 + 4x2 + x6 ≤ 250 x1,x2,x6 ≥ 0
x3 = 80 − x1 − x2 − 12x6
x4 = 100 − 2x1 − x2 − x6
x5 = 40 − x1 − x6
x7 = 250 − 3x1 − 4x2 − x6
z = 0 + 3x1 + 2x2 + x6
in the optimal dictionary→makex7basic→expresx7using the dictionary→new dictionary
x7=250−3x1−4x2−x6=250−3(20+x3−x4−12x6)−4(60−2x3+x4)−x6=−50+5x3−x4+12x6 x1 = 20 + x3 − x4 − 12x6
x2 = 60 − 2x3 + x4
x5 = 20 − x3 + x4 − 12x6
z = 180 − x3 − x4 − 12x6
→
x1 = 20 + x3 − x4 − 12x6
x2 = 60 − 2x3 + x4
x5 = 20 − x3 + x4 − 12x6
x7 = −50 + 5x3 − x4 + 12x6 z = 180 − x3 − x4 − 12x6
If the resulting dictionary is feasible, then it is also optimal (we don’t changez, all coeffs still non-positive) However, the resulting dictionary may be infeasible if some basic variable is negative (herex7<0)
→to recover optimal solution, we use Dual Simplex Method.
8
Other Simplex Methods
8.1 Dual Simplex Method
– we use when the dictionary is infeasible but dually feasible – same as Simplex on the Dual without dealing with the Dual directly – useful when adding new constraints to quickly recover optimal solution
Recall: for every (basic) solution of the Primal, we have shadow prices that we can assign to each item (constraint) in such a way that the total value of items in shadow prices is exactly the value of the solution
P R I M A L
Max 3x1 + 2x2 x1 + x2 ≤ 80 2x1 + x2 ≤ 100
x1 ≤ 40
x1,x2 ≥ 0
x1 = 40 − x5
x3 = 40 − x2 + x5 x4 = 20 − x2 + 2x5 z = 120 + 2x2 − 3x5
Solution:x1=40,x2=0,x3=40, x4=20,x5=0of valuez=120 Shadow prices:π1=0,π2=0,π3=3 80π1+100π2+40π3 = 120 = 3x1+2x3
D U A L
Min 80y1 + 100y2 + 40y3 y1 + 2y2 + y3 ≥ 3
y1 + y2 ≥ 2
y1,y2,y3 ≥ 0
Note: the above shadow prices→an infeasible solution in Dual 80π1 + 100π2 + 40π3 = 120 the same value as Primal
π1 + 2π2 + π3 = 3 ≥3 π1 + π2 = 0 6≥2 π1=0≥0,π2=0≥0,π3=3≥0
Shadow prices corresponding to a non-optimal feasible solution of the Primal are infeasible in the Dual Shadow prices corresponding to an optimal solution of the Primal are optimal in the Dual.
Shadow prices for an infeasible solution of the Primal may or may not be feasible in the Dual.
A solution of the Primal is dually feasible if the corresponding shadow prices are feasible in the dual.
Dual Simplex Algorithm
We maintain that the current solution is dually feasible but may itself be infeasible x1 = 20 + x3 − x4 − 12x6
x2 = 60 − 2x3 + x4
x5 = 20 − x3 + x4 − 12x6
x7 = −50 + 5x3 − x4 + 12x6 z = 180 − x3 − x4 − 12x6
How do we know that the solution is dually feasible?
. . . if all coefficients of variables inzare non-positive (Why? Because the coeffs of slack variables are shadow prices
while the coeffs of non-slack variables are reduced costs)
←is dually feasible 52
8.1. DUAL SIMPLEX METHOD 53
Dually feasible solution
Is feasible? Optimal
solution NO
Improve the solution
YES
cannot be improved LP is Infeasible
Sincex7 < 0, the solution is infeasible. We want to make it feasible. In order to do that, we need to increase the value ofx7. We makex7non-basic (value=0).
We can do this by increasing one of the non-basic variables (x3orx4orx6), i.e., by making one of the non-basic variables basic. Clearly,x4is not good for this since increasingx4makesx7only more negative (x4appears in the equation forx7with negative coefficient).
So we must increasex3orx6. But we also must make sure that this results in dually feasible dictionary, i.e., the resulting coefficients of non-basic variables inzare non-positive.
Ifx6enters the basis, thenx7=−50+5x3−x4+12x6 → 12x6=50−5x3+x4+x7and so
z=180−x3−x4−12x6=180−x3−x4−12×2(50−5x3+x4+x7) =130+4x3−2x4−x7
Ifx3enters the basis, thenx7=−50+5x3−x4+12x6 → 5x3=50+x4−12x6+x7and so
z=180−x3−x4−12x6=180−1× 15(50+x4−12x6+x7)−x4−12x6=170−65x4−25x6−15x7
Whenx6enters the basis, the solution is not dually feasible, while it is whenx3enters the basis. How do we find out?
Both substitutions are a result of adding tozsome multiple∆of the equation forx7. z=180−x3−x4−12x6−∆(
0
z }| {
50−5x3+x4−12x6+x7)
= (180−50∆) + (−1+5∆
| {z }
≤0
)x3+ (−1−∆
| {z }
≤0
)x4+ (−12+12∆
| {z }
≤0
)x6+ (−∆)
| {z }
≤0
x7
−1+5∆≤0
−1−∆≤0
−12 + 12∆ ≤ 0
−∆≤0
→
∆≤ 15
−1≤∆
∆≤1 0≤∆
→ 0≤∆≤ 15 → for∆= 15 the bound is tight forx3
→x3disappears fromz→enters the basis
Ratio test
Simplified procedure: compare coefficients of non-basic variables inzand inx7, choose smallest ratio x3:coeff−1x3inzand5x3inx7, ratio1
5 =0.2 x4:coeff−1x4inzand−1x4inx7, no constraint
(negative coeff inx7) x6:coeff−12x6inzand12x6inx7, ratio
12 12
=1
x1 = 20 + x3 − x4 − 12x6
x2 = 60 − 2x3 + x4
x5 = 20 − x3 + x4 − 12x6
x7 = −50 + 5x3 − x4 + 12x6 z = 180 − 1x3 − x4 − 12x6
ratio forx3: 1
5 =0.2
(again watch-out: we only consider this ratio because the coefficient ofx3is positive)
54 CHAPTER 8. OTHER SIMPLEX METHODS
Smallest ratio15forx3→x3enters x7=−50+5x3−x4+12x6
→ x3=10+15x4−101x6+15x7
x1 = 30 − 45x4 − 35x6 + 15x7 x2 = 40 + 35x4 + 15x6 − 25x7
x3 = 10 + 15x4 − 101x6 + 15x7 x5 = 10 + 45x4 − 25x6 − 15x7
z = 170 − 65x4 − 25x6 − 15x7
Shadow prices:
π1=0,π2= 65, π3=0,π4= 15 optimal solution to
the Dual.
(Why?) Solution is feasible (and dually feasible)→optimal solution found.
Summary
Starting with a dually feasible solution:
1. Find a basic variablexiof negative value.
2. If no suchxiexists→stop, the solution is feasible→optimal solution found.
3. Ratio test: in the dictionary, in the equation forxi, find a non-basic variablexjsuch that
• xjappears with positive coefficientain the equation forxi
• the ratio c
a is smallest possible (where−cis the coefficient ofxjinz)
4. If no suchxjexists→stop, no feasible solution exists→report that LP is Infeasible.
5. Pivotxjinto the basis,xileaves the basis.
(the resulting dictionary is guaranteed to be dually feasible) 6. Repeat.