The simplex method provides a systematic algebraic procedure for moving from one extreme point to an adjacent one while improving the function value. The method was developed by George Dantzig in 1947 and has been widely used since then.
The first step is to convert the LP problem into standard form as in (4.6), with b≥ 0. By introducing slack variables,x3,x4, andx5, the problem defined in (4.4) then becomes
maximize 2x1+ x2
subject to 2x1− x2+x3 =8 (1)
x1+2x2+x4=14 (2)
−x1+ x2+x5 =4 (3) x1≥0, x2≥0, x3≥0, x4≥0, x5≥0
Denoting cT = [2, 1, 0, 0, 0], xT = [x1, x2, x3, x4, x5], bT = [8, 14, 4], and A=
2 −1 1 0 0
1 2 0 1 0
−1 1 0 0 1
, the problem can be stated in the form of (4.6). The three equality constraints are defined in terms of five variables. If these equations are linearly independent (Ais of rank 3), a solution can be obtained if any two (=5 – 3) variables are set equal to zero. This solution is called abasic solution. IfAis anm rown+mcolumn matrix, wherenis the original dimension of the problem andm is the number of constraints, a basic solution is obtained by settingnvariables equal to zero. If all the variables of the basic solution are nonnegative, then the solution is abasic feasible solution(bfs). Every basic feasible solution is an extreme point of the feasible polytope.
The simplex method is initiated when:
(i) a basic feasible solution is obtained,
(ii) the objective is written only in terms of the nonbasic variables.
For the problem with LE constraints, the standard form given by (4.6) is canonical. If we set x1 = 0 and x2 = 0, then x3 = 8 (= b1), x4 = 14 (= b2),
4.7 The Simplex Method – Starting with LE (≤) Constraints 147 x5 =4 (=b3). The basic feasible solution is, thus, readily obtained: the slacks are basic, the original variables are nonbasic. Moreover, the cost row is in terms of the original or nonbasic variables.1 This starting point is the cornerAinFig. 4.3. Now, sincex1has a larger coefficient we decide to move alongx1fromA. We have, thus, decided thatx1should turn into a basic variable or, in other words, we sayx1enters the basis. In the simplex method, the first row is set as negative of the objective function row for maximization, and we look for the variable with the largest nega- tive coefficient (explained in detail subsequently). The simplex method is initiated with the following tableau
First Tableau (Corresponds to PointA):
x1 x2 x3 x4 x5 rhs
f −2 −1 0 0 0 0
x3 [ 2] −1 1 0 0 8
x4 1 2 0 1 0 14
x5 −1 1 0 0 1 4
Asx1is increased, we look for the largest value that still retains feasibility. This is checked by performing theratio test. For each constraintiwith a positive coeffi- cientai1, we evaluate the ratiobi/ai1. Among these ratios, we identify the smallest value. This row is now referred to as the pivot row and the column corresponding to the variable entering the basis is the pivot column. In the problem at hand, we com- pare 8/2 and 14/1. The first row is the pivot row. The third coefficient is negative, and this does not control the upper limit onx1. The basic variable corresponding to this row isx3. Pivot element is [2]. Thus,x3leaves the basis. That is,x3becomes zero, which, in view of the fact that it is the slack in constraint 1, means that we move to a point where the constraint becomes active. InFig. 4.3, we move fromA toB. Elementary row operations (ERO) as discussed in detail in Chapter 1 are now performed for pivot element at row 1 and column 1. The idea of the operation is to make the pivot location equal to 1 by dividing the entire equation by the pivot value, and making the other coefficients in that column equal to zero by adding an appro- priate multiple of the row to each of the other constraint equations. The elementary row operation is performed on the row corresponding to the objective function also.
The second tableau is as follows:
1 Significance of cost row coefficients are explained further under the heading “Optimality Conditions for Problems with LE Constraints.”
Second Tableau (Corresponds to PointB):
x1 x2 x3 x4 x5 rhs
f 0 −2 1 0 0 8
x1 1 −12 12 0 0 4
x4 0 5
2
−12 1 0 10
x5 0 12 12 0 1 8
Now the most negative coefficient in the first row is−2 corresponding to vari- ablex2. Thus,x2 enters the basis. The ratio test 10/(5/2) and 8/(1/2) shows thatx4
should leave the basis. Pivot element is 5/2. After performing the elementary row operations, the third tableau becomes
Third Tableau (Corresponds to PointC):
x1 x2 x3 x4 x5 rhs
f 0 0 35 45 0 16
x1 1 0 25 15 0 6
x2 0 1 −15 25 0 4
x5 0 0 35 −15 1 6
Now all the coefficients in the first row are positive, and it yields the max- imum value of the function f = 16. The solution is x1 = 6, x2 = 4, x3 = 0, x4=0,x5=6.
In the program SIMPLEX, the first row is set as the last row. Further, the ratio test needs to be implemented with a small tolerance as: B(I)/(A(I,ICV)+1E− 10),where ICV is the incoming variable.
Minimization
IfcTxis to be minimized, it is equivalent to maximization of−cTx. Taking the nega- tive coefficients of the maximization problem, the first row has just thecvalues. The right-hand side will be obtained as negative of the function value. We need to apply a final change in sign for the function value.
4.7 The Simplex Method – Starting with LE (≤) Constraints 149 Adegenerate basic feasible solutionis one in which one or more variables in the basis have a value of zero. In this situation, since nonbasic variables are at zero, bringing zero level basic variable into nonbasic set does not improve the function value. However, degenerate solutions do not pose problems in the working of the simplex method. Some aspects of recycling are discussed in the next section.
Steps Involved in the Simplex Method for LE Constraints
Step 1: Create the initial tableau in standard form, with negative of the objec- tive coefficients for maximization problem (original coefficients for minimiza- tion).
Step 2: Choose the incoming variable j corresponding to the most negative objective row coefficient (pivot column). If there is no negative coefficient, the optimum has been attained.
Step 3: Perform the ratio testbi/aijfor positive coefficients2 aijand determine the rowicorresponding to the least ratio (pivot row).If there is no positive coefficient in the column, the solution is unbounded.
Step 4: Perform the elementary row operations to make the pivot equal to 1 and other coefficients in the column equal to 0, including the first row. Go to Step 2.
Optimality Conditions and Significance of Cost Row Coefficients
In the preceding text, slacksx3,x4,x5were chosen as basic to start with, point A in Fig. 4.3, which resulted in the objectivef=2x1+x2to be automatically expressed in terms of nonbasic variables. This is desired because we may then scan the coeffi- cients to see which nonbasic variable, currently at zero value, should enter the basis so as to increase the objective function value. However, for the objective maximize cTx, it was stated earlier, without explanation that –cmust be inserted in the cost row in the simplex tableau. The reasoning behind this has to do with optimality and is now explained. Once the initial tableau is defined, the steps above will lead to an optimum (or detection of infeasibility or unboundedness).
Recall from the earlier chapters that the gradient of a function is normal to its contour and points in the direction of increasing function value. Now, it is evident fromFig. 4.3that pointAis not an maximum. Observing the direction ofcand the feasible region, any “move” along thex1- orx2-axis will increasefwhile staying in . More generally, sincecis the gradient off, any direction of travel that points into the half-space towardcwill increasef. At pointB, a move fromBtoCwill improve the objective, at pointE, a move from pointEto D, and at pointDa move from
2 In a computer code, a small tolerance must be used to decide whenai jis positive.
d
D c
Figure 4.4. Nonoptimality at pointDin Fig. 4.3.
DtoC. At pointC, there is no move that will improve the objective indicating that Cis optimum.Figure 4.4illustrates the situation at pointD – any direction vector in the dotted cone will improvefwhile staying in, although the simplex method will choosedas in the figure, and thus, move along the boundary, from vertex to vertex.
Karusch–Kuhn–Tucker or KKT conditions describe optimality or nonoptimal- ity at a point generally so as to be applicable in N-dimensions. While these are explained in greater detail in Chapter 5, they are also discussed below to illumi- nate the choice and significance of cost row coefficients. For maximization, the cost gradient is expressed as a linear combination ofactiveconstraint gradients as
c=
i∈I
μiNi (4.7)
whereI=index set of active constraints at a point, andNiis the gradient or normal of theith active constraint (either ai or−ei),μi are Lagrange multipliers. Equa- tion(4.7)states thatclies in theconvex coneformed by the gradients of the active constraints.
Optimality with LE (≤) constraints: Equation(4.7) must be satisfied with all μi ≥0,i ∈ I, and the point must be feasible.
At point A, active constraint gradients are [−10],[−10]. Thus, c=[21] are expressed in terms of the constraint gradients as [21]= −2[−01]−1[−01], from which we haveμ1 = −2,μ2 = −1. The cost row coefficients play the role ofμi. Hence, as per KKT conditions, pointAis not optimal.
At point D, we can write [21]=μ1[−11]+μ2[12], which, upon solving, yields μ1= −1,μ2=1, hence, not optimal.
At point C, we can write [21]=μ1[12]+μ2[−21], which gives μ1 = 0.8, μ2=0.6, hence, optimal. PointCis a “KKT point.”
[What are the Lagrange multipliers at pointsBandE? Relate multipliers atB with the simplex tableau earlier]
Geometric Interpretation of the Pivot Operation
The choice of the variable entering the basis, based on most negative reduced cost coefficient (step 2 earlier) can be interpreted asa move based on maximum expected
4.7 The Simplex Method – Starting with LE (≤) Constraints 151 increase in objective function(as we choose the most negative cost row coefficient to enter the basis), until pointCis reached. Thus, at pointA, sincec=[2, 1]T, the
“slope” or gradient of the objective fromAtoBequals 2 while fromAtoEequals 1. Thus, based on information at pointAonly, the move fromAtoBcorresponds tomaximum expected increasein objective function, and theactive bound constraint x2 becomes inactive (i.e., x2 becomes positive). However, theactual increase also depends on the distance fromAtoBor fromAtoE, which will be known only after the pivot operation.
Further, the choice of the variable that leaves the basis is essentially the deter- mination of a variable that changes from a positive value to zero. This reduces to determination of which constraint becomesactive. If the variable is an original vari- able, then theboundbecomes active. If the variable corresponds to a slack, then the corresponding inequality constraint becomes active. The ratio test in step 3 earlier determines theclosest constraint that intersects the pathfrom the existing vertex to the next.
In summary, incoming variable (step 2) determines adirection of movementinx- space, while the outgoing variable (step 3) determines thestep sizeto the boundary of the feasible polytope.
KKT Conditions
KKT conditions in Eq.(4.7) may be restated by associating zero value Lagrange multipliers with inactive constraints. That is, fori∈/ I, implyinggi <0, we require μi=0. This can be enforced by thecomplementarity conditionμigi=0.
Problem
maximize cTx subject to Ax≤b
x≥0 (4.8)
KKT Conditions
Feasibility Ax+y=b (4.9)
Optimality c=ATv−u (4.10)
Complementarity uTx+vTy=0 (4.11)
Nonnegativity x≥0, y≥0, u≥0, v≥0 (4.12) Note that feasibilityAx≤bhas been expressed asAx+y=b,y≥0. Direct solu- tion of the KKT conditions Eqs.(4.9)–(4.12)is difficult owing to the complemen- tarity conditions, or equivalently, owing to the task of determining the active set.
The KKT conditions define a complementary problem. The linear complementary problem (LCP) for quadratic programming (QP) has a very similar structure, as discussed in a later section.