y x
A
y
B
y x
x
C
Fig. 4.1.Convex and nonconvex sets.
Lemma 4.9.Let E be an affine space. The following statements are true.
(a) Intersections of convex sets are convex: if {Cγ}γ∈Γ is a family of convex sets in E, then∩γ∈ΓCγ is a convex set.
(b) Minkowski sums of convex sets are convex: if {Ci}ki=1 is a set of convex sets, then their Minkowski sum
C1+· · ·+Ck:={x1+· · ·+xk :xi∈Ci, i= 1, . . . , k} is a convex set.
(c) An affine image of a convex set is convex: ifC ⊆E is a convex set and T : E → F is an affine map from E into another affine space F, then T(C)⊆F is also a convex set.
Proof. These statements are all easy to prove; we prove only (a). Letx, y∈ C := ∩γ∈ΓCγ. For each γ ∈ Γ, we have x, y ∈ Cγ, and since Cγ is convex, [x, y]⊆Cγ; therefore, [x, y]⊆Cand Cis a convex set. ut Definition 4.10.Let {xi}k1 be a finite set of points in an affine spaceE. A convex combinationof {xi}k1 is any point of the form
k
X
i=1
λixi, λi≥0,
k
X
i=1
λi= 1.
Let A⊆E be a nonempty set. The convex hullof A is the set of all convex combinations of points fromA, that is,
co(A) :=
k X
i=1
λixi : xi∈A,
k
X
i=1
λi= 1, λi≥0, k≥1
.
Theorem 4.11.LetA6=∅ be a subset of an affine spaceE. Thenco(A)is a convex set; in fact,co(A)is the smallest convex set containingA.
Proof. The proof is essentially a repeat of the proof of Lemma 4.2, but we now make the additional requirements that 0< α <1 and that {λi}k1, {µj}l1 be nonnegative in that proof. It suffices to note that all the affine combinations
now become convex combinations. ut
4.2 Convex Sets 91 The following result is an immediate consequence of the above theorem.
Corollary 4.12.If C is a convex set in an affine space E, thenco(C) =C, that is, all convex combinations of elements fromC lie inC,
λi≥0, xi∈C, i= 1, . . . , k,
k
X
i=1
λi= 1 =⇒
k
X
i=1
λixi∈C.
The following theorem, due to Carath´eodory [53], is a fundamental result in convexity in finite-dimensional vector spaces, and has many applications, including in optimization.
Theorem 4.13. (Carath´eodory)Let A be a nonempty subset of an affine spaceE. Every element ofco(A)can be represented as a convex combination of affinely independent elements from A.
Consequently, if n= dim(aff(A))<∞, then every element of co(A) can be represented as a convex combination of at mostn+ 1elements from A; in other words,
co(A) =nn+1X
i=1
λixi:xi∈A, λi≥0, i= 1, . . . , n+ 1,
n+1
X
i=1
λi = 1o . Proof. Let
x=
k
X
i=1
λixi∈co(A), where
k
X
i=1
λi= 1, λi>0. (4.3) If{xi}k1 is affinely independent, then {xi−x1}k2 is linearly independent and k−1≤n; thus,k≤n+ 1, and the theorem is proved.
Suppose that{xi}k1 is affinely dependent. It follows from Lemma 4.6 that there exist scalars{δi}k1 such that
k
X
i=1
δixi= 0,
k
X
i=1
δi= 0, (δ1, . . . , δk)6= 0. (4.4) If we subtract from (4.3)εtimes (4.4) (ε >0), we obtain
x= (λ1−εδ1)x1+· · ·+ (λk−εδk)xk,
k
X
i=1
(λi−εδi) = 1. (4.5) Since Pk
i=1δi = 0, there exist positive and negative scalars δi. If δi ≤ 0, thenλi−εδi ≥0 remains nonnegative for all ε≥0; however, ifδi >0, then λi−εδi≥0 if and only ifε≤λi/δi. Therefore, if we setε= min{λi/δi:δi>
0}, thenxremains a convex combination in (4.5), but has at least one fewer term. We can continue this process until the vectors{x2−x1, . . . , xk−x1}in the representation (4.3) are linearly independent. When we halt, we will have
k≤n+ 1. ut
We immediately have the following.
Corollary 4.14.IfC is a nonempty subset of ann-dimensional vector space E, then every element of co(A) can be represented as a convex combination of at most n+ 1 elements fromA.
Corollary 4.15.If C is a nonempty compact subset of a finite-dimensional affine spaceE, then so is the setco(C).
Proof. It follows from Theorem 4.13 that co(C) =
n+1 X
i=1
λixi:λi≥0, xi∈C, i= 1, . . . , n+ 1,
n+1
X
i=1
λi= 1
,
wheren= dim(C). Consider a sequence{xk}∞1 in co(C), where xk=
n+1
X
i=1
λkixki.
SinceC is compact, the sequence {xk1} has a convergent subsequencexk1j → x1 ∈ C. Next, let the sequence {xk2j} have convergent subsequence xk2jl → x2∈C, and so on. Eventually, we can find a subsequencekj such that
j→∞lim xkij =xi∈C for all i= 1, . . . , n+ 1.
Using the same arguments, we can assume that limj→∞λkij =λi≥0 for all i= 1, . . . , n+ 1. Then we havePn+1
i=1 λi= 1, and xkj →
n+1
X
i=1
λixi∈co(C).
This proves that co(C) is compact. ut
An alternative proof runs as follows: Let
∆n:=
(λ1, . . . , λn+1) :λi≥0, i= 1, . . . , n+ 1,
n+1
X
i=1
λi= 1
be the standard unit simplex inRn+1, and consider the map
∆n×C× · · · ×C
| {z }
n+1 times
→E,
given by
4.2 Convex Sets 93
T(λ1, . . . , λn+1, x1, . . . , xn+1) =
n+1
X
i=1
λixi.
Note that the image of T is co(C). Since the map T is continuous and the domain ofT is compact (∆n and Care compact), we conclude that co(C) is compact.
We also record here the following elementary results.
Lemma 4.16.LetEbe an affine space in a normed vector space. IfC⊆Eis a convex set, then its closureCis also a convex set. IfC1, C2⊆E are convex sets,C1 is compact, and C2 is closed, thenC1+C2 is a closed, convex set.
Proof. To prove the first statement, define the convex set
C:={z:kz−xk< , x∈C}={x+u:x∈C, kuk< }=C+B(0).
Note thatC :=∩>0C, because a pointz ∈E lies inC if and only if given > 0, there exists a point x ∈ C such that kz−xk < . It follows from Lemma 4.9 thatCis a convex set.
To prove the second statement, let {zk}∞k=1 be a sequence in C1+C2
converging to a pointz. Writezk=xk+yk withxk∈C1 andyk∈C2. Since C1 is compact, there exists a subsequencexki →x∈C1. Since zki →z, yki
must converge to the point y := z−x∈ C2. Thus, z = x+y ∈ C1+C2,
proving thatC1+C2 is a closed set. ut
4.2.1 Convex Cones
Definition 4.17.A set K in a vector space E is called a cone if tx ∈ K whenever t > 0 and x ∈ K. If K is also a convex set, then it is called a convex cone.
Lemma 4.18.A setK in a vector space E is a convex cone if and only if x, y∈K andt >0 =⇒ tx∈K, x+y∈K. (4.6) Proof. Letx, y∈ K. IfK is a convex cone, then (x+y)/2 ∈K, since K is convex, andx+y = 2((x+y)/2)∈K, since K is a cone. This proves (4.6).
Conversely, if 0< t <1 and (4.6) holds, then (1−t)x+ty∈K, proving that
K is a convex set. ut
Many concepts and results for convex sets have analogues for convex cones.
Definition 4.19.Let {xi}k1 be a finite set of points in a vector space E. A positive combination of{xi}k1 is any point of the form
k
X
i=1
λixi, λi>0, i= 1, . . . , k.
Let A⊆E be a nonempty set. The convex conical hull of A is the set of all positive combinations of points fromA, that is,
cone(A) :=
k X
i=1
λixi : xi∈X, λi>0, k≥1
.
Theorem 4.20.LetAbe a nonempty set in a vector space E. Thencone(A) is the smallest convex cone containing A. If K is a convex cone, then cone(K) =K.
This is proved in exactly the same way as Theorem 4.11. In fact, the proof here is somewhat simpler, since the weights {λi} in a positive combination are not required to sum to one.
Theorem 4.21. (Carath´eodory) Let A be a nonempty subset of a vector space E. Every element of cone(A) can be represented as a positive com- bination of linearly independent elements from A. Consequently, if n = dim(span(A)) < ∞, then every element of cone(A) can be represented as a positive combination of at mostnelements from A. In other words,
cone(A) = n
X
i=1
λixi:xi∈A, λi>0, i= 1, . . . , n
. Proof. The proof is essentially the same as in the affine case. Let
x=
k
X
i=1
λixi∈cone(A), where allλi>0. (4.7) If the vectors{xi}k1 are linearly dependent, then there exist scalars {δi}ki=1, not all zero, such that
k
X
i=2
δixi= 0.
If we subtract from (4.3)εtimes this equation, we obtain x= (λ1−εδ1)x1+· · ·+ (λk−εδk)xk.
The rest of the proof is completed using the same arguments in the proof of
Theorem 4.13. ut
Corollary 4.22.IfC is a nonempty subset of ann-dimensional vector space E, then every element ofcone(A)can be represented as a positive combination of at most nelements from A.