Vectors in Rn
The algebraic idea of a vector inR2orR3is defined in§1.1, in which we asked you to consider what would be involved in generalizing the operations of vector addition, scalar multiplication, etc., to n-dimensional vectors, where n can be arbitrary. We explore some of the details of such a generalization next.
DEFINITION 6.1 AvectorinRnis an orderedn-tuple of real numbers. We usea=(a1,a2, . . . ,an) as our standard notation for a vector inRn.
EXAMPLE 1 The 5-tuple (2,4,6,8,10) is a vector inR5. The (n+1)-tuple (2n,2n−2,2n−4, . . . ,2,0) is a vector inRn+1, wherenis arbitrary. ◆ Exactly as is the case inR2orR3, we call two vectorsa=(a1,a2, . . . ,an) and b=(b1,b2, . . . ,bn)equaljust in caseai =bifori =1,2, . . . ,n. Vector addition and scalar multiplication are defined in complete analogy with Definitions 1.3 and 1.4: Ifa=(a1,a2, . . . ,an) andb=(b1,b2, . . . ,bn) are two vectors inRn and k∈Ris any scalar, then
a+b=(a1+b1,a2+b2, . . . ,an +bn) and
ka=(ka1,ka2, . . . ,kan).
The properties of vector addition and scalar multiplication given in §1.1 hold (with proofs that are no different from those in the two- and three-dimensional cases). Similarly, the dot product of two vectors inRnis readily defined:
a·b=a1b1+a2b2+ · · · +anbn.
The dot product properties given in§1.3 continue to hold inndimensions; we leave it to you to check that this is so.
What wecannotdo in dimensions larger than three is to develop a pictorial representation for vectors as arrows. Nonetheless, the power of our algebra and analogy does allow us to define a number of geometric ideas. We define thelength of a vector ina∈Rn by using the dot product:
a =√ a·a. Thedistancebetween two vectorsaandbinRn is
Distance betweenaandb= a−b.
We can even define theanglebetween two nonzero vectors by using a generalized version of equation (4) of§1.3:
θ=cos−1 a·b a b.
Here a,b∈Rn andθ is taken so that 0≤θ ≤π. (Note: At this point in our discussion, it is not clear that we have
−1≤ a·b a b ≤1,
which is a necessary condition if our definition of the angleθ is to make sense.
Fortunately, the Cauchy–Schwarz inequality—formula (1) that follows—takes care of this issue.) Thus, even though we are not able to draw pictures of vectors inRn, we can nonetheless talk about what it means to say that two vectors are perpendicular or parallel, or how far apart two vectors may be. (Be careful about this business. We aredefiningnotions of length, distance, and angle entirely in
terms of the dot product. Results like Theorem 3.3 have no meaning inRn, since the ideas of angles between vectors and dot products are not independent.)
There is no simple generalization of the cross product. However, see Exer- cises 39–42 at the end of this section for the best we can do by way of analogy.
We can create a standard basis of vectors in Rn that generalize the i, j, k-basis inR3. Let
e1 =(1,0,0, . . . ,0), e2 =(0,1,0, . . . ,0),
...
en =(0,0, . . . ,0,1). Then it is not difficult to see (check for yourself ) that
a=(a1,a2, . . . ,an)=a1e1+a2e2+ · · · +anen. Here are two famous (and often handy) inequalities:
Cauchy–Schwarz inequality. For all vectorsaandbinRn, we have
|a·b| ≤ a b. (1)
PROOF Ifn=2 or 3, this result is virtually immediate in view of Theorem 3.3.
However, in dimensions larger than three, we do not have independent notions of inner products and angles, so a different proof is required.
First note that the inequality holds if eitheraorbis0. So assume thataand bare nonzero. Then we may define the projection ofbontoajust as in§1.3:
projab= a·b a·a
a=ka.
Herekis, of course, the scalara·b/a·a. Letc=b−ka(so thatb=ka+c).
Then we havea·c=0, since
a·c=a·(b−ka)
=a·b−ka·a
=a·b− a·b a·a
a·a
=a·b−a·b
=0.
We leave it to you to check that the “Pythagorean theorem” holds, namely, that the following equation is true:
b2 =k2a2+ c2. Multiply this equation bya2=a·a. We obtain
a2b2 = a2k2a2+ a2c2
= a2 a·b a·a
2
a2+ a2c2
=(a·a) a·b a·a
2
(a·a)+ a2c2
=(a·b)2+ a2c2. Now, the quantitya2c2is nonnegative. Hence,
a2b2 ≥(a·b)2.
Taking square roots in this last inequality yields the result desired. ■ ka
a b c
Figure 1.80 The geometry behind the proof of the Cauchy–Schwarz inequality.
The geometric motivation for this proof of the Cauchy–Schwarz inequality comes from Figure 1.80.1
The triangle inequality. For all vectorsa,b∈Rn we have
a+b ≤ a + b. (2)
PROOF Strategic use of the Cauchy–Schwarz inequality yields a+b2 =(a+b)·(a+b)
=a·a+2a·b+b·b
≤a·a+2a b +b·b by (1)
= a2+2a b + b2
=(a + b)2.
Thus, the result desired holds by taking square roots, since the quantities on both
sides of the inequality are nonnegative. ■
a
b
a + b Figure 1.81 The triangle inequality visualized.
In two or three dimensions the triangle inequality has the following obvious proof from which the inequality gets its name: Sincea,b, anda+bcan be viewed as the lengths of the sides of a triangle, inequality (2) says nothing more than that the sum of the lengths of two sides of a triangle must be at least as large as the length of the third side, as demonstrated by Figure 1.81.
Matrices
We had a brief glance at matrices and determinants in§1.4 in connection with the computation of cross products. Now it’s time for another look.
A matrix is defined in§1.4 as a rectangular array of numbers. To extend our discussion, we need a good notation for matrices and their individual entries. We used the uppercase Latin alphabet to denote entire matrices and will continue to do so. We shall also adopt the standard convention and use the lowercase Latin alphabet and two sets of indices (one set for rows, the other for columns) to identify matrix entries. Thus, the generalm×nmatrix can be written as
A=
⎡
⎢⎢
⎣
a11 a12 · · · a1n a21 a22 · · · a2n ... ... . .. ...
am1 am2 · · · amn
⎤
⎥⎥
⎦= (shorthand) (ai j).
1 See J. W. Cannon,Amer. Math. Monthly96(1989), no. 7, 630–631.
The first indexalwayswill represent the row position and the second index, the column position.
Vectors inRn can also be thought of as matrices. We shall have occasion to write the vectora=(a1,a2, . . . ,an) either as arow vector(a 1×nmatrix),
a=
a1 a2 · · · an , or, more typically, as acolumn vector(ann×1 matrix),
a=
⎡
⎢⎢
⎣ a1
a2
... an
⎤
⎥⎥
⎦.
We did not use double indices since there is only a single row or column present.
It will be clear from context (or else indicated explicitly) in which form a vector awill be viewed. Anm×nmatrix Acan be thought of as a “vector of vectors”
in two ways: (1) asmrow vectors inRn,
A=
⎡
⎢⎢
⎢⎢
⎣
a11 a12 · · · a1n a21 a22 · · · a2n
...
am1 am2 · · · amn
⎤
⎥⎥
⎥⎥
⎦,
or (2) asncolumn vectors inRm,
A=
⎡
⎢⎢
⎣
⎡
⎢⎢
⎣ a11 a21 ... am1
⎤
⎥⎥
⎦
⎡
⎢⎢
⎣ a12 a22 ... am2
⎤
⎥⎥
⎦ · · ·
⎡
⎢⎢
⎣ a1n a2n ... amn
⎤
⎥⎥
⎦
⎤
⎥⎥
⎦.
We now define the basic matrix operations. Matrix addition and scalar mul- tiplication are really no different from the corresponding operations on vectors (and, moreover, they satisfy essentially the same properties).
DEFINITION 6.2 (MATRIXADDITION) LetAandBbe twom×nmatrices.
Then theirmatrix sum A+Bis them×nmatrix obtained by adding cor- responding entries. That is, the entry in theith row and jth column ofA+B isai j+bi j, whereai j andbi j are thei jth entries ofAandB, respectively.
EXAMPLE 2 If A=
1 2 3
4 5 6
and B=
7 0 −1
−2 5 0
, then
A+B=
8 2 2
2 10 6
.
However, if B=
7 1
5 3
, then A+Bis not defined, since B does not have
the same dimensions asA. ◆
Properties of matrix addition. For allm×nmatricesA,B, andCwe have 1. A+B= B+A(commutativity);
2. A+(B+C)=(A+B)+C (associativity);
3. Anm×nmatrixO(thezero matrix) with the property thatA+O = A for allm×nmatrices A.
DEFINITION 6.3 (SCALARMULTIPLICATION) If Ais anm×nmatrix and k∈Ris any scalar, then theproductk Aof the scalarkand the matrix Ais obtained by multiplying every entry in Abyk. That is, thei jth entry ofk A iskai j (whereai jis thei jth entry of A).
EXAMPLE 3 If A=
1 2 3
4 5 6
, then 3A=
3 6 9 12 15 18
. ◆
Properties of scalar multiplication. If Aand B are anym×n matrices andkandl are any scalars, then
1. (k+l)A=k A+l A(distributivity);
2. k(A+B)=k A+k B(distributivity);
3. k(l A)=(kl)A=l(k A).
We leave it to you to supply proofs of these addition and scalar multiplication properties if you wish.
Just as defining products of vectors needed to be “unexpected” in order to be useful, so it is with defining products of matrices. To a degree, matrix multipli- cation is a generalization of the dot product of two vectors.
DEFINITION 6.4 (MATRIXMULTIPLICATION) Let Abe anm×n matrix andBann× pmatrix. Then thematrix product A Bis them× pmatrix whosei jth entry is the dot product of theith row of Aand the jth column ofB(considered as vectors inRn). That is, thei jth entry of
⎡
⎢⎢
⎢⎢
⎢⎣
a11 a12 · · · a1n
... ...
[ai1 ai2 · · · ai n]
... ...
am1 am2 · · · amn
⎤
⎥⎥
⎥⎥
⎥⎦
⎡
⎢⎢
⎣ b11 b21 ... bn1
· · ·
· · ·
⎡
⎢⎢
⎣ b1j b2j
... bn j
⎤
⎥⎥
⎦
· · ·
· · · b1p b2p
... bnp
⎤
⎥⎥
⎦
is
ai1b1j +ai2b2j+ · · · +ai nbn j = (more compactly) n
k=1
ai kbk j.
EXAMPLE 4 If A=
1 2 3
4 5 6
and B=
⎡
⎣ 0 1
7 0
2 4
⎤
⎦,
then the (2,1) entry ofA Bis the dot product of the second row of Aand the first column ofB:
(2,1) entry=
4 5 6
·
⎡
⎣ 0 7 2
⎤
⎦=(4)(0)+(5)(7)+(6)(2)=47.
The full productA Bis the 2×2 matrix 20 13
47 28
. On the other hand,B Ais the 3×3 matrix
⎡
⎣ 4 5 6
7 14 21
18 24 30
⎤
⎦.
◆
Order matters in matrix multiplication. To multiply two matrices we must have
Number of columns of left matrix=number of rows of right matrix.
In Example 4, the products A B and B A are matrices of different dimensions;
hence, they could not possibly be the same. A worse situation occurs when the matrix product is defined in one order and not the other. For example, ifAis 2×3 andBis 3×3, thenA Bis defined (and is a 2×3 matrix), butB Ais not. However, even if both products A BandB Aare defined and of the same dimensions (as is the case ifAandBare bothn×n, for example), it is in general still true that
A B= B A.
Despite this negative news, matrix multiplication does behave well in a number of respects, as the following results indicate:
Properties of matrix multiplication. Suppose A, B, andC are matrices of appropriate dimensions (meaning that the expressions that follow are all defined) and thatkis a scalar. Then
1. A(BC)=(A B)C;
2. k(A B)=(k A)B= A(k B);
3. A(B+C)= A B+ AC;
4. (A+B)C = AC+BC.
The proofs of these properties involve little more than Definition 6.4, although the notation can become somewhat involved, as in the proof of property 1.
One simple operation on matrices that has no analogue in the real number system is thetranspose. The transpose of anm×nmatrixAis then×mmatrix
AT obtained by writing the rows of Aas columns. For example, if A=
1 2 3
4 5 6
, then AT =
⎡
⎣ 1 4
2 5
3 6
⎤
⎦.
More abstractly, thei jth entry ofAT isaj i, the j ith entry ofA.
The transpose operation turns row vectors into column vectors and vice versa.
We also have the following results:
(AT)T = A, for any matrixA. (3)
(A B)T = BTAT, whereAism×nandBisn× p. (4) The transpose will largely function as a notational convenience for us. For example, considera,b∈Rnto be column vectors. Then the dot producta·bcan be written in matrix form as
a·b=a1b1+a2b2+ · · · +anbn =
a1 a2 · · · an
⎡
⎢⎢
⎣ b1
b2
... bn
⎤
⎥⎥
⎦=aTb.
EXAMPLE 5 Matrix multiplication is defined the way it is so that, roughly speaking, working with vectors or quantities involving several variables can be made to look as much as possible like working with a single variable. This idea will become clearer throughout the text, but we can provide an important example now. Alinear functionin a single variable is a function of the form f(x)=ax wherea is a constant. The natural generalization of this to higher dimensions is alinear mapping F:Rn →Rm,F(x)= Ax, where A is a (constant)m×n matrix andx∈Rn. More explicitly, F is a function that takes a vector in Rn (written as a column vector) and returns a vector inRm(also written as a column).
That is,
F(x)= Ax=
⎡
⎢⎢
⎣
a11 a12 · · · a1n a21 a22 · · · a2n
... ... . .. ...
am1 am2 · · · amn
⎤
⎥⎥
⎦
⎡
⎢⎢
⎣ x1 x2 ... xn
⎤
⎥⎥
⎦.
The functionFhas the properties thatF(x+y)=F(x)+F(y) for allx,y∈Rn andF(kx)=kF(x) for all x∈Rn,k∈R. These properties are also satisfied by f(x)=ax, of course. Perhaps more important, however, is the fact that linear mappings behave nicely with respect to composition. SupposeFis as just defined andG:Rm→Rp is another linear mapping defined byG(x)= Bx, whereBis a p×mmatrix. Then there is a composite functionG◦F:Rn →Rpdefined by
G◦F(x)=G(F(x))=G(Ax)= B(Ax)=(B A)x
by the associativity property of matrix multiplication. Note that B Ais defined and is ap×nmatrix. Hence, we see that the composition of two linear mappings is again a linear mapping. Part of the reason we defined matrix multiplication the
way we did is so that this is the case. ◆
EXAMPLE 6 We saw that by interpreting equation (1) in§1.2 inndimensions, we obtain parametric equations of a line inRn. Equation (2) of§1.5, the equation
for a plane in R3 through a given point (x0,y0,z0) with given normal vector n= Ai+Bj+Ck, can also be generalized tondimensions:
A1(x1−b1)+ A2(x2−b2)+ · · · +An(xn−bn)=0.
If we letA=(A1,A2, . . . ,An), b=(b1,b2, . . . ,bn) (“constant” vectors), and x=(x1,x2, . . . ,xn) (a “variable” vector), then the aforementioned equation can be rewritten as
A·(x−b)=0 or, consideringA,b, andxasn×1 matrices, as
AT(x−b)=0.
This is the equation for ahyperplane in Rn through the point b with normal vectorA. The pointsxthat satisfy this equation fill out an (n−1)-dimensional
subset ofRn. ◆
At this point, it is easy to think that matrix arithmetic and the vector geometry ofRn, although elegant, are so abstract and formal as to be of little practical use.
However, the next example, from the field of economics,2 shows that this is not the case.
EXAMPLE 7 Suppose that we havencommodities. If the price per unit of the ith commodity is pi, then the cost of purchasing xi (>0) units of commodity i is pixi. If p=(p1, . . . ,pn) is the price vector of all the commodities and x=(x1, . . . ,xn) is thecommodity bundlevector, then
p·x= p1x1+p2x2+ · · · +pnxn
represents the total cost of the commodity bundle.
Now suppose that we have an exchange economy, so that we may buy and sell items. If you have an endowment vectorw=(w1, . . . , wn), wherewi is the amount of commodityi that you can sell (trade), then, with prices given by the price vectorp, you can afford any commodity bundlexwhere
p·x≤p·w. We may rewrite this last equation as
p·(x−w)≤0.
In other words, you can afford any commodity bundle x in the budget set {x|p·(x−w)≤0}. The equationp·(x−w)=0 defines abudget hyperplane
passing throughwwith normal vectorp. ◆
Determinants
We have already defined determinants of 2×2 and 3×3 matrices. (See§1.4.) Now we define the determinant of anyn×n(square) matrix in terms of determi- nants of (n−1)×(n−1) matrices. By “iterating the definition,” we can calculate any determinant.
2 See D. Saari, “Mathematical complexity of simple economics,”Notices of the American Mathematical Society42(1995), no. 2, 222–230.
DEFINITION 6.5 Let A=(ai j) be ann×nmatrix. Thedeterminantof Ais the real number given by
|A| =(−1)1+1a11|A11| +(−1)1+2a12|A12| + · · · +(−1)1+na1n|A1n|, where Ai j is the (n−1)×(n−1) submatrix of Aobtained by deleting the ith row and jth column ofA.
EXAMPLE 8 If A=
⎡
⎢⎣
1 2 1 3
−2 1 0 5
4 2 −1 0
3 −2 1 1
⎤
⎥⎦, then
A12=
⎡
⎢⎣
1 2 1 3
−2 1 0 5
4 2 −1 0
3 −2 1 1
⎤
⎥⎦=
⎡
⎣ −2 0 5
4 −1 0
3 1 1
⎤
⎦.
According to Definition 6.5,
det
⎡
⎢⎣
1 2 1 3
−2 1 0 5
4 2 −1 0
3 −2 1 1
⎤
⎥⎦=(−1)1+1(1) det
⎡
⎣ 1 0 5
2 −1 0
−2 1 1
⎤
⎦
+(−1)1+2(2) det
⎡
⎣ −2 0 5
4 −1 0
3 1 1
⎤
⎦
+(−1)1+3(1) det
⎡
⎣ −2 1 5
4 2 0
3 −2 1
⎤
⎦
+(−1)1+4(3) det
⎡
⎣ −2 1 0
4 2 −1
3 −2 1
⎤
⎦
=(1)(1)(−1)+(−1)(2)(37)+(1)(1)(−78) +(−1)(3)(−7)
= −132. ◆
The determinant of the submatrixAi jofAis called thei jthminorofA, and the quantity (−1)i+j|Ai j|is called thei jthcofactor. Definition 6.5 is known as cofactor expansionof the determinant along the first row, since detAis written as the sum of the products of each entry of the first row and the corresponding cofactor (i.e., the sum of the termsa1j times (−1)i+j|Ai j|).
It is natural to ask if one can compute determinants by cofactor expansion along other rows or columns ofA. Happily, the answer isyes(although we shall not prove this).
Convenient Fact. The determinant of Acan be computed by cofactor ex- pansion along any row or column. That is,
|A| =(−1)i+1ai1|Ai1| +(−1)i+2ai2|Ai2| + · · · +(−1)i+nai n|Ai n| (expansion along theith row),
|A| =(−1)1+ja1j|A1j| +(−1)2+ja2j|A2j| + · · · +(−1)n+jan j|An j| (expansion along the jth column).
EXAMPLE 9 To compute the determinant of
⎡
⎢⎢
⎢⎣
1 2 0 4 5
2 0 0 9 0
7 5 1 −1 0
0 2 0 0 2
3 1 0 0 0
⎤
⎥⎥
⎥⎦,
expansion along the first row involves more calculation than necessary. In partic- ular, one would need to calculate four 4×4 determinants on the way to finding the desired 5×5 determinant. (To make matters worse, these 4×4 determinants would, in turn, need to be expanded also.) However, if we expand along the third column, we find that
detA=(−1)1+3(0) detA13+(−1)2+3(0) detA23+(−1)3+3(1) detA33
+(−1)4+3(0) detA43+(−1)5+3(0) detA53
=detA33
=
1 2 4 5
2 0 9 0
0 2 0 2
3 1 0 0
.
There are several good ways to evaluate this 4×4 determinant. We’ll expand about the bottom row:
1 2 4 5
2 0 9 0
0 2 0 2
3 1 0 0
=(−1)4+1(3)
2 4 5
0 9 0
2 0 2
+(−1)4+2(1)
1 4 5
2 9 0
0 0 2
=(−1)(3)(−54)+(1)(1)(2)
=164. ◆
Of course, not all matrices contain well-distributed zeros as in Example 9, so there is by no means always an obvious choice for an expansion that avoids much calculation. Indeed, one does not compute determinants of large matrices by means of cofactor expansion. Instead, certain properties of determinants are used to make hand computations feasible. Since we shall rarely need to consider determinants larger than 3×3, we leave such properties and their significance to the exercises. (See, in particular, Exercises 26 and 27.)