Rank and Equivalence of Matrices

66 5 The Echelon Form and the Rank of Matrices

P A^TQ^T =

Ir 0r,n−r

V 0m−r,n−r

for some matrixV ∈ K^m−r,r. Ifr =m, thenV = [ ]. In the following, for simplicity, we omit the sizes of the zero matrices. The matrix

Y :=

Ir 0

−V Im−r

∈ K^m,m

is invertible with

Y⁻¹= Ir 0

V Im−r

∈ K^m,m.

Thus,

Y P A^TQ^T = I_r 0

0 0

,

and withZ := P^TY^T ∈G Lm(K)we obtain

Q A Z = Ir 0

0 0

. (5.5)

⇒: Suppose that (5.5) holds for A ∈ K^n,m and matrices Q ∈ G Ln(K)and Z ∈G Lm(K). Then with (3a) we obtain

rank(A)=rank(A Z Z⁻¹)≤rank(AZ)≤rank(A),

and thus, in particular, rank(A) = rank(A Z). Due to the invariance of the echelon form (and hence the rank) under left-multiplication with invertible matrices (cp. Corollary5.9), we get

rank(A)=rank(A Z)=rank(Q A Z)=rank Ir 0

0 0

=r.

(2) IfA∈ K^n×n,Q∈GLn(K)andZ ∈GLm(K), then the invariance of the rank under left-multiplication with invertible matrices and(3a)can again be used for showing that

rank(A)=rank(Q A Z Z⁻¹)≤rank(Q A Z)=rank(A Z)≤rank(A), and hence, in particular, rank(A)=rank(Q A Z).

(4) If rank(A) = r, then by (1)there exist matrices Q ∈ G L_n(K) and Z ∈ G Lm(K)withQ A Z =

Ir 0 0 0

. Therefore,

68 5 The Echelon Form and the Rank of Matrices

rank(A)=rank(Q A Z)=rank I_r 0

0 0

=rank I_r 0

0 0 T

=rank((Q A Z)^T)

=rank(Z^TA^TQ^T)=rank(A^T).

(3b) Using(3a)and(4), we obtain

rank(A)=rank(A^T)=rank(C^TB^T)≤rank(C^T)=rank(C).

(5) LetA=BCwithB∈ K^n,ℓ,C∈ K^ℓ,m. Then by(3a), rank(A)=rank(BC)≤rank(B)≤ℓ.

Let, on the other hand, rank(A) = r ≤ ℓ. Then there exist matrices Q ∈ G Ln(K)andZ ∈G Lm(K)withQ A Z=

I_r 0 0 0

. Thus, we obtain

A=

Q⁻¹

Ir 0r,ℓ−r

0n−r,r 0n−r,ℓ−r

Ir 0r,m−r

0ℓ−r,r 0ℓ−r,m−r

Z⁻¹

=:BC,

whereB∈ K^n,ℓandC∈ K^ℓ,m. ⊓⊔

Example 5.12 The matrix

A=

⎡

⎣

0 2 1 3 3 0 2 0 1 1 0 2 0 1 1

⎤

⎦∈Q^3,5

from Example5.3has the echelon form

⎡

⎢⎣

0 1 0 ¹₂ ¹₂ 0 0 1 2 2 0 0 0 0 0

⎤

⎥⎦.

Since there are two pivot positions, we have rank(A)=2. Multiplying Afrom the right by

B=

⎡

⎢⎢

⎢⎢

⎣

1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0−1−1 0 0 0−1−1

⎤

⎥⎥

⎥⎥

⎦

∈Q^5,5,

yieldsA B=0∈Q^3,5, and hence rank(A B)=0<rank(A).

Assertion (1) in Theorem5.11motivates the following definition.

Definition 5.13 Two matrices A,B ∈ K^n,m are called equivalent, if there exist matricesQ∈G Ln(K)andZ ∈G Lm(K)withA=Q B Z.

As the name suggests, this defines an equivalence relation on the setK^n,m, since the following properties hold:

• Reflexivity:A=Q A ZwithQ=I_nandZ =I_m.

• Symmetry: IfA=Q B Z, thenB=Q⁻¹A Z⁻¹.

• Transitivity: IfA=Q₁B Z₁andB=Q₂C Z₂, thenA=(Q₁Q₂)C(Z₂Z₁).

The equivalence class ofA∈K^n,mis given by [A] =

Q A Z|Q∈G Ln(K)andZ ∈G Lm(K) .

If rank(A)=r, then by (1) in Theorem5.11we have

Ir 0r,m−r

0n−r,r 0n−r,m−r

= Ir 0

0 0

∈ [A]

and, therefore,

Ir 0

0 0 = [A].

Consequently, the rank of Afully determines the equivalence class[A]. The matrix Ir 0

0 0

∈K^n,m

is called theequivalence normal formof A. We obtain

K^n,m=

min{n,m}

r=0

I_r 0

0 0 , where I_r 0

0 0

I_ℓ0

0 0 =Ø, ifr=ℓ.

Hence there are 1+min{n,m}pairwise distinct equivalence classes, and I_r 0

0 0

∈ K^n,m

r =0,1, . . . ,min{n,m}

is a complete set of representatives.

From the proof of Theorem4.9we know that(K^n,n,+,∗)for n ≥ 2 is a non- commutative ring with unit that contains non-trivial zero divisors. Using the equiv- alence normal form these can be characterized as follows:

• If A ∈ K^n,n is invertible, then Acannot be a zero divisor, since then A B = 0 implies thatB=0.

70 5 The Echelon Form and the Rank of Matrices

• If A ∈ K^n,n \ {0}is a zero divisor, then Acannot be invertible, and hence 1 ≤ rank(A) =r <n, so that the equivalence normal form of Ais not the identity matrixIn. LetQ,Z ∈G Ln(K)be given with

Q A Z = Ir 0

0 0

.

Then for every matrix

V :=

0r,r 0r,n−r

V₂₁ V₂₂

∈ K^n,n

andB:=Z V we have

A B=Q⁻¹ Ir 0

0 0

0r,r 0r,n−r

V₂₁ V₂₂

=0.

IfV =0, thenB=0, sinceZ is invertible.

Exercises

(In the following exercisesK is an arbitrary field.) 5.1 Compute the echelon forms of the matrices

A= 1 2 3

2 4 48

∈Q^2,3, B = 1 i

i 1

∈C^2,2, C =

⎡

⎢⎢

⎣

1 i −i0 0 0 0 1 5 0−6i0 0 1 0 0

⎤

⎥⎥

⎦∈C^4,4,

D=

⎡

⎣ 1 0 1 1 0 1

⎤

⎦∈(Z/2Z)^3,2, E =

⎡

⎣ 1 0 2 0 2 0 1 1 1 2 0 2

⎤

⎦∈(Z/3Z)^3,4.

(Here for simplicity the elements ofZ/nZare denoted bykinstead of[k].) State the elementary matrices that carry out the transformations. If one of the matrices is invertible, then compute its inverse as a product of the elementary matrices.

5.2 LetA= α β

γ δ

∈ K^2,2withαδ=βγ. Determine the echelon form of Aand a formula forA⁻¹.

5.3 Let A = 1 A₁₂

0 B

∈ K^n,n withA₁₂ ∈ K^1,n−1andB ∈ K^n−1,n−1. Show that A∈G Ln(K)if and only ifB ∈G L_n−1(K).

5.4 Consider the matrix

A=

t+1 t−1

t−1 t² t² t+1

t−1 t+1

!

∈(K(t))^2,2,

where K(t) is the field of rational functions (cp. Exercise 3.19). Examine whetherAis invertible and determine, if possible,A⁻¹. Verify your result by computingA⁻¹AandA A⁻¹.

5.5 Show that ifA∈G Ln(K), then the echelon form of[A, In] ∈K^n,2n is given by[In, A⁻¹].

(The inverse of an invertible matrixAcan thus be computed via the transfor- mation of[A, In]to its echelon form.)

5.6 Two matrices A,B ∈ K^n,mare calledleft equivalent, if there exists a matrix Q∈G L_n(K)withA=Q B. Show that this defines an equivalence relation on K^n,mand determine a most simple representative for each equivalence class.

5.7 Prove Lemma5.7.

5.8 DetermineLU-decompositions (cp. Theorem5.4) of the matrices

A=

⎡

⎢⎢

⎣ 1 2 3 0 4 0 0 1 5 0 6 0 0 1 0 0

⎤

⎥⎥

⎦, B =

⎡

⎢⎢

⎣

2 0−2 0

−4 0 4−1 0−1−1−2

0 0 1 1

⎤

⎥⎥

⎦ ∈ R^4,4.

If one of these matrices is invertible, then determine its inverse using itsLU- decomposition.

5.9 Let Abe the 4×4 Hilbert matrix(cp. the MATLAB-Minute above Defini- tion5.6). Determine rank(A). DoesAhave anLU-decomposition as in The- orem5.4withP =I₄?

5.10 Determine the rank of the matrix

A=

⎡

⎣

0 α β

−α 0 γ

−β −γ 0

⎤

⎦∈R^3,3

in dependence ofα,β,γ∈R.

5.11 LetA,B ∈K^n,nbe given. Show that

rank(A)+rank(B)≤rank A C

0 B

for allC∈K^n,n. Examine when this inequality is strict.

5.12 Leta,b,c∈R^n,1. (a) Determine rank(ba^T).

(b) LetM(a,b):=ba^T −ab^T. Show the following assertions:

(i) M(a,b)= −M(b,a)andM(a,b)c+M(b,c)a+M(c,a)b=0, (ii) M(λa+µb,c)=λM(a,c)+µM(b,c)forλ,µ∈R,

(iii) rank(M(a,b))=0 if and only if there existλ,µ∈Rwithλ=0 or µ=0 andλa+µb=0,

(iv) rank(M(a,b))∈ {0,2}.

Chapter 6

Linear Systems of Equations

Solving linear systems of equations is a central problem of Linear Algebra that we discuss in an introductory way in this chapter. Such systems arise in numerous applications from engineering to the natural and social sciences. Major sources of linear systems of equations are the discretization of differential equations and the linearization of nonlinear equations. In this chapter we analyze the solution sets of linear systems of equations and we characterize the number of solutions using the echelon form from Chap.5. We also develop an algorithm for the computation of the solutions.

Definition 6.1 Alinear system (of equations)over a fieldKwithnequations inm unknowns x₁, . . . ,xmhas the form

a₁₁x₁+. . .+a_1mxm=b₁, a₂₁x₁+. . .+a_2mx_m=b₂,

... an1x1+. . .+anmxm=bn

or

Ax =b,

where thecoefficient matrix A = [a_{i j}] ∈ K^n,mand theright hand side b= [b_i] ∈ K^n,1 are given. Ifb =0, then the linear system is calledhomogeneous, otherwise non-homogeneous. Everyx ∈ K^m,1withAx =bis called asolutionof the linear system. All thesex form thesolution setof the linear system, which we denote by L(A,b).

The next result characterizes the solution setL(A,b)of the linear systemAx =b using the solution setL(A,0)of the associated homogeneous linear systemAx =0.

© Springer International Publishing Switzerland 2015

J. Liesen and V. Mehrmann,Linear Algebra, Springer Undergraduate Mathematics Series, DOI 10.1007/978-3-319-24346-7_6

73

Lemma 6.2 Let A ∈ K^n,m and b ∈ K^n,1 with L(A,b) = Øbe given. Ifx ∈ L(A,b), then

L(A,b)=x+L(A,0):= {x+z|z∈L(A,0)}.

Proof Ifz∈L(A,0), and thusx+z∈x+L(A,0), then A(x+z)=Ax+Az=b+0=b.

Hencex+z∈L(A,b), which shows thatx+L(A,0)⊆L(A,b).

Let nowx₁∈L(A,b)and letz:=x₁−x. Then Az=Ax₁−Ax=b−b=0,

i.e.,z∈L(A,0). Hencex₁=x+z∈x+L(A,0), which shows thatL(A,b)⊆

x+L(A,0).

We will have a closer look at the set L(A,0): Clearly, 0 ∈ L(A,0) = Ø. If z∈L(A,0), then for allλ∈ K we have A(λz)=λ(Az)=λ·0 =0, and hence

λz∈L(A,0). Furthermore, forz₁,z₂∈L(A,0)we have A(z₁+z₂)= Az₁+Az₂=0+0=0,

and hencez₁+z₂ ∈L(A,0). Thus,L(A,0)is a nonempty subset ofK^m,1that is closed under scalar multiplication and addition.

Lemma 6.3 If A∈ K^n,m, b∈ K^n,1and S ∈ K^n,n, thenL(A,b)⊆L(S A,Sb).

Moreover, if S is invertible, thenL(A,b)=L(S A,Sb).

Proof Ifx ∈ L(A,b), then also S Ax = Sb, and thusx ∈ L(S A,Sb), which shows thatL(A,b) ⊆ L(S A,Sb). IfS is invertible andy ∈ L(S A,Sb), then S Ay=Sb. Multiplying from the left withS⁻¹yieldsAy=b. Sincey∈L(A,b),

we haveL(S A,Sb)⊆L(A,b).

Consider the linear system of equations Ax =b. By Theorem5.2we can find a matrix S ∈ G Ln(K)such thatS Ais in echelon form. Letb = [bi] := Sb, then L(A,b) = L(S A,b)by Lemma6.3, and the linear system S Ax =b takes the

form ⎡

⎢⎢

⎢⎢

⎢⎢

⎢⎢

⎢⎢

⎣

1 ⋆ 0 0 0

1 ⋆

0 ⋆

1 ...

⋆

0 0 . .. 0

0 0 1

0

⎤

⎥⎥

⎥⎥

⎥⎥

⎥⎥

⎥⎥

⎦ x=

⎡

⎢⎢

⎢⎢

⎢⎢

⎢⎢

⎢⎣ b₁

... bn

⎤

⎥⎥

⎥⎥

⎥⎥

⎥⎥

⎥⎦ .

6 Linear Systems of Equations 75 Suppose that rank(A)=r, and let j₁,j₂, . . . ,jrbe the pivot columns. Using a right- multiplication with a permutation matrix we can move therpivot columns ofS Ato the firstrcolumns. This is achieved by

P^T := [ej1, . . . ,ejr,e1, . . . ,ej1−1,ej1+1, . . . ,ej2−1,ej2+1, . . . ,ejr−1,ejr+1, . . . ,em] ∈K^m,m, which yields

A:=S A P^T =

I_r A₁₂ 0n−r,r 0n−r,m−r

,

where A₁₂ ∈ K^r,m−r. Ifr =m, then we haveA₁₂ = [ ]. This permutation leads to a simplification of the following presentation, but it is usually omitted in practical computations.

Since P^TP = Im, we can writeS Ax =b as (S A P^T)(P x) =b, or Ay =b, which has the form

⎡

⎢⎢

⎢⎢

⎢⎢

⎣

I_r A₁₂

0n−r,r 0n−r,m−r

⎤

⎥⎥

⎥⎥

⎥⎥

⎦

=A:=S A P ^T

⎡

⎢⎢

⎢⎢

⎢⎢

⎢⎢

⎣ y1

... y_r y_r+1

... ym

⎤

⎥⎥

⎥⎥

⎥⎥

⎥⎥

⎦

=y:=P x

=

⎡

⎢⎢

⎢⎢

⎢⎢

⎢⎢

⎣ b₁

... b_r b_r+1

... bn

⎤

⎥⎥

⎥⎥

⎥⎥

⎥⎥

⎦

=b:=Sb

. (6.1)

The left-multiplication ofxwithPjust means a different ordering of the unknowns x1, . . . ,xm. Thus, the solutions ofAx =bcan be easily recovered from the solutions of Ay =b, and vice versa: We havey ∈ L(A,b)if and only ifx := P^Ty ∈ L(S A,b)=L(A,b).

The solutions of (6.1) can now be determined using theextended coefficient matrix [A,b] ∈K^n,m+1,

which is obtained by attachingb as an extra column to A. Note that rank(A) ≤ rank([A,b]), with equality if and only ifb_r+1= · · · =bn =0.

If rank(A) < rank([A,b]), then at least one ofb_r+1, . . . ,bn is nonzero. Then (6.1) cannot have a solution, since all entries ofAin the rowsr+1, . . . ,nare zero.

If, on the other hand, rank(A) =rank([A,b]), thenb_r+1 = · · · =bn =0 and (6.1) can be written as

⎡

⎢⎣ y₁

... yr

⎤

⎥⎦=

⎡

⎢⎣ b₁

... br

⎤

⎥⎦−A₁₂

⎡

⎢⎣ y_r+1

... ym

⎤

⎥⎦. (6.2)

This representation implies, in particular, that b^(m):= [b₁, . . . ,br,0, . . . ,0

m−r

]^T ∈L(A,b)=Ø.

From Lemma6.2we know thatL(A,b)=b^(m)+L(A,0). In order to determine L(A,0)we setb₁= · · · =b_r =0 in (6.2), which yields

L(A,0)=

[y₁, . . . ,ym]^T |y_r+1, . . . ,ymarbitrary and (6.3) [y1, . . . ,yr]^T =0−A12[yr+1, . . . ,ym]^T

. Ifr =m, then A₁₂ = [ ],L(A,0)= {0}and thus|L(A,b)| =1, i.e., the solution of Ay=bis uniquely determined.

Example 6.4 For the extended coefficient matrix

[A,b] =

⎡

⎣

1 0 3b₁ 0 1 4b2

0 0 0b₃

⎤

⎦∈Q^3,4

we have rank(A)=rank([A,b])if and only ifb3 =0. Ifb3 =0, thenL(A,b)=Ø.

Ifb3 =0, thenAy =bcan be written as y₁

y₂

= b₁

b₂

− 3

4

[y₃].

Hence,b⁽³⁾= [b₁,b₂,0]^T ∈L(A,b)and L(A,0)=

[y₁,y₂,y₃]^T |y₃arbitrary and[y₁,y₂]^T = −[3,4]^T[y₃] . Summarizing our considerations we have the following algorithm for solving a linear system of equations.

Algorithm 6.5 LetA∈K^n,mandb∈ K^n,1be given.

(1) DetermineS∈G Ln(K)such thatS Ais in echelon form and defineb:=Sb.

(2a) If rank(S A) <rank([S A,b]), thenL(S A,b)=L(A,b)=Ø.

(2b) Ifr=rank(A)=rank([S A,b]), then defineA:=S A P^T as in (6.1).

We haveb^(m)∈L(A,b)andL(A,b)=b^(m)+L(A,0), whereL(A,0)is determined as in (6.3), as well asL(A,b)= {P^Ty|y∈L(A,b)}.

Since rank(A) = rank(S A) = rank(A)and rank([A,b]) = rank([S A,b]) = rank([A,b]), the discussion above also yields the following result about the different cases of the solvability of a linear system of equations.

6 Linear Systems of Equations 77

Corollary 6.6 For A∈ K^n,mand b∈K^n,1the following assertions hold:

(1) Ifrank(A) <rank([A,b]), thenL(A,b)=Ø.

(2) Ifrank(A)=rank([A,b])=m, then|L(A,b)| =1(i.e., there exists a unique solution).

(3) Ifrank(A)=rank([A,b]) <m, then there exist many solutions. If the field K has infinitely many elements (e.g., when K=Q, K =Ror K=C), then there exist infinitely many pairwise distinct solutions.

The different cases in Corollary6.6will be studied again in Example10.8.

Example 6.7 LetK =Qand consider the linear system of equationsAx =bwith

A=

⎡

⎢⎢

⎢⎢

⎣ 1 2 2 1 0 1 0 3 1 0 3 0 2 3 5 4 1 1 3 3

⎤

⎥⎥

⎥⎥

⎦, b=

⎡

⎢⎢

⎢⎢

⎣ 1 0 2 3 2

⎤

⎥⎥

⎥⎥

⎦.

We form[A,b]and apply the Gaussian elimination algorithm in order to transform Ainto echelon form:

[A,b]

⎡

⎢⎢

⎢⎢

⎣

1 2 2 1 1 0 1 0 3 0 0−2 1−1 1 0−1 1 2 1 0−1 1 2 1

⎤

⎥⎥

⎥⎥

⎦

⎡

⎢⎢

⎢⎢

⎣

1 2 2 1 1 0 1 0 3 0 0 0 1 5 1 0 0 1 5 1 0 0 1 5 1

⎤

⎥⎥

⎥⎥

⎦

⎡

⎢⎢

⎢⎢

⎣

1 2 2 1 1 0 1 0 3 0 0 0 1 5 1 0 0 0 0 0 0 0 0 0 0

⎤

⎥⎥

⎥⎥

⎦

⎡

⎢⎢

⎢⎢

⎣

1 0 2 −5 1 0 1 0 3 0 0 0 1 5 1 0 0 0 0 0 0 0 0 0 0

⎤

⎥⎥

⎥⎥

⎦

⎡

⎢⎢

⎢⎢

⎣

1 0 0−15−1

0 1 0 3 0

0 0 1 5 1

0 0 0 0 0

0 0 0 0 0

⎤

⎥⎥

⎥⎥

⎦

= [S A|b].

Here rank(S A) = rank([S A,b]) = 3, and hence there exist solutions. The pivot columns are ji = i for i = 1,2,3, so that P = P^T = I4 and A = S A. Now S Ax =bcan be written as

⎡

⎣ x₁ x₂ x3

⎤

⎦=

⎡

⎣

−1 0 1

⎤

⎦−

⎡

⎣

−15 3 5

⎤

⎦[x₄].

Consequently,b⁽⁴⁾ = [−1,0,1,0]^T ∈ L(A,b)andL(A,b)=b⁽⁴⁾+L(A,0), where

L(A,0)=

[x₁, . . . ,x₄]^T |x₄ arbitrary and[x₁,x₂,x₃]^T = −[−15,3,5]^T[x₄] .

Exercises

6.1 Find a fieldKand matricesA∈K^n,m,S∈ K^n,nandb∈K^n,1withL(A,b)= L(S A,Sb).

6.2 DetermineL(A,b)for the followingAandb:

A=

⎡

⎣1 1 1 1 2−1 1−1 6

⎤

⎦∈R^3,3, b=

⎡

⎣ 1

−2 3

⎤

⎦∈R^3,1,

A=

⎡

⎣1 1 1 0

1 2 −1 −1

1 −1 6 2

⎤

⎦∈R^3,4, b=

⎡

⎣ 1

−2 3

⎤

⎦∈R^3,1,

A=

⎡

⎢⎢

⎣

1 1 1

1 2 −1

1 −1 6

1 1 1

⎤

⎥⎥

⎦∈R^4,3, b=

⎡

⎢⎢

⎣ 1

−2 3 1

⎤

⎥⎥

⎦∈R^4,1,

A=

⎡

⎢⎢

⎣

1 1 1

1 2 −1

1 −1 6

1 1 1

⎤

⎥⎥

⎦∈R^4,3, b=

⎡

⎢⎢

⎣ 1

−2 3 0

⎤

⎥⎥

⎦∈R^4,1.

6.3 Letα∈Q,

A=

⎡

⎣ 3 2 1 1 1 1 2 1 0

⎤

⎦∈Q^3,3, bα=

⎡

⎣ 6 3 α

⎤

⎦∈Q^3,1.

DetermineL(A,0)andL(A,bα)in dependence ofα.

6.4 LetA ∈ K^n,mandB ∈ K^n,s. Fori =1, . . . ,sdenote byb_i theith column of B. Show that the linear system of equations A X =Bhas at least one solution

X ∈K^m,sif and only if

rank(A)=rank([A,b₁])=rank([A,b₂])= · · · =rank([A,bs]).

Find conditions under which this solution is unique.

6 Linear Systems of Equations 79 6.5 Let

A=

⎡

⎢⎢

⎢⎢

⎣ 0 β₁ α₂ 0 . ..

. .. . ..β_n α_n 0

⎤

⎥⎥

⎥⎥

⎦

∈ K^n,n, b=

⎡

⎢⎣ b₁

... bn

⎤

⎥⎦∈ K^n,1

be given withβ_i,α_i =0 for alli. Determine a recursive formula for the entries of the solution of the linear systemAx=b.

Determinants of Matrices

The determinant is a map that assigns to every square matrix A∈ R^n,n, whereRis a commutative ring with unit, an element of R. This map has very interesting and important properties. For instance it yields a necessary and sufficient condition for the invertibility of A ∈ R^n,n. Moreover, it forms the basis for the definition of the characteristic polynomial of a matrix in Chap.8.