Advanced Linear Algebra
Advanced Linear Algebra
References:
[A] Howard Anton ＆ Chris Rorres. (2000). Elementary Linear
Algebra. New York: John Wiley & Sons, Inc
[B] Setya Budi, W. 1995. Aljabar Linear. Jakarta: Gramedia
R. Rosnawati
Jurusan Pendidikan Matematika FMIPA UNY
Inner Product Spaces
Inner Product Spaces
1
Inner Products
2
Angle and Orthogonality in Inner
Product Spaces
3
Gram-Schmidt Process;
3
Gram-Schmidt Process;
QR-Decomposition
4
Best Approximation; Least Squares
5
Least Squares Fitting to Data
6
Function Approximation; Fourier
θ
3 Gram
3 Gram--Schmidt Process; QR
Schmidt Process;
QR--Decomposition
QR
4. Best Approximation; Least
4. Best Approximation; Least
Least squares solutions to A
Least squares solutions to Ax
x
=
=
b
6 Function Approximation;
6 Function Approximation;
Diagonalization & Quadratic
Diagonalization & Quadratic
Forms
Forms
1
Orthogonal Matrices
2
Orthogonal Diagonalization
3
Quadratic Forms
3
Quadratic Forms
4
Optimization Using Quadratic
Forms
5
Hermitian, Unitary, and Normal
Central conics in standard
Central conics in standard
4 Optimization Using Quadratic
4 Optimization Using Quadratic
5 Hermitian, Unita and Normal
5 Hermitian, Unita and Normal
Unitarily Diagonalizing
Unitarily Diagonalizing
I. Linear Transformations
I. Linear Transformations
General Linear Transformations
Isomorphisms
Compositions and Inverse
Compositions and Inverse
Transformations
Matrices for General Linear
Transformations
Dilation and Contraction
Dilation and Contraction
Compositions and Inverse
Compositions and Inverse
Matrix of Compositions
Matrix of Compositions
Eigenvalues and Eigenvectors
Eigenvalues and Eigenvectors
Definition 1: A nonzero vector x is an eigenvector (or characteristic vector) of a square matrix A if there exists a scalar λ such that Ax = λx. Then λ is an eigenvalue (or characteristic value) of A.
Note: The zero vector can not be an eigenvector even though A0 = λ0. But λ
Geometric interpretation of
Geometric interpretation of
Eigenvalues and Eigenvectors
Eigenvalues and Eigenvectors
An n×n matrix A multiplied by n×1 vector x results in another n×1 vector y=Ax. Thus A can be considered as a
transformation matrix.
In general, a matrix acts on a vector by changing both its magnitude and its direction. However, a matrix may act on
certain vectors by changing only their magnitude, and leaving certain vectors by changing only their magnitude, and leaving their direction unchanged (or possibly reversing it). These
vectors are the eigenvectors of the matrix.
A matrix acts on an eigenvector by multiplying its magnitude by a factor, which is positive if its direction is unchanged and
6.2 Eigenvalues
6.2 Eigenvalues
Let x be an eigenvector of the matrix A. Then there must exist an eigenvalue λ such that Ax = λx or, equivalently,
Ax - λx = 0 or
(A – λI)x = 0
If we define a new matrix B = A – λI, then
Bx = 0
λ
λ λ
λ
λ
λ
Bx = 0
If B has an inverse then x = B-10 = 0. But an eigenvector cannot be zero.
Thus, it follows that x will be an eigenvector of A if and only if B
does not have an inverse, or equivalently det(B)=0, or
det(A – λI) = 0
Example 1: Find the eigenvalues of
two eigenvalues: 1, 2
Note: The roots of the characteristic equation can be repeated. That is, λ1 = λ2 λ λ 5 1 12 2 A ) 2 )( 1 ( 2 3 12 ) 5 )( 2 ( 5 1 12 2
2
λ λ λ λ λ λ λ λ
λI A
6.2 Eigenvalues: examples
6.2 Eigenvalues: examples
Note: The roots of the characteristic equation can be repeated. That is, λ1 = λ2 =…= λk. If that happens, the eigenvalue is said to be of multiplicity k.
Example 2: Find the eigenvalues of
λ = 2 is an eigenvector of multiplicity 3.
Example 1 (cont.): 0 0 4 1 4 1 12 3 ) 1 ( :
1 I A
λ
, 4 0
4 2 1 2
1
x x t x t
x
6.3 Eigenvectors
6.3 Eigenvectors
To each distinct eigenvalue of a matrix A there will correspond at least one eigenvector which can be found by solving the appropriate set of homogenous equations. If λi is an eigenvalue then the corresponding eigenvector xi is the solution of (A – λiI)xi = 0
0 , 1 4 , 4 0 4 2 1 1 2 1 2 1 t t x x t x t x x x x 0 0 3 1 3 1 12 4 ) 2 ( :
2 I A
λ 0 , 1 3 2 1
2
s s
x x
Example 2 (cont.): Find the eigenvectors of
Recall that λ = 2 is an eigenvector of multiplicity 3.
Solve the homogeneous linear system represented by
0 0 0 0 0 0 0 0 0 0 1 0 ) 2 ( 2 1 x x x A I x
6.3 Eigenvectors
6.3 Eigenvectors
2 0 0 0 2 0 0 1 2 A λLet . The eigenvectors of = 2 are of the form
s and t not both zero. 0 0 0
0 x3
t x
s
x
3 1 , , 1 0 0 0 0 1 0 3 2 1
s t
6.4 Properties of Eigenvalues and Eigenvectors
6.4 Properties of Eigenvalues and Eigenvectors
Definition: The trace of a matrix A, designated by tr(A), is the sum of the elements on the main diagonal.
Property 1: The sum of the eigenvalues of a matrix equals the trace of the matrix.
Property 2: A matrix is singular if and only if it has a zero eigenvalue.
λ λ
Property 2: A matrix is singular if and only if it has a zero eigenvalue.
Property 3: The eigenvalues of an upper (or lower) triangular matrix are the elements on the main diagonal.
6.4 Properties of Eigenvalues and
6.4 Properties of Eigenvalues and
Eigenvectors
Eigenvectors
Property 5: If λ is an eigenvalue of A then kλ is an eigenvalue of kA where k is any arbitrary scalar.
Property 6: If λ is an eigenvalue of A then λk is an eigenvalue of Ak for any positive integer k.
λ λ
λ λ
λ λ
Ak for any positive integer k.
Property 8: If λ is an eigenvalue of A then λ is an eigenvalue of AT.
6.5 Linearly independent eigenvectors
6.5 Linearly independent eigenvectors
Theorem: Eigenvectors corresponding to distinct (that is, different) eigenvalues are linearly independent.
Theorem: If λ is an eigenvalue of multiplicity k of an n n matrix A then the number of linearly independent eigenvectors of A associated with λ is given by m = n - r(A- λI). Furthermore, 1 ≤ m ≤ k.
λ
λ λ
Example 2 (cont.): The eigenvectors of = 2 are of the form
s and t not both zero.
= 2 has two linearly independent eigenvectors
, 1 0 0 0 0 1 0 3 2 1
s t
