Structured inverse least square problem

where D^H =−D∈C^k×k, Z^H =−Z ∈C(n−k)×(n−k).

We mention that the results obtained above can easily be extended to the case when S⊂ {J,L}.

Theorem 2.5.2. Let S=Herm. Then for anyx, b∈Cⁿ we have αS= |im(x^Hb)|

kxk .

Proof: Assume that x, b∈ Cⁿ. Construct a unitary matrix Q= [x/kxk Q1] ∈ C^n×n such that Q^H₁x= 0.Then construct the Hermitian matrixA=Q

"

a11 a^H₁ a1 A1

#

Q^H.Then we obtain

minA∈SkAx−bk²_F = min

A∈S

°°

°°

°

"

a11kxk −x^Hb/kxk a1kxk −Q^H₁b

#°°

°°

°

2

F

=|a11kxk−x^Hb/kxk|²+ min

a1∈Kⁿ⁻¹ka1kxk−Q^H₁ bk²_F. Choosea1=Q^H₁ b/kxk.Next mina11∈R|a11kxk −x^Hb/kxk|² is minimized whena11= ^re(x_kxk^H2^b). Consequently, we obtainαS=|im(x^Hb)|/kxk.Next we have,

A = Q

"

re(x^Hb)

kxk² (Q^H₁b/kxk)^H Q^H₁b/kxk A1

# Q^H

= re(x^Hb)

kxk⁴ xx^H+ 1

kxk²[xb^H(I−xx^H) + (I−xx^H)bx^H] + (I−xx^H)Z(I−xx^H) where Z=Z^H ∈C^n×n.¥

In a similar fashion we can obtain the solution of SILSP for skew-Hermitian matrices as follows.

Theorem 2.5.3. Let S=skew-Herm. Then for anyx, b∈Cⁿ we have αS= |re(x^Hb)|

kxk . Proof: The proof is similar to Theorem 2.5.2.¥

Now assume thatS=J or S=Lwhere J andLare the Jordan algebra and Lie algebra corresponding to orthosymmetric bilinear / sesquilinear scalar product h·,·iM. Then its ev- ident by Theorem 2.5.1, Theorem 2.5.2 and Theorem 2.5.3 thet whenever S(x, b) = ∅ for a givenx, b∈Cⁿ\ {0} then

minA∈SkAx−bk=















|hx, biM|

kxk2 , if (M A)^T =−M A

|imhx, biM|

kxk₂ , if (M A)^H =M A

|rehx, biM| kxk2

, if (M A)^H =−M A.

Now we consider SILSP for matrices. Before that we prove the following result which will be used in the subsequent development.

Lemma 2.5.4. Let α, β >0andb1, b2∈C.Thenminx∈C(|xα−b1|²+|xβ−b2|²)is given by x= αb1+βb2

α²+β² .

Proof: Assume thatx=x1+ix2, b1=b11+ib12, b2=b21+ib22∈C.Then define φ(x1, x2) = |xα−b1|²+|xβ−b2|²

= (α²+β²)(x²₁+x²₂) + (b²₁₁+b²₁₂) + (b²₂₁+b²₂₂)−2α(x1b11+x2b12)−2β(x1b21+x2b22).

Setting ∂φ(x1, x2)

∂xi = 0, i= 1,2 we obtain the stationary points x1=αb11+βb21

α²+β² , x2=αb12+βb22

α²+β² . This gives the relative minimum as the Hessian matrix







∂²φ

∂x²₁

∂²φ

∂x1∂x2

∂²φ

∂x₂∂x₁

∂²φ

∂x²₂





=

à 2(α²+β²) 0 0 2(α²+β²)

!

is positive definite. Consequently we obtain the desired result.¥

Theorem 2.5.5. LetSbe the space of symmetric matrices andX, B∈K^n×k withrank(X) = r. Assume that the SVD of X = UΣV^H where Σ =

"

Σ1 0

0 0

#

,Σ1 = diag(σ1, . . . , σr), σ1 >

. . . > σr>0, U = [U1 U2], V = [V1 V2] . Then

minA∈SkAX−BkF =kK◦(U₁^TBV1Σ1+ Σ1V₁^TB^TU1)Σ1−U₁^TBV1k²_F+kU₁^TBV2k²_F+kU₂^TBV2k²_F is attained at

A=U1[K◦U₁^TBV1Σ1+K◦Σ1V₁^TB^TU1]U₁^H+(BX^†)^T(I−XX^†)+(I−XX^†)^TBX^†+(I−XX^†)^TZ(I−XX^†) where Z^T =Z ∈C^r×r, K = [kij], kij= _σ2¹

i+σ_j² and◦denotes the Hadamard product.

Proof: Assume thatX, K ∈K^n×k with rank(X) =r.Consider the SVDX =UΣV^H.Define a symmetric linear mapA : Range(X)⊕Range(X)^⊥ →Range(X)⊕Range(X)^⊥. From the SVD of X given above it is easily seen that (U1, U2) and (U1, U2) are bases of Range(X)⊕ Range(X)^⊥ and Range(X)⊕Range(X)^⊥ respectively. The block matrix representation ofA is of the form

A= h

U1 U2

i"

A11 A^T₁₂ A12 A22

# "

U₁^H U₂^H

#

where A11, A12, A22are of compatible sizes. Consequently we obtain kAX−Bk²_F = kU^TAU U^HX−U^TBk²_F

=

°°

°°

°

"

A11 A^T₁₂ A12 A22

# "

U₁^HX 0

#

−

"

U₁^TB U₂^TB

#°°

°°

°

2

F

= kA11U₁^HX−U₁^TBk²_F +kA12U₁^HX−U₂^TBk²_F.

Then

A12∈Cmin^(n−r)×rkA12U₁^HX−U₂^TBk²_F = min

A12∈C^(n−r)×rkA12U₁^HUΣV^H−U₂^TBk²_F

= min

A12∈C^(n−r)×rkA12U₁^HUΣ−U₂^TBVk²_F

= min

A12∈C^(n−r)×rkA12Σ1−U₂^TBV1k²_F+kU₂^TBV2k²_F. MoreoverkA12Σ1−U₂^TBV1k²_F is minimized if and only ifA12Σ1−U₂^TBV1= 0 i.e. if and only ifA12=U₂^TBV1Σ⁻¹₁ .Similarly we obtain

A11∈C^r×rmin,A^T₁₁=A11

kA11U₁^HX−U₁^TBk²_F = min

A11∈C^r×r,A^T₁₁=A11

kA11Σ1−U₁^TBV1k²_F+kU₁^TBV2k²_F. Further assume thatA11= [aij], U₁^TBV1= [bij].Consequently we have,

kA11Σ1−U₁^TBV1k²_F = Xr

j≤i,j=1

Xr

i=1

(|aijσi−bij|²+|ajiσj−bji|²)

= Xr

j≤i,j=1

Xr

i=1

(|aijσi−bij|²+|aijσj−bji|²)

The desired minimum can be obtained by minimizing|aijσi−bij|²+|aijσj−bji|²,for all i, j= 1 :r. By Lemma 2.5.4 the minimum can be obtained by

aij= σibij+bjiσj

σ²_i +σ²_j , aij =aji,∀i, j= 1 :r.

HencekA11Σ1−U₁^TBV1k²_F can be minimized by

A11=K◦(U₁^TBV1Σ1+ Σ1V₁^TB^TU1) whereK= [kij], kij =_σ2¹

i+σ_j² and◦denotes the Hadamard product. Consequently, we obtain

A = U

"

K◦(U₁^TBV1Σ1+ Σ1V₁^TB^TU1) Σ⁻¹₁ V₁^TB^TU2

U₂^TBV₁Σ⁻¹₁ A₂₂

# U^H

= U1[K◦(U₁^TBV1Σ1+ Σ1V₁^TB^TU1)]U₁^H+U1Σ⁻¹₁ V₁⁻¹B^TU2U₂^H+U1U₂^TBV1Σ⁻¹₁ U₁^H+U2A22U₂^H

= U1[K◦(U₁^TBV1Σ1+ Σ1V₁^TB^TU1)]U₁^H+ (BX^†)^T(I−XX^†) + (I−XX^†)^TBX^† +(I−XX^†)^TZ(I−XX^†), Z^T =Z∈C^r×r

which gives the desired result.¥

Now we consider skew-symmetric matrices.

Theorem 2.5.6. Let S be the space of skew-symmetric matrices and X, B ∈ K^n×k with rank(X) =r.Assume that the SVD ofX=UΣV^HwhereΣ =

"

Σ1 0

0 0

#

,Σ1= diag(σ1, . . . , σr), σ1>

. . . > σr>0, U = [U1 U2], V = [V1 V2] . Then

αS=kK◦(U₁^TBV1Σ1−Σ1V₁^TB^TU1)Σ1−U₁^TBV1k²_F +kU₁^TBV2k²_F +kU₂^TBV2k²_F

is attained at

A=U1[K◦U₁^TBV1Σ1−K◦Σ1V₁^TB^TU1]U₁^H−(BX^†)^T(I−XX^†)+(I−XX^†)^TBX^†+(I−XX^†)^TZ(I−XX^†) where Z^T =−Z∈C^r×rK= [kij], kij= _σ2¹

i+σ_j² and◦denotes the Hadamard product.

Proof: The proof is similar to the proof for symmetric case.¥

Next, we consider Hermitian matrices.

Theorem 2.5.7. LetSbe the space of Hermitian matrices andX, B∈K^n×k withrank(X) = r. Assume that the SVD of X = UΣV^H where Σ =

"

Σ1 0

0 0

#

,Σ1 = diag(σ1, . . . , σr), σ1 >

. . . > σr>0, U = [U1 U2], V = [V1 V2] . Then

αS=kK◦(U₁^HBV1Σ1+ Σ1V₁^HB^HU1)Σ1−U₁^HBV1k²_F +kU₁^HBV2k²_F+kU₂^HBV2k²_F is attained at

A=U1[K◦U₁^HBV1Σ1+K◦Σ1V₁^HB^HU1]U₁^H+(BX^†)^H(I−XX^†)+(I−XX^†)BX^†+(I−XX^†)Z(I−XX^†) where Z^H=Z ∈C^r×r, K = [kij], kij= _σ2¹

i+σ_j² and◦ denotes the Hadamard product.

Proof: Assume thatX, K ∈K^n×k with rank(X) =r.Consider the SVDX =UΣV^H.Define a Hermitian linear map A : Range(X)⊕Range(X)^⊥ →Range(X)⊕Range(X)^⊥. From the SVD of X given above it is easily seen that (U1, U2) and (U1, U2) are bases of Range(X)⊕ Range(X)^⊥ and Range(X)⊕Range(X)^⊥ respectively. The block matrix representation ofA is of the form

A=U

"

A11 A^H₁₂ A12 A22

# U^H. Consequently we obtain

kAX−Bk²_F = kU^HAU U^HX−U^HBk²_F

=

°°

°°

°

"

A11 A^H₁₂ A₁₂ A₂₂

# "

U₁^HX 0

#

−

"

U₁^HB U₂^HB

#°°

°°

°

2

F

= kA11U₁^HX−U₁^HBk²_F +kA12U₁^HX−U₂^HBk²_F. Then,

A12∈Cmin^(n−r)×rkA12U₁^HX−U₂^HBk²_F = min

A12∈C^(n−r)×rkA12U₁^HUΣV^H−U₂^HBk²_F

= min

A12∈C^(n−r)×rkA12U₁^HUΣ−U₂^HBVk²_F

= min

A12∈C^(n−r)×rkA12Σ1−U₂^HBV1k²_F+kU₂^HBV2k²_F. Now kA12Σ1−U₂^HBV1k²_F is minimized if and only ifA12Σ1−U₂^HBV1= 0 i.e. if and only if A12=U₂^HBV1Σ⁻¹₁ .Similarly we have,

A11∈C^r×rmin,A^H₁₁=A11

,kA11U₁^HX−U₁^HBk²_F = min

A11∈C^r×r,A^H₁₁=A11

kA11Σ1−U₁^HBV1k²_F+kU₁^HBV2k²_F.

Further letA= [aij], U₁^HBV1= [bij]∈C^r×r. Consequently we have kA11Σ1−U₁^HBV1k²_F =

Xr

j≤i,j=1

Xr

i=1

(|aijσi−bij|²+|aijσj−bji|²)

= Xr

j≤i,j=1

Xr

i=1

(|aijσi−bij|²+|aijσj−bji|²).

Now the desired minimum can be obtained by minimizing|a_ijσ_i−b_ij|²+|a_ijσ_j−b_ji|²for all i, j= 1 :r. By Lemma 2.5.4 the minimum can be obtained by

aij = σibij+σjbji

σ_i²+σ_j² , aji=aij,∀i, j= 1 :r.

Hence we obtain that kA11Σ1−U₁^HBV1k²_F is minimized by A11=K◦(U₁^HBV1Σ1+ Σ1V₁^HB^HU1) whereK= [kij], kij =_σ2¹

i+σ_j² and◦denotes the Hadamard product. Consequently, we obtain

A = U

"

K◦(U₁^HBV1Σ1−Σ1V₁^HB^HU1) +Σ⁻¹₁ V₁^HB^HU2

U₂^HBV₁Σ⁻¹₁ A₂₂

# U^H

= U1[K◦(U₁^HBV1Σ1+ Σ1V₁^HB^HU1)]U₁^H+U1Σ⁻¹₁ V₁⁻¹B^HU2U₂^H+U1U₂^HBV1Σ⁻¹₁ U₁^H+U2A22U₂^H

= U1[K◦(U₁^HBV1Σ1+ Σ1V₁^HB^HU1)]U₁^H+ (BX^†)^H(I−XX^†) + (I−XX^†)BX^† +(I−XX^†)^HZ(I−XX^†), Z^H =Z ∈C^r×r

which gives the desired result.¥

Now consider skew-Hermitian matrices.

Theorem 2.5.8. Let S be the space of skew-Hermitian matrices and X, B ∈ K^n×k with rank(X) =r.Assume that the SVD ofX=UΣV^HwhereΣ =

"

Σ1 0

0 0

#

,Σ1= diag(σ1, . . . , σr), σ1>

. . . > σr>0, U = [U1 U2], V = [V1 V2] . Then

αS=kK◦(U₁^HBV1Σ1−Σ1V₁^HB^HU1)Σ1−U₁^HBV1k²_F +kU₁^HBV2k²_F+kU₂^HBV2k²_F is attained at

A=U1[K◦U₁^HBV1Σ1−K◦Σ1V₁^HB^HU1]U₁^H−(BX^†)^H(I−XX^†)+(I−XX^†)BX^†+(I−XX^†)Z(I−XX^†) where Z^H=−Z∈C^r×r, K= [kij], kij = _σ2¹

i+σ²_j and◦ denotes the Hadamard product.

Proof: The proof is similar to the proof for Hermitian case.¥

Now consider S ∈ {J,L}. Then for any given X, B ∈ C^n×k we have kAX −BkF = kM AX−M BkF. Therefore the SILSP problem can be resolved for S just replacing B by M B. Further the matrixA∈Swhich produces the minimum, can be obtained by redefining it as M A.

Chapter 3

Structured backward errors and pseudospectra of structured

matrix pencils

Structured backward perturbation analysis plays an important role in the accuracy assessment of computed eigenelements of structured eigenvalue problems. We undertake a detailed struc- tured backward perturbation analysis of approximate eigenelements of linearly structured matrix pencils. The structures we consider include, for example, symmetric, skew-symmetric, Hermitian, skew-Hermitian, even, odd, palindromic and Hamiltonian matrix pencils. We also analyze structured backward errors of approximate eigenvalues and structured pseudospectra of structured matrix pencils.

3.1 Introduction

Backward perturbation analysis determines the smallest perturbation for which a computed solution is an exact solution of the perturbed problem. On the other hand, condition num- bers measure the sensitivity of solutions to small perturbations in the data of the problem.

Thus, backward errors when combined with condition numbers provide an approximate upper bounds on the errors in the computed solutions.

With a view to preserving structures and their associated properties, structured preserv- ing algorithms for structured eigenproblems have been proposed in the literature (see, for example, [9, 10, 18, 46, 74, 75] and the references therein). Consequently, there is a growing interest in the structured perturbation analysis of structured eigenproblems (see, for exam- ple, [16, 38, 51, 54, 81, 95] for sensitivity analysis of structured eigenproblems).

The main purpose of this chapter is to undertake a detailed structured backward perturba- tion analysis of approximate eigenelements of linearly structured matrix pencils. Needless to mention that structured backward errors when combined with structured condition numbers provide an approximate upper bounds on the errors in the computed eigenelements. Hence structured backward perturbation analysis plays an important role in the accuracy assessment of approximate eigenelements of structured pencils. Further, it also plays an important role in the selection of an optimum structured linearization of a structured matrix polynomial.

This assumes significance due to the fact that linearization is a standard approach to solving a polynomial eigenvalue problem (see, for example, [39, 41] and the references therein).

We consider regular matrix pencils of the form L(λ) =A+λB,whereAandB are square matrices of size n. We assume L to be linearly structured, that is, L to be an element of a real or a complex linear subspace S of the space of pencils. More specifically, we consider ten special classes of linearly structured pencils, namely, T-symmetric, T-skew-symmetric, T-odd, T-even, T-palindromic, H-Hermitian, H-skew-Hermitian, H-even and H-odd and H-palindromic. These structures, defined in the next section, are prototypes of structured pencils which occur in many applications (see, [40, 75] and the references therein). We also considerSto be the space of pencils whose coefficient matrices are elements of Jordan and/or Lie algebras associated with the scalar product (x, y)7→y^TM xor (x, y)7→y^HM x,whereM is unitary and M^T =±M or M^H =±M. For example, whenM :=

"

0 I

−I 0

#

, the Lie and Jordan algebras associated with the scalar product (x, y) 7→y^HM x consist of Hamiltonian and skew-Hamiltonian matrices, respectively. The structures so considered encompass a wide variety of structured pencils and, in particular, includes pencils whose coefficient matrices are Hamiltonian and skew-Hamiltonian. We show, however, that analyzing these wide classes of structured pencils ultimately boils down to analyzing one of the ten special classes of structured pencils considered above. Consequently, we consider these ten special classes of structured pencils and investigate structured backward perturbation analysis of approximate eigenelements.

So, letSbe the space of pencils having one of the ten structures. Let L∈Sand (λ, x)∈ C×Cⁿ withx^Hx= 1.Then we define the structured backward errorη^S(λ, x,L) of (λ, x) by

η^S(λ, x,L) := inf{|||4L|||:4L∈S and L(λ)x+4L(λ)x= 0}.

Here the pencil norm|||L|||is given by|||L|||:=p

kAk²+kBk²,where L(z) =A+zB andk · k is either the spectral norm or the Frobenius norm on C^n×n. The main contributions of this chapter are as follows.

Given (λ, x)∈ C×Cⁿ with x^Hx= 1 and L∈ S, we show that there is a pencil K∈ S such that L(λ)x+ K(λ)x= 0.Consequently, η^S(λ, x,L) <∞. We determine η^S(λ, x,L) and construct a pencil4L∈Ssuch that|||4L|||=η^S(λ, x,L) and L(λ)x+4L(λ)x= 0.Moreover, we show that4L is unique for the Frobenius norm onC^n×nbut there are infinitely many such 4L for the spectral norm onC^n×n.Further, for the spectral norm, we show how to construct all such 4L. In either case, we show that if K ∈ S is such that L(λ)x+ K(λ)x = 0 then K =4L + (I−xx^H)^∗N(I−xx^H) for some N∈S,where (I−xx^H)^∗ denotes the transpose or the conjugate transpose of (I−xx^H) depending upon the structure defined byS.Furthermore, we show that the unstructured backward errorη(λ, x,L) of (λ, x) is a lower bound ofη^S(λ, x,L) and is attained by η^S(λ, x,L) for certain λ ∈ C. However, η(λ, x,L) 6=η^S(λ, x,L) for most λ∈C.

Next, we consider structured pseudospectra of structured matrix pencils. It is a well known fact that pseudospectra of matrices and matrix pencils are powerful tools for sensitivity and perturbation analysis (see, [100] and the references therein). We consider structured and

unstructured²-pseudospectra

σ^S_²(L) :={λ∈C:η^S(λ,L)≤²} andσ²(L) :={λ∈C:η(λ,L)≤²}

of L,whereη^S(λ,L) := min

x^Hx=1η^S(λ, x,L) andη(λ,L) := min

x^Hx=1η(λ, x,L),respectively, are struc- tured and unstructured backward errors of an approximate eigenvalue λ. When L is T- symmetric or T-skew-symmetric pencils, we show that η^S(λ,L) = η(λ,L) for the spectral norm andη^S(λ,L) =√

2η(λ,L) for the Frobenius norm. Consequently, for these structures, we show that σ^S_²(L) = σ²(L) for the spectral norm and σ^S_²(L) = σ_²/^√₂(L) for the Frobe- nius norm. For the rest of the structures, we show that there is a set Ω ⊂ C such that σ^S_²(L)∩Ω =σ²(L)∩Ω. For example, Ω =R when L is H-Hermitian or H-skew-Hermitian and Ω =iRwhen L isH-even orH-odd. Often the spectrum of L is symmetric with respect to Ω.When Ω does not contain an eigenvalue of L,it is of practical importance to determine the smallest perturbation 4L∈S of L such that L +4L has an eigenvalue in Ω. We show how to construct such a 4L.Indeed, we show that the equalityσ^S_²(L)∩Ω =σ²(L)∩Ω plays a crucial role in the construction of such a4L.