Chapter IV: A Unified Theory of Union-of-Subspaces Representations of
4.5 From Subspaces to Dictionaries
x(n)=
8
Õ
i=1
xi(n) (4.13)
where, for eachi, xi(n)indicates the component alongVi. Due to the lack of linear independence among these subspaces, we can obtain an infinite number of such decompositions of the same signal. Notice that the LCM property does not apply to the decomposition in Fig. 4.5(a). The LCM property, if blindly applied, estimates the period to belcm(2,3,6,8) = 24, which is incorrect. This is because theVP’s are not linearly independent.
Next, let us decompose the same signal along subspaces that offer unique periodic decompositions. We used the Ramanujan subspaces{S1,S2, . . . ,S8}in Fig. 4.5(c) and the Natural Basis subspaces{T1,T2, . . . ,T8} in Fig. 4.5(d). The Natural Basis subspaces are an example of non-orthogonal NPSs, proposed in [23]. The decom- positions shown in Fig. 4.5 are the only possible decompositions ofx(n)along these subspaces. In Fig. 4.5(c), the subspaces with non-zero components areS4 andS8, while in Fig. 4.5(d), they areT2,T4andS8. Notice that the LCM property correctly predicts the period of the signal as 8 in each case. In these plots, we have ignored the period 1 components, since it is just DC.
7
x6(n) x5(n) x4(n) x3(n) x2(n)
x8(n) x(n)
x7(n) x6(n) x5(n) x4(n) x3(n) x2(n)
x8(n) x(n)
x7(n) x6(n) x5(n) x4(n) x3(n) x2(n)
x8(n) x(n)
x7(n) x6(n) x5(n) x4(n) x3(n) x2(n)
(a)
(b)
(c)
(d)
1 8 16
1 8 16
1 8 16
1 8 16
Figure 4.5: Parts (a) and (b): Two different decompositions of a period 8 signal onto{V1,V2, . . . ,V8}. Part (a) involvesV2,V3,V6andV8, while Part (b) involves V2, V4 andV8. Clearly, it is difficult to determine the component periods in the signal using these subspaces. Notice that the LCM property results in an incorrect period estimate in Part (a). Parts (c) and (d) use subspaces that offer unique periodic decompositions: Ramanujan Subspaces in Part (c) and the Natural Basis subspaces in Part (d). Both involve only subspaces with period 8 and its divisors. The LCM property correctly identifies the period as 8 in both these cases. Please see Sec. 4.4 for a discussion.
new property of the Farey dictionary. In the next section, we use this new definition of periodic dictionaries to answer the questions mentioned above.
We will first discuss the connection between dictionaries and the subspace models discussed in the previous sections. Suppose we want to estimate the period of a signalx(n), with the prior knowledge that the period lies in the range 1 ≤ P ≤ Pmax. The main idea in the previous sections is that we can design a set of subspaces T= {T1,T2, . . . ,TPmax}such thatx(n)lies in their union. To find the period of x(n), we need to find the exact subset of these subspaces that are involved in spanning x(n).
A popular approach to this problem is to use dictionaries. Let us assume that we haveN consecutive samples ofx(n)available to us in the form:
x=[x(0),x(1), . . . ,x(N−1)]T (4.14) For everyi, letRidenote a basis forTi. We can form a dictionaryA, whose columns are the signals in∪iRi, truncated to the data lengthN. That is:
A= [R1R2 . . . RPmax] (4.15)
Then, the following system of equations will always have a solution fory:
x=Ay (4.16)
By looking at the locations of the non-zero entries in y, we can find out those columns of A (and hence the subspaces in T) that are involved in spanning x(n).
This can be used to estimate the period, as was done for NPDs in Chapter 3.
Typically though, for practical data lengths, Ais a fat matrix. So (4.16) can have multiple solutions fory. How do we solve for the one that involves subspaces with period P and its divisors? Several techniques have been proposed in the past for this, ranging from the sparsity based techniques from the compressive sensing world ([11], [38]–[41] etc.), to simplel2norm convex programs with closed form solutions [23] (Chapter 3).
Periodic Dictionaries - A General Definition
We begin our analysis with the following general definition of a periodic dictionary, that, in particular, captures all the previously proposed dictionaries of [25], [26] and [23] as special examples.
the signals inB. ♦ Notice that the signals in a periodic dictionary are infinitely long, defined over all values of the time index n. One may ask an important question in this regard. In practical applications, since we only have finite duration signals, requiring that our dictionaries span the whole of x(n) may be restrictive. If x(n) was available for n ∈ [1,2, . . . ,N], then, compared to the number of dictionaries that can span the whole ofx(n), we might be able to construct many more dictionaries that spanx(n) only over that finite lengthN duration.
Indeed, there can be many such dictionaries that work for specific data lengths.
But again, in practice, the length of the available data might be arbitrary, or even unknown a priori. So it is desirable to have dictionaries that work for arbitrary data lengths, in which case, it is necessary that it span periodic signals completely, for all time indicesn∈Z. If a dictionaryBcan span a periodic signalx(n)over alln ∈Z, then in particular, it can also span this signal overnbelonging to any subset ofZ. Examples of Periodic Dictionaries
Definition 4.5.1 captures all the previously proposed examples of dictionaries that span periodic signals. For example, in [25], Nakashizuka constructed a periodic dictionary as follows. The signals obtained by periodically extending the columns of aP×Pidentity matrix form a basis forVP. By collecting together such columns from identity matrices of all sizes from 1 to Pmax, a periodic dictionaryB can be obtained. An example forPmax =4 is shown below:
B=
... ... ... ... ... ... ... ... ... ...
1 1 0 1 0 0 1 0 0 0
1 0 1 0 1 0 0 1 0 0
1 1 0 0 0 1 0 0 1 0
1 0 1 1 0 0 0 0 0 1
1 1 0 0 1 0 1 0 0 0
1 0 1 0 0 1 0 1 0 0
... ... ... ... ... ... ... ... ... ...
(4.17)
The first column has period 1, the second and third columns have period 2, fourth to sixth have period 3 and so on. We refer to the columns as atoms. It will be shown in the following that this dictionary has a lot of redundancy. The number of atoms inBis given by:
N(Pmax)=
Pmax
Õ
P=1
P = Pmax× (Pmax+1)
2 (4.18)
We will refer to this dictionary as the Identity dictionary.
Notice that, instead of using the identity matrix to generate a basis forVP, one may use any full rankP×Pmatrix. In [26], the authors chose theP×PDFT matrix for this purpose. Doing so reduces the number of atoms in the dictionary significantly, since many of the DFT matrices of different sizes give rise to the same columns. It was shown in [26] that removing the repeating copies gives a dictionary of size:
N(Pmax)=
Pmax
Õ
P=1
φ(P)= 3P2max
π2 +O(PmaxlogPmax) (4.19) This dictionary was named as the Farey dictionary in [26]. The difference between (4.18) and (4.19) goes as O(Pmax2 ) [24]. We will now derive an important new property of the Farey dictionary:
Theorem 4.5.1. Uniqueness of The Farey Dictionary: A set of complex exponen- tials5Bwill be a periodic dictionary of order Pmax, if and only if, for each period P in 1 ≤ P ≤ Pmax, B contains the φ(P) unique complex exponentials that are periodic with periodP, namely{ej2πkP n:gcd(k,P)=1}. ♦ Proof. Consider a particularPin 1 ≤ P ≤ Pmax. One can show that among the set of all possible complex exponentials, there are exactlyφ(P) complex exponentials with period P. These are in fact the set{ej2πkP n : gcd(k,P) = 1}. SupposeBdoes not contain one of theseφ(P)complex exponentials. Then clearly,Bdoes not have a basis forVP since none of the other complex exponentials can span this missing complex exponential.
Moreover, if for eachPin 1 ≤ P ≤ Pmax, all theφ(P)P-periodic complex exponen- tials are present inB, then the set of all complex exponentials with periods that are divisors ofPare in fact the columns of the P×PDFT matrix. So they will form a
basis forVP. 5 5 5
5A complex exponential is a signal of the formx(n)=ejωn
on sinusoidal frequency estimation, such as [42] and [43]. These works target the problem of estimating the frequencies in a mixture of complex exponentials.
Dictionaries consisting of complex exponentials as atoms were used for this purpose, with their frequencies lying on a uniform grid over 0 to 2π. Since any periodic signal is a mixture of complex exponentials (Fourier series), in principle, one might consider using such techniques for period estimation. However, as shown in Theorem 4.5.1, a complex-exponential dictionary for periodic signals has to have a nonuniform grid as demonstrated in Fig. 4.1.
In [23], the authors showed that the Farey dictionary is an example of a more general set of dictionaries called the Nested Periodic Dictionaries (NPDs). They are based on the NPMs (Sec. 3.1). Consider an NPM Awith number of columns
=lcm(1,2, . . . ,Pmax). LetBbe the set of signals obtained by periodically extending those columns of A that have periods in the range 1 to Pmax. It follows from the properties of NPMs that B is actually a periodic dictionary of order Pmax. Such dictionaries were called as Nested Periodic Dictionaries (NPDs). NPDs have the same size as a Farey dictionary. In fact the Farey dictionary is also an NPD, since the DFT matrix is itself an NPM [23]. Notice that the NPDs have exactly φ(P)signals with periodPfor everyPin the range 1≤ P ≤ Pmax by construction. An example, the Natural Basis dictionary for Pmax = 4, is shown below. Note its smaller size compared to (4.17).
B=
... ... ... ... ... ...
1 1 1 0 1 0
1 0 0 1 0 1
1 1 0 0 0 0
1 0 1 0 0 0
1 1 0 1 1 0
1 0 0 0 0 1
... ... ... ... ... ...
(4.20)