Fast Fourier Transforms

2.3 Trigonometric Interpolation

2.3.2 Fast Fourier Transforms

whereN = 2M + 1andN = 2M, respectively, satisfy Ψ(x_k) =f_k, k= 0,1, . . . N−1,

forx_k = 2πk/N if and only if the coefficients ofΨ(x)are given by A_h= 2

N

N−1 k=0

f_kcoshx_k= 2 N

N−1 k=0

f_kcos2πhk N , B_h= 2

N

N−1 k=0

f_ksinhx_k = 2 N

N−1 k=0

f_ksin2πhk N .

Proof. Only the expressions for A_h, B_h remain to be verified. For by (2.3.1.4)

A_h=β_h+β_N−h= 1 N

N−1 k=0

f_k(e^−hixk+e^{−(N−h)ixk}),

B_h=i(β_h−β_N−h) = i N

N−1 k=0

f_k(e^−hixk−e^{−(N−h)ixk}),

and the substitutions (2.3.1.3) yield the desired expressions.

Note that if the support ordinatesf_kare real, then so are the coefficients A_h,B_h in (2.3.1.12).

ofN²multiplications, putting it out of reach for even high-speed electronic computers for those large values ofN necessary for a sufficiently accurate discrete representation of the above integrals. The discovery [Cooley and Tukey (1965); it is remarkable that similar techniques were already used by Gauss] of a method for rapidly evaluating (on the order of NlogN multiplications) all expressions (2.3.2.1) for large special values of N has therefore opened up vast new areas of applications. This method and its variations are called fast Fourier transforms.For a detailed treatment see Brigham (1974) and Bloomfield (1976).

There are two main approaches, the originalCooley-Tukey method and one described by Gentleman and Sande (1966), commonly called theSande- Tukey method.

Both approaches rely on an integer factorization ofN and decompose the problem accordingly into subproblems of lower degree. These decom- positions are then carried out recursively. This works best when

N = 2ⁿ, n >0 integer.

We restrict our presentation to this most important and most straightfor- ward case, although analogous techniques will clearly work for the more genereal caseN =N₁N₂· · ·N_n,N_m integer.

The Cooley-Tukey approach is best understood in terms of the inter- polation problem described in the previous section 2.3.1. SupposeN = 2M and consider the two interpolating phase polynomialsq(x) andr(x) of order M =N/2 with

q(x_2h) =f_2h, r(x_2h) =f_2h+1, h= 0, . . . , M−1.

The phase polynomial q(x) interpolates all support points of even index, whereas the phase polynomial ˆr(x) =r(x−2π/N) =r(x−π/M) interpo- lates all those of odd index. Since

e^Mixk =e^2πiMk/N =e^πik=

+1, k even,

−1, k odd,

the complete interpolating phase polynomialp(x) is now readily expressed in terms of the two lower-order phase polynomialsq(x) andr(x):

(2.3.2.2) p(x) =q(x)

1 +e^Mix 2

+r(x−π/M)

1−e^Mix 2

. This suggests the followingnstep recursive scheme. Form≤n, let

M = 2^m−1 and R= 2^n−m. Stepmthen consists of determiningR phase polynomials

p^(m)_r =β^(m)_r,0 +β_r,1^(m)e^ix+· · ·+β_r,2M^(m) ₋₁e^(2M−1)ix, r= 0, . . . , R−1,

from 2Rphase polynomialsp^(m−1)_r (x),r= 0,. . ., 2R−1, using the recursion (2.3.2.2):

2p^(m)_r (x) =p^(m−1)_r (x)(1 +e^Mix) +p^(m−1)_R+r (x−π/M)(1−e^Mix).

This relation gives rise to the following recursion between the coefficients of the above phase polynomials:

(2.3.2.3)

2β_r,j^(m)=β_r,j^(m−1)+β_R+r,j^(m−1)ε^j_m 2β_r,M+j^(m) =β_r,j^(m−1)+β_R+r,j^(m−1)ε^j_m







r= 0, . . . , R−1, j= 0, . . . , M−1, where

ε_m:=e^−2πi/2m, m= 0, . . . , n.

The recursion is initiated by putting

β_k,0⁽⁰⁾:=f_k, k= 0, . . . , N−1, and terminates with

β_j:=β_0,j⁽ⁿ⁾, j= 0, . . . , N−1.

This recursion typifies the Cooley-Tukey method.

The Sande-Tukey approach chooses a clever sequence of additions in the sums "_N−1

k=0 f_ke^−jixk. Again with M = N/2, we assign to each termf_ke^−jixk an opposite termf_k+Me^−jixk+M. Summing respective oppo- site terms in (2.3.2.1) producesN sums ofM =N/2 terms each. Splitting those N sums into two sets, one for even indicesj = 2hand one for odd indices j = 2h+ 1, will lead to two problems of evaluating expressions of the form (2.3.2.1), each problem being of reduced degreeM =N/2.

Using the abbreviation

ε_m=e^−2πi/2m

again, we can write the expressions (2.3.2.1) in the form N β_j =

N−1

k=0

f_kε^jk_n , j= 0,1, . . . , N−1.

Here nis such that N = 2ⁿ. Distinguishing between even and odd values ofj and combining opposite terms gives

N β_2h=

N−1

k=0

f_kε^2hk_n =

M−1

k=0

(f_k+f_k+M)ε^hk_n−1=:

M−1 k=0

f_kε^hk_n−1

N β_2h+1=

N−1

k=0

f_kε^(2h+1)k_n =

M−1 k=0

((f_k−f_k+M)ε^k_n)ε^hk_n−1=:

M−1 k=0

f_kε^hk_n−1

forh= 0,. . .,M −1 andM :=N/2, sinceε²_n =ε_n−1,ε^M_n =−1. Here f_k =f_k+f_k+M

f_k=

f_k−f_k+M ε^k_n

'

k= 0, . . . , M−1.

In order to iterate this process for m=n,n−1,. . ., 0, we let M :=

2^m−1,R:= 2^n−mand introduce the notation

f_r,k^(m), r= 0, . . . , R−1, k= 0, . . . ,2M−1,

withf_0,k⁽ⁿ⁾=f_k,k= 0,. . .,N−1.f_0,k⁽ⁿ⁻¹⁾andf_1,k⁽ⁿ⁻¹⁾represent the quantities f_k andf_k, respectively, which were introduced above. In general we have, withM = 2^m−1andR= 2^n−m,

(2.3.2.4) N β_jR+r=

2M−1

k=0

f_r,k^(m)ε^jk_m, r= 0,1, . . . , R−1, j= 0,1, . . .2M−1,

with the quantitiesf_r,k^(m)satisfying the recursions:

(2.3.2.5)

f_r,k^(m−1)=f_r,k^(m)+f_r,k+M^(m) f_r+R,k^(m−1)= (f_r,k^(m)−f_r,k+M^(m) )ε^k_m











m=n, n−1, . . . ,1, r= 0,1, . . . , R−1, k= 0,1, . . . , M−1.

Proof.Suppose (2.3.2.4) is correct for somem≤n, and letM:=M/2 = 2^m−2, R := 2R = 2^n−m+1. For j = 2h and j = 2h+ 1, respectively, we find by combining opposite terms

N β_hR+r=N β_jR+r=

M−1 k=0

(f_r,k^(m)+f_r,k+M^(m) )ε^jk_m =

2M−1 k=0

f_r,k^(m−1)ε^hk_m−1,

N β_hR+r+R=N β_jR+r=

M−1 k=0

(f_r,k^(m)−f_r,k+M^(m) )ε^jk_m

=

M−1 k=0

(f_r,k^(m)−f_r,k+M^(m) )ε^k_mε^hk_m−1=

2M−1 k=0

f_r+R,k^(m−1)ε^hk_m−1, wherer= 0,. . .,R−1,j= 0, . . ., 2M −1.

The recursion (2.3.2.5) typifies the Sande-Tukey method. It is initiated by

f_0,k⁽ⁿ⁾:=f_k, k= 0, . . . , N −1, and terminates with

β_r:= 1

Nf_r,0⁽⁰⁾, r= 0,1, . . . , N−1.

Returning to the Cooley-Tukey method for a more detailed algorithmic formulation, we are faced with the problem of arranging the quantitiesβ^(m)_r,j in a one-dimensional array:

β[κ(m, r, j)] :=˜ β_r,j^(m), κ= 0, . . . , N−1.

Among suitable maps (m, r, j) → κ(m, r, j), the following is the most straightforward:

κ= 2^mr+j; m= 0, . . . , n, r= 0, . . . ,2^n−m−1, j= 0, . . . ,2^m−1.

It has the advantage, that the final results are automatically in the correct order. However, two arrays ˜β[ ], ˜β[ ] are necessary to accommodate the left- and right-hand sides of the recusion (2.3.2.3). We can make do with only one array ˜β[ ] if we execute the transformations “in place”, that is, if we let each pair of quantitiesβ_r,j^(m),β_r,M^(m)_+j occupy the same positions in β[ ] as the pair of quantities˜ β_r,j^(m−1), β_R+r,j^(m−1), from which the former are computed. In this case, however, the entries in the array ˜β[ ] are being permuted, and the maps which assign the positions in ˜β[ ] as a function of integersm, r,j become more complicated. Let

τ =τ(m, r, j)

be a map with the above mentioned replacement properties, namely β[τ(m, r, j)] =˜ β_r,j^(m)

with (2.3.2.6)

τ(m, r, j) =τ(m−1, r, j)

τ(m, r, j+ 2^m−1) =τ(m−1, r+ 2^n−m, j) ' 



m= 1, . . . , n,

r= 0, . . . ,2^n−m−1, j = 0, . . . ,2^m−1−1, and

(2.3.2.7) τ(n,0, j) =j, j= 0, . . . , N−1.

The last condition means that the final resultβ_j will be found in position j in the array ˜β:β_j = ˜β[j].

The conditions (2.3.2.6) and (2.3.2.7) define the mapτ recursively. It remains to determine it explicitly. To this end, let

t=α₀+α₁·2 +· · ·+α_n−1·2ⁿ⁻¹, α_j ∈ {0,1},

be the binary representation of an integert, 0≤t <2ⁿ. Then putting (2.3.2.8) ρ(t) :=α_n−1+α_n−2·2 +· · ·+α₀·2ⁿ⁻¹

defines a permutation of the integerst= 0,. . ., 2ⁿ−1 calledbit reversal.The bit-reversal permutation is symmetric, i.e.ρ(ρ(t)) =t.

In terms of the bit-reversal permutation ρ, we can express τ(m, r, j) explicitly:

(2.3.2.9) τ(m, r, j) =ρ(r) +j,

for allm= 0, 1,. . .,n,r= 0, 1,. . ., 2^n−m−1,j= 0, 1,. . ., 2^m−1.

Proof.If again

t=α₀+α₁·2 +· · ·+α_n−1·2ⁿ⁻¹, α_j ∈ {0,1}, then by (2.3.2.6) and (2.3.2.7)

t=τ(n,0, t) =

τ(n−1,0, t) ifα_n−1= 0, τ(n−1,0, t−2ⁿ⁻¹) ifα_n−1= 1.

Thus

t=τ(n,0, t) =τ(n−1, α_n−1, α₀+· · ·+α_n−2·2ⁿ⁻²), and, more generally,

t=τ(n,0, t) =τ(m, α_n−1+· · ·+α_m·2^n−m−1, α₀+· · ·+α_m−1·2^m−1) for allm=n−1,n−2,. . ., 0. Forr=α_n−1+· · ·+α_m·2^n−m−1, we find

ρ(r) =α_m·2^m+· · ·+α_n−1·2ⁿ⁻¹

andt=ρ(r) +j, where j=α₀+· · ·+α_m−1·2^m−1. By the symmetry of bit reversal,

τ(m, ρ(¯r), j) = ¯r+j,

where ¯ris a multiple of 2^m, 0≤r <¯ 2ⁿ, and 0≤j <2^m. Observe that if 0≤j <2^m−1, then

t=τ(m, ρ(¯r), j) =τ(m−1, ρ(¯r), j) = ¯r+j,

t¯=τ(m, ρ(¯r), j+ 2^m−1) =τ(m−1, ρ(¯r) + 2^n−m, j) = ¯r+j+ 2^m−1 mark a pair of positions in ˜β[ ] which contain quantities connected by the Cooley-Tukey recursions (2.3.2.3).

In the following pseudo-algol formulation of the classical Cooley- Tukey method, we assume that the array ˜β[ ] is initialized by putting

β[ρ(k)] :=˜ f_k, k= 0, . . . , N−1,

whereρis the bit-reversal permutation (2.3.2.8). This “scrambling” of the initial values can also be carried out “in place”, because the bit-reversal permutation is symmetric and consists, therefore, of a sequence of pairwise interchanges or “transpositions”. In addition, we have deleted the factor 2 which is carried along in the formulas (2.3.2.3), so that finally

β_j := 1 N

β[j],˜ j= 0, . . . , N −1.

The algorithm then takes the form form:= 1 step1 until n do

begin for j:= 0 step 1 until 2^m−1−1 do begin e:=ε^j_m;

for ¯r:= 0 step 2^m until 2ⁿ−1 do beginu:= ˜β[¯r+j]; v:= ˜β[¯r+j+ 2^m−1]×e;

β˜[¯r+j] :=u+v; ˜β[¯r+j+ 2^m−1] :=u−v end

end end ;

If the Sande-Tukey recursions (2.3.2.5) are used, there is again no prob- lem if two arrays of lengthN are available for new and old values, respec- tively. However, if the recursions are to be carried out “in place” in a single array ˜f[ ], then we must again map index triplesm,r,jinto single indices τ. This index map has to satisfy the relations

τ(m−1, r, k) =τ(m, r, k), τ(m−1, r+ 2^n−m, k) =τ(m, r, k+ 2^m−1)

form=n,n−1,. . ., 1,r= 0, 1,. . ., 2^n−m−1,k= 0, 1,. . ., 2^m−1−1.

If we assume

τ(n,0, k) =k fork= 0, . . . , N −1,

that is, if we start out with the natural order, then these conditions are precisely the conditions (2.3.2.6) and (2.3.2.7) written in reverse. Thus τ =τ(m, r, k) is identical to the index map τ considered for the Cooley- Tukey method.

In the following pseudo-algolformulation of the Sande-Tukey method, we assume that the array ˜f[ ] has been initialized directly with the values f_k:

f˜[k] :=f_k, k= 0,1, . . . , N−1.

However, the final results have to be “unscrambled” using bit reversal, β_j:= 1

N

f˜[ρ(j)], j= 0, . . . , N−1 :

form:=n step −1 until 1 do

begin for k:= 0 step1 until 2^m−1−1 do begin e:=ε^k_m;

for ¯r:= 0 step 2^m until 2ⁿ−1 do begin

u:= ˜f[¯r+k]; v:= ˜f[¯r+k+ 2^m−1];

f˜[¯r+k] :=u+v; ˜f[¯r+k+ 2^m−1] := (u−v)×e end

end end ;

If all valuesf_k, k= 0,. . .,N−1, are real andN = 2M is even, then the problem of evaluating the expressions (2.3.2.1) can be reduced in size by putting

g_h:=f_2h+i f_2h+1, h= 0,1, . . . , M−1, and evaluating the expressions

γ_j := 1 M

M−1

h=0

g_he^−2πijh/M, j= 0,1, . . . , M−1.

The desired valuesβ_j,j = 0,. . .,N−1, can be expressed in terms of the valuesγ_j,j= 0,. . .,M −1. Indeed, one has withγ_M :=γ₀

(2.3.2.10) β_j= 1

4(γ_j+ ¯γ_M−j) + 1

4i(γ_j−γ¯_M−j)e^−2πij/N, j= 0,1, . . . , M β_N−j= ¯β_j, j= 1,2, . . . , M−1.

Proof.It is readily verified that 1

4(γ_j+ ¯γ_M−j) = 1 N

M−1

h=0

f_2he^−2πij2h/N, 1

4i(γ_j−γ¯_M−j) = 1 N

M−1

h=0

f_2h+1e−2πij(2h+1)/N+2πij/N.

In many cases, particularly if all values f_k are real, one is actually interested in the expressions

A_j:= 2 N

N−1

k=0

f_kcos2πjk

N , B_j:= 2 N

N−1 k=0

f_ksin2πjk N

forj= 0, 1,. . .,M, which occur, for instance, in Theorem (2.3.1.12). The valuesA_j,B_j are connected with the corresponding values forβ_j via the relations (2.3.1.4).