This thesis would never have been possible without the support and guidance of my friends, family, colleagues and advisors. Additionally, I would never have been able to succeed at Caltech if it weren't for the people at Northwestern who taught me to love research.
Background
Historical Framework 1
The discovery of turbo codes led to a flurry of research interest in the field, and in particular to the rediscovery of Gallager's 1963 work [26] on low density parity check (LDPC) codes. Although Gallager's work was largely forgotten due to the limited computing power of his time, some interesting developments occurred.
Scope of This Thesis
Finally, in Chapter 6, we will discuss methods to estimate the performance of a code without resorting to extensive simulation.
The Basic Communications System
Channel
Common Channel Models Used in This Thesis
An additive white Gaussian noise (AWGN) channel is defined by the signal-to-noise ratio Eb/N0, the ratio of the energy in one bit of data to the energy of the surrounding noise. Gaussian noise is a reasonable approximation of the type of noise that occurs in deep space communications [4].
Error-Correcting Codes
The rate of a code R =k/n is the ratio of the amount of information contained in a word (k bits) to the length of the code (n bits). In terms of a Tanner graph, a word is a codeword if the subsets associated with each control node have parity.
The Weight Enumerator
Ensembles
- The Shannon Ensemble
- Regular LDPC Codes
- Litsyn and Shevelev’s LDPC Ensembles
- RA Code Ensembles
- Protograph Code LDPC Ensembles
- Codes Used in this Thesis
The spectral shape is the limit of the logarithm of this equation, and is given in the following equation, In this chapter, we will show that the spectral shape of an ensemble is closely related to that of the Shannon ensemble.
Review of the Pless Identities and Dual Codes
According to the Pless Identities, the first d⊥ central moments are equivalent to the first d⊥ central moments of the Shannon ensemble. For simplicity, we will use Ψ(s) to denote the moment generating function of the codeword weight in a particular code or string and Ψ0(s) to denote the moment generating function of the codeword weight in the Shannon Ensemble.
A Legendre Transform Theorem for Spectral Shapes*Shapes*
In words, this means that the function associated with the asymptotic cumulant generating function of the codeword weight and the convex hull of the spectral shape are Legendre transforms of each other. From the distribution of q by V, we can transform into the distribution of p by J, where for j =w(v),pj =mjq(v) andP.
Connecting the Pless Identities to the Spectral Shape*Shape*
- The Cumulant-Generating Function
- The Legendre Transform
We are now ready to use Theorem 2.2 and take the Legendre transform of F(x) and F0(x), which gives the convex hulls of the logarithms of the weights G(s) and G0(s). As in Fa´a di Bruno's formula for composition of functions, the first derivatives of a parametrically defined function depend only on a1, a2,. The Legendre transformation sets up a bijection between i moment space and xin spectral shape space as s=G0(x) and x=F0(s) [2].
This is the case for Litsyn and Shevelev's [33] ensembles C, D and G, which were introduced in chapter one and will be further studied in the following chapter.
Conclusions*
Since the minimum distance of the dual code is not greater than the minimum rank weight of the parity check matrix, this suggests that ensembles whose parity check matrices have high rank weight have spectral shapes similar to that of the Shannon ensemble. The minimum distance of the LDGM code is clearly not greater than the smallest row weight in its generator matrix. In the limit, when the length of the code approaches infinity, the minimum distance is precisely the minimum row weight of the generator matrix.
Thus, the minimum dual weight is the minimum row weight of the parity check matrix.
Regular Ensemble*
The minimum distance of the LDGM code is obviously not greater than k, since each of the lines is a codeword and has a weight of k. Using our formula for parametric derivatives discussed in Chapter 4, we can calculate the derivatives of the spectral shape E(θ) with respect to θ. However, if we subtract the derivatives of the entropy function and the factorial, we can say something about the radius.
If we let bi be the ratio between the ith derivative of the spectral shape and the entropy function, then the radius of convergence of the series is 12lim(bi)−1/i.
Litsyn and Shevelev’s Ensembles*
- Ensemble C
- Ensemble E*
- Ensemble G
With only fifteen values we cannot define the limit precisely, but the series appears to approach somewhere close to 0.6, which indicates that the radius of convergence of the Taylor series is about 0.3. We expect the derivatives of the spectral shape at θ = 1/2 to match those of the Shannon ensemble to this extent. In the neighborhood around θ = 12 the difference between the spectral shape and the shifted entropy function h(x)−(1−R) is of order (1−2θ)k∗ where k∗ is the smallest element inK, and the smallest row weight of the parity check matrix.
Thus, as expected, the derivatives of the spectral shape will be identical to those of the Shannon ensemble up to the k∗th derivative.
RA Codes*
We know that the degree of tangent, which in this case is four, must be at least the minimum weight of the duplicate code. However, this is a case where the degree of tangent actually exceeds the minimum distance of the duplicate code. To calculate the true minimum weight of the dual against the RA codes, we need to examine the Tanner plot representation of the codes, shown in Figure 3.7.
The spectral shape of the RA code matched the Shannon cast to the fourth degree, suggesting that it may not be the smallest dual weight that matters, but the smallest dual weight for which the number of words is important relative to n.
Protograph Codes*
- Newton’s and Broyden’s Methods
- Specific Protograph Enumerators*
For a vector x∈Ω, W(x)1 is the weight of the first 3 components of x, and W(x)2 is the weight of the last 4 components of x. We solve the constrained maximization problem (1.11) for the spectral form of the protograph ensemble by taking the derivative of the Lagrangian. In our case, we know that the solution is unique due to the Legendre transformation relation.
We expect the minimum weight of a dual code to be the same as the minimum weight of a row of the protomatrix.
![Table 3.3: Example of x and W (x) for [3, 4] Protograph](https://thumb-ap.123doks.com/thumbv2/123dok/10415309.0/63.918.200.771.420.832/table-3-3-example-x-w-x-protograph.webp)
Conclusions
As mentioned in Chapter 1, one of the advantages of our method for calculating protograph spectral shapes is that it can be easily modified to calculate the growth rate of stop set enumerators. If each row in the parity check matrix has weight k, then there are at least O(n) words with weight k in the binary code. If this is the case, then the results in this section support a stronger hypothesis than Theorem 2.3, i.e. that the ith derivative of the spectral shape of the ensemble at θ = 1/2 is exactly the same as the derivative of the spectral shape of the Shannon ensemble, if the number of words in the double ensemble of weight i is o(n).
This would suggest that, while the minimum double weight is useful for predicting the spectral shape of the primordial ensemble, knowing even a few more points in the double adder provides a much more accurate description of the primordial code's spectral shape will make possible.
Introduction and Notation
To construct these sequences we need the concept of set theory and number theory partitions. We will use the uppercase letter Λ to refer to the former, and lowercase λ to refer to the latter. Finally, let N(λ) be the number of set theory partitions that collapse to number theory partition λ.
Fa´ a di Bruno and Function Composition
Parametric Functions*
Implicit Functions*
With each subsequent differentiation, the number of derivatives x increases by one, as we might expect. The sum of these ordered pairs will be the pair (n, k) We will assume the convention that in each pair, the first element corresponds to x derivatives and the second to y. We will assume that the first pair corresponds to the off derivatives, and we will call this first pairµ.
As a result, the number of y-derivatives (k) must also be equal to the number of terms involving g.
Bounded Distance and Maximum Likelihood Decoding
Finally, the last section discusses the effects of adding an input buffer to the decoding system. Using the weight counter, or spectral form, bounds on the maximum likelihood decoding error probability can be found. A maximum likelihood decoder must find the distance between the received word and each of the codewords in order to choose the smallest one.
So while the maximum likelihood decoder can correct most errors, its complexity grows exponentially with the length of the codewords.
Introduction to Message Passing Decoding
- Message Passing on the BEC
- Message Passing on Other Channels
In the upper right, the third control node sends a message to the second variable node, letting it know that it is zero. This is an essential element of evidence for the success of message forwarding decoding and will be discussed further in Section 6.3. The outgoing message at each end leaving a mutable node is the sum of all incoming messages to the other ends connected to that mutable node.
At a control node, the outgoing message u is given in terms of the incoming messages on other edges, x and y, by equation (5.1).
Iterative Decoders with Input Buffers*
- The Model
- Mathematical Analysis
- Latency
- Sensitivity to the Quality of the Distribution
- Optimization of the Iteration Cap
- Performance Curves
- Conclusion
Since each iteration of the decoder takes a constant amount of time, an iteration can be viewed as a unit of time; we consider the average number of iterations needed to decode a word and the number of iterations between word arrivals. Note that when the repetition cap is small, the WER of the buffered decoder is limited by the performance of the fixed repetition decoder. For convenience, a graph of the average number of iterations to decode at each Eb/N0 value accompanies each sequence of graphs.
Note also that, as predicted by the latency analysis, the decoder's word error rate in the buffer, even with an additional buffer containing only one word, is nearly optimal when the number of iterations between received words is greater than twice the average number of iterations for decoding.
Applications of the Spectral Shape*
The approximate values of θs, the value of θ at which the growth rate of the stopping set counter crosses zero, are also tabulated. An entry of "-" in the table is used for ensembles whose spectral shapes are never negative. The minimum possible signal-to-noise ratio is given by the speed of the code, so it is reasonable to compare codes of the same speed.
In this section, we compared the zero crossings of the spectral shape, the growth rate of the stop set enumerator, and the performance limit of AWGN for several simple protographic codes.
Density Evolution on the BEC
- Simulated Annealing*
- Simulated Annealing with Constraints*
At each change in the annealing process, the value of the spectral shape at θ is calculated. However, if the value of the spectral shape at this point is positive, the change is always rejected. Calculation of the zero crossing of the spectral shape is difficult and cannot be performed for every simulated annealing.
The density evolution threshold on the BEC appears to be negatively correlated with the zero crossing of the spectral shape.
Cycles and Girth
- Gallager’s Method for Regular Codes
- Protographs Codes — An Example*
- Protograph Codes in General*
- Conclusions
For any code of length n, there exists a point where the number of nodes in the tree exceeds n. We denote by Tc(L) the number of control nodes connected to the first Lrow of variable nodes. At each layer, we count the number of variables of type j that came from check nodes of type i.
We then calculate the number of control nodes of type i that come from these variables of type j.
