1 0 0 0 0 0 0 2 1 2 1 2 0 1 0 0 0 0 1 0 2 1 1 2 0 0 1 0 0 0 2 2 0 1 1 1 0 0 0 1 0 0 1 1 1 0 1 1 0 0 0 0 1 0 1 2 2 2 0 1 0 0 0 0 0 1 2 1 2 2 1 0
Determine the parameters of the ternary code with this check matrix.
What are the parameters after erasing one column?
Compare with the sphere-packing bound.
Observe that the entries ofσ(y)are the dot products of y and the rows ofH.
In particular, y∈ C if and only ifσ(y) = 0 is the 0vector.
When will two vectors have the same syndrome vector,σ(y) =σ(y′)? This means 0 = σ(y)−σ(y′) = σ(y−y′), in other words,y−y′ ∈ C.The sets of vectors with the same syndrome vector are calledcosetsofC.
3.25 Definition. A cosetof the linear codeC is a set of vectors of the form C+y. Two vectors are in the same coset if they have the same syndrome vector. The number of different cosets isqn−k (if C is an[n, k]q code).
What does this have to do with decoding? If y =x+err is the received vector and we decode it (hopefully) as y7→x=y−err, thenσ(y) =σ(err).
We are therefore looking for a vector with small weight in the same coset as y.These vectors are our candidates forerr.
Here is how syndrome decoding works: in each of the qn−k cosets of C, find a vector of smallest weight. Call this element the coset leader. Ify is received, the following steps are executed:
1. Compute the syndrome vector σ(y).
2. In the list find the coset leaderǫof the corresponding coset.
3. Decodey7→y−ǫ.
Use the extended binary Hamming code [8,4,4]2to illustrate. Consider the matrix
G=H =
1000 0111 0100 1011 0010 1101 0001 1110
.
Check again our basic facts: the first four columns show thatGgenerates a 4-dimensional codeC.As any two rows are orthogonal, we haveC ⊆ C⊥,the dimension showsC =C⊥, a self-dual code. We can therefore considerGalso as a check matrixH for the same code. Cis contained in the sum-0 code (all rows ofGhave weight 4), and it is clear that there is no codeword of weight 2.
This shows thatCis an [8,4,4]2code. Observe that the columns ofH consist of all 4-tuples of weights 1 or 3.Observe also thatC contains the all-1 vector (the sum of all rows ofG). It follows that there can be no codewords of weight 6 inC.
The number of cosets is 16.One coset isCitself, with coset leader the 0 word.
Each vector of weight 1 is the unique coset leader in its coset (as there are no words of weight 2 inC). This gives us eight more cosets and their leaders.
The corresponding syndrome vectors are columns of H,hence all vectors of weights 1 or 3 in F4
2. If a coset contains a vector of weight 2, then this is the minimum weight in this coset. The six vectors of weight 2 with support
in the second half of the coordinates are in pairwise different cosets. This gives us six more cosets. The corresponding syndrome vectors are the vectors of weight 2.Only one coset is still missing. It corresponds to the syndrome vector (1,1,1,1). We can choose (1,0,0,0,1,0,0,0) as representative. This gives us the following list of coset leaders and syndrome vectors:
coset leader syndrome vector
00000000 0000
10000000 1000
01000000 0100
00100000 0010
00010000 0001
00001000 0111
00000100 1011
00000010 1101
00000001 1110
00001100 1100
00001010 1010
00001001 1001
00000110 0110
00000101 0101
00000011 0011
10001000 1111
Our code corrects one error. This translates to the fact that each vector of weight ≤1 is the uniquely determined leader of its coset. As we choose leaders also from cosets which do not contain a unique leader, an attempt is made to decode also in these cases when more than one error occurred. If this idea is followed systematically, we arrive atlist decoding, a topic which is developed in Section 23.1.
Assume y= 11000011 is received. Computeσ(y) = 1111.The correspond- ing coset leader is 10001000.We decodey7→01001011.
1. LetH be a check matrix of the [n, k, d]q codeC. 2. Thesyndrome vectorofy∈Fn
q isσ(y) =yHt (the entries ofσ(y) are the dot products ofy and the rows ofH).
3. y, y′ ∈Fn
q are in the samecosetofC ify−y′∈ C,equivalently ifσ(y) =σ(y′).
4. Aleaderof a coset is a vector of smallest weight.
5. syndrome decoding:
6. Establish a list of leaders of allqn−k cosets.
7. Decodey7→x=y−z,wherezis the leader of the coset containingy.
Exercises 3.7
3.7.1. Use the decoding scheme for [8,4,4]2 given in this section to decode the following received vectors:
y= 00110011, y = 11111010, y= 10001001.
3.7.2. Find coset leaders for the sum-0 codes[n, n−1,2]q. 3.7.3. Find coset leaders for the Hamming code[7,4,3]2.
3.7.4. Find a check matrix of a quaternary code[6,3,4]4. Find coset leaders for all cosets.
Chapter 4
Singleton bound and Reed-Solomon codes
Basic concepts: Singleton bound, Lagrange interpolation, Vandermonde matrices. Latin square codes and OA codes.
Covering arrays and software testing.
Here comes our second general bound on codes. Consider a code (n, M, d)q
and arrange the codewords in an array (with M rows and n columns). The trick is the following: pick somed−1 coordinates, say the last ones. Forget these coordinates (in other words, project to the firstn−d+ 1 coordinates).
These shorter strings are still different (if two of them agreed, then the code- words they were derived from would have distance ≤ d−1, which is not possible by the definition ofd). It follows thatM cannot be larger than the number of all q-ary (n−d+ 1)-tuples. We have proved the following bound (see Singleton [190]):
4.1 Theorem (Singleton bound). Let C be aq-ary code of lengthn.Then
|C| ≤qn−d+1.
Codes satisfying this bound with equality are traditionally calledMDS codes.
Here the lettersMDSstand for maximum distance separable.
In the linear case this simplifies:
4.2 Corollary. Each linear code [n, k, d]q satisfiesk+d≤n+ 1.
Next we construct a very important family of MDS codes.
Before this can be done, recall the concept of a polynomial. A polynomial (in one variable) with coefficients in the fieldK is an expression of the form
f(X) =anXn+an−1Xn−1+· · ·+a1X+a0.
71
Ifan6= 0,we callnthedegreeoff(X).Each polynomial defines a polynomial function (u∈K is mapped to the evaluation f(u)). Call u∈ K a root(or a zero) of f(X) iff(u) = 0.It is a basic fact that a polynomial of degreen cannot have more thann roots. The reason is that, whenever f(u) = 0, we can write f(X) as a product f(X) = (X−u)g(X), where g(X) has degree n−1; see Exercises 4.7 to 4.11, where the reader is also asked to prove some of the basic facts concerning a very handy and elementary tool, thegeometric series.
We describe an array whose rows are the codewords. Let the rows be parametrized by the polynomials p(X) with coefficients inFq,of degree < k (these form a vector space of dimension k with the monomials Xi, i = 0,1, . . . , k−1 as basis). The columns are parametrized by the elements u∈ Fq. The entry in row p(X) and column uis the evaluation p(u). Obviously this defines a linear q-ary code of length q (because linear combinations of polynomials of degree< kare again polynomials of degree< k).
4.3 Definition. Fork≤qthe words of the Reed-Solomon codeRS(k, q) of dimensionkoverFq are parametrized by the polynomials p(X)with coeffi- cients inFq of degree< k,the coordinates byu∈Fq.The corresponding entry isp(u).The code RS(k, q) has lengthq and dimension k.
Let us consider some examples, at firstRS(2,3).The columns (coordinates) are indexed by the elements 0,1,2 of the field; the rows of the generator matrix are indexed by 1 (a constant polynomial) andX.The corresponding generator matrix ofRS(2,3) is
1 1 1 0 1 2
.A nonzero polynomial of degree≤1 (a linear combination of 1 and X) has at most one root, so the minimum weight of the corresponding codeword is ≥3−1 = 2. We conclude that RS(2,3) is a [3,2,2]3 code.
ConsiderRS(2,4).A generator matrix is
1 1 1 1 0 1ω ω
.Again, a polynomial of degree ≤1 has at most one zero, sod≥4−1 = 3. It is clear thatd= 3, so RS(2,4) is a [4,2,3]4 code. The Reed-Solomon code RS(2,4) has been displayed in Section 3.3.
A generator matrix of RS(3,5) is
1 1 1 1 1 0 1 2 3 4 0 1 4 4 1
(the last row corresponds toX2,the entry in column uisu2). A polynomial of degree≤2 has at most 2 roots, so each codeword has weight≥5−2 = 3.It follows thatRS(3,5) is a [5,3,3]5code.
Finally,RS(4,7),has generator matrix
1 1 1 1 1 1 1 0 1 2 3 4 5 6 0 1 4 2 2 4 1 0 1 1 6 1 6 6
.
A polynomial of degree ≤3 has at most three roots, so d= 7−3 = 4 and RS(4,7) is a [7,4,4]7 code. In fact it is clear thatd= 4 and not larger. The
polynomialX(X−1)(X−2) has degree 3 and three roots. The corresponding codeword therefore has weight 7−3 = 4. It is (0,0,0,6,3,4,1).As X(X − 1)(X−2) =X3+ 4X2+ 2X,this should be the last row plus four times the third row plus the double of the second row of our generator matrix. This is indeed the case.
Matrices of the type that we obtain as generator matrices of Reed-Solomon codes (each column indexed by some field element u,the column consisting of the consecutive powers ofu) are known asVandermonde matrices.
It is clear now how to determine the minimum distance of Reed-Solomon codes in general. The number of roots of a nonzero polynomial of degree< k is≤k−1.It follows that the minimum weight is≥q−(k−1) =q−k+ 1.
As there are polynomials of degreek−1 withk−1 different roots (products of different linear factors), we have equality. This shows that RS(k, q) is a [q, k, q−k+ 1]q code. Comparison with the Singleton bound Theorem 4.1 shows that we have equality: the Reed-Solomon codes are MDS codes.
4.4 Theorem. The Reed-Solomon code RS(k, q) is a[q, k, q−k+ 1]q code.
Reed-Solomon codes are MDS codes.
The strength of the Reed-Solomon codes is as large as it could possibly be. Readers who knowLagrange interpolationwill realize this right away.
Lagrange interpolation shows that, for every k ≥ 1, there is precisely one polynomial of degree < k which takes on given values at k field elements.
This statement is valid over any field. As it is of independent interest and rather famous, we formulate and prove it here in this form:
4.5 Theorem(Lagrange interpolation). Let F be a field. A nonzero polyno- mial p(X)∈F[X] of degreenhas at mostnroots.
Ifndifferent elementsx1, x2, . . . , xn inF andnarbitrary elementsy1, . . . , yn
inF are given, then there is exactly one polynomial p(X)of degree< nwith coefficients in F which satisfies
p(xi) =yi, i= 1,2, . . . , n.
PROOF Ifp(x) = 0,thenp(X) is divisible by the linear factorX−x.If p(X) hasn+ 1 different roots x1, x2, . . . , xn+1,thenp(X) is divisible by the product of the corresponding linear factors. If the degree of p(X) is ≤n, it follows thatp(X) is the 0 polynomial. This proves the first claim. Consider the second claim. We prove at first the uniqueness of the polynomial. In fact, ifp1(X) andp2(X) are polynomials both satisfying our conditions, then p1(X)−p2(X) has n different roots, hence is the 0 polynomial, by the first claim. The existence of an interpolating polynomial is proved by writing it down:
p(X) = Xn
i=1
yi
Q
j6=i(X−xj) Q
j6=i(xi−xj).
Theorem 4.5 is particularly obvious, at least over the real numbers, for small values ofk.There is precisely one constant polynomial having some prescribed value at a certain point (what else, if it is constant?) For k = 2 we obtain the statement that two points with differentx-coordinates are on a uniquely determined (nonvertical) line. Things get less obvious at k = 3 : there is precisely one quadratic polynomial interpolating three points with different x-coordinates; in other words, there is precisely one parabola through these three points. Here it is understood that the parabola may degenerate to a line (if the interpolating polynomial has degree<2).
In our context we do not need this algebraic proof for Lagrange interpola- tion. We can prove it using duality. What is the dual ofRS(k, q)? If Lagrange interpolation is correct, it should be a (q−k)-dimensional code of minimum distance k+ 1, hence an MDS code again. The only possible candidate we know is the (q−k)-dimensional Reed-Solomon code. The following theorem confirms this suspicion.
4.6 Theorem.
RS(k, q)⊥ =RS(q−k, q).
PROOF Observe that RS(k, q) and RS(q−k, q) have complementary dimensions, so it suffices to show they are orthogonal. RS(k, q) is generated by theXi, i < k; the codeRS(q−k, q) is generated by theXj, j < q−k.The dot product of the corresponding codewords is S =P
uui+j. Let l =i+j.
Observel≤q−2.There is some 06=c∈Fq such thatcl6= 1,as otherwise the polynomialXl−1 of degreelwould haveq−1> lroots, which is impossible.
We haveclS =S,ascuruns through all nonzero elements ofFq whenudoes.
It follows that (cl−1)S= 0.Ascl6= 1,it follows that S= 0.
4.7 Theorem. RS(k, q)is an OA1(k, q, q).
PROOF The dual ofRS(k, q) isRS(q−k, q),of minimum distancek+ 1.
By duality this means that RS(k, q) has strength k. As it hasqk elements, the claim follows.
Observe that the interpolation property (there is exactly one polynomial such that. . .) of Theorem 4.5 is equivalent to the fact that the Reed-Solomon code is an OA of index 1.
Asq-ary Reed-Solomon codes have lengthq,we need a relatively large field to obtain interesting codes. Examples for parameters of Reed-Solomon codes are
[5,3,3]5, [7,3,5]7, [8,4,5]8, [8,5,4]8, [16,8,9]16, . . .
Consider 8-ary Reed-Solomon codes. We need the fieldF8.Let us just follow the general construction procedure: as 8 = 23, we need an irreducible poly- nomial f(X) of degree 3 with coefficients in F2. The constant term cannot vanish (the polynomial would be divisble by X otherwise). This leaves us with four candidates:
X3+ 1, X3+X2+ 1, X3+X+ 1 andX3+X2+X+ 1.
The first and the last of these candidates are reducible, as they satisfyf(1) = 0. These polynomials are therefore divisible by X −1 = X + 1. The two remaining candidates are irreducible. We choose
f(X) =X3+X2+ 1.
Denote byǫthe image ofX inF8.The elements ofF8are the polynomials of degree≤2 inǫ
F8={0,1, ǫ, ǫ+ 1, ǫ2, ǫ2+ 1, ǫ2+ǫ, ǫ2+ǫ+ 1}
This defines the addition. The multiplication is determined by the basic equation (equivalent to the choice off(X))
ǫ3=ǫ2+ 1.
We can multiply any two elements ofF8.The basic equation allows us to get rid of all exponents>2.Consider the powers ofǫ
ǫ3=ǫ2+ 1, ǫ4=ǫ3+ǫ=ǫ2+ǫ+ 1.
In the same manner we can determine all powers ofǫ
ǫ5=ǫ3+ǫ2+ǫ=ǫ+ 1, ǫ6=ǫ2+ǫ andǫ7= 1.
We see once again that knowledge of the irreducible polynomial is equivalent to knowledge of the field.
To write down generator matrices of the 8-ary Reed-Solomon codes, let 0,1, ǫ, . . . , ǫ6 parametrize the columns, in this order. Rows are parametrized by the monomials 1, X, X2, . . . .
The following generator matrix forRS(4,8) is obtained:
G=
1 1 1 1 1 1 1 1 0 1 ǫ ǫ2ǫ3ǫ4ǫ5ǫ6 0 1ǫ2ǫ4ǫ6 ǫ ǫ3ǫ5 0 1ǫ3ǫ6ǫ2ǫ5 ǫ ǫ4
.
We know thatRS(4,8) is a self-dual code [8,4,5]8.Also, the Reed-Solomon codes of different dimensions are contained in each other. The first three rows ofGgenerateRS(3,8),the first two rows generateRS(2,8) andRS(1,8) is the repetition code.
Latin square codes and OA codes
In general, we concentrate on linear codes, as they are easier to construct and to work with. However, there are a number of situations when very good nonlinear codes can be constructed.
Consider an OA1(t, n, q). It has qt rows. Let us interpret these rows as words of a q-ary code of length n. Assume two of these codewords agree in t coordinates. Then we have the contradiction that, in the projection onto certaint coordinates, at-tuple occurs more than once. This shows that any two rows of the orthogonal array are at Hamming distance≥n−t+ 1. The rows of anOA1(t, n, q) therefore form an (n, qt, n−t+ 1)q code. Comparison with Theorem 4.1 shows that we have an MDS code. It is clear that the converse is true as well.
4.8 Theorem. The following are equivalent:
• AnOA1(t, n, q),
• An MDS code(n, qt, n−t+ 1)q.
Recall from Section 3.4 that an OA1(2, n, q) is equivalent to a set of n− 2 mutually orthogonal Latin squares of order q. Mutually orthogonal Latin squares have been studied for a long time and for a variety of reasons. A survey is in [106]. Theorem 4.8 shows that mutually orthogonal Latin squares are a source of optimal codes. For example, there is a pair of orthogonal Latin squares of order 10,so an MDS code (4,100,3)10 exists. The existence of 3 MOLS of order 10 is in doubt. This is a famous open problem. It is equivalent to the existence of a (5,100,4)10code.
Another famous special case is the nonexistence of a pair of orthogonal Latin squares of order 6, equivalently the nonexistence of a (4,36,3)6 code. This was conjectured by L. Euler in 1782. In terms of recreational mathematics it is known as the problem of the 36 officers. Nonexistence was shown by Tarry in 1901 [205]. Stinson’s short proof in [199] is coding theoretic.
Covering arrays and software testing
Assume we wish to test a certain software for accuracy. If there are 12 input parameters involved, each with 16 possible parameter values, the total number of possible test runs is 1612= 248.This clearly is too much to test, in particular if each single test should be expensive and time consuming. How should the set of test runs be chosen? Let us visualize this set as an array
with 12 columns and entries from a 16 set. Each row corresponds to a test run.
One frequently used approach is the following: choose the array of test runs such that, for every pair (or for every triple) of parameters, each pos- sible combination of parameter values does actually occur at least once (is covered). Generalizing from pairwise or triplewise covering tot-wise covering for arbitraryt,this leads to the following definition:
4.9 Definition. Acovering array of strengtht(parametersCA(N;t, n, q)) is an array with N rows and n columns, with entries from aq-set, such that in the projection onto any t columns each possible t-tuple of entries appears at least once (is covered).
In order to save time and money in software testing, we wish to construct covering arrays, most often of strength 2 or 3, with a minimum number of rows.
It is clear thatN ≥qt,with equality if and only if there is anOA1(t, n, q); in other words, covering arrays are generalizations of orthogonal arrays of index 1.In the case of the parameters considered above, we are lucky, asRS(2,16) and RS(3,16) yield (after throwing away four columns) OA1(2,12,16) and OA1(3,12,16),respectively.
Typically, we will not be that lucky. For example, in the binary caseq= 2, Reed-Solomon codes will not help us at all. Here is an (optimal)CA(10; 3,5,2) (see Chateauneuf and Kreher [47]):
0 0 0 0 0 1 1 1 1 1 1 0 0 1 1 0 1 1 0 0 1 1 0 0 1 0 0 1 1 0 1 1 0 1 0 0 0 1 0 1 1 0 1 0 0 0 1 0 1 1
Most diverse methods are being used for the construction of covering ar- rays, from combinatorial design theory and coding theory to heuristic search methods. Also, various names are employed, from qualitatively indepen- dent partitions(R´enyi [174]) tot-independent setsand
universal sets(see Goldreich [95]).
Remarks
Reed-Solomon codes were introduced in Reed and Solomon [173]. They are among the most widely used codes. Some applications are discussed in Wicker and Bhargava [219]. There is an attractive application in compact disc tech- nology, where shortened Reed-Solomon codes [32,28,5]256 and [28,24,5]256
play a key role. This is also described in Hoffman et al. [116].
1. Pick a prime-powerq and a dimensionk≤q.The Reed-Solomon codeRS(k, q) is defined as follows:
2. The coordinates are parametrized by the elementsu∈Fq. 3. The codewords are parametrized by the polynomials
p(X) with coefficients inFq,of degree< k.
4. The codeword parametrized byp(X) has as entries the evaluationsp(u).
5. RS(k, q) is a code [q, k, q−k+ 1]q and an orthogonal arrayOA1(k, q, q).
Exercises 4
4.1. Prove the following slight generalization of Lagrange interpolation:
Let a ≥ 0 be a natural number, x1, x2, . . . , xn different nonzero ∈ F and y1, . . . , yn arbitrary inF.Then there is exactly one polynomialp(X)of degree
< nwith coefficients inF which satisfies
xaip(xi) =yi, i= 1,2, . . . , n.
4.2. Leta≥0be a natural number andqa prime-power. Define aq-ary code of length q−1 with coordinates indexed by 0 6=u ∈ Fq and with codewords indexed by the polynomials p(X)of degree< k,wherek < q.The entry of the codeword indexed by p(X)in coordinate uisuap(u).
Determine the dimension, minimum distance and strength of this code.
4.3. Prove: The dual of a linear MDS code is an MDS code.
4.4. Let C be an [n, k, d]q-code. Determine the parameters of C⊥ as an or- thogonal array. Derive the Singleton bound (in the case of linear codes).
4.5. Give another simple proof of the Singleton bound for linear[n, k, d]q codes based on Gauß elimination (no calculations needed).
4.6. In order to understand why covering arrays are also known as qualita- tively independent partitions, consider theCA(10; 3,5,2)which we gave as an example. Interpret the rows as the ground set X (of10 elements).Then every column describes a partition ofX into two parts.
Express the defining property of a strength2covering array in terms of this family of partitions.
4.7. Prove the following basic fact: if f(X)is a polynomial with coefficients in the field F,andu∈F such thatf(u) = 0,thenf(X)can be written in the formf(X) = (X−u)g(X), whereg(X)is a polynomial.
4.8. The finitegeometric seriesis the identity (X−1)
n−1X
i=0
Xi=Xn−1.Prove this identity.
4.9. Let q be a complex number of absolute value|q|<1. Prove X∞
n=0
qn = 1 +q+q2+· · ·= 1/(1−q).This is the infinite geometric series.
4.10. Show the identity(X−u)
n−1X
i=0
Xiun−1−i=Xn−un. 4.11. Solve Exercise 4.7 using the geometric series.