5.1.8. Let C be an[n, k, d]q code, x= (x1, . . . , xw,0,0, . . .)∈ C⊥ a codeword of weight w in the dual code. Apply shortening to C with respect to the first w coordinates. We know that the resulting code has parameters (at least) [n−w, k−w, d]q.
Show that this shortened code has in fact better parameters.
This is known asconstruction Y1.
5.1.9. From Section 3.4 we know the Simplex codes. Their duals are the Hamming codes. Determine the parameters of the codes obtained by applying constructionY1 to the Simplex codes.
5.1.10. Show that a code with minimum distanced > q cannot be MDS.
5.1.11. Prove that the Simplex codes meet the Griesmer bound with equality.
5.1.12. Prove the following generalization of the residual code construction, which is used in Dodunekov [72] and Groneick and Grosse [100]:
Let C be an [n, k, d]q code and v ∈ C a codeword of weight w. Assume d−w+⌈w/q⌉>0.
Then there is an [n−w, k−1, d−w+⌈w/q⌉]q code.
LetQ=qr.The fieldFQ containsFq as a subfield. Start from an arbitrary linear Q-ary code C with parameters [N, K, D]Q. Each codeword is an N- tuple (x1, x2, . . . , xN),wherexi∈FQ.NowFQ is a vector space of dimension r over Fq. Every r-dimensional code over Fq is an r-dimensional Fq-vector space as well. Here is the idea of concatenation: fix a codeDwith parameters [n, r, d]qand a one-to-one mapping, a vector space isomorphismα:FQ−→ D. In order to obtain the new q-ary concatenated code, we replace each field element u ∈ FQ with its image α(u) ∈ Frq. The images of codewords of C (Q-aryN-tuples) areq-arynN-tuples.
5.8 Definition. Let Q= qr and C be a linear Q-ary code [N, K, D]Q (the outer code). LetD be a linear [n, r, d]q (theinner code). Choose an Fq- isomorphism
α:FQ−→ D ⊆Fnq.
The words of theconcatenated codeare in bijection with the words ofC.If (x1, x2, . . . , xn)∈ C, then the corresponding word of the concatenated code is (α(x1), . . . , α(xN)).
The length of the concatenated code is nN. It has as many codewords as the outer code C.This number isQK =qrK.The dimension of the concate- nated code is therefore rK. Let α(x), x ∈ C be a nonzero codeword. AsC has minimum weight D, there are at least D coordinates such thatxi 6= 0.
For these coordinates, α(xi) is a nonzero word of the inner code D and has therefore weight≥d.It follows thatα(x) has weight≥dD.
5.9 Theorem. The concatenated code of an outer code [N, K, D]Q, where Q=qr,and an inner code[n, r, d]q,is a linearq-ary code[nN, rK, dD]q.
Use concatenation to derive binary codes from quaternary. The inner codes must then be two-dimensional binary codes. Use the Reed-Solomon code [4,2,3]4 as the outer code. As the inner code use the sum-0 code [3,2,2]2.Its codewords are 000,110,101,011.We can choose
α: 07→000, 17→110, ω7→101, ω27→011.
Consider the codeword ofRS(2,4) parametrized by the polynomial X.It is (0,1, ω, ω).Its image underαis 000110101011 of weight 6.The concatenated code is a [12,4,6]2code.
Assume we want to use our [8,4,4]2as the inner code. The outer code will then have to be a 16-ary code. The Reed-Solomon codes yield [16, k,17−k]16
for all k ≤ 16. Concatenation yields binary codes [128,4k,68−4k]2 for all k ≤ 16. We see that we can construct binary codes of arbitrary lengths by using concatenation with a Reed-Solomon code defined over a large extension field as the outer code and a suitable short inner code.
Let us see how to obtain a generator matrix of the concatenated code. LetG be a generator matrix of the inner code. Denote its rows byz1, z2, . . . , zr.We
need another fact from field theory (see Section 3.2): whenFqr is constructed from the base fieldFq,using an irreducible polynomialf(X) of degreerandǫ isX (modf(X)), then 1, ǫ, . . . , ǫr−1is a basis ofFqr as a vector space overFq. Recall thatǫr can be written as a linear combination of these smaller powers ofǫ.
We can chooseαsuch that
α: 1→z1, ǫ→z2, . . . , ǫr−1→zr.
Let nowM be a generator matrix of the outer code, with rowsR1, R2, . . . , RK. The rows of a generator matrix of the concatenated code are the images under αof
R1, ǫR1, . . . , ǫr−1R1, . . . , RK, ǫRK, . . . , ǫr−1RK.
Consider our first example. Our “large” field F4 has basis 1, ω.The rows of a generator matrix of the inner code [3,2,2]2 can be chosen asz1= 110 and z2= 101.According to our recipe, this gives us
α: 1→110, ω→101.
The rest is determined by linear combinations: α(0) = 000 of course and α : ω = 1 +ω →110 + 101 = 011. This is what we had before. We know how a generator matrix of the outer [4,2,3]4 can be determined. AsR1 = (1,1,1,1), R2= (0,1, ω, ω),this yields the following generator matrix for the concatenated [12,4,6]2:
R1→110110110110 ωR1→101101101101 R2→000110101011 ωR2→000101011110
The
(u, u+v)-construction
This is another simple and useful recursive construction. It seems to go back to Sloane and Whitehead [194] and can be generalized to the case of not necessarily linear codes. The basic idea is described by its name.
We start from two linear codesC1,C2,which are defined over the same field and have the same lengthn.Letube a typical element ofC1 andv a typical element of C2. The code we construct has length 2n. Its typical codeword is f(u, v) = (u, u+v).It is rather clear that the resulting code has as dimension the sum of the dimensions ofC1andC2.The point is to control the minimum distance.
5.10 Theorem. Let Ci, i= 1,2 be codes[n, ki, di]q, whered1≤d2. We can construct a code [2n, k1+k2, d]q, whered≥M in{d2,2d1}.
PROOF We construct the code C as the image of f : C1⊕ C2 −→ F2nq defined byf(u, v) = (u, u+v).Clearly,fis one-to-one (iff(u1, v1) =f(u2, v2);
then the first coordinate section shows u1 = u2, the second section shows u1+v1=u2+v2,and, as we know already thatu1=u2,this impliesv1=v2).
It follows thatdim(C) =k1+k2.
In order to bound the minimum weight, it suffices to distinguish two cases.
Ifv= 0, u6= 0,thenwt(u, u)≥2d1.Ifv6= 0,thenwt(u, u+v)≥wt(v)≥d2. This is the decisive observation. In fact, whenever vi 6= 0, it must be that either ui6= 0 or (u+v)i6= 0.
As an example, start from an [6,3,3]2and apply Theorem 5.10 recursively, with the repetition code in the role ofC2.This yields codes
[12,4,6]2, [24,5,12]2, [48,6,24]2,in general [6·2i,3 +i,3·2i]2.
Start from the trivial code [2,2,1]2. Using the same method as above, we obtain [4,3,2]2, [8,4,4]2, [16,5,8]2and in general
[2i, i+ 1,2i−1]2for alli≥1.This yields another construction for the extended binary Hamming code [8,4,4]2. All these codes meet the Griesmer bound Theorem 5.7 with equality.
5.11 Corollary. The existence of a linear [n, k, n/2]q code implies [2in, k+ i,2i−1n]q codes for alli≥1.
PROOF This is the special case of Theorem 5.10 whend1≥n/2 and the second code is the repetition code.
1. LetC be a [N, K, D]Q code, whereQ=qr. 2. LetDbe an [n, r, d]q code.
3. Concatenation produces an [nN, rK, dD]q code.
4. We can construct good codes of arbitrary length over small fields by using concatenation with Reed-Solomon codes over extension fields as outer codes.
5. Let codes [n, k1, d1]q and [n, k2, d2]q be given.
6. The (u, u+v)-construction produces a code [2n, k1+k2, d]q,where d≥M in{d2,2d1}.
7. There are families of binary codes [6·2i,3 +i,3·2i]2 and [2i, i+ 1,2i−1]2.
Exercises 5.2
5.2.1. UseRS(4,8) and[7,3,4]2 in a concatenation scheme.
What are the parameters of the resulting concatenated code?
5.2.2. We have already met the following matrix in Exercise 3.4.6. It gen- erates a code [6,3,4]4, which is known as the hexacode. We want to apply concatenation with the hexacode as the outer code. List at least two param- eters of binary codes that can be used as inner codes, and the parameters of the resulting concatenated codes.
1 0 0 1ω ω 0 1 0 1ω ω 0 0 1 1 1 1
5.2.3. Use concatenation with Reed-Solomon codes over F2r as outer codes and binary sum-0 codes as inner codes. Determine the parameters of the resulting binary codes.
5.2.4. Consider concatenation with Reed-Solomon codes as outer codes and a ternary[10,4,6]3 as the inner code. What are the resulting parameters?
5.2.5. Construct a48-dimensional binary linear code by applying concatena- tion to a Reed-Solomon code as the outer code and [23,12,7]2 as the inner code.
What are the parameters of this concatenated code?
5.2.6. Show how to construct a code[128,12,56]2by concatenation. What do you use as outer and inner codes?
5.2.7. Show how to construct a code[27,10,10]3by concatenation.
What do you use as outer and inner codes?
5.2.8. An[11,4,5]2 was promised in Section 2.3.
Construct it by concatenation.
5.2.9. The Reed-Muller codes RM(a, r) are binary codes of length 2r, minimum distance2r−a and dimension
Xa
i=0
r i
.
Use the (u, u+v)-construction to obtain codes with these parameters. Here RM(0, r)is the repetition code andRM(r, r)is the ambient space.
5.2.10. Give several parameters of good short binary linear codes among the family constructed in the third exercise above.
5.2.11. Give several examples of Reed-Muller codes with good parameters.
5.2.12. Construct[2i×14,4 +i,2i−1×14]2 codes for alli ≥1. This yields in particular [28,5,14]2,[56,6,28]2,[112,7,56]2.
5.2.13. Construct a [240,8,120]2 code.
Chapter 6
Universal hashing
Basic concepts: The concept of hashing. Almost universal hash classes and error-correcting codes. Error detection.
So far we have concentrated on one important area of applications for codes, transmission of messages over noisy channels. We mentioned early in this text that the applications of the concept of codes are manifold and certainly not limited to this historically first area. It is time to take a step in this direction.
An important concept in theoretical computer science ishash functions.
The idea is to find a functionf :X −→Y from a “large” setX to a “small”
setY with one of the following properties:
• Givenx∈X,it is computationally infeasible to find x′6=xsuch thatf(x′) =f(x).
• It is computationally infeasible to findx6=x′ such thatf(x′) =f(x).
Applications in cryptography are described in Stinson [200]. To give just one example, when a long document is to be signed electronically, one applies at first a hash function to produce a much shorter string (a digest of the original document) and then applies the signature scheme to this hash value.
The notion of a hash function is very problematic. For one thing, it is clear that numerous collisions (pairsx, x′ such thatf(x′) =f(x)) will happen. We want to make it computationally impossible for an opponent to find any. How can one guarantee that this is the case? The answer is: one cannot. In fact, whenever a new hash function is proposed, it takes just a couple of years to break it, that is, to describe a method which results in collisions.
Universal hashing was designed to find a way out of this dilemma, at least for certain applications. Just as for hash functions themselves, there are numerous variants. Let us fix notation:
93
6.1 Definition. An (n, M)q array is an array (a matrix) with n rows and M columns, where each entry is taken from an alphabetA of sizeq. We also interpret each row of the array as a mapping f :X −→ A, where |X|=M (the elements ofX parametrize the columns). In this perspective we speak of an (n, M)q hash family.
The idea behind universal hashing is to use a carefully chosen family of hash functions instead of a single hash function. Whenever a hash function has to be applied, it will be chosenat randomfrom the hash family. The expected behavior of this method is determined by structural properties of the hash family. Recall the notion of a probability space as introduced in Section 3.5.
6.2 Definition. A probability space Ωis uniform if each element ofΩ has the same probability 1/|Ω|.
As we intend to choose the rows of our arrays (the functions from our hash family) at random, it is natural to consider the rows as elements of a uniform sample space: each of the n rows (functions) has probability 1/n.The idea that collisions should be unlikely translates into the following requirement:
whenever two different columns (elementsx6=x′ of the ground set) are fixed and a rowf is chosen at random, the probability thatf(x′) =f(x) should be small. This leads to the following definition:
6.3 Definition. An(n, M)q hash family isǫ-universalfor some0< ǫ <1if, for any two distinct elements (columns)x′, x∈X,the number ν of functions (rows) f such that f(x′) =f(x)satisfies ν
n ≤ǫ; in other words:
the probability of collision is≤ǫ.
Here is a toy example:
01234 01234 01234 01234 01234 01234 12340 23401 34012 40123 01234 23401 40123 12340 34012 01234 34012 12340 40123 23401 01234 40123 34012 23401 12340
We see this as a (5,25)5 hash family, or as a family of five functions from a 25-set to a 5-set. It can be checked that each pair of columns contains at most one collision. For example, a collision of the first column and some other column is equivalent to an entry 0 in that other column. As each column except the first contains at most one entry 0,we have checked pairs involving the first column. Our array is 15-universal.
It is now easy to interpret this notion from the point of view of the Hamming distance. Pick two columns. A collision is a coordinate (row) where they agree. The Hamming distance is the number of coordinates where there isno collision. Let d be the Hamming distance between the two columns. The probability of collision is then n−dn = 1−d/n.This shows how to interpret the
notion of collision from our coding point of view: see the columns as words of a q-ary code of lengthnand minimum distanced.The array is thenǫ-universal, whereǫ= 1−d/n.We have established the following general equivalence:
6.4 Theorem. The following are equivalent:
• Anǫ-universal(n, M)q hash family, and
• An(n, M, d)q code, whereǫ= 1−d/n.
In fact, in our toy example we simply chose the columns as the codewords of the Reed-Solomon code RS(2,5).Asd= 4,we haveǫ= 1−4/5 = 1/5.
The notion of anǫ-universal hash family really is nothing new for us, but it gives a different interpretation of codes. In this application very small alphabets are not interesting at all. Also, as the ground setX has to be very large, the length n cannot be small. The probabilityǫof collision has to be small. This means thatd/nhas to be close to 1.It follows that the dimension k of our code (in the linear case) must be relatively small. As M = qk we see again that q cannot be all that small. Fortunately, we know already a source of good long codes over large alphabets: the Reed-Solomon codes. If they should not suffice, we can use concatenation with Reed-Solomon codes as outer codes.
Assume we want to hash documents of 220 bits (about 1 million bits). As we want to be able to hash any document of this length, we have a ground set of size 2220.The hashed value should have a length of only 32 bits. This means that the alphabet has sizeq= 232.Can we use a q-ary Reed-Solomon codeRS(k, q)? We needqk≥2220, ork≥215.The probability of collision is thenǫ <215/232= 2−17.This describes a (232,2220)232 hash family, which is 2−17−universal. When using Reed-Solomon codes, we will always haven=q.
The idea of universal hashing in the computer science literature seems to go back to two influential papers [43, 218] by J. L. Carter and M. N. Wegman.
Section 16.7 has more on universal hashing, its applications and links to codes.
An application to error detection
Here is an application ofǫ−universal hash families for the purposes of error detection. Assume we send the long message xto the receiver via a rather good channel. Let y be the received message. In most cases, no errors will occur, hencey=x.In order to be able to detect errors (not to correct them), the following method can be used: we choose a functionf at random from our ǫ−universal family and sendf as well asf(x) to the receiver, using a public channel which will certainly not commit errors. This will not cost all too much, as these messages are much shorter thanx(in the example abovexhas 220bits, whereasf andf(x) are bitstrings of length 32 each). Iff(y)6=f(x), then the receiver knows that a transmission error has happened. He will ask for retransmission in this case. Iff(y) =f(x),the message will be accepted as correct. The probability that this assumption is in error is then limited by ǫ.
Exercises 6
6.1. Determine the parameters of Reed-Solomon codes when interpreted asǫ−universal hash families.
6.2. Describe an (8,16)2 hash family, which is 12-universal.
6.3. Find a 25-universal(10,81)3 hash family. Hint: this code has been con- structed in Chapter 3. It is closely related to the ternary Golay code.
6.4. Does a 12-universal (6,9)2 hash family exist?
6.5. Construct a (256,25617)256 hash family, which is 161-universal.
6.6. Explain why a code [4000,5,3996]4096 exists and determine the parameters of the universal hash family derived from it.
Chapter 7
Designs and the binary Golay code
Basic concepts: Cyclic codes. The binary Golay code.
Combinatorialt-designs, the large Witt design, projective planes.
A fruitful source of good codes is the theory of cyclic codes. A code is called cyclic if the cyclic shift of any codeword is a codeword again. As an example, consider the binary 23-tuple
z1= (1,1,0,0,0,1,1,1,0,1,0,1,0,0,0,0,0,0,0,0,0,0,0)
In order to obtain its first cyclic shift, we write first the final entry 0,followed by the first 22 entries:
z2= (0,1,1,0,0,0,1,1,1,0,1,0,1,0,0,0,0,0,0,0,0,0,0).
The next cyclic shift is
z3= (0,0,1,1,0,0,0,1,1,1,0,1,0,1,0,0,0,0,0,0,0,0,0,0)
and so on. Denote by C the cyclic binary linear code generated by z1. It is clear that the first 12 cyclic shifts z1, z2, . . . , z12 are linearly independent (why?), butz13 is a linear combination ofz1, z2, . . . , z12.It follows thatC has dimensionk= 12 and that a matrix withz1, z2, . . . , z12as rows is a generator matrix ofC.
7.1 Definition. Thebinary Golay codeG23is the binary linear cyclic code generated by z1 above and its cyclic shifts.
Here is the generator matrixG:
97
1 1 0 0 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 0 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 1 0 1 0 1
The cyclic structure is clearly visible. The addition of a parity check bit to each row ofGyields a generator matrix of the so-called extended binary Golay code [24,12,8]2.It is easily checked that this codeG24is self-dual: G24=G24⊥. In fact, because of the cyclicity, it suffices to check that the first row of the generator matrix is orthogonal to all rows. All rows of the generator matrix ofG24have weight 8.This together with self-duality suffices to prove that all words ofG24 have weights divisible by 4 (see Exercise 7.2). We wish to show that weight 4 does not occur.
The reader is urged to find a generator matrix of the formM = (I|P) ofG24. HereIis the (12,12) unit matrix andPis a (12,12) matrix (see Exercise 7.3).
This is not hard, as the left half ofGis already in triangular form. It turns out that all rows ofM have weight 8.By Lemma 2.14 a check matrix is (Pt|I).
AsG24 is self-dual, this is also a generator matrix.
Assume now v ∈ G24, wt(v) = 4. Writev = (vL, vR),where vL is the left half ofvandvR is the right half. Our generator matrices show that only the 0-word hasvL= 0 orvR= 0.Assumewt(vL) = 1.ThenvLis a row ofM.This is impossible, as rows ofM have weight 8.Ifwt(vR) = 1, vLis a column ofP.
These have weight 7. This is a contradiction. The only remaining possibility is wt(vL) = wt(vR) = 2.It follows thatv is the sum of two rows of M.It is easy to check that none of these 122
= 66 vectors has weight 4.
7.2 Theorem. The binary Golay codeG23is a[23,12,7]2code. It is a perfect code. The extended binary Golay code is a self-dual[24,12,8]2code.
The presence of the all-1 word 1 (see Exercise 7.1) shows that the only possible weights of codewords of G24 are 0,8, 12,16,24 and that there are as many words of weight 16 as there are words of weight 8.Define Ai to be the number of codewords of weighti in G24. You are challenged to find the weight distribution ofG24 (see Exercise 7.4). In particular, it turns out that A8= 759.
This can serve as a motivation for a basic structure of modern discrete mathematics,combinatorial designs.
7.3 Definition. Let V be a set of v objects and B a family of subsets of V, where each such subset (ablock) haskelements. We say thatBis at-design with replication numberλif the following holds:
Each set T ⊂V of t elements is in precisely λ blocks. The parameters of such a t-design are written t−(v, k, λ). In the case of λ= 1one speaks of a Steiner systemand writesS(t, k, v)instead oft−(v, k,1).
We can derive a 5-design from our code G24, as follows: As underlying set, use the 24 coordinates. Identify each word of weight i with a subset of cardinalityi: those coordinates where the entry is 1. Take as blocks the 759 subsets (calledoctads) corresponding to the words of weight 8.This defines a Steiner systemS(5,8,24) (see Exercise 7.5).
7.4 Theorem. The 759octads of the extended binary Golay code[24,12,8]2
are the blocks of a designS(5,8,24).It is known as thelarge Witt design.
We have seen that the cyclic code G23 and its lengthening G24 are highly interesting codes. In fact cyclic codes are one of the most widely used families of codes. The traditional theory of cyclic codes uses notation and results from ring theory (rings are an important type of algebraic structure). A different approach uses deeper knowledge of finite fields. The theory of cyclic codes is a core topic of Part II, in particular Chapters 12 and 13.
A particularly important family of designs is theprojective geometries.
In Part II we are going to make heavy use of these structures. The geometric description of linear codes, another core topic of Part II, is based on projective geometries (see in particular Chapter 17).
We start by describing the projective planesP G(2, q): start from an arbi- trary finite field Fq. Consider the space V =F3q, a three-dimensional space.
Define
• points: the one-dimensional subspaces ofV,
• lines: the two-dimensional subspaces ofV.
AsV hasq3−1 nonzero vector, and each one-dimensional space hasq−1 nonzero vectors, the total number of points is (q3−1)/(q−1) =q2+q+ 1.
The same counting method shows that each line has (q2−1)/(q−1) =q+ 1 points.
The most important property of this geometry is the following: any two different points are on precisely one common line. This fact is clear, as any two one-dimensional spaces generate a two-dimensional space. We conclude that P G(2, q), the classical projective plane of order q, is a designS(2, q+ 1, q2+q+ 1).
Design theory is an important branch of modern combinatorial theory.
The basic axiom is a uniformity property reminiscent of the definition of orthogonal arrays. In fact, one of the origins of combinatorialdesign theory is in statistics and in particular in thedesign of experiments.