Chapter IV: Causal joint source-channel coding with feedback
4.4 Joint source-channel coding reliability function
In the achievability proof, we fix a sequence of codes with instantaneous encoding for transmitting the firstk symbols of a DSS,k = 1,2, . . ., over a non-degenerate DMC with feedback, evaluate the asymptotic behavior of the code sequence as k → ∞, and conclude the achievability ofE(R)(4.38).
For any (fully accessible) DS, the JSCC reliability function (4.38) is achievable by the MaxEJS code [45, Sec. IV-C], and is achievable by the SED code [45, Sec.
V-B] if the channel is a symmetric binary-input DMC (AppendixC.7).
For any DSS withf =∞, including the DS (4.3), the buffer-then-transmit code for k source symbols that achievesE(R)(4.38) operates as follows. It waits until the k-th symbol arrives at timetk, and at times t ≥ tk+ 1, applies a JSCC reliability function-achieving code with block encoding forksymbolsSkof a (fully accessible) DS with priorPSk (4.1) (e.g., the MaxEJS code or the SED code [45]). The buffer- then-transmit code achieves (see details in AppendixC.8)
E(R)≥C1
1− H
C + 1 f
R
, (4.39)
which reduces toE(R)(4.38) forf = ∞. Indeed,f = ∞means that the arrival time tk is negligible compared to the blocklength. The buffer-then-transmit code fails to achieveE(R)(4.38) iff <∞.
For any DSS withf <∞that satisfies the assumptions(a)–(b)in Theorem9, the code with instantaneous encoding fork source symbols that achievesE(R)(4.38) implements the instantaneous encoding phase (Section4.3) at timest = 1,2, . . . , tk and operates as a JSCC reliability function-achieving code with block encoding for ksymbolsSkof a (fully accessible) DS with priorPSk|Ytk at timest ≥tk+ 1, where Y1, . . . , Ytk are the channel outputs generated in the instantaneous encoding phase.
For example, we can insert the instantaneous encoding phase before the MaxEJS code (or the SED code for symmetric binary-input DMCs). See AppendixC.9.
Assumption (a) holds with H = H for any information stable source since such sources satisfy 1nlog P 1
Sn(Sn)
−→i.p. H [102]. For example, H =H(S)if the source emits i.i.d. symbols. Assumption(b)in Theorem9implies
f ≥ C
H (4.40)
since H(Y|X) ≥ log2p1
max and H ≥ H. The symbol arriving rate constraint (4.40) ensures that all coding rates R < HC are achievable. Otherwise, if (4.40)
is not satisfied and the DSS has pS,max < 1, the rate region achievable by any code with instantaneous encoding is limited to R ≤ f. The limitation arises because decodingSkbefore the final arrival timetkresults in a non-vanishing error probability (Appendix C.12). For example, if the DSS emits i.i.d. symbols with entropy rateH = 1nat per symbol arriving at the encoder every1000channel uses, and the DMC has capacity C = 1nat per channel use, then the achievable rate is limited by 10001 symbols per channel use, which is far less than Shannon’s JSCC limit HC = 1symbol per channel use.
Since the (fully accessible) DS (4.3) is a special DSS, Theorem 9gives the JSCC reliability function (4.38) for a fully accessible source. It generalizes Burnashev’s reliability function [40] to the classical JSCC context, and generalizes Truong and Tan’s excess-distortion reliability function [87] at zero distortion to the DS with memory and to all ratesR < HC.
Remarkably, Theorem9establishes that the JSCC reliability function for a streaming source (satisfying assumptions(a)–(b)) is equal to that for a fully accessible source.
This is surprising as this means that revealing source symbols only causally to the encoder has no detrimental effect on the reliability function.
While the instantaneous encoding phase in Section 4.3 achieves E(R) (4.38), in fact, any coding strategy during the symbol arriving period that satisfies
k→∞lim
I(Sk;Ytk) tk
=C (4.41)
achievesE(R)(4.38) when followed by a JSCC reliability function-achieving code with block encoding. This is because (C.42b)–(C.42c) in the achievability proof in AppendixC.9always hold for such a coding strategy. For equiprobably distributed q-ary source symbols that arrive at the encoder one by one at consecutive timest= 1,2, . . . , kand a symmetricq-input DMC, uncoded transmission during the symbol arriving period t = 1,2, . . . , k satisfies (4.41) and thus constitutes an appropriate instantaneous encoding phase for that scenario. If q = 2, this corresponds to the systematic transmission phase in [47]. Furthermore, even if the instantaneous encoding phase in Section4.3drops the randomization (4.25)–(4.30) and transmits Zt(4.29) as the channel input, it continues to satisfy the sufficient condition (4.41) under a more conservative condition than (4.37) (see Remark2below).
Remark 2. Fix a non-degenerate DMC with the maximum and the minimum channel transition probabilities pmax and pmin, and fix a (q,{tn}∞n=1) DSS with maximum
symbol arriving probabilitypS,max<1and symbol arriving ratef <∞. If the DSS satisfies
(b′) the symbol arriving rate is large enough:
f > 1 logp 1
S,max
log 1
pmin −log 1 pmax
, (4.42)
then the instantaneous encoding phase in Section 4.3 that transmits the non- randomized Zt (4.29) as the channel input at each time t = 1,2, . . . , tk satisfies (4.41), which means that it achieves E(R)(4.38), the JSCC reliability function for streaming, when followed by a JSCC reliability function-achieving code with block encoding.
Proof sketch. We show that under assumption (b′), all source priors θi(yt−1), i ∈ [q]N(t), converge pointwise to zero intduring the symbol arriving periodt ∈[1, tk] ask → ∞. The convergent source priors and the partitioning rule (4.24) imply that the group priors converge pointwise to the capacity-achieving distributionPX∗. Since the encoder transmits a group index without randomization as the channel input, the channel input distribution converges to the capacity-achieving distribution, yielding (4.41). See AppendixC.13for details.
Note that the result of Remark2does not require assumption (a)sincePSn(sn)≤ (pS,max)n,∀sn∈[q]n, already implies that it holds withH ←logp 1
S,max. Since H ≥ logp 1
S,max and log p1
min ≥ H(PY∗), assumption (b′) is stricter than assumption (b). The increase of the threshold is because 1) the channel output distributionPY∗ in (C.52b) is replaced byPYt|Yt−1(·|·)≥pmin(C.64); 2) in the proof of Remark 2, we show that all the source priors convergepointwiseto zero (C.67) during the symbol arriving period ask → ∞using the upper boundPSn|Sn−1(·|·)≤ pS,max, whereas in Theorem9, we only need that the source prior of the true symbol sequence convergesin probabilityto zero (C.55).