Streaming with random arrivals

Chapter IV: Causal joint source-channel coding with feedback

4.6 Streaming with random arrivals

Second, we observe that during the symbol arriving periodt = 1,2. . . , t_k, the in- stantaneous SED code corresponds to dropping the randomization step of the instan- taneous encoding phase in Section4.3. This is because for a symmetric binary-input DMC, (4.43) impliesπ_x^′(y^t−1)≤P_X^∗(x^′) = ¹₂, thus any partition{G_x(y^t−1)}x∈{0,1}

that satisfies the instantaneous SED rule (4.43)–(4.44) also satisfies the partitioning rule in (4.24). Therefore, Remark 2 implies that the instantaneous SED code at timest= 1,2, . . . , t_ksatisfies the sufficient condition (4.41) under assumption(b^′). As we have discussed in the proof sketch of Theorem9, a JSCC reliability function- achieving code with instantaneous encoding can be obtained by preceding a JSCC reliability function-achieving code with block encoding by an instantaneous en- coding phase that satisfies (4.41). The two observations above imply that the instantaneous SED code achievesE(R)(4.38) in the setting of Theorem10.

4.6 Streaming with random arrivals

A new source symbol arrives at timet+ 1according to the probability distribution

P_SN(t+1)|S^N(t). (4.53)

Since at most one symbol can arrive at any time, the conditional probability distribution P_SN(t+1)|S^N(t)(·|s) can only place non-zero masses at S^N(t+1) = s, S^N(t+1) = s⊞s^′ for s^′ ∈ [q]. We assume that both the encoder and the decoder know the symbol arriving probability distribution (4.53), the decoder does not know the exact realizations of the symbol arriving times, and the first symbol arrives at τ1 ≜1.

We define a code with instantaneous encoding that we use to transmit a DSS with random arrivals over a non-degenerate DMC with feedback.

Definition 22(A (k, R, ϵ) code with instantaneous encoding for random arrivals). Fix aq-ary DSS with random arrivals and fix a non-degenerate DMC (4.8)with a single-letter channel transition probability P_Y|X: X → Y. A (k, R, ϵ)code with instantaneous encoding for random arrivals consists of:

1. A sequence of encoding functions f_t: Q_t× Y^t−1 → X, t = 1,2, . . . that the encoder uses to form the channel inputsX_t(4.12).

2. A sequence of decoding functionsg_t,t = 1,2, . . . defined in Definition20-2.

3. A stopping time η_k defined in Definition 20-3, which satisfies both the rate constraintR(4.14)and the error constraintϵ(4.15).

For any R > 0, the minimum error probability achievable by rate-R codes with instantaneous encoding for random arrivals and message lengthkis given by

˜

ϵ^∗(k, R)≜min{ϵ: ∃(k, R, ϵ)code with instantaneous encoding for random arrivals}.

For transmitting a DSS with random arrivals over a non-degenerate DMC with noiseless feedback via a code with instantaneous encoding for random arrivals, we define the JSCC reliability function forrandomstreaming as

E(R)˜ ≜ lim

k→∞

R

k log 1

˜

ϵ^∗(k, R). (4.54)

If the symbol arriving times are deterministic, a (k, R, ϵ)code with instantaneous encoding for random arrivals reduces to a(k, R, ϵ)code with instantaneous encoding in Definition 20, and the JSCC reliability function for random streaming E(R)˜ reduces to the JSCC reliability function for streamingE(R)(4.17).

Instantaneous SED code for random arrivals

We generalize the instantaneous SED code in Section 4.5 to a DSS with random arrivals. The key is to allow the encoder and the decoder to track the priors and the posteriors of all possible sequences that could have arrived by timet. We fix aq-ary DSS with random arrivals.

To generalize the anytime instantaneous SED code in Section 4.5 to a DSS with random arrivals, we replace alphabet [q]^N(t) in Section 4.5 by alphabet Q_t that contains all possible source sequences that could have arrived at the encoder by time t. As a consequence, at timest = 1,2, . . ., for all sequencesi ∈ Q_t, the priors are updated as

θi(y^t−1) = X

j∈Qt−1

P_S^N(t)_|S^N(t−1)(i|j)ρj(y^t−1); (4.55)

the encoder and the decoder partitionQ_tinto groups{G_x(y^t−1)}x∈{0,1}that satisfy the instantaneous SED rule (4.43)–(4.44); the posteriors of all sequences inQ_tare updated as (4.45).

To restrict the anytime instantaneous SED code for random arrivals described above to transmit only the first k symbols, we replace the alphabet Q_t that contains all possible sequences that could have arrived by timet by the alphabetQ_min{t,k} that stops evolving at timest > k; we equip the code with the stopping timeη_kin (4.49) and the MAP decoder in (4.46).

Joint source-channel coding reliability function for random streaming

We derive the JSCC reliability function for random streaming E(R)˜ (4.54) using the instantaneous SED code for random arrivals. Similar to (4.35), we denote

˜

p_S,max≜ max

t∈N,s∈Q_tP_SN(t)|S^N(t−1)(s|s) +P_SN(t)|S^N(t−1)(s|s⊟), (4.56) wheres⊟is the sequence after truncating the last (newest) symbol of sequences. Theorem 11. Fix a non-degenerate symmetric binary-input DMC with channel capacityC(4.10), maximum KL divergenceC₁(4.11), maximum channel transition probability p_max (4.33), and minimum channel transition probability p_min (4.34). Fix a DSS with random arrivals that emits symbols S1, S2, . . . at a sequence of random symbol arriving times τ1 < τ2 < . . . with entropy rateH > 0 (4.2) and

˜

p_S,max<1. If the DSS with random arrivals satisfies

(c) the symbol arriving timesτ₁, τ₂, . . . are bounded:∃functionh(·) :Z⁺ →Z⁺, such thath(n) = o(n)and

τ_n≤n+h(n), n= 1,2, . . .; (4.57) (d) the entropy rate of the symbol arriving times is zero, i.e.,

n→∞lim

H(τⁿ)

n = 0; (4.58)

(e) assumption(b^′)in Remark2is satisfied withf ←1,p_S,max←p˜_S,max; then, the JSCC reliability function for random streaming is equal to

E(R) =˜ C₁

1− H CR

, 0< R < C

H. (4.59)

Proof sketch. The converse proof is in Appendix C.16: we show that converse bounds on the JSCC reliability function for a fully accessible source apply toE(R)˜ . The achievability proof is in AppendixC.17: we show that the detrimental effect on the reliability function due to the randomness in the symbol arriving times vanishes as the source lengthk → ∞.

Assumptions(c)–(d) posit that the symbol arriving times of the DSS have limited randomness. An example of such a source emits symbols as follows: among the first n source symbols, n = 1,2, . . ., there are h^′(n) ≤ h(n) symbols that can randomly select their symbol arriving times within h(n) time options, and the remaining symbols arrive at deterministic times so that the n-th symbol arriving time is bounded as (4.57). The symbol arriving times in the example satisfies (4.58), see Appendix C.20. Such a source could appear in a time-slotted communication scenario: a source emits packets with most packets arriving at the encoder at deterministic times and a few packets arriving with random and bounded delays due to system deficiencies.

Theorem11 establishes that the JSCC reliability function for a DSS with random symbol arriving times satisfying assumptions (c)–(e) is equal to that for a DSS with deterministic symbol arriving times, i.e.,E(R) =˜ E(R). This means that even though the decoder does not know the exact symbol arriving times, the instantaneous SED code for random arrivals achieves E(R)˜ (4.38) as if the decoder knew the symbol arriving times.

While the instantaneous SED code for random arrivals achievesE(R)˜ , in fact, any coding strategy at timest= 1,2, . . . , k+h(k)that satisfies (c.f. (4.41))

k→∞lim

I S^k;Y^k+h(k)

k+h(k) =C (4.60)

achieves E(R)˜ when followed by the SED code. This is because plugging (4.60) into (C.96) givesE(R)˜ (4.59).

Relaxing assumption (c) and dropping assumptions (d)–(e) in Theorem 11, we obtain an achievability bound onE(R)˜ .

Proposition 4. Fix a non-degenerate symmetric binary-input DMC with channel capacity C (4.10) and maximum KL divergence C₁ (4.11), and fix a DSS with random arrivals that emits symbols S₁, S₂, . . . at a sequence of random symbol arriving timesτ₁ < τ₂ < . . . with entropy rateH >0(4.2). If the symbol arriving times satisfies assumption(c)with the right side of (4.57)relaxed toE[τ_n] +h(n), then, the JSCC reliability function for random streaming is lower bounded as

E˜(R)≥C₁ 1−lim sup

k→∞

H S^k|Y^E[τk]+h(k)

kC + E[τ_k] k

! R

!

, (4.61)

whereY1, Y2, . . . are the channel outputs in response to the channel inputs generated by the encoder of the instantaneous SED code for random arrivals.

Proof. AppendixC.21.

In the setting of Proposition4, a buffer-then-transmit code that idles the transmissions at timest= 1, . . . , τ_kand operates as a code with block encoding at timet≥τ_k+ 1 only achieves (c.f. (4.39))

E(R)˜ ≥C1

1−

H

C + lim sup

k→∞

E[τ_k] k

R

. (4.62)

The achievability bound in (4.61) is larger than or equal to the achievability bound in (4.62) sinceS^kandY^E[τk]+h(k)are not independent. This means that in terms of achievable error exponent, the instantaneous SED code for random arrivals performs no worse than the best buffer-then-transmit code.