Code Reduction Results

Chapter V: From Code Reduction to Capacity Reduction

5.3 Code Reduction Results

5.2 Dependent Sources

In the classical network coding setting, independent sources are considered.

We employ dependent sources [25] to understand the rates achievable when dependent sources are allowed.

For a blocklengthn, a vector ofnδ-dependent sources of rateR= (R₁, ..., R_|S|) is a random vector W= (W₁,· · · , W_k), where W_i ∈F^nR2 ^i+nδ such that

X

i∈[k]

H(W_i)−H(W)≤nδ

and H(Wi) ≥ nRi for all i∈ [k]. For δ = 0, the set of nδ-dependent sources only includes random variables that are independent and uniformly distributed over their supports.

We also consider linearly dependent sources [42, Section 2.2]. A vector of nδ-linearly-dependent sources are nδ-dependent sources that are pre-specified linear combinations of underlying independent processes [42]. We denote such an underlying independent process by U, and we assume that U is uniformly distributed overF^nR2 ^U. Thus, eachWi can be expressed as

W_i =U T_i,

where each T_i is an nR_U ×n(R_i+δ)matrix over F2.

We now define communication with nδ dependent sources. Network I is (R, , n, δ)-feasible if there exists a set of nδ-dependent source variables W and a network code C = {{f_e},{g_t}} with blocklength n such that opera- tion of codeC on source message random variables W yields error probability Pe⁽ⁿ⁾ ≤ . When the feasibility vector does not include the parameter for dependence (i.e., (R, , n)), we assume that the sources are independent.

Lemma 5.3.1. For any network I and rate vector R,

1. [43, Corollary 5.1] For any blocklength n and any , δ ≥ 0, if I is (R, , n, δ)-feasible, then by adding a cooperation facilitator, the network I_λ^cf is(R−ρ,0, n,0)-feasible, where λ andρ tend to zero as andδ tend to zero and n goes to infinity.

2. For any blocklength n and any , λ, δ ≥ 0, if I_λ^bf is (R, , n, δ)-feasible, then for dependent sources, I is (R−ρ,0, n, δ⁰)-feasible, where δ⁰ and ρ tend to zero as , λ, δ tend to zero.

3. For any blocklengthn and any, δ≥0, ifI is(R, , n, δ)-linearly-feasible for an nδ-linearly-dependent source, then by adding a cooperation facili- tator, the network I_λ^cf is (R−ρ, , n,0)-feasible, where λ and ρ tend to zero as δ tend to zero.

Proof. (2) Let I_λ^bf be (R, , n, δ)-feasible. By Lemma 5.3.1(1), I_λ+λ^bf ∗ is (R− ρ^∗,0, n,0)-feasible, whereλ^∗ and ρ^∗ tend to zero asand δ tend to zero and n tends to infinity. (Adding a λ^∗ cooperation facilitator to I_λ^bf is equivalent to increasing the bottleneck capacity of I_λ^bf from λ to λ+λ^∗.) Let λ⁰ = λ+λ^∗, and letC be an (R,0, n,0)-feasible network code for I_λ^bf0.

Define Zb to be the value being sent on the bottleneck edge (ssu, sre) of the broadcast facilitator in I_λ^bf0 under the operation of C. The conditional entropy H(W|Z_b) can be expanded as

H(W|Z_b) = X

γ⁰∈F^nλ2 ⁰

p(Z_b =γ⁰)H(W|Z_b =γ⁰).

By an averaging argument, there exists a γ such that H(W|Z_b = γ) ≥ H(W|Z_b). We therefore have the following inequalities:

H(W|Z_b =γ)≥H(W|Z_b)

≥H(W)−H(Z_b)

≥H(W)−nλ⁰. (5.1)

For each i∈[k],

H(W_i|Z_b =γ)≥H(W|Z_b =γ)−H((W_j, j ∈[k]\ {i})|Z_b =γ)

≥H(W)−nλ− X

j∈[k]\{i}

nR_j

!

(5.2)

=H(W_i)−nλ⁰.

Inequality (5.2) follows from bound on the support size of W_j and equa- tion (5.1).

X

i∈[k]

H(Wi|Zb =γ)−H(W|Zb =γ)≤ X

i∈[k]

(nRi)−H(W) +nλ⁰ (5.3)

≤nλ⁰

Inequality (5.3) follows from bound on support size of W.

Define the conditional random variable W^∗ = W|_Zb=γ. The alphabet size of W^∗ is the same as W, where for each i ∈ [k], |Wi| = 2^nRi. We then have an nλ⁰-dependent sourceW^∗ such that I is (R−λ⁰,0, n, λ⁰)-feasible.

(3) LetW= (W₁,· · · , W_k) denote thenδ-linearly dependent source (See Sec- tion 5.2) with U as the underlying random process. Therefore, the source is the result of a linear transformation of U, namely,

L_U :F^nR2 ^U → Y

i∈[k]

F^nR2 ⁱ. In particular, we can express each source W_i as

W_i =U L_i, where each L_i is an nR_U by nR_i matrix overF2.

For any matrix M, denote by M(i) the ith column of M and by CS(M) the column space of M. For a row vector W_i, denote by W_i,j the jth element of Wi. For vector spaces V and W, let V +W denote the direct sum of V and W, (i.e., {v+w|v ∈ V, w ∈ W}). Let Bi ⊆[nRi] be index sets such that for B ={(i, j) :i∈[k], j ∈B_i},

[

(i,j)∈B

{L_i(j)}

is the minimum spanning set of the vector spaceP

i∈[k]CS(F_i). Thus the rank of LU equals |B|.

LetB¯_i = [nR_i]\B_i, then for each(i, j)such thati∈[k], j ∈B¯_i, we can express U L_i(j) as linear combinations of the spanning set, i.e.,

U L_i(j) = X

i⁰∈[k]

X

j⁰∈B_i0

η_iji⁰_j⁰U L_i⁰(j⁰),

where each ηiji⁰j⁰ is a coefficient in F².

I

s

⁰

1

s

⁰_k

t

₁

t

_k

s

₁

s

_k

s

_su

s

_re

W

⁰

1

W

_k⁰

W

^∗

1

W

_k^∗

X

_α

Figure 5.1: A schematic for generating dependent variables using a cooperation facilitator.

Next, we describe a scheme to turn the dependent source code for I into a code for I_δ^cf that would operate on independent sources at the cost of a small loss in rate. Let W⁰ = (W₁⁰,· · ·, W_k⁰) be a set of independent sources for I_δ^cf where each W_i⁰ is uniformly distributes in F^|B2 ^i|. With the help of the cooperation facilitator, we first transformW⁰ into a set of dependent variables W^∗ = (W₁^∗,· · · , W_k^∗)using a full rank linear transformation

L^∗ : Y

i∈[k]

F^n|B2 ^i| → Y

i∈[k]

F^nR2 ⁱ

before applying the encoders of I (See Figure 5.1). Due to the topology of I_δ^cf, we need a distributed scheme to applyL^∗. We therefore need a two-step procedure to generate W^∗. At the first step, the cooperation facilitator first broadcast a common messageX_α =f_α(W),

f_α : Y

i∈[k]

F^n|B2 ^i|→F^nδ2 ,

to each of the old source nodes s₁,· · · , s_k in I_δ^cf. In the second step, each s_i apply a local linear transformation

L^∗_i :F^n|B2 ^i|×F^nδ2 →F^nR2 ⁱ, that maps (W_i⁰, X_α) toW_i^∗.

To ensure correct operation of the code, L^∗ is designed such that the distri- bution and the support set of W^∗ equals that of W. By assumption of the validity of the code, each terminaltiwill be able to reconstructW_i^∗with overall error probability less than. Finally, to allow for each terminal to reconstruct W_i⁰ fromW_i^∗, we also requireW_i⁰ to be a function of W_i^∗. Thus, an appropriate choice of L^∗ and λ will yield an (R−δ, , n,0)-linearly-feasible code for I_δ^cf. In what follows, we give the definitions off_α and{L_i}that satisfy the require- ments.

• LetB¯ ={(i, j) :i ∈[k], j ∈B¯_i}. Recall that W_i,j denotes the jth bit of W_i. The function f_α is defined such that X_α = f_α(W⁰) = (α_i,j,(i, j) ∈ B), where¯

α_i,j = X

i⁰∈[k]

X

j⁰∈B⁰_i

η_iji⁰_j⁰W_i⁰⁰_,j⁰.

The rate of the CF required to broadcast this function can be bounded as follow,

|B|¯ =X

i∈[k]

|B_i| −H(W)≤

X

i∈[k]

nR_i

−

X

i∈[k]

nR_i

+nδ =nδ.

• For each i ∈ [k], fix an arbitrary ordering of elements in B_i. define an index mapping function

β_i :B_i →[|B_i|]

that mapsj to iif j is the ith element inB_i. The linear transformation L^∗_i that maps W_i⁰ to W_i^∗ is defined as follows:

W_i,j^∗ =





 W_i,β⁰

i(j) j ∈B_i α_i,j j ∈B¯_i.

It can be verified that there exists a inverse map fromW_i^∗ toW_i⁰ for each i∈[k], and the resulting L^∗ is full rank.

Finally, we show that W^∗ and W has the same distribution and support set.

Observe that W is uniformly distributed over its support since it is a linear function ofU which is uniformly distributed overF^nR2 ^U. This is due to the fact that the pre-image of any element in the co-domain of L_U has the same size.

Similarly, since W⁰ is uniformly distributed, W^∗ is also uniformly distributed over its support. Letw= (w₁,· · ·, w_k)∈ W_sp, thenw⁰ = (w₁⁰,· · · , w_k⁰)defined as follows satisfies L^∗_U(w⁰) =w,

w_i,j⁰ =







w_i,j j ∈B_i

P

i⁰∈[k]

P

j⁰∈B_i⁰ηiji⁰j⁰wi,j j ∈B¯i.

Thus, the support set ofWis contained in that ofW^∗. Finally, since bothL_U and L^∗ has the same rank, the support set of W equals W^∗. I_δ^cf is therefore (R−δ, , n,0)-linearly-feasible.

5.4 Representative Topologies for The Edge Removal Statement