PROOF OF PROPOSITION 1

Suppose that, given 𝑆_𝑡−1, our algorithm has generated the set 𝒜 of disjoint augmentations. Let 𝒜^∗ denote a near optimal set of augmentations satisfying 𝑟_𝒘(𝑆_𝑡−1⊕ 𝒜^∗) ≥^𝑘−1

𝑘+1𝑟_𝒘^∗, which consists of d disjoint augmentations {𝐴₁, 𝐴₂, . . , 𝐴_𝑑 }. For each 𝐴_𝑖, let 𝑛_𝑖 denote an (arbitrary) endpoint of 𝐴_𝑖 if 𝐴_𝑖 is a path, or an arbitrary node in 𝐴_𝑖 if 𝐴_𝑖 is a cycle. Consider an event 𝓔 under our algorithm such that all the following are true:

⚫ the set of self-selected seeds equals {𝑛₁, 𝑛₂. . . , 𝑛_𝑑 },

⚫ all seed nodes 𝑛_𝑖 select its intended size as 𝑠𝑖𝑧𝑒(𝐴_𝑖), and

⚫ the generated set of augmentation 𝒜 equals 𝒜^∗.

Since it is clear that Pr{𝒜 = 𝒜^∗ | 𝑆𝑡−1} ≥ Pr{𝓔 | 𝑆_𝑡−1}, we focus on a lower bound of Pr{𝓔 | 𝑆𝑡−1}.

Note that i) the seed selection of {𝑛_𝑖} occurs with probability 𝑝^𝑑 (1 − 𝑝)^𝑉−𝑑 , where 𝑉 = |𝒱|, and ii) the probability that all seeds 𝑛_𝑖 independently select its intended size of 𝑠𝑖𝑧𝑒(𝐴_𝑖) is 𝑘^−𝑑 . iii) Finally, we compute the probability that the generated set of augmentation 𝒜 equals 𝒜^∗, which

occurs when, at each iteration, each active node selects its corresponding neighbor that belongs to 𝒜^∗. Each selection has probability 𝛴⁻¹ , where Σ is the maximum node degree, and total 𝑉 selections will be made under the algorithm. Thus, combining the three probabilities, we have that

Pr{𝓔 |𝑆_𝑡−1} ≥ 𝑝^𝑑 (1 − 𝑝)^𝑉−𝑑 ⋅ 𝑘^−𝑑⋅ 𝛴^−V≥ min {1, ( 𝑝 1 − 𝑝)

𝑉

} (1 − 𝑝 𝑘Σ )

𝑉

. (11)

setting 𝛿 = min {1, ( ^𝑝

1−𝑝)^𝑉} (^1−𝑝

𝑘Σ)^𝑉 completes the proof. □

C. PROOF OF LEMMA 2

At time 𝑡 > 0, define 𝑙_𝑡= ⌈^4|ℒ|2(|ℒ|+1) ln 𝑡

Δ_min^𝛼 ⌉. Suppose that a non-near-optimal matching 𝑆 ∈ 𝒮̅_𝒘^𝛼 such that 𝜏̂_𝑆(𝑡) ≥ 𝑙_𝑡 is scheduled. It also implies 𝜏̂_𝑖 (𝑡) ≥ 𝑙_𝑡 for all links 𝑖 ∈ 𝑆. Also, consider an arbitrary near-optimal matching 𝑆^′∈ 𝒮_𝒘^𝛼, and let 𝑆 = {𝑒₁, 𝑒₂, . . . , 𝑒_𝐴} and 𝑆^′= {𝑒₁ , 𝑒₂ , . . . , 𝑒_𝐵 }, where 𝐴 = |𝑆| and 𝐵 = |𝑆^′|, respectively. We consider the event that the total index sum over links in 𝑆 is greater than that over links in 𝑆^′.

𝕀{𝑟_𝒘̅𝑡(𝑆) ≥ 𝑟_𝒘̅𝑡(𝑆^′)} = 𝕀 {∑ 𝑤̅_{𝑒𝑖,𝑡}

𝐴

𝑖=1 ≥ ∑ 𝑤̅_𝑒

𝑗′,𝑡 𝐵

𝑗=1 }

≤ 𝕀 { max

𝑙_𝑡≤𝑐₁,…,𝑐_𝐴<𝑡∑ (𝑤̂_{𝑒𝑖,𝑐𝑖}+ √(|ℒ| + 1) ln 𝑡 𝑐𝑖

)

𝐴 𝑖=1

≥ max

0<𝑐₁^′,…,𝑐_𝐵^′<𝑡∑ (𝑤̂_𝑒

𝑗′,𝑐_𝑗^′+ √(|ℒ| + 1) ln 𝑡 𝑐_𝑗^′ )

𝐵

𝑗=1 }

(12)

which is bounded by ∑ ⋯ ∑ ∑^𝑡_𝑐 ⋯

1′=1 ∑^𝑡_𝑐 𝕀{ℤ}

𝐵′=1 𝑡𝑐_𝐴=1

𝑡𝑐₁=1 , where event ℤ is defined as

∑ (𝑤̂_{𝑒𝑖,𝑐𝑖}+ √(|ℒ| + 1) ln 𝑡

𝑐_𝑖 )

𝐴

𝑖=1 ≥ ∑ (𝑤̂_𝑒

𝑗′,𝑐_𝑗^′+ √(|ℒ| + 1) ln 𝑡 𝑐_𝑗^′ )

𝐵

𝑗=1 .

For the ease of explanation, we set 𝑤̂_{𝑒𝑖,𝑐𝑖} to denote the sample mean of link 𝑒_𝑖 scheduled 𝑐_𝑖 times.

We further decompose ℤ into {𝔸_𝑖}, {𝔹_𝑗} and ℂ as

𝔸_𝑖: 𝑤̂_{𝑒𝑖,𝑐𝑖}− √(|ℒ| + 1) ln 𝑡

𝑐_𝑖 ≥ 𝑤_𝑒𝑖 (13)

𝔹_𝑗: 𝑤̂_𝑒

𝑗′𝑐_𝑗^′+ √(|ℒ| + 1) ln 𝑡 𝑐_𝑗^′ ≤ 𝑤_𝑒

𝑗′

ℂ: ∑ 𝑤_𝑒

𝑗′

𝐵

𝑗=1 − ∑ 𝑤_𝑒𝑖

𝐴

𝑖=1 < 2 ∑ (√(|ℒ| + 1) ln 𝑡

𝑐_𝑖 )

𝐴 𝑖=1

Note that if 𝕀{ℤ } = 1, we should have ∑^𝐴_𝑖=1𝕀{𝔸_𝑖}+ ∑^𝐵_𝑗=1𝕀{𝔹_𝑗}+ 𝕀{ℂ} ≥ 1, which can be easily shown by contradiction. The probability of each event of 𝔸_𝑖 and 𝔹_𝑗 can be bounded by the Chernoff-Hoeffding bound as

Pr {𝑤̂_{𝑒𝑖,𝑐𝑖}− √(|ℒ| + 1) ln 𝑡

𝑐_𝑖 ≥ 𝑤_𝑒𝑖 } ≤ 𝑡^{−2(|ℒ|+1)},

Pr { 𝑤̂_𝑒

𝑗′𝑐_𝑗^′+ √(|ℒ| + 1) ln 𝑡 𝑐_𝑗^′ ≤ 𝑤_𝑒

𝑗′ } ≤ 𝑡^{−2(|ℒ|+1)},

(14)

Further, we can show Pr{ℂ} = 0 if 𝑐_𝑖 ≥ ⌈^4|ℒ|2(|ℒ|+1) ln 𝑡

Δ_min^𝛼 ⌉ for all links 𝑒_𝑖 ∈ 𝑆 as follows. Suppose that ℂ occurs. Then we should 0 > ∑ 𝑤_𝑒

𝑗′

𝐵𝑗=1 − ∑^𝐴_𝑖=1𝑤_𝑒𝑖− 2 ∑ (√(|ℒ|+1) ln 𝑡 𝑐_𝑖 )

𝐴𝑖=1 . The right term is no smaller than 𝛼 · 𝑟_𝒘^∗ − max

𝑆∈𝒮̅_𝒘^𝛼𝑟_𝒘(𝑆) − Δ^𝛼_min , which equals 0 by definition of Δ_min^𝛼 , resulting in a contradiction.

Using these and taking expectation on (12), we have

Pr{𝑟_𝒘̅𝑡(𝑆) ≥ 𝑟_𝒘̅𝑡(𝑆^′)} ≤ ∑ ⋯ ∑ ∑ ⋯

𝑡

𝑐₁^′=1 ∑ Pr{ℤ}

𝑡 𝑐_𝐵^′=1 𝑡

𝑐_𝐴=1 𝑡

𝑐₁=1

≤ ∑ ⋯ ∑ ∑ ⋯

𝑡

𝑐₁^′=1 ∑ (∑ Pr{𝔸_𝑖}

𝐴

𝑖=1 + ∑ Pr{𝔹_𝑗}

𝐵

𝑗=1 )

𝑡 𝑐_𝐵^′=1 𝑡

𝑐_𝐴=1 𝑡

𝑐₁=1

≤ 𝑡^𝐴+𝐵⋅ 2|ℒ|𝑡^{−2(|ℒ|+1)}≤ 2|ℒ|𝑡⁻².

□

D. PROOF OF PROPOSITION 2

Overall, we show that the number of explorations to non-near optimal matchings is bounded. To this end, we consider a sequence of time points where a non-near-optimal matching is sufficiently played at each point. They serve as a foothold to count the total number of plays of non-near- optimal matchings.

To begin with, for the horizon time T > 0, let 𝑙𝑇 = ⌈^4|ℒ|2(|ℒ|+1) ln 𝑡

Δ_min^𝛼 ⌉, and let 𝑇′ denote the first time when all non-near-optimal matchings are sufficiently (i.e., more than 𝑙_𝑇 times) explored, i.e.,

𝑇^′= min{𝑡 | 𝜏̂_𝑆,𝑡 ≥ 𝑙_𝑇 for all 𝑆 ∈ 𝒮̅_𝒘^𝛼}.

(𝟏) When 𝑇^′≤ 𝑇 : Let 𝒮̅_𝒘^𝛼 = {𝑆¹, 𝑆², … , 𝑆^𝑀} with 𝑀 = |𝒮̅_𝒘^𝛼|. Further we define 𝒮̅(𝑡) = {𝑆 ∈ 𝒮̅_𝒘^𝛼 | 𝜏̂_𝑆,𝑡 ≥ 𝑙_𝑇}, which is the set of non-near-optimal matchings that are scheduled sufficiently many times by time 𝑡 , and 𝒮(𝑡) = 𝒮̅_𝒘^𝛼− 𝒮̅(𝑡) denotes the set of not-yet-sufficiently-scheduled non-near- optimal matchings. Also, let 𝑇^𝑛 denote the time when matching 𝑆^𝑛 is sufficiently scheduled, i.e., 𝜏̂_𝑆^𝑛(𝑇^𝑛) = 𝑙_𝑇. Without loss of generality, we assume 𝑇¹ < 𝑇²< ⋯ < 𝑇^𝑀= 𝑇^′.

To apply the decomposition inequality (9), we need to estimate the expected value of ∑_{𝑆∈𝒮̅𝒘}^𝛼𝜏̂_𝑆,𝑇^′, which can be written as

∑ 𝜏̂_𝑆,𝑇^′

𝑆∈𝒮̅_𝒘^𝛼 = ∑ ∑ 𝕀{𝑆_𝑡 = 𝑆}

𝑇^′ 𝑡=1 𝑆∈𝒮̅_𝒘^𝛼

= 𝑙_𝑇𝑀 + ∑ ∑ ∑ 𝕀{𝑆_𝑡= 𝑆}

𝑆∈𝒮̅(𝑇^𝑛) 𝑇^𝑛+1

𝑡=𝑇^𝑛 𝑀−1

𝑛=1

(15)

Hence, we need to estimate ∑_{𝑆∈𝒮̅𝒘}^𝛼Pr{𝑆_𝑡= 𝑆} for 𝑡 ∈ (𝑇^𝑛, 𝑇^𝑛+1], which can be obtained as in the following lemma.

LEMMA 5. For each 𝑡 ∈ (𝑇^𝑛, 𝑇^𝑛+1], we have ∑ Pr{𝑆_𝑡 = 𝑆}

𝑆∈𝒮̅_𝒘^𝛼 ≤ (1 − 𝛿) ⋅ ∑ Pr{𝑆_𝑡−1= 𝑆}

𝑆∈𝒮̅_𝒘^𝛼

(16) + Pr{𝑆_𝑡−1∈ 𝒮(𝑇^𝑛)} + (|𝒮̅(𝑇^𝑛)| + 𝛿) ⋅ 2|ℒ|𝑡⁻²

Proof. We first divide the case into three exclusive sub-cases based on the previous schedule 𝑆_𝑡−1: events 𝔸 = {𝑆_𝑡−1∈ 𝒮_𝒘^𝛼}, 𝔹 = {𝑆_𝑡−1∈ 𝒮(𝑇^𝑛)} and ℂ = {𝑆_𝑡−1 ∈ 𝒮̅(𝑇^𝑛)}, and. Then we have ∑ Pr{𝑆𝑡 = 𝑆}

𝑆∈𝒮̅(𝑇^𝑛)

= ∑ Pr{𝑆_𝑡 = 𝑆 | 𝔸} ⋅ Pr{𝔸}

𝑆∈𝒮̅(𝑇^𝑛) (17)

+ ∑ Pr{𝑆_𝑡= 𝑆 | 𝔹}

𝑆∈𝒮̅(𝑇^𝑛) ⋅ Pr{𝔹} (18)

+ ∑ Pr{𝑆𝑡 = 𝑆 | ℂ} ⋅ Pr{ℂ}

𝑆∈𝒮̅(𝑇^𝑛) (19)

Let 𝒜_𝑡 denote the set of augmentations chosen under our algorithm at time 𝑡. We can obtain a bound on (17) as

∑ Pr{𝑆_𝑡= 𝑆 | 𝔸} ⋅ Pr{𝔸}

𝑆∈𝒮̅(𝑇^𝑛)

.

≤ ∑ Pr{𝑟_𝒘̅𝑡(𝑆) ≥ 𝑟_𝒘̅𝑡(𝑆_𝑡−1) | 𝔸} ⋅ Pr{𝔸}

𝑆∈𝒮̅(𝑇^𝑛)

≤ |𝒮̅(𝑇^𝑛)| ⋅ 2|ℒ|𝑡⁻² (20)

where the last inequality comes from Lemma 2. The result holds for all 𝑡 ∈ (𝑇^𝑛, 𝑇^𝑛+1]. For the second term (18), we have

∑ Pr{𝑆_𝑡 = 𝑆 | 𝔹}

𝑆∈𝒮̅(𝑇^𝑛) ⋅ Pr{𝔹} ≤ Pr{𝔹} (21)

Finally, the third term (19) denotes the probability to transit from a sufficiently-played non-near- optimal matching to a sufficiently-played non-near-optimal matching, and thus we have

∑ Pr{𝑆_𝑡 = 𝑆 | ℂ} ⋅ Pr{ℂ}

𝑆∈𝒮̅(𝑇^𝑛)

≤ ∑ Pr{𝑆_𝑡∈ 𝒮̅(𝑇^𝑛) | 𝑆_𝑡−1 = 𝑆} ⋅ Pr{𝑆_𝑡−1= 𝑆}

𝑆∈𝒮̅(𝑇^𝑛) .

Letting 𝑆^′= 𝑆 ⊕ 𝒜_𝑡, the conditional probability can be derived as Pr{𝑆_𝑡 ∈ 𝒮̅(𝑇^𝑛) | 𝑆_𝑡−1= 𝑆}

≤ Pr{𝑆_𝑡 ∈ 𝒮̅(𝑇^𝑛) | 𝑆_𝑡−1 = 𝑆, 𝑆^′∈ 𝒮_𝒘^𝛼} ⋅ Pr{𝑆^′ ∈ 𝒮_𝒘^𝛼} + Pr{𝑆^′ ∈ 𝒮̅_𝒘^𝛼}

≤ Pr{𝑆𝑡 ∈ 𝒮̅(𝑇^𝑛) | 𝑆_𝑡−1 = 𝑆, 𝑆^′∈ 𝒮𝒘𝛼} ⋅ 𝛿 + 1 − 𝛿

≤ Pr{𝑟_𝒘̅𝑡(𝑆) ≥ 𝑟_𝒘̅𝑡(𝑆_𝑡−1) | 𝑆 ∈ 𝒮̅_𝒘^𝛼, 𝑆^′∈ 𝒮_𝒘^𝛼} ⋅ 𝛿 + 1 − 𝛿

≤ 2|ℒ|𝑡⁻²⋅ 𝛿 + 1 − 𝛿.

where the second inequality comes from Proposition 1, the equality holds since 𝑆_𝑡 should be 𝑆 (otherwise, 𝑆_𝑡 = (𝑆 ⊕ 𝒜_𝑡 ) ∉ 𝒮̅(𝑇^𝑛)) and thus 𝑆 should have the larger weight sum to be chosen by the augmentation algorithm, and the last inequality comes from Lemma 2. Hence, the third term (19) can be upper bounded by

∑ Pr{𝑆_𝑡−1= 𝑆} ⋅ (2𝛿|ℒ|𝑡⁻²+ 1 − 𝛿)

𝑆∈𝒮̅(𝑇^𝑛)

≤ 2𝛿|ℒ|𝑡⁻²+ (1 − 𝛿) ∑ Pr{𝑆_𝑡−1 = 𝑆}

𝑆∈𝒮̅(𝑇^𝑛) ,

(22)

for all 𝑡 ∈ (𝑇^𝑛, 𝑇^𝑛+1].

The result can be obtained by combining (20), (21), and (22). □

In order to apply Lemma 5 to (15), we rewrite it in a recursive form. Let 𝜂 = 1 − 𝛿, 𝐻_𝑛= (|𝒮̅(𝑇^𝑛)| + 𝛿) ⋅ 2|ℒ| = (𝑛 + 𝛿) · 2|ℒ|, and 𝛩_𝑛(𝑡) = ∑_{𝑆∈𝒮̅(𝑇}𝑛)Pr{𝑆_𝑡 = 𝑆}. We have a recursive form of (16) as

𝛩_𝑛(𝑡) ≤ 𝜂^{𝑡−𝑇𝑛}𝛩_𝑛(𝑇^𝑛) (23) +𝐻_𝑛∑ 𝜂^𝑡−𝑖𝑖⁻²

𝑡 𝑖=𝑇^𝑛+1

(24)

+ ∑ 𝜂^𝑡−𝑖⋅

𝑡

𝑖=𝑇^𝑛+1 Pr{𝑆_𝑡−1∈ 𝒮(𝑇^𝑛)}. (25) By summing it over 𝑡 ∈ (𝑇^𝑛, 𝑇^𝑛+1] on the both sides, we obtain the following lemma.

LEMMA 6. The total number of times that sufficiently played non-near-optimal matchings are selected during (𝑇^𝑛, 𝑇^𝑛+1] is bounded by

∑ 𝛩_𝑛(𝑡)

𝑇^𝑛+1 𝑖=𝑇^𝑛+1

≤1

𝛿(1 +𝜋²

6 𝐻_𝑛+ 𝔼 [∑ 𝜏_𝑆,𝑛+1

𝑆∈𝒮(𝑇^𝑛) ]), (26)

where 𝜏𝑆,𝑛+1 denote the number of time slots that 𝑆 is scheduled in (𝑇^𝑛, 𝑇^𝑛+1].

The proof of Lemma 6 follows the same line of analysis as [17] and included for the completion.

Proof. By summing up (25) over 𝑡 ∈ (𝑇^𝑛, 𝑇^𝑛+1], we have

∑ ∑ 𝜂^𝑡−𝑖⋅

𝑡

𝑖=𝑇^𝑛+1 Pr{𝑆_𝑡−1∈ 𝒮(𝑇^𝑛)}

𝑇^𝑛+1 𝑡=𝑇^𝑛

= ∑ Pr{𝑆_𝑡−1∈ 𝒮(𝑇^𝑛)}

𝑇^𝑛+1

𝑡=𝑇^𝑛 ⋅ (∑ 𝜂^𝑖

𝑇^𝑛+1−𝑡

𝑖=0 )

≤ 1

1 − 𝜂𝔼 [∑ 𝕀{𝑆_𝑡−1∈ 𝒮(𝑇^𝑛)}

𝑇^𝑛+1

𝑡=𝑇^𝑛 ]

=1

𝛿𝔼 [∑ 𝜏_𝑆,𝑛+1

𝑆∈𝒮(𝑇^𝑛) ], (27)

where the inequality holds because ∑^𝑇_𝑖=0^{𝑛+1−𝑡}𝜂^𝑖 ≤ ¹

1−𝜂, Pr{𝑆_𝑡−1 ∈ 𝒮(𝑇^𝑛)} = 𝔼[𝕀{𝑆_𝑡−1∈ 𝒮(𝑇^𝑛)}], and 𝕀{𝑆_𝑇^𝑛∈ 𝒮(𝑇^𝑛)} = 0 since the matching scheduled at 𝑇^𝑛 does not belong to 𝒮(𝑇^𝑛) by definition.

Also, by summing up (24) over (𝑇^𝑛, 𝑇^𝑛+1], we have

∑ 𝐻_𝑛∑ 𝜂^𝑡−𝑖⋅

𝑡 𝑖=𝑇^𝑛+1

𝑖⁻²≤ 𝐻_𝑛⋅1 𝛿⋅𝜋²

6 .

𝑇^𝑛+1 𝑡=𝑇^𝑛

(28) Similarly, we take the sum of (23) as

∑ 𝜂^{𝑡−𝑇𝑛}𝛩_𝑛(𝑇^𝑛)

𝑇^𝑛+1 𝑡=𝑇^𝑛

≤ ∑ 𝜂^{𝑡−𝑇𝑛}

𝑇^𝑛+1 𝑡=𝑇^𝑛

≤1

𝛿 (29)

Combining (27), (28), and (29), we obtain the result. □

Now, by taking expectation on (15), we can obtain the expected total number of times that non- near optimal matchings are selected up to time 𝑇^′(≤ 𝑇) as

∑ 𝔼[𝜏̂_𝑆(𝑇^′)]

𝑆∈𝒮̅_𝒘^𝛼 = 𝑙_𝑇𝑀 + ∑ ∑^𝑇 𝛩_𝑛(𝑡)

𝑛+1

𝑡=𝑇^𝑛 𝑀−1

𝑛=1

≤ 𝑙_𝑇𝑀 +1

𝛿∑ (1 +𝐻_𝑛𝜋²

6 + 𝔼 [∑ 𝜏_𝑆,𝑛+1

𝑆∈𝒮(𝑇^𝑛) ])

𝑀−1 𝑛=1

≤ 𝑙_𝑇𝑀 +𝑀

𝛿 (1 +|ℒ|𝛿𝜋²

6 +|ℒ|(𝑀 − 1)𝜋²

6 + 𝑙_𝑇).

The last inequality holds since (i) ∑^𝑀−1_𝑛=1𝐻_𝑛= ∑^𝑀−1_𝑛=1(𝑛 + 𝛿) ⋅ 2|ℒ| ≤ 𝑀 ⋅ |ℒ| ⋅ (2𝛿 + (𝑀 − 1)), and (ii) ∑_{𝑆∈𝒮(𝑇}𝑛)𝜏_𝑆,𝑛+1 is the total number that the matchings that have been chosen less than 𝑙_𝑇 up to 𝑇^𝑛 are chosen during (𝑇^𝑛, 𝑇^𝑛+1] and thus results in ∑𝑀−1∑_{𝑆∈𝒮(𝑇}^𝑛)𝜏_𝑆,𝑛+1

𝑛=1 ≤ ∑^𝑀_𝑘=2𝑙_𝑇 ≤ 𝑙_𝑇𝑀.

From 𝑀 ≤ |𝒮| − 1, we have

∑ 𝔼[𝜏̂_𝑆(𝑇^′)]

𝑆∈𝒮̅_𝒘^𝛼 ≤ 𝐷₁⋅ ln 𝑇

(Δ_min^𝛼 )²+|𝒮| − 1

𝛿 (1 +|ℒ|𝛿𝜋²

6 +|ℒ|(|𝒮| − 2)𝜋²

6 )

(30) where 𝐷₁= (1 +¹

𝛿) ⋅ 4|ℒ|²(|ℒ| + 1) ⋅ (|𝒮| − 1).

This provides a bound on the number of times that non-near optimal matchings are selected up to 𝑇^′. For the rest time, 𝑡 ∈ (𝑇^′, 𝑇], we need to compute ∑^𝑇_𝑡=𝑇′Pr{𝑆𝑡 ∈ 𝒮̅𝒘𝛼}. Let 𝑆^′ = 𝑆𝑡−1⊕ 𝒜𝑡. Since next schedule 𝑆_𝑡 is either 𝑆_𝑡−1 and 𝑆^′ under the algorithm, we divide the event {𝑆_𝑡∈ 𝒮̅_𝒘^𝛼} into three sub-cases based on 𝑆_𝑡−1 and 𝑆^′, and compute the probability as

Pr{𝑆_𝑡 ∈ 𝒮̅_𝒘^𝛼} = Pr{𝑆^′∈ 𝒮̅_𝒘^𝛼, 𝑆_𝑡−1∈ 𝒮̅_𝒘^𝛼}

+ Pr{𝑆^′ ∈ 𝒮̅_𝒘^𝛼, 𝑆_𝑡−1 ∈ 𝒮_𝒘^𝛼, 𝑟_𝒘(𝑆^′) ≥ 𝑟_𝒘(𝑆_𝑡−1)}

+ Pr{𝑆^′ ∈ 𝒮_𝒘^𝛼, 𝑆_𝑡−1 ∈ 𝒮̅_𝒘^𝛼, 𝑟_𝒘(𝑆^′) ≤ 𝑟_𝒘(𝑆_𝑡−1)}

This leads to

Pr{𝑆_𝑡 ∈ 𝒮̅_𝒘^𝛼} ≤ Pr{𝑆^′∈ 𝒮̅_𝒘^𝛼 | 𝑆_𝑡−1 ∈ 𝒮̅_𝒘^𝛼} ⋅ Pr {𝑆_𝑡−1∈ 𝒮̅_𝒘^𝛼}

+ Pr{𝑟_𝒘(𝑆^′) ≥ 𝑟_𝒘(𝑆_𝑡−1) | 𝑆^′ ∈ 𝒮̅_𝒘^𝛼, 𝑆_𝑡−1∈ 𝒮_𝒘^𝛼} + Pr{𝑟_𝒘(𝑆^′) ≤ 𝑟_𝒘(𝑆_𝑡−1) | 𝑆^′∈ 𝒮_𝒘^𝛼, 𝑆_𝑡−1∈ 𝒮̅_𝒘^𝛼}.

From Proposition 1, we have Pr{𝑆^′∈ 𝒮̅_𝒘^𝛼 | 𝑆_𝑡−1∈ 𝒮̅_𝒘^𝛼} = 1 − Pr{𝑆^′∈ 𝒮_𝒘^𝛼 | 𝑆_𝑡−1∈ 𝒮̅_𝒘^𝛼} ≤ 1 − 𝛿 = 𝜂. Since 𝜏̂_𝑆(𝑡) ≥ 𝑙_𝑇 for all 𝑆 and 𝑡 ∈ (𝑇^′, 𝑇], Lemma 2 provides an upper bound 2|ℒ|𝑡⁻² on each conditional probability in the second and the third terms. As a result, we can obtain

Pr{𝑆_𝑡 ∈ 𝒮̅_𝒘^𝛼} ≤ 𝜂 Pr{𝑆_𝑡−1∈ 𝒮̅_𝒘^𝛼} + 4|ℒ|𝑡⁻². By extending the inequality in a recursive manner down to𝑇^′, we obtain that

Pr{𝑆_𝑡∈ 𝒮̅_𝒘^𝛼} ≤ 𝜂^{𝑡−𝑇′}Pr{𝑆_𝑇^′ ∈ 𝒮̅_𝒘^𝛼} + 4|ℒ| ∑ 𝜂^𝑡−𝑖𝑖⁻²

𝑡 𝑡=𝑇^′+1

= 𝜂^{𝑡−𝑇′}+4|ℒ| ∑ 𝜂^𝑡−𝑖𝑖⁻²

𝑡 𝑡=𝑇^′+1

where the last equality holds since Pr{𝑆_𝑇^′ ∈ 𝒮̅_𝒘^𝛼} = 1 from the definition of 𝑇^′. Summing over 𝑡 ∈ (𝑇^′, 𝑇] on the both sides, and from 𝜂 = 1 − 𝛿, we have

∑ Pr{𝑆_𝑡 ∈ 𝒮̅_𝒘^𝛼}

𝑇

𝑡=𝑇^′+1 ≤1 − 𝛿

𝛿 +2|ℒ|𝜋²

3𝛿 . (31)