Appendix

8.3 Future research

9.1.5 Appendix

Example 4.1.17. Let H₁ = H₂ = (ℓ₂(R,∥.∥_ℓ2), where ℓ₂(R) := {x = (x₁, x₂, ...), x_i ∈ R : P∞

i=1|x_i|² < +∞} endowed with the norm ∥x∥_ℓ2 := (P∞

i=1|x_i|²)¹² , ∀ x ∈ ℓ₂(R).

Now, consider the Split Variational Inequality Problem (SVI_qP) with nonempty closed and convex subsets C and Q of H1 and H2, respectively, where C and Q are defined by

C:={x∈ℓ₂(R) :∥x−b∥_ℓ2 ≤3} and Q:={x∈ℓ₂(R) :∥x−b∥_ℓ2 ≤1},

respectively. We set J_γ^B_n =P_C and J_λ^G_n = P_Q. Then, the projection operators P_C and P_Q

have the following explicit formulas P_C(x) :=

(3_∥x−b∥^x−b

ℓ2

+b, ∥x−b∥ℓ2 >3,

x, ∥x−b∥_ℓ2 ≤3, and P_Q(x) :=

( _x−b

∥x−b∥_ℓ

2

+b, ∥x−b∥ℓ2 >1, x, ∥x−b∥_ℓ2 ≤1, where b= 1,¹₂,¹₃, ...

. Also, let T :ℓ₂(R)→ℓ₂(R) be defined by T x=

0, x1,x₂ 2 ,x₃

3, ...

, ∀ x∈ℓ2(R).

Then,T is a bounded linear operator with adjointT^∗, defined byT^∗y= y₂,^y₂³,^y₃⁴, ...

, ∀y∈ ℓ₂(R).

Define the mapping A, D:ℓ₂(R)→ℓ₂(R) by A(x) =

x₁e^−x21,0,0, ...

and D(x) =

5x₁e^−x21,0,0, ...

, respectively.

Now, let B = ∂δ_C and G = ∂δ_Q in Theorem 4.1.10 (where δ_C and δ_Q are the indicator functions) and use

Tol_n := 1

2 ∥x_n−P_C(x_n−γ_nAx_n)∥²+∥T x_n−P_Q(T x_n−λ_nDT x_n)∥²

< ϵ for stopping criterion, where ϵ= 10⁻⁴.

We consider four cases for our numerical experiments:

Case 1: Take x₀ = −7,1,−¹₇, ...

and x₁ = 2,1,¹₂, ...

Case 2: Take x₀ = 8,1,¹₈, ...

and x₁ = (1,0.1,0.01, ...) Case 3: Take x₀ = 9,1,¹₉, ...

and x₁ = (−2,0.2,−0.02, ...) Case 4: Take x₀ = −6,1,−¹₆, ...

and x₁ = −3,1,−¹₃, ...

.

The numerical results are displayed in Figure 4.2 and Table 4.1.17.

Table 4.1.17. Numerical Results for Example 4.1.17

Case 1 Case 2 Case 3 Case 4

Iter. CPU Time Iter. CPU Time Iter. CPU Time Iter. CPU Time

KR 26 0.0117 26 0.0098 26 0.0093 25 0.0085

Izu 57 0.0075 57 0.0074 58 0.0076 56 0.0071

Thn 620 0.0153 1248 0.0173 478 0.0150 1759 0.0189

She 42 0.0061 43 0.0070 43 0.0065 42 0.0069

IzR 169 0.0129 194 0.0121 168 0.0034 203 0.0126

Prop 41 0.0117 41 0.0132 41 0.0132 41 0.0132

10⁰ 10¹ 10² 10³ Iteration number (n)

10^-5 10^-4 10^-3 10^-2 10^-1 10⁰ 10¹

Errors

Kazmi & Rizvi Alg.

Izuchukwu et al. Alg.

Thong et al. Alg.

Shehu & Ogbuisi Alg.

Izuchukwu, Riech et al. Alg.

Proposed Alg.

10⁰ 10¹ 10² 10³

Iteration number (n) 10^-5

10^-4 10^-3 10^-2 10^-1 10⁰ 10¹

Errors

Kazmi & Rizvi Alg.

Izuchukwu et al. Alg.

Thong et al. Alg.

Shehu & Ogbuisi Alg.

Izuchukwu, Riech et al. Alg.

Proposed Alg.

10⁰ 10¹ 10² 10³

Iteration number (n) 10^-5

10^-4 10^-3 10^-2 10^-1 10⁰ 10¹

Errors

Kazmi & Rizvi Alg.

Izuchukwu et al. Alg.

Thong et al. Alg.

Shehu & Ogbuisi Alg.

Izuchukwu, Riech et al. Alg.

Proposed Alg.

10⁰ 10¹ 10² 10³ 10⁴

Iteration number (n) 10^-5

10^-4 10^-3 10^-2 10^-1 10⁰ 10¹

Errors

Kazmi & Rizvi Alg.

Izuchukwu et al. Alg.

Thong et al. Alg.

Shehu & Ogbuisi Alg.

Izuchukwu, Riech et al. Alg.

Proposed Alg.

Figure 4.2. Top left: Case I; Top right: Case II; Bottom left: Case III; Bottom right:

Case IV.

Step 1. Given the iterates x_n−1 and x_n (n ≥1), choose α_n such that 0≤α_n ≤α¯_n, where

¯ α_n :=







min _n−1

n+α−1,_∥x ^ϵn

n−xn−1∥ , if x_n ̸=x_n−1

n−1

n+α−1, otherwise.

(4.85)

Step 2. Set

w_n =x_n+α_n(x_n−xn−1).

Then, compute

un =wn−γnGwn and yn =J_λ^B_n(I−λnA)un = (I+λnB)⁻¹(I−λnA)un, (4.86) where

γ_n+1=

(minn _µ∥u

n−wn∥

∥Gun−Gwn∥, γ_no

, if Gu_n̸=Gw_n,

γ_n, otherwise

(4.87)

and

λ_n+1 =

(minn _η∥u

n−y_n∥

∥Aun−Ayn∥, λ_no

, if Au_n ̸=Ay_n,

λ_n, otherwise.

(4.88)

Step 3. Compute

x_n+1 = (1−β_n−θ_n)u_n+β_nz_n, (4.89) where z_n:= (1−ψ_n)u_n+ψ_n

y_n+λ_n A(u_n)−A(y_n)

.

Stopping criterion: If y_n =u_n =w_n =x_n, then stop, otherwise, set n :=n+ 1 and go back to Step 1.

Remark 4.2.4. λ_n+1 ≤λ_n ∀n ∈N and lim

n→∞λ_n ≥min{λ₁,_L^η}.

Indeed, from (4.88), it is obvious that λ_n+1 ≤ λ_n ∀n ∈ N. Furthermore, since A is L-Lipschitz continuous, we get in the case of Aun̸=Ayn, that

λ_n+1 = min

η∥u_n−y_n∥

∥Au_n−Ay_n∥, λ_n

≥minnη L, λ_no

,

which by induction implies that λ_n ≥min{λ₁,_L^η}. This gives that the limit of {λ_n} exists and lim

n→∞λ_n≥min{λ₁,_L^η}.

Since G is 2∥T∥²-Lipschitz continuous, then in a similar manner, we have from (4.87) that γ_n+1 ≤γ_n ∀n∈N and lim

n→∞γ_n≥minn

γ₁,_2∥T^µ_∥2

o .

We now show that the stopping criterion of Algorithm 4.2.3 is valid.

Lemma 4.2.5. If y_n =u_n =w_n=x_n in Algorithm 4.2.3, then x_n∈Γ.

Proof. Letyn =un =wn=xn. Then, from (4.86), it is clear thatxn=J_λ^B_n(xn−λnA(xn)).

This implies thatx_n ∈(A+B)⁻¹(0). From (4.86), we also obtain thatx_n =x_n−γ_nGx_n= x_n−γ_nT^∗(I−S)T x_n,which implies that T^∗(I−S)T x_n = 0. That is,

ST x_n =T x_n+ ¯z, where T^∗z¯= 0. Now, let z ∈Γ, then we obtain that

∥T x_n−T z∥² =∥T x_n−T z∥²+ 2⟨x_n−z, T^∗z⟩¯

=∥T xn−T z∥²+ 2⟨T xn−T z,z⟩¯

=∥T x_n−T z+ ¯z∥²− ∥¯z∥²

=∥ST x_n−T z∥²− ∥¯z∥²

≤ ∥T x_n−T z∥²− ∥¯z∥²,

which implies that ∥¯z∥ = 0. That is, ¯z = 0. Hence ST x_n = T x_n, which gives that T x_n∈F(S).Therefore, x_n∈Γ.

Lemma 4.2.6. Let Assumption 4.2.2 hold and let {x_n} be a sequence generated by Algo- rithm 4.2.3. Then {x_n} is bounded.

Proof. From (4.88), we have that

∥Au_n−Ay_n∥ ≤ η

λ_n+1∥u_n−y_n∥ ∀n ∈N (4.90) holds for both Au_n=Ay_n and Au_n̸=Ay_n. Now, let x^∗ ∈Γ. Then, by Step 3, we obtain

∥zn−x^∗∥² =∥(1−ψn)un+ψnyn+ψnλn A(un)−A(yn)

−x^∗∥²

=∥(1−ψ_n)(u_n−x^∗) +ψ_n(y_n−x^∗) +ψ_nλ_n A(u_n)−A(y_n)

∥²

= (1−ψn)²∥un−x^∗∥²+ψ_n²∥yn−x^∗∥²+ψ_n²λ²_n∥A(un)−A(yn)∥² + 2ψ_n(1−ψ_n)⟨u_n−x^∗, y_n−x^∗⟩

+ 2ψ_n(1−ψ_n)λ_n⟨u_n−x^∗, A(u_n)−A(y_n)⟩+ 2ψ_n²λ_n⟨y_n−x^∗, A(u_n)−A(y_n)⟩. (4.91) Using Lemma 2.5.18 (i), we obtain

2⟨u_n−x^∗, y_n−x^∗⟩=∥u_n−x^∗∥²+∥y_n−x^∗∥²− ∥u_n−y_n∥². (4.92) Combining (4.91) and (4.92), we get

∥zn−x^∗∥² = (1−ψn)∥un−x^∗∥²+ψn∥yn−x^∗∥²+ψ²_nλ²_n∥A(un)−A(yn)∥²

−ψ_n(1−ψ_n)∥u_n−y_n∥²+ 2ψ_n(1−ψ_n)λ_n⟨u_n−x^∗, A(u_n)−A(y_n)⟩

+ 2ψ²_nλ_n⟨y_n−x^∗, A(u_n)−A(y_n)⟩. (4.93)

Now, from (4.86), we obtain Ay_n+ 1

λ_n(u_n−λ_nAu_n−y_n)∈(A+B)y_n.

Since x^∗ ∈Γ, we have that 0∈(A+B)(x^∗). Hence, we obtain from Lemma 2.5.15that

⟨y_n−u_n−λ_n A(y_n)−A(u_n)

, y_n−x^∗⟩ ≤0, which implies that

⟨y_n−u_n, y_n−x^∗⟩ ≤λ_n⟨A(y_n)−A(u_n), y_n−x^∗⟩. (4.94) Now, observe that

∥y_n−x^∗∥² ≤ ∥y_n−u_n∥²+∥u_n−x^∗∥²+ 2⟨y_n−u_n, u_n−x^∗⟩

≤ ∥y_n−u_n∥²+∥u_n−x^∗∥²−2⟨y_n−u_n, y_n−u_n⟩+ 2⟨y_n−u_n, y_n−x^∗⟩

≤ ∥u_n−x^∗∥²− ∥y_n−u_n∥²−2λ_n⟨A(u_n)−A(y_n), y_n−x^∗⟩, (4.95) where the last inequality follows from (4.94). Substituting (4.95) into (4.93), and using (4.90), we obtain

∥z_n−x^∗∥² ≤(1−ψ_n)∥u_n−x^∗∥²+ψ_n

∥u_n−x^∗∥² − ∥u_n−y_n∥²

−2λn⟨A(un)−A(yn), yn−x^∗⟩

−ψn(1−ψn)∥un−yn∥²+ψ_n²λ²_n∥A(un)−A(yn)∥² + 2ψ_n(1−ψ_n)λ_n⟨u_n−x^∗, A(u_n)−A(y_n)⟩+ 2ψ_n²λ_n⟨y_n−x^∗, A(u_n)−A(y_n)⟩

=∥u_n−x^∗∥²−ψ_n(2−ψ_n)∥u_n−y_n∥²+ψ_n²λ²_n∥A(u_n)−A(y_n)∥²

−2ψ_nλ_n⟨A(u_n)−A(y_n), y_n−x^∗⟩+ 2ψ²_nλ_n⟨y_n−x^∗, A(u_n)−A(y_n)⟩

+ 2ψ_n(1−ψ_n)λ_n⟨u_n−x^∗, A(u_n)−A(y_n)⟩

=∥u_n−x^∗∥²−ψ_n(2−ψ_n)∥u_n−y_n∥²+ψ_n²λ²_n∥A(u_n)−A(y_n)∥² + 2ψ_n(1−ψ_n)λ_n⟨A(u_n)−A(y_n), u_n−y_n⟩

≤ ∥u_n−x^∗∥²−ψ_n(2−ψ_n)∥u_n−y_n∥²+ψ²_nλ²_n η²

λ²_n+1∥u_n−y_n∥² + 2ψ_n(1−ψ_n)λ_n η

λ_n+1∥u_n−y_n∥²

=∥un−x^∗∥²−ψn

2−ψn−2(1−ψn)η λ_n

λ_n+1 −ψnη² λ²_n λ²_n+1

∥un−yn∥². (4.96) Since the limit of {λ_n} exists (see Remark 4.2.4 (d)), then lim

n→∞λ_n = lim

n→∞λ_n+1. Also, by Assumption 4.2.2 (2)(d), there exists ψ ∈ (0,1] such that ψ = lim

n→∞ψ_n. Thus, we obtain that

n→∞lim

2−ψ_n−2(1−ψ_n)η λ_n

λ_n+1 −ψ_nη² λ²_n λ²_n+1

= (1−η)(2−ψ+ψη)>0. (4.97) Hence, there exists n₀ ∈ N such that 2−ψ_n−2(1−ψ_n)η_λ^λn

n+1 −ψ_nη²_λ^λ2²ⁿ n+1

>0 ∀n ≥n₀. Thus, we obtain from (4.96) that

∥z_n−x^∗∥ ≤ ∥u_n−x^∗∥ ∀n ≥n₀. (4.98)

Observe from Lemma2.5.25and Lemma2.5.16(ii), (iii), that (I−γ_nG) isγ_n∥T∥²-averaged.

That is, (I−γ_nG) = (1−β_n)I+β_nV_n, ∀ n∈ N, whereβ_n =γ_n∥T∥² ∈(0,1) and V_n is nonexpansive for all n ∈N. Therefore, we can rewriteu_n from (4.86) as:

u_n= (1−β_n)w_n+β_nV_nw_n, n ≥1. (4.99) Thus, we obtain that

∥un−x^∗∥² = (1−βn)∥wn−x^∗∥²+βn∥Vnwn−x^∗∥²−βn(1−βn)∥Vnwn−wn∥²

≤ ∥w_n−x^∗∥²−β_n(1−β_n)∥V_nw_n−w_n∥² (4.100)

≤ ∥w_n−x^∗∥².

Now, from Step 1 and Assumption 4.2.2 (2), we have that α_n∥x_n−x_n−1∥ ≤ϵ_n ∀n ∈ N, which implies that

α_n

θ_n∥x_n−xn−1∥ ≤ ϵ_n

θ_n →0, as n→ ∞. (4.101)

Hence, there exists M₁ >0 such that α_n

θ_n∥xn−xn−1∥ ≤M1 ∀ n ∈N. (4.102) Thus, from Step 2, we obtain

∥w_n−x^∗∥ ≤ ∥x_n−x^∗∥+α_n∥x_n−xn−1∥

=∥x_n−x^∗∥+θ_nα_n

θ_n∥x_n−x_n−1∥

≤ ∥xn−x^∗∥+θnM1, (4.103)

and so,

∥u_n−x^∗∥ ≤ ∥w_n−x^∗∥ ≤ ∥x_n−x^∗∥+θ_nM₁. (4.104) From (4.89), we obtain

∥x_n+1−x^∗∥=∥(1−β_n−θ_n)u_n+β_nz_n−x^∗∥

=∥(1−β_n−θ_n)(u_n−x^∗) +β_n(z_n−x^∗)−θ_nx^∗∥

≤ ∥(1−β_n−θ_n)(u_n−x^∗) +β_n(z_n−x^∗)∥+θ_n∥x^∗∥. (4.105) Note that

∥(1−βn−θn)(un−x^∗) +βn(zn−x^∗)∥²

= (1−β_n−θ_n)²∥u_n−x^∗∥²+ 2(1−β_n−θ_n)β_n⟨u_n−x^∗, z_n−x^∗⟩ +β_n²∥z_n−x^∗∥²

≤(1−β_n−θ_n)²∥u_n−x^∗∥²+ 2(1−β_n−θ_n)β_n∥u_n−x^∗∥.∥z_n−x^∗∥ +β_n²∥z_n−x^∗∥²

≤(1−β_n−θ_n)²∥u_n−x^∗∥²+ (1−β_n−θ_n)β_n∥u_n−x^∗∥² + (1−β_n−θ_n)β_n∥z_n−x^∗∥² +β_n²∥z_n−x^∗∥²

≤(1−βn−θn)(1−θn)∥un−x^∗∥²+ (1−θn)βn∥zn−x^∗∥² (4.106)

≤(1−β_n−θ_n)(1−θ_n)∥u_n−x^∗∥²+ (1−θ_n)β_n∥u_n−x^∗∥²

= (1−θ_n)²∥u_n−x^∗∥²,

which implies from (4.104) and (4.105) that

∥x_n+1−x^∗∥ ≤(1−θ_n)∥x_n−x^∗∥+θ_n(1−θ_n)M₁+θ_n∥x^∗∥

≤(1−θn)∥xn−x^∗∥+θn(∥x^∗∥+M1).

It follows from Lemma 2.5.24 that {x_n} is bounded.

Lemma 4.2.7. Let{x_n}be a sequence generated by Algorithm4.2.3such that Assumption 4.2.2 holds. If there exists a subsequence {x_nk} of {x_n} which converges weakly to a point z ∈H1 and lim

n→∞∥un_k−yn_k∥= 0 = lim

n→∞∥un_k −xn_k∥. Then, z ∈Γ.

Proof. From Step 2 and (4.101), we have

∥w_n−x_n∥=θ_nα_n

θ_n∥x_n−xn−1∥ →0, as n→ ∞. (4.107) Let {x_nk} be a subsequence of{x_n} which is weakly convergent to a point z ∈H₁. Then, the subsequences {u_nk},{y_nk} and {w_nk} are weakly convergent toz ∈H₁.

We first show that T z ∈ F(S). To do this, recall from Remark 4.2.4 that lim

n→∞γ_n ≥ minn

γ₁,_2∥T^µ_∥2

o

>0. Let lim

n→∞γ_n=γ >0,since {T^∗(I−S)T w_nk}is bounded, we get that

∥(I−γ_nkT^∗(I−S)T)w_nk−(I−γT^∗(I−S)T)w_nk∥=|γ_nk−γ|∥T^∗(I−S)T w_nk∥ →0, as k → ∞.

That is, lim

k→∞∥u_nk − (I −γT^∗(I −S)T)w_nk∥ = 0, which implies from (4.107) and our hypothesis that

k→∞lim ∥w_nk− I −γT^∗(I−S)T

w_nk∥= 0. (4.108)

Thus, we obtain from Lemma2.5.55, thatz ∈F I−γT^∗(I−S)T

.Hence, using the same line of argument as in the proof of Lemma 4.2.5, we obtain that T z ∈F(S).

Now, let (v, w)∈G(A+B),then w−Av ∈B(v).Also, we obtain from (4.86) that 1

λ_nk u_nk −λ_nkAu_nk −y_nk

∈B(y_nk).

Thus, by the monotonicity of B, we obtain that

⟨v −y_nk, w−Av− 1

λ_nk u_nk −λ_nkAu_nk −y_nk

⟩ ≥0. (4.109)

Using (4.109) and the monotonicity ofA, we obtain

⟨v−y_nk, w⟩ ≥ ⟨v−y_nk, Av+ 1

λ_nk(u_nk−y_nk)−Au_nk⟩

=⟨v−y_nk, Av−A(y_nk)⟩+⟨v−y_nk, A(y_nk)−A(u_nk)⟩

+⟨v−yn_k, 1

λ_nk(un_k −yn_k)⟩

≥ ⟨v−y_nk, A(y_nk)−A(u_nk)⟩+⟨v−y_nk, 1 λn_k

(u_nk−y_nk)⟩. (4.110)

Recall that lim

k→∞λ_nk ≥ min{λ₁,_L^η} > 0, lim

k→∞∥u_nk −y_nk∥ = 0 and by the Lipschitz continuity of A, lim

k→∞∥A(u_nk)−A(y_nk)∥ = 0. Hence, passing limit as k → ∞in (4.110), we obtain

⟨v−z, w⟩ ≥0.

Also, by Lemma 2.5.15, A+B is maximal monotone, thus, 0 ∈ (A+B)z. This together with the fact that T z ∈F(S) gives that z ∈Γ.

Theorem 4.2.8. Suppose that Assumption4.2.2holds. Then, the sequence{x_n}generated by Algorithm 4.2.3 converges strongly to x^∗ ∈Γ, where

∥x^∗∥= min{∥z∥:z ∈Γ}.

Proof. Let x^∗ ∈Γ. Then, we obtain from (4.98), (4.100) and Step 2 that

∥(1−βn)un+βnzn−x^∗∥² =∥(1−βn)(un−x^∗) +βn(zn−x^∗)∥²

= (1−βn)²∥un−x^∗∥²+β_n²∥zn−x^∗∥²+ 2(1−βn)βn⟨un−x^∗, zn−x^∗⟩

≤(1−β_n)²∥u_n−x^∗∥² +β_n²∥z_n−x^∗∥²+ 2(1−β_n)β_n∥u_n−x^∗∥.∥z_n−x^∗∥

≤(1−β_n)²∥u_n−x^∗∥² +β_n²∥z_n−x^∗∥² + (1−β_n)β_n ∥u_n−x^∗∥²+∥z_n−x^∗∥²

= (1−β_n)∥u_n−x^∗∥²+β_n∥z_n−x^∗∥²

≤ ∥w_n−x^∗∥². (4.111)

On the other hand, we have

∥w_n−x^∗∥² =∥x_n−x^∗∥²+ 2α_n⟨x_n−x^∗, x_n−xn−1⟩+α²_n∥x_n−xn−1∥²

≤ ∥x_n−x^∗∥²+ 2α_n∥x_n−x^∗∥.∥x_n−x_n−1∥+α²_n∥x_n−x_n−1∥²

=∥xn−x^∗∥²+αn∥xn−xn−1∥(2∥xn−x^∗∥+αn∥xn−xn−1∥)

≤ ∥x_n−x^∗∥²+ 3α_n∥x_n−xn−1∥M₂, (4.112) for some M₂ >0. Combining (4.111) and (4.112), we obtain

∥(1−β_n)u_n+β_nz_n−x^∗∥² ≤ ∥x_n−x^∗∥²+ 3α_n∥x_n−xn−1∥M₂. (4.113) Thus, from Step 3 and Lemma 2.5.18(ii), we obtain

∥x_n+1−x^∗∥² =∥(1−θ_n)

(1−β_n)u_n+β_nz_n−x^∗

−

θ_nβ_n(u_n−z_n) +θ_nx^∗

≤(1−θ_n)²∥(1−β_n)u_n+β_nz_n−x^∗∥²−2⟨θ_nβ_n(u_n−z_n) +θ_nx^∗, x_n+1−x^∗⟩

≤(1−θ_n)∥(1−β_n)u_n+β_nz_n−x^∗∥²+ 2⟨θ_nβ_n(u_n−z_n), x^∗ −x_n+1⟩ + 2θn⟨x^∗, x^∗−xn+1⟩

≤(1−θ_n)∥(1−β_n)u_n+β_nz_n−x^∗∥²+ 2θ_nβ_n∥u_n−z_n∥.∥x^∗−x_n+1∥

+ 2θ_n⟨x^∗, x^∗−x_n+1⟩. (4.114)

Therefore, using (4.113) and (4.114), we have

∥x_n+1−x^∗∥² ≤(1−θ_n)∥x_n−x^∗∥²+ 3(1−θ_n)α_n∥x_n−xn−1∥M₂ + 2θ_nβ_n∥u_n−z_n∥.∥x^∗−x_n+1∥+ 2θ_n⟨x^∗, x^∗−x_n+1⟩

= (1−θ_n)∥x_n−x^∗∥² +θ_nh

3α_n

θ_n(1−θ_n)∥x_n−x_n−1∥M₂+ 2β_n∥u_n−z_n∥.∥x^∗−x_n+1∥ + 2⟨x^∗, x^∗ −x_n+1⟩i

= (1−θ_n)∥x_n−x^∗∥²+θ_nd_n, ∀n≥n₀, (4.115) whered_n:=h

3α_n

θ_n(1−θ_n)∥x_n−x_n−1∥M₂+ 2β_n∥u_n−z_n∥.∥x^∗−x_n+1∥+ 2⟨x^∗, x^∗−x_n+1⟩i . To show that the sequence n

∥x_n−x^∗∥o

converges to zero, by Lemma2.5.55, it is enough to show that lim sup

k→∞

d_nk ≤0 (where{d_nk}is a subsequence of{d_n}), for every subsequence n∥x_nk−x^∗∥o

of n

∥x_n−x^∗∥o

satisfying lim inf

k→∞

∥x_nk+1−x^∗∥ − ∥x_nk −x^∗∥

≥0. (4.116)

Now, suppose thatn

∥x_nk−x^∗∥o

is a subsequence of n

∥x_n−x^∗∥o

such that (4.116) holds.

Then, lim inf

k→∞

∥x_nk+1−x^∗∥²− ∥x_nk−x^∗∥²

= lim

k→∞inf

(∥x_nk+1−x^∗∥ − ∥x_nk−x^∗∥)(∥x_nk+1−x^∗∥+∥x_nk−x^∗∥)

≥0. (4.117) Now, by Step 3, we obtain

∥x_n+1−x^∗∥² =∥(1−β_n−θ_n)(u_n−x^∗) +β_n(z_n−x^∗)−θ_nx^∗∥²

=∥(1−β_n−θ_n)(u_n−x^∗) +β_n(z_n−x^∗)∥²−2θ_n⟨(1−β_n−θ_n)(u_n−x^∗) +β_n(z_n−x^∗), x^∗⟩+θ_n²∥x^∗∥²

≤ ∥(1−β_n−θ_n)(u_n−x^∗) +β_n(z_n−x^∗)∥²+θ_nM₃, (4.118) for some M₃ >0. Substituting (4.106) into (4.118), we have

∥x_n+1−x^∗∥² ≤(1−β_n−θ_n)(1−θ_n)∥u_n−x^∗∥²+ (1−α_n)β_n∥z_n−x^∗∥²+θ_nM₃. (4.119)

Using (4.96), (4.112) and (4.119), we obtain

∥x_n+1−x^∗∥²

≤(1−β_n−θ_n)(1−θ_n)∥u_n−x^∗∥²+ (1−θ_n)β_n ∥u_n−x^∗∥²

−ψ_n

2−ψ_n−2(1−ψ_n)η λn

λ_n+1 −ψ_nη² λ²_n λ²_n+1

∥u_n−y_n∥²

!

+θ_nM₃

= (1−θ_n)²∥u_n−x^∗∥²−(1−θ_n)β_nψ_n

2−ψ_n−2(1−ψ_n)η λ_n

λ_n+1 −ψ_nη² λ²_n λ²_n+1

∥u_n−y_n∥² +θ_nM₃

≤(1−θ_n)²

∥x_n−x^∗∥²+ 3α_n∥x_n−xn−1∥M₂

−(1−θ_n)β_nψ_n

2−ψ_n−2(1−ψ_n)η λ_n λ_n+1

−ψ_nη² λ²_n λ²_n+1

∥u_n−y_n∥²+θ_nM₃

≤ ∥x_n−x^∗∥²+θ_n

3α_n θn

∥x_n−xn−1∥M₂+M₃

−(1−θ_n)β_nψ_n

2−ψ_n−2(1−ψ_n)η λ_n λn+1

−ψnη² λ²_n λ²_n+1

∥un−yn∥².

Hence, by (4.102) and (4.117), we obtain lim sup

k→∞

(1−θ_nk)β_nkψ_nk

2−ψ_nk −2(1−ψ_nk)η λ_nk

λ_nk+1 −ψ_nkη² λ²_nk λ²_n

k+1

∥u_nk −y_nk∥²

≤lim sup

k→∞

(

∥x_nk −x^∗∥²− ∥x_nk+1−x^∗∥²+θ_nk

3M₁M₂+M₃ )

= lim sup

k→∞

n∥x_nk−x^∗∥²− ∥x_nk+1−x^∗∥²o

=−lim inf

k→∞

n∥x_nk+1−x^∗∥²− ∥x_nk−x^∗∥²o

≤0, which implies from (4.97) that

k→∞lim ∥u_nk−y_nk∥= 0. (4.120) Using (4.119), (4.98) and (4.100), we obtain

∥xn+1−x^∗∥² ≤(1−βn−θn)(1−θn)∥un−x^∗∥²+ (1−θn)βn∥zn−x^∗∥²+θnM3

≤(1−θ_n)²∥u_n−x^∗∥²+θ_nM₃

≤ ∥w_n−x^∗∥²−β_n(1−β_n)∥V_nw_n−w_n∥²+θ_nM₃

≤ ∥x_n−x^∗∥² + 3α_n∥x_n−xn−1∥M₂−β_n(1−β_n)∥V_nw_n−w_n∥²+θ_nM₃.

Thus, by (4.117) we obtain lim sup

k→∞

h

β_nk(1−β_nk)∥V_nkw_nk −w_nk∥²i

≤lim sup

k→∞

"

∥x_nk −x^∗∥²− ∥x_nk+1−x^∗∥²

+θ_nk

3M₁M₂+M₃

#

=−lim inf

k→∞

h∥x_nk+1−x^∗∥²− ∥x_nk −x^∗∥²i

≤0, which implies that

k→∞lim ∥Vn_kwn_k−wn_k∥= 0. (4.121) Thus, we obtain from (4.99) that

k→∞lim ∥u_nk −w_nk∥= lim

k→∞β_nk∥V_nkw_nk −w_nk∥= 0. (4.122) Using (4.122) and (4.107), we obtain

k→∞lim ∥u_nk −x_nk∥= 0. (4.123) Observe from Step 3 and (4.120) that

∥z_nk −y_nk∥= (1−ψ_nk)∥u_nk −y_nk∥+ψ_nk∥λ_nk(Au_nk −Ay_nk)∥

≤(1−ψn_k)∥un_k −yn_k∥+ψn_kλn_kL∥un_k −yn_k∥ →0 as k → ∞. (4.124) Thus, from (4.120), we obtain

∥z_nk −u_nk∥= 0. (4.125)

Also, from Step 3, we obtain

∥x_nk+1−u_nk∥ ≤β_nk∥z_nk−u_nk∥+θ_nk∥u_nk∥ →0 as k→ ∞. (4.126) Thus, we obtain from (4.123) and (4.126) that

k→∞lim ∥x_nk+1 −x_nk∥= 0. (4.127) Since

xn_k is bounded, there exists a subsequence

xn_kj of

xn_k which converges weakly to some z ∈H₁, such that

lim sup

k→∞

⟨x^∗, x^∗−x_nk⟩= lim

j→∞⟨x^∗, x^∗−x_nkj⟩=⟨x^∗, x^∗ −z⟩. (4.128) Also, we obtain from (4.120), (4.123) and Lemma4.2.7 that z ∈Γ.

Thus, since x^∗ =P_Γ0, we obtain from (4.128) that lim sup

k→∞

⟨x^∗, x^∗−x_nk⟩=⟨x^∗, x^∗−z⟩ ≤0,

which implies from (4.127) that lim sup

k→∞

⟨x^∗, x^∗−x_nk+1⟩ ≤0. (4.129) Now, recall that dn_k :=

h 3αn_k

θ_nk(1−θn_k)∥xn_k−xn_k−1∥M2+ 2βn_k∥un_k−zn_k∥.∥x^∗−xn_k+1∥+ 2⟨x^∗, x^∗−x_nk+1⟩i

.

Thus, by (4.129), (4.101) and (4.125), we obtain lim sup

k→∞

d_nk ≤ 0. Hence, we get that

n→∞lim ∥x_n−x^∗∥= 0. Therefore, we conclude that{x_n} converges strongly to x^∗ =P_Γ0.

Some Corollaries

If we set B =NC in Algorithm 4.2.3, then we obtain from the definition of normal cone (see (2.5.44)) thatJ_λ^NC(I−λA) = P_C(I−λA).In this case, we know that (A+N_C)⁻¹(0) = V IP(C, A). Thus, we obtain the following consequent result.

Algorithm 4.2.9. Initialization: Choose sequences{θ_n}^∞_n=1, {β_n}^∞_n=1,{ϵ_n}^∞_n=1 and{ψ_n}^∞_n=1 such that the conditions from Assumption 4.2.2 (2) hold and let γ₁, λ₁ > 0, µ, η ∈ (0,1), α≥3 and x0, x1 ∈H1 be given arbitrarily.

Iterative Steps: Set n := 1.

Step 1. Given the iterates x_n−1 and x_n (n ≥1), choose α_n such that 0≤α_n ≤α¯_n, where

¯ α_n :=







min _n−1

n+α−1,_∥x ^ϵn

n−xn−1∥ , if x_n ̸=x_n−1

n−1

n+α−1, otherwise.

(4.130)

Step 2. Set

w_n =x_n+α_n(x_n−xn−1).

Then, compute

u_n =w_n−γ_nGw_n and y_n=P_C(I−λ_nA)u_n, (4.131) where

γ_n+1=

(minn _µ∥u

n−wn∥

∥Gu_n−Gw_n∥, γ_no

, if Gu_n̸=Gw_n,

γn, otherwise

(4.132)

and

λ_n+1 =

(minn _η∥u

n−y_n∥

∥Aun−Ayn∥, λ_no

, if Au_n ̸=Ay_n,

λ_n, otherwise.

(4.133)

Step 3. Compute

x_n+1 = (1−β_n−θ_n)u_n+β_nz_n, (4.134) where z_n:= (1−ψ_n)u_n+ψ_n

y_n+λ_n A(u_n)−A(y_n)

.

Stopping criterion: If y_n =u_n =w_n =x_n, then stop, otherwise, set n :=n+ 1 and go back to Step 1.

Corollary 4.2.10. Suppose that Assumption 4.2.2 holds with B = N_C and Γ := {z ∈ V IP(C, A) : T z ∈ F(S)}. Then, the sequence {x_n} generated by Algorithm 4.2.9 con- verges strongly to x^∗ ∈Γ, where ∥x^∗∥= min{∥z∥:z ∈Γ}.

If A≡0, then we obtain the following consequent result.

Algorithm 4.2.11. Initialization: Choose sequences {θ_n}^∞_n=1, {β_n}^∞_n=1, {ϵ_n}^∞_n=1, {ψ_n}^∞_n=1 such that the conditions from Assumption 4.2.2 (2) hold and letγ1 >0, µ∈(0,1), α≥3 and x₀, x₁ ∈H₁ be given arbitrarily.

Iterative Steps: Set n := 1.

Step 1. Given the iterates xn−1 and x_n (n ≥1), choose α_n such that 0≤α_n ≤α¯_n, where

¯ α_n :=







min _n−1

n+α−1,_∥x ^ϵn

n−xn−1∥ , if x_n ̸=xn−1 n−1

n+α−1, otherwise.

(4.135)

Step 2. Set

w_n =x_n+α_n(x_n−xn−1).

Then, compute

u_n=w_n−γ_nGw_n and y_n=J_λ^B

nu_n, (4.136)

where inf

n≥1λ_n≥λ >0 and γ_n+1=

(minn _µ∥u

n−wn∥

∥Gu_n−Gw_n∥, γ_no

, if Gu_n̸=Gw_n.

γn, otherwise

(4.137)

Step 3. Compute

x_n+1 = (1−β_n−θ_n)u_n+β_nz_n, (4.138) where zn:= (1−ψn)un+ψnyn.

Stopping criterion: If y_n =u_n =w_n =x_n, then stop, otherwise, set n :=n+ 1 and go back to Step 1.

Corollary 4.2.12. Suppose that Assumption 4.2.2 holds with A ≡ 0 and Γ := {z ∈ B⁻¹(0) : T z ∈ F(S)}. Then, the sequence {x_n} generated by Algorithm 4.2.11 converges strongly to x^∗ ∈Γ, where ∥x^∗∥= min{∥z∥:z ∈Γ}.

Remark 4.2.13. If we set H₁ =H₂ =H and T =I =S in Algorithms 4.2.3, 4.2.9 and 4.2.11, we recover relaxed inertial methods for solving the classical MVIP (see [17]), VIP (see [14]) and NPP (see [126]), respectively. Furthermore, under appropriate settings, Al- gorithms 4.2.3, 4.2.9and 4.2.11reduce to relaxed inertial methods for solving the classical SMVIP of Moudafi [171], SVIP of Censor et al. [54] and SCNPP, respectively.

In this section, we discuss the numerical behavior of our proposed methods; Algorithm 4.2.3, Algorithm 4.2.9 and Algorithm 4.2.11. Furthemore, we compare them with related strong convergent methods in the literature; namely, the method of Tian and Jiang [229]

and the method of Chidume and Nnakwe [64, Algorithm (3.1)] (other related methods in the literature are mostly weak convergent methods).

The codes for the numerical analysis are written in Matlab 2016 (b) and performed on a personal computer with an Intel(R) Core(TM) i5-2600 CPU at 2.30GHz and 8.00 Gb- RAM. In Tables 1-4, “Iter.” means the number of iterations while “CPU” means the CPU time in seconds. In our numerical computation, we randomly choose the relaxation stepsize ψ_n and the starting points x₀, x₁ ∈H₁ (see the cases below). We also choose the parameters (randomly) γ₁ >0, µ, η ∈ (0,1), α≥ 3 (the choices of these parameters will be discussed in Remark4.2.17), and the control sequencesθ_n= _n+1¹ , β_n= ¹₂−θ_n,α_n = ¯α_n and ϵ_n = _n^θ0.01ⁿ for Algorithm 4.2.3, Algorithm 4.2.9, Algorithm 4.2.11; λ_n = _2L¹ for Algo- rithm Tian and Jiang [229], Chidume and Nnakwe [64, Algorithm (3.1)]; λ_n = _3n+7²ⁿ for Algorithm 4.2.11; γn = _2||T¹_||2 for Algorithm Tian and Jiang [229], Chidume and Nnakwe [64, Algorithm (3.1)]; and α_n= _n+1¹ for Algorithm Tian and Jiang [229].

Furthermore, we define TOL_n:= ¹₂ ||x_n−J_λ^B(x_n−λAx_n)||²+||T x_n−ST x_n||²

for Algo- rithm 4.2.3; TOL_n := ¹₂(||x_n−P_C(x_n−λAx_n)||²+||T x_n−ST x_n||²) for Algorithm 4.2.9, Algorithm Tian and Jiang [229], Chidume and Nnakwe [64, Algorithm (3.1)];

Number of iterations

0 100 200 300 400 500

SNR

5 10 15 20 25 30 35

Algorithm 3.2 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Original Cameraman Blurred Cameraman

Algorithm 3.2 Tian & Jiang Alg. Chidume Alg.

Number of iterations

0 100 200 300 400 500

SNR

5 10 15 20 25 30

Algorithm 3.2 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Original MRI Blurred MRI

Algorithm 3.2 Tian & Jiang Alg. Chidume Alg.

Figure 4.3. Comparison for Numerical results for image restoration problem 4.140: Top:

Cameraman; Bottom: MRI.

TOL_n:= ¹₂ ||x_n−J_λ^Bx_n||²+||T x_n−ST x_n||²

for Algorithm4.2.11; and use the stopping criterion TOL_n < ε for the iterative processes, where ε is the predetermined error. Note that if TOL_n= 0, then x_n is a solution of the problem under consideration.

Example 4.2.14. Let H₁ =H₂ =L₂([0,1]) be equipped with the inner product

⟨x, y⟩= Z 1

0

x(t)y(t)dt ∀ x, y ∈L₂([0,1]) and ∥x∥:=

s Z 1

0

|x(t)|²dt ∀x, y,∈L₂([0,1]).

Now, define the operators A, B :L₂([0,1])→L₂([0,1]) by Ax(t) =

Z 1 0

x(t)−

2tse^t+s e√

e²−1

cosx(s)

!

ds+ 2te^t e√

e²−1, x∈L2([0,1]), Bx(t) = max{0, x(t)}, t∈[0,1].

Then A is Lipschitz continuous and monotone, and B is maximal monotone on L₂([0,1]).

Let T :L₂([0,1]) →L₂([0,1]) be defined by T x(s) =

Z 1 0

κ(s, t)x(t)dt ∀ x∈L₂([0,1]),

where κ is a continuous real-valued function defined on[0,1]×[0,1]. Then, T is a bounded linear operator with adjoint

T^∗x(s) = Z 1

0

κ(t, s)x(t)dt ∀ x∈L₂([0,1]).

Let S:L₂([0,1])→L₂([0,1]) be defined by Sx(t) =

Z 1 0

tx(s)ds, t∈[0,1].

Then, S is nonexpansive. Indeed, we have

|Sx(t)−Sy(t)|² =| Z 1

0

t x(s)−y(s)

|² ≤ Z 1

0

t|x(s)−y(s)|ds

!2

≤ Z 1

0

|x(s)−y(s)|²ds=∥x−y∥².

Now, let C = {x ∈ L₂([0,1]) : ⟨y, x⟩ ≤ b}, where y = e^t + 1 and b = 10, then C is a nonempty closed and convex subset of L₂([0,1]). Thus, we define the metric projection P_C as:

P_C(x) =

(_{b−⟨y,x⟩}

||y||² y+x, if ⟨y, x⟩> b, x, if ⟨y, x⟩ ≤b.

For Algorithm Tian and Jiang [229], we define h:L₂([0,1])→L₂([0,1]) by hx(t) =

Z 1 0

t

2x(s)ds, t ∈[0,1].

Then, h is a contraction.

We now consider the following cases for the relaxation stepsize ψ_n and the starting points x₀, x₁:

Case 1: Take x₁(t) = t³−3, x₀(t) =t and ψ_n = _2n+5ⁿ . Case 2: Take x₁(t) = sin(t), x₀(t) =t+ 1 and ψ_n= _n+10ⁿ . Case 3: Take x₁(t) = cot(t), x₀(t) = sin(t) and ψ_n= _n+10ⁿ . Case 4: Take x₁(t) = e^t, x₀(t) =t²+t+ 1 and ψ_n= _18n+1²ⁿ .

The numerical results are displayed in Table 4.2.14 and Figure 4.4.

Table 4.2.14. Numerical results for Example 4.2.14 with ε= 10⁻⁷.

Cases Algorithm

4.2.9

Algorithm

4.2.11 Tian Jiang Chidume

Nnakwe

1 CPU

Iter.

15.8066 64

7.2564 51

21.7214 114

35.9767 158

2 CPU

Iter.

17.8827 39

7.6196 32

61.6392 96

137.3976 140

3 CPU

Iter.

20.4864 42

9.6206 35

55.5797 102

144.6658 145

4 CPU

Iter.

22.8433 82

10.4064 63

46.0686 109

56.3887 153

Example 4.2.15. Let H₁ = H₂ =ℓ₂, where ℓ₂ = ℓ₂(R) := {x = (x₁, x₂, ..., x_i, ...), x_i ∈ R :

∞

P

i=1

|x_i|² < ∞}, with inner product ⟨., .⟩ : ℓ₂ ×ℓ₂ → R defined by ⟨x, y⟩ := P∞ i=1x_iy_i and the norm ∥.∥ :ℓ₂ → R by ∥x∥ :=pP∞

i=1|x_i|², where x ={x_i}^∞_i=1 and y= {y_i}^∞_i=1. Define the mapping A :ℓ₂ → ℓ₂ by Ax =

x1+|x1|

2 ,^x2+|x₂ ^2|, ...,^xi+|x₂ ^i|, ...

∀x ∈ℓ₂. Then, A is Lipschitz continuous and monotone (see [14]).

Let B : ℓ₂ →ℓ₂ be defined by B(x) = (2x₁,2x₂, ...,2x_i, ...) ∀x∈ℓ₂. Then, B is maximal monotone.

Let T :ℓ₂ →ℓ₂ be defined by T x= 0, x₁,^x₂²,^x₃³, ...

, for all x∈ ℓ₂. Then, T is a bounded linear operator on ℓ₂ with adjoint T^∗y= y₂,^y₂³,^y₃⁴, ...

for all y∈ℓ₂.

Now, C = {x ∈ ℓ₂ : ||x−a||_ℓ2 ≤ r}, where a = (1,¹₂,¹₃,· · ·) and r = 3. Then C is a

Number of iterations

0 20 40 60 80 100 120 140 160

TOLn

10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2 10^-1 10⁰ 10¹

Algorithm 3.2 Algorithm 4.4 Algorithm 4.6 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Number of iterations

0 20 40 60 80 100 120 140

TOLn

10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2 10^-1 10⁰

Algorithm 3.2 Algorithm 4.4 Algorithm 4.6 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Number of iterations

0 50 100 150

TOLn

10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2 10^-1 10⁰

Algorithm 3.2 Algorithm 4.4 Algorithm 4.6 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Number of iterations

0 20 40 60 80 100 120 140 160

TOLn

10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2 10^-1 10⁰ 10¹

Algorithm 3.2 Algorithm 4.4 Algorithm 4.6 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Figure 4.4. The behavior of TOL_n with ε = 10⁻⁷ for Example 4.2.14: Top Left: Case 1;

Top Right: Case 2; Bottom Left: Case 3; Bottom Right: Case 4.

nonempty closed and convex subset of ℓ₂. Thus, P_C(x) =

(x, if ||x−a||ℓ2 ≤r,

x−a

||x−a||_ℓ

2

r+a, otherwise.

Furthermore, we define the mappings S, h : ℓ₂ → ℓ₂ by Sx = (0, x₁, x₂, . . .) and hx = 0, ^x₂¹, ^x₂²,· · ·

for all x∈ℓ₂, and consider the following cases for the relaxation stepsize ψ_n and the starting points x₀, x₁:

Case 1: Take x₁ = (1,¹₂,¹₃,· · ·), x₀ = (¹₂,¹₅,₁₀¹,· · ·) and ψ_n= _2n+5ⁿ . Case 2: Take x₁ = (¹₂,¹₅,₁₀¹,· · ·), x₀ = (1,¹₂,¹₃,· · ·) and ψ_n= _n+10ⁿ . Case 3: Take x1 = (1,¹₄,¹₉,· · ·), x0 = (¹₂,¹₄,¹₈,· · ·) and ψn = _n+10ⁿ . Case 4: Take x₁ = (¹₂,¹₄,¹₈,· · ·), x₀ = (1,¹₄,¹₉,· · ·) and ψ_n = _18n+1²ⁿ . The numerical results are displayed in Table 4.2.15 and Figure 4.5.

Table 4.2.15. Numerical results for Example 4.2.15 with ε= 10⁻⁸.

Cases Algorithm

4.2.9

Algorithm

4.2.11 Tian Jiang Chidume

Nnakwe

1 CPU

Iter.

0.0566 76

0.0325 56

1.0297 449

1.0889 509

2 CPU

Iter.

0.0449 67

0.0208 51

1.0290 449

1.0679 506

3 CPU

Iter.

0.0394 74

0.0223 55

1.0268 449

1.0719 510

4 CPU

Iter.

0.0389 83

0.0199 60

1.0290 449

1.0676 510

Example 4.2.16. In many practical problems, it is important if the minimum-norm so- lutions of such problems can be found. Such problems can be formulated mathematically as (see, for instance [206, Example 3.4]):

Find x^∗ ∈H such that ∥x^∗∥= min{∥x∥:x∈H}, (4.141) where H is a real Hilbert space. Note that (4.141) can be reformulated as the following particular variational inequality problem:

Find x^∗ ∈H such that ⟨x^∗, x^∗−x⟩ ≤0, ∀x∈H. (4.142) Now, let H₁ =H₂ =L₂([a, b]), C ={x ∈L₂([a, b]) : ⟨a, x⟩=b} and Q ={x∈ L₂([a, b]) :

⟨a, x⟩ ≥b}, for some a∈L₂([a, b])− {0} and b ∈R.

Number of iterations

0 100 200 300 400 500 600

TOLn

10^-10 10^-8 10^-6 10^-4 10^-2 10⁰ 10²

Algorithm 3.2 Algorithm 4.4 Algorithm 4.6 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Number of iterations

0 100 200 300 400 500 600

TOLn

10^-10 10^-8 10^-6 10^-4 10^-2 10⁰ 10²

Algorithm 3.2 Algorithm 4.4 Algorithm 4.6 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Number of iterations

0 100 200 300 400 500 600

TOLn

10^-10 10^-8 10^-6 10^-4 10^-2 10⁰ 10²

Algorithm 3.2 Algorithm 4.4 Algorithm 4.6 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Number of iterations

0 100 200 300 400 500 600

TOLn

10^-10 10^-8 10^-6 10^-4 10^-2 10⁰ 10²

Algorithm 3.2 Algorithm 4.4 Algorithm 4.6 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Figure 4.5. The behavior of TOL_n with ε = 10⁻⁸ for Example 4.2.15: Top Left: Case 1;

Top Right: Case 2; Bottom Left: Case 3; Bottom Right: Case 4.

Then,x^∗ minimizes||.||+δ_C if and only if0∈∂(||.||+δ_C) (x^∗)andT x^∗ minimizes||.||+δ_Q if and only if 0 ∈ ∂(||.||+δ_Q) (T x^∗), where δ_C (defined as δ_C(x) = 0 if x ∈ C and +∞, otherwise) and δ_Q denote the indicator functions of C and Q, respectively. Now, if we set in (4.84), A = 0, B = (||.||+δ_C) and F(S) = ∂(||.||+δ_Q), then problem (4.84) becomes the following problem:

Find x^∗ ∈C such that x^∗ =argmin{∥x∥:x∈C}, (4.143) and such that

T x^∗ ∈Q solves T x^∗ =argmin{∥y∥:y∈Q}. (4.144) It is known that the solution to problem (4.143)-(4.144) is a minimum-norm solution (see [206, Example 3.4]).

Now, for the numerical computation, we define the bounded linear operatorT as in Example 4.2.14. We also choose the starting points and relaxation stepsize as in Case 1-Case 4 of Example 4.2.14. The numerical results are reported in Table 4.2.16 and Figure 4.6.

Table 4.2.16. Numerical results for Example 4.2.16 with ε= 10⁻⁸.

Cases Algorithm

4.2.9

Algorithm

4.2.11 Tian Jiang Chidume

Nnakwe

1 CPU

Iter.

19.8207 121

15.4169 101

29.0280 136

50.9212 182

2 CPU

Iter.

26.5295 92

25.3138 83

87.9523 119

187.6199 165

3 CPU

Iter.

41.3551 95

34.5019 93

82.0133 125

204.0348 170

4 CPU

Iter.

25.2010 110

22.7278 99

77.0343 132

96.5969 178

Remark 4.2.17. We now summarize this section by highlighting some of the observations from the numerical results.

ˆ During the numerical computations, we observed that irrespective of the choices of the parameters γ1, λ1 > 0, µ, η ∈ (0,1) and α ≥ 3, the number of iteration does not change and no significant difference in the CPU time. Therefore, we randomly choose these parameters.

ˆ Throughout the experiments, we see clearly that both in CPU time and number of iterations, Algorithm 4.2.11 outperforms Algorithm 4.2.3and Algorithm 4.2.9. This is expected since Algorithm 4.2.11 does not require any evaluation of A and the stepsize λ_n in Algorithm 4.2.11 does not involve any form of “inner loop” during implementation, which are required in Algorithm 4.2.3 and Algorithm 4.2.9.

Number of iterations

0 50 100 150 200

TOLn

10^-10 10^-8 10^-6 10^-4 10^-2 10⁰ 10²

Algorithm 4.4 Algorithm 4.6 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Number of iterations

0 50 100 150 200

TOLn

10^-9 10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2 10^-1 10⁰

Algorithm 4.4 Algorithm 4.6 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Number of iterations

0 50 100 150 200

TOLn

10^-9 10^-8 10^-7 10^-6 10^-5 10^-4 10^-3 10^-2 10^-1 10⁰

Algorithm 4.4 Algorithm 4.6 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Number of iterations

0 50 100 150 200

TOLn

10^-10 10^-8 10^-6 10^-4 10^-2 10⁰ 10²

Algorithm 4.4 Algorithm 4.6 Tian & Jiang Alg.

Chidume & Nnakwe Alg.

Figure 4.6. The behavior of TOL_n with ε = 10⁻⁸ for Example 4.2.16: Top Left: Case 1;

Top Right: Case 2; Bottom Left: Case 3; Bottom Right: Case 4.

ˆ It can also be seen from the tables and figures that Algorithm 4.2.3 and Algorithm 4.2.9 perform better than Algorithm Tian and Jian [229] and Algorithm (3.1) of Chidume and Nnakwe [64]. This is also expected due to the presence of both re- laxation and inertial steps in our methods. We can also see from the cases that irrespective of the choices of the relaxation stepsize, our proposed methods converge more than twice as fast as the other methods.

Chapter 5

Inertial Type Algorithms for Solving

Variational Inequality Problems and

Fixed Point Problems

Assumption 5.2.1. Suppose that the following conditions hold:

1. The set C is defined by

C ={x∈H | c(x)≤0};

where c:H →R satisfies the following conditions:

(a) c : H → R is a continuously differentiable convex function such that c^′(.) is M-Lipschitz continuous;

(b) c(x) is weakly lower semicontinuous (w-lsc) on H.

2. (a) A : H → H is monotone and L-Lipschitz continuous with constant L > 0.

However, prior knowledge of the Lipschitz constant is not required for the im- plementation of the proposed algorithm.

(b) T : H → H is a K-Lipschitz continuous quasi-pseudo-contractive mapping, which is demiclosed at zero and with K ≥1.

(c) The solution set Ω = F(T)∩VI(C, A) is nonempty.

(d) D:H →H is a strongly positive bounded linear operator with coefficientγ¯ and f :H →H is a contraction with coefficient ρ∈(0,1) such that 0< γ < ^γ¯_ρ. 3. {α_n}^+∞_n=1, {β_n}^+∞_n=1 and {γ_n}^+∞_n=1 are positive sequences satisfying the following con-

ditions:

(a) α_n ∈ (0,1), lim

n→+∞α_n = 0, P+∞

n=1α_n = +∞, lim

n→+∞

γ_n

α_n = 0,0 < c₁ ≤ β_n ≤ c₂ <1.

(b) {ϕ_n} ⊂(0,1] such that lim

n→+∞ϕ_n=ϕ∈(0,1],0< ξ < µ < ¹

1+√ 1+K². (c) 0< δ <p

(1−ϕ)(2−ϕ) + (1−κ²)−(1−ϕ) for some κ >0.

Algorithm 5.2.2.

Initialization: Choose sequences {α_n}^+∞_n=1, {β_n}^+∞_n=1 and{γ_n}^+∞_n=1 such that the conditions from Assumption 5.2.1(3) hold and let λ₁ >0, α≥3and x₀, x₁ ∈H be given arbitrarily.

Set n = 1.

Iterative Steps:

Step 1. Given the iterates x_n−1 and x_n(n≥1), choose θ_n such that 0≤θ_n ≤θ¯_n, where θ¯_n=

(min _n−1

n+α−1,_∥x ^γn

n−xn−1∥ , if x_n ̸=xn−1,

n−1

n+α−1, otherwise. (5.1)

Step 2 Compute

w_n=x_n+θ_n(x_n−x_n−1),

Step 3 Construct the half-space

C_n ={x∈H :c(w_n) +⟨c^′(w_n), x−w_n⟩ ≤0}.

and compute

yn =PCn(wn−λnAwn), where

λ_n+1 =











minn _δ∥w

n−yn∥

∥Awn−Ayn∥+∥c^′(wn)−c^′(yn)∥, λ_no

, if ∥Aw_n−Ay_n∥

+∥c^′(w_n)−c^′(y_n)∥ ̸= 0,

λ_n, otherwise.

(5.2)

Step 4. Compute

tn := (1−ϕn)wn+ϕn

yn+λn Awn−Ayn

. Step 5. Compute

x_n+1 =α_nγf(w_n) + (I −α_nD)

(1−β_n)t_n+β_nVt_n , where

V= (1−ξ)I+ξT (1−µ)I+µT . Set n =n+ 1 and go back to Step 1.

Remark 5.2.3. (a) Our method only requires one projection onto a half-space Cn, which can be easily computed. Hence, our method is less computationally expensive than several other methods in the literature (see for example, [44, 104]), for solving variational inequal- ity problems. By the definition of the subgradient, we observe that C ⊂Cn.

(b) The step size {λ_n}given by (5.2) is generated at each iteration by some simple compu- tations. Thus, {λ_n} is easily implemented and does not depend on the Lipschitz constant of the cost operator.

(c) Step 1of Algorithm 5.2.2 is also easily implemented since the value of∥x_n−xn−1∥ is known before choosing θ_n.

Remark 5.2.4. Observe that from (5.2) in Algorithm 5.2.2, we have λ_n+1 ≤ λ_n ∀n ≥1.

Moreover, since A is L-Lipschitz continuous and c^′ is K-Lipschitz continuous, we obtain in the case of ∥Awn−Ayn∥+∥c^′(wn)−c^′(yn)∥ ̸= 0 in Algorithm 5.2.2, that

λ_n+1 = min

δ∥w_n−y_n∥

∥Aw_n−Ay_n∥+∥c^′(w_n)−c^′(y_n)∥, λ_n

≥min δ

L+K, λ_n

, which by induction, implies that {λ_n} is bounded below by min _δ

L+K, λ₁ . Since {λ_n} is monotone nonincreasing, it follows that the limit exists, and lim

n→+∞λn ≥ min{_L+K^δ , λ1}>

0.

Remark 5.2.5. By Assumption (5.2.1) 3(a), one can easily verify from (5.1) that

n→+∞lim θn||xn−xn−1||= 0 and lim

n→+∞

θn

α_n||xn−xn−1||= 0. (5.3) Lemma 5.2.6. Let p ∈ Ω and {x_n} be a sequence generated by Algorithm 5.2.2 under Assumption 5.2.1. Then, we have the following inequality:

∥t_n−p∥² ≤ ∥w_n−p∥²−ϕ_n

2−ϕ_n−δ² λ²_n

λ²_n+1 −2(1−ϕ_n)δ λ_n

λ_n+1 −ρ²

∥w_n−y_n∥². (5.4) Proof. Let p∈ Ω. Then, p ∈C ⊂ C_n. Since y_n =P_Cn(w_n−λ_nAw_n) and p ∈C_n, then by the characterization of PCn we have

⟨w_n−λ_nAw_n−y_n, y_n−p⟩ ≥0, or equivalently

2⟨wn−yn, yn−p⟩ −2λn⟨Awn−Ayn, yn−p⟩ −2λn⟨Ayn, yn−p⟩ ≥0. (5.5) Next, applying Lemma 2.5.18(i), we get

2⟨w_n−y_n, y_n−p⟩=∥w_n−p∥²− ∥w_n−y_n∥² − ∥y_n−p∥². (5.6) By the monotonicity of A, we obtain

⟨Ay_n, y_n−p⟩=⟨Ay_n−Ap, y_n−p⟩+⟨Ap, y_n−p⟩

≥ ⟨Ap, y_n−p⟩. (5.7)

Substituting (5.6) and (5.7) in (5.5), we have

∥y_n−p∥² ≤ ∥w_n−p∥²− ∥w_n−y_n∥²−2λ_n⟨Aw_n−Ay_n, y_n−p⟩+ 2λ_n⟨Ap, p−y_n⟩.

(5.8) Next, using the definition of t_n and Lemma 2.5.18, we obtain

∥t_n−p∥² =∥(1−ϕ_n)w_n+ϕ_ny_n+ϕ_nλ_n(Aw_n−Ay_n)−p∥²

=∥(1−ϕ_n)(w_n−p) +ϕ_n(y_n−p) +ϕ_nλ_n(Aw_n−Ay_n)∥²

= (1−ϕ_n)²∥w_n−p∥²+ϕ²_n∥y_n−p∥²+ϕ²_nλ²_n∥Aw_n−Ay_n∥²

+ 2ϕ_n(1−ϕ_n)⟨w_n−p, y_n−p⟩+ 2λ_nϕ_n(1−ϕ_n)⟨w_n−p, Aw_n−Ay_n⟩ + 2λ_nϕ²_n⟨y_n−p, Aw_n−Ay_n⟩

= (1−ϕn)²∥wn−p∥²+ϕ²_n∥yn−p∥²+ϕ²_nλ²_n∥Awn−Ayn∥² +ϕ_n(1−ϕ_n)

∥w_n−p∥²+∥y_n−p∥²− ∥w_n−y_n∥²

+ 2λ_nϕ_n(1−ϕ_n)⟨w_n−p, Aw_n−Ay_n⟩+ 2λ_nϕ²_n⟨y_n−p, Aw_n−Ay_n⟩

= (1−ϕn)∥wn−p∥² +ϕn∥yn−p∥²−ϕn(1−ϕn)∥wn−yn∥² +ϕ²_nλ²_n∥Aw_n−Ay_n∥²

+ 2λ_nϕ_n(1−ϕ_n)⟨w_n−p, Aw_n−Ay_n⟩+ 2λ_nϕ²_n⟨y_n−p, Aw_n−Ay_n⟩. (5.9)

Using (5.8) in (5.9), we get

∥t_n−p∥² ≤(1−ϕ_n)∥w_n−p∥²+ϕ_n

∥w_n−p∥²− ∥w_n−y_n∥²−2λ_n⟨Aw_n−Ay_n, y_n−p⟩

+ 2λ_n⟨Ap, p−y_n⟩

−ϕ_n(1−ϕ_n)∥w_n−y_n∥²+ϕ²_nλ²_n∥Aw_n−Ay_n∥² + 2ϕn(1−ϕn)λn⟨wn−p, Awn−Ayn⟩+ 2ϕ²_nλn⟨yn−p, Awn−Ayn⟩

=∥w_n−p∥²−ϕ_n(2−ϕ_n)∥w_n−y_n∥²+ϕ²_nλ²_n∥Aw_n−Ay_n∥²

+ 2λ_nϕ_n(1−ϕ_n)⟨Aw_n−Ay_n, w_n−y_n⟩+ 2λ_nϕ_n⟨Ap, p−y_n⟩. (5.10) Now, we consider the following two cases:

Case 1: Ap= 0. If Ap= 0, then from (5.10), we have

∥t_n−p∥² ≤ ∥w_n−p∥²−ϕ_n(2−ϕ_n)∥w_n−y_n∥² +ϕ²_nλ²_n∥Aw_n−Ay_n∥² + 2λ_nϕ_n(1−ϕ_n)⟨Aw_n−Ay_n, w_n−y_n⟩

≤ ∥w_n−p∥²−ϕ_n(2−ϕ_n)∥w_n−y_n∥² +ϕ_nδ² λ²_n

λ²_n+1∥w_n−y_n∥² + 2ϕn(1−ϕn)δ λ_n

λ_n+1∥w_n−yn∥²

=∥w_n−p∥²−ϕ_n

2−ϕ_n−δ² λ²_n

λ²_n+1 −2(1−ϕ_n)δ λ_n λ_n+1

∥w_n−y_n∥²

≤ ∥w_n−p∥²−ϕ_n

2−ϕ_n−δ² λ²_n

λ²_n+1 −2(1−ϕ_n)δ λ_n λn+1

−κ²

∥w_n−y_n∥², for some κ >0,which is the required inequality.

Case 2: Ap̸= 0.

By Lemma 2.5.11, it follows that p∈∂C and there exists κ >0 such that Ap=−κc^′(p).

Observe that c(p) = 0 since p∈∂C. By the subdifferential inequality (2.54), we get c(y_n)≥c(p) +⟨c^′(p), y_n−p⟩

= −1

κ ⟨Ap, y_n−p⟩, which implies that

⟨Ap, p−y_n⟩ ≤κc(y_n). (5.11) Since yn∈Cn, we get

c(w_n) +⟨c^′(w_n), y_n−w_n⟩ ≤0. (5.12) Again, by applying the subdifferential inequality (2.54), we have

c(y_n) +⟨c^′(y_n), w_n−y_n⟩ ≤c(w_n). (5.13) Adding (5.12) and (5.13), we have

c(y_n)≤ ⟨c^′(y_n)−c^′(w_n), y_n−w_n⟩. (5.14)