Convergence of partially asynchronous block
quasi-Newton methods for nonlinear systems of equations
Jian-Jun Xu∗
Department of Mathematics, Temple University, Philadelphia, PA 19122, United States Received 4 May 1998; received in revised form 7 October 1998
Abstract
In this paper, a partially asynchronous block Broyden method is presented for solving nonlinear systems of equations of the formF(x) = 0. Sucient conditions that guarantee its local convergence are given. In particular, local convergence is shown when the JacobianF′(x∗) is an H-matrix, wherex∗ is the zero point ofF. The results are extended to Schubert’s method. A connection with discrete Schwarz alternating procedure is also shown. c1999 Elsevier Science B.V. All rights reserved.
Keywords:Parallel iterative methods; Asynchronous iterations; Quasi-Newton methods; Schwarz alternating procedure
1. Introduction
Chazan and Miranker rst introduced an asynchronous method for the solution of nonsingular linear system of equations of the form Ax=b [9]. They proved that the asynchronous point method converges to the solution if and only if A is an H-matrix [29]. This idea was extended to nonlinear systems of equations, and there is now a considerable understanding on the convergence properties of asynchronous iterative methods for nonlinear problems. Many authors have concentrated on xed point problems of the form x=G(x). Convergence properties have been shown if G satises some contraction properties; see, e.g., [3–5, 13]. The xed point mapping G does not always arise directly from the formulation of physical problems, however, and when it does, the contracting properties are not always easily veriable.
In this paper, we consider the problem of nding x∗ which satises the following nonlinear system
of equations:
F(x) = 0; (1.1)
where F= (f1; : : : ; fn)T is a nonlinear operator from Rn into itself.
∗E-mail: [email protected].
In the literature, one can nd two general approaches for the asynchronous solutions of (1.1). First, for an asynchronous implementation of Newton’s method, one can use two processes, one updating the iterates, and the other evaluating the Jacobian matrix; see [3, 7, 20]. Also see [31] for the analogue of Broyden’s method. More recently, asynchronous block methods, or more general multisplitting methods were proposed. In [2, 6] the nonlinear term was treated explicitly because of the weak nonlinearity. In [14, 22], convergence properties were shown where F is an M-function, see also [12, 15] for point methods. In [12, 14, 15, 22], exact solutions for subproblems were assumed. The use of Newton-type methods for approximate solutions of the subproblems and local convergence theorems for the corresponding asynchronous methods were given in [1, 27, 32].
In this paper, we consider partially asynchronous block quasi-Newton methods. Suppose F and x in (1.1) are conformally partitioned as follows:
F= (FT
1; : : : ; FLT)T; x= (xT1; : : : ; xLT)T;
where Fi:Rn→Rni; Fi= (fi1; : : : ; fini)
T; x
i∈Rni; xi= (xi1; : : : ; xini)
T; i= 1; : : : ; L: Let S
i={i1; : : : ; ini},
the partition chosen is such that SL
i=1Si={1; : : : ; n}; Si∩Sj=∅; i6=j; i; j= 1; : : : ; L. This partition
may correspond to a decomposition of the domain of a nonlinear partial dierential equations, see Section 5.
The system (1.1) can be rewritten as
Fl(x1; : : : ; xl; : : : ; xL) = 0; l= 1; : : : ; L: (1.1)′
We consider the following nonlinear block method. Given initial values of x= (xT
1; : : : ; xTL)T, repeat
the following procedure until convergence: For l= 1; : : : ; L
Solve for yl in Fl(x1; : : : ; xl−1; yl; xl+1; : : : ; xL) = 0;
for all l∈S⊆{1; : : : ; L};
Set xl=yl for all l∈S:
(1.2)
In (1.2) the subset S can be chosen in a dynamic fashion. The classical nonlinear block-Jacobi method and nonlinear block-Gauss–Seidel method [4, 23] are two examples of such selections. For the purpose of parallel processing, the nonlinear block Jacobi method is nearly ideal, since up to L processors can each perform one of the iterations in (1.2) with the others simultaneously. It is synchronous in the sense that to begin the computation of the next iterate, each processor has to wait until all processors have completed their current iteration. In practice we need to add a synchronization mechanism to ensure that the algorithm is carried out correctly. The time necessary to carry out the synchronization mechanism, as well as the time a processor must wait until its data is ready to continue the calculation, adds an overhead to the computation. By removing this synchronization and letting the processors continue their calculations according to the information currently available, we obtain the so-called asynchronous parallel methods; see, e.g., [3, 4, 9].
The advantages of quasi-Newton methods for (1.1) are that they involve evaluating only O(n) scalar functions in each iteration step, which is much cheaper than Newton’s method, they do not involve computation of the derivative of F, and they still have superlinear convergence; see, e.g. [10, 11] and the references given therein. In this paper, we consider the use of quasi-Newton methods instead of Newton’s method. We will consider two quasi-Newton methods: Broyden’s method [8] and Schubert’s method [24].
The main purpose of this paper is to give sucient conditions which guarantee local convergence of partially asynchronous block quasi-Newton methods. To this end, we assume in the rest of the paper that F and the partition satisfy the following conditions:
F is continuously dierentiable on an open convex set ⊂Rn: (1.3)
There exists a constant ¿0 such that
kF′(x)−F′(x∗)k6kx−x∗k; for x∈: (1.4)
All the matrices @Fi(x∗)=@xi; i= 1; : : : ; L; are nonsingular, moreover
(|D(x∗)−1(F′(x∗)−D(x∗))|)¡1; (1.5)
where D(x∗) = diag(@F
1(x∗)=@x1; : : : ; @FL(x∗)=@xL) is a block diagonal matrix, and we use the
nota-tion
@Fi(x)
@xj
=
@fi1(x)
@xj1
· · · @fi1(x)
@xjnj
· · · ·
@fini(x)
@xj1
· · · @fini(x)
@xjnj
i; j= 1; : : : ; L;
and
F′(x) =
@F1(x)
@x1
· · · @F1(x)
@xL
· · · ·
@FL(x)
@x1
· · · @FL(x)
@xL
:
Remark 1. Conditions (1.3) and (1.4) are standard. Condition (1.5) is natural for the convergence. Consider the linear case, i.e., F(x) =Ax−b, and let F and x be partitioned as before. Suppose that
A−1 exists. Since F′(x) =A, condition (1.5) is necessary and sucient for the convergence of the
asynchronous block method for the linear equations F(x) = 0; see, e.g., [4, 9].
Remark 2. Condition (1.5) holds when the Jacobian matrix F′(x∗) is an H-matrix, since F′(x∗) =
D(x∗)−(D(x∗)−F′(x∗)) is an H-splitting of F′(x∗); see e.g., [5, 15]. On the other hand, if each
H-matrix. The class of H-matrices includes M-matrices, and also the class of strictly or irreducibly diagonally dominant matrices [29].
In Section 2, we present both the computational and mathematical models of partially asynchronous block Broyden’s method. The local convergence theorem is proved in Section 3 under conditions (1.3)–(1.5). In Section 4, we point out that local convergence is still guaranteed if Schubert’s method [24] is used instead of Broyden’s method. The connection of the block method with the Schwarz alternating procedure (see, e.g., [18, 19, 25, 30]) is discussed in Section 5.
The weighted maximum norms is used as an important tool in proofs of asynchronous iterative methods; see, e.g., [3, 4, 6, 9, 13, 17, 26, 27]. Given an vectorw∈Rn; w ¿0. Letw= (wT
1; : : : ; wTL)T
be partitioned conformally with x, then we dene
kxkw= max{kxikwi;16i6L}= max
(
|xij|
wij
;16j6ni;16i6L
)
:
Given a matrix A∈Rn×n. Let A= (A
ij)L×L be partitioned conformally with F′(x), then the norms
k · kw;k · kwi induce the following matrix norm:
kAkw= max
kAxk
w
kxkw
: x6= 0; x∈Rn
:
We extend the weighted maximum norms for rectangular matrices as follows:
k(Ai1; : : : ; AiL)kwi= max
k(A
i1; : : : ; AiL)xkwi kxkw
: x6= 0; x∈Rn
and
kAijkwi= max
(
kAijxjkwi kxjkwj
: xj6= 0; xj∈Rnj
)
:
We immediately have
kAii(Ai1; : : : ; AiL)kwi6kAiikwi· k(Ai1; : : : ; AiL)kwi; (1.6)
k|A|kw=kAkw; k|(Ai1; : : : ; AiL)|kwi=k(Ai1; : : : ; AiL)kwi; (1.7)
kAkw= max{k(Ai1; : : : ; AiL)kwi;16i6L}; (1.8)
and
kAijkwi6k(Ai1; : : : ; AiL)kwi; 16i; j6L: (1.9)
(1:5)′ All matrices @F
i(x∗)=@xi; i= 1; : : : ; L; are nonsingular, moreover, there exists 0¡1, and a
vector w ¿0 such that kD(x∗)−1(F′(x∗)−D(x∗))k
w¡ 0:
2. Partially asynchronous block Broyden method
Broyden’s method [8] for (1.1) is as follows:
x(k+ 1) =x(k)−B(k)−1F(x(k));
B(k+ 1) =B(k) + (1=z(k)Tz(k))·(y(k)−B(k)z(k))z(k)T; k= 1;2; : : : ;
where y(k) =F(x(k+ 1))−F(x(k)); z(k) =x(k+ 1)−x(k);
where x(0) and B(0) are initial approximation to the solution x∗ and initial approximation the Jacobian F′(x∗), respectively.
We will rst describe a computational model for partially asynchronous block Broyden’s method, i.e., how the computer is actually programmed to execute its instructions, then we give a mathematics model, i.e., how the execution is described mathematically in order to analyze convergence [26].
Let us as before assume that there are L processors. We design L processes. Each processor perform computation in one process. In each process, say the lth process, we use one step of Broyden’s method to update the lth block xl in the lth subproblem in (1.2), l= 1; : : : ; L. More
precisely, the computational model can be written as the following pseudo-code: Given an initial guess x, and initial matrices Bl; l= 1; : : : ; L.
Process l(l= 1; : : : ; L) Initialization for x; Bl
Step 1: Compute Fl:=Fl(x);
Step 2: Set x:=xl; Fl:=Fl;
Step 3: Compute xl:=xl−B−l 1Fl;
Step 4: Send xl to the other processes;
Step 5: Check the termination criterion, if satised then stop;
Step 6: Receive the values of x (except xl) from the other processes;
Step 7: Compute Fl:=Fl(x);
Step 8: Compute z:=xl−xl; y:=Fl−Fl;
Step 9: Compute Bl:=Bl+ (1=zTz)·(y−Blz)zT;
Step 10: Goto step 2;
where Bl;Fl;x; y; z are local variables, only x is a global variable. In each cycle of a process, say
the lth process, one step of Broyden’s update is performed for a subproblem of nl dimension. The
L processes can be done in parallel. There is no explicit synchronization among the processes.
Remark 3. Since Broyden update is a rank-one update, the Sherman–Morrison formula can be used to update the inverse of Bl. Let Hl=Bl−1, then initialization for Bl is changed forHl, Steps 3 and 9
are rewritten as follows while the other steps remain unchanged: Step 3′: Compute x
l:=xl−Hl·Fl;
Step 9′: H
One can also uses QR factorization technique to solve the linear system in step 3; see, e.g., [11]. We give the following mathematical model in order to analyze the convergence.
Let the initial approximations x(0) = (x1(0)T; : : : ; xL(0)T)T, and Bi(0); i= 1; : : : ; Lbe given, then for
k= 0;1; : : : ; i= 1; : : : ; L;
xi(k+ 1) =xi(si(k))−Bi(k)−1Fi(u(k)) ifi∈I(k)
xi(k+ 1) =xi(k) else
Bi(k+ 1) =Bi(k) + (1=zi(k)Tzi(k))·(yi(k)−Bi(k)zi(k))zi(k)T ifi∈I(k)
Bi(k+ 1) =Bi(k) else;
whereu(k) = (x1(s1(k))T; : : : ; xL(sL(k))T)T;
yi(k) =Fi(x(k + 1))−Fi(v(k; i));
v(k; i) = (x1(k+ 1)T; : : : ; xi−1(k+ 1)T;
xi(si(k))T; xi+1(k+ 1)T; : : : ; xL(k+ 1)T)T;
zi(k) =xi(k+ 1)−xi(si(k));
where I(k) and sj(k) satisfy the following conditions
(2.1) for some xed integer p ¿0; k−p ¡ sj(k)6k; j= 1; : : : ; L; k= 0;1; : : :
(2.2) each I(k); k= 0;1; : : : are subset of {1; : : : ; L}, and the set {k: j∈I(k)} is unbounded for all j= 1; : : : ; L.
Conditions (2.1) and (2.2) may be interpreted as follows: At each instant of time k, all the ith block component xi; i∈I(k), are updated while the remaining are unchanged. The updating uses the
rst block component of x(s1(k)), the second block component of x(s2(k)), etc. Then the matrices
Bi; i∈I(k), are updated by using the new results. Each pair of block component and matrix is
updated innitely many often, and no update uses a value of a component which was produced by an update p or more steps previously.
The assumption of the existence of uniform bound p for delays appeared in the rst asynchronous model in [9]. It is not an additional constraint in most practical implementations [3]. This type of asynchronous model was called partially asynchronous method in [4].
We will refer the above mathematical model as partially asynchronous block Broyden’s (PABB) method.
If we take si(k) =k; I(k) ={1; : : : ; L} for all k and i, then PABB method describes block-Jacobi
Broyden method.
If we take si(k) =k; I(k) ={k+ 1 (modL)} for all k, then PABB method describes block-Gauss–
Seidel Broyden method.
3. Local convergence theorem
Theorem 3.1. Suppose that (1:3)–(1:5) are satised; then there exist two constants ¿0; ¿0;
such that if kx(0)−x∗kw6 for w ¿0 dened in (1:5)′; kBi(0)−(@Fi(x∗))=@xik26; i= 1; : : : ; L;
then the sequence {x(k)} generated by PABB method converges to x∗.
Before we give the proof of Theorem 3.1, we need several Lemmas. Because of the norm equiv-alence, we may assume that the norm in (1.4) is k · kw, and there exist two constants 1; 2¿0
such that
kAk261kAkwˆ; kAkwˆ62kAk2; (3.1)
where ˆw=w if A∈Rn×n, and ˆw=w
i if A∈Rni×n(orRni×nj).
Lemma 3.2 (Ortega and Rheinboldt [23]). IfF:Rn→Rm is Gateaux-dierentiable in an open
exists S(x∗; 1) ={x:kx−x∗kw61} ⊂; such that
kD(x∗)−1(F′(x)−D(x))kw¡ 0; forx∈S(x∗; 1)
thus by (1.8), (3.3) holds.
while by (3.1), (3.2)
thus we have proved Lemma 3.6.
Proof of Theorem 3.1. Suppose that for some xed ¿0; we have
We use mathematical induction to show the following inequalities:
kxi(k+ 1)−x∗i kwi6rkx(s(k))−x∗kw if i∈I(k);
and as a byproduct we get
Thus if i∈I(m), the uniform bound p for the delays, each block component is updated at least once for every 2p
If i =∈I(m);
Bi(m+ 1)−
@Fi(x∗)
@xi
2
=
Bi(m)−
@Fi(x∗)
@xi
2
62:
Therefore (3.8) holds for all k; and then (3.9) holds for all k; thus {x(k)} convergences to x∗. By Remark 2, we have the following corollary.
Corollary 3.7. Suppose that (1.3)–(1.4) are satised; and F′(x∗) is an H-matrix; then for any
partitioning of F and x; there exist two constants ¿0; ¿0; such that if kx(0)−x∗kw6 for
w ¿0 dened in (1.5)′; kB
i(0)−(@Fi(x∗))=@xik26; i= 1; : : : ; L; the sequence {x(k)} generated by
PABB method converges to x∗.
4. Extension to Schubert’s method
Schubert’s method [24] is a variation of Broyden’s method which is of interest in the case that the Jacobian is sparse. In this variation, the approximate Jacobian is forced to have the same sparsity pattern as the Jacobian.
Given a vector function F= (f1; : : : ; fm)T∈Rm → Rm; we dene its sparsity pattern as a matrix
PF= (pij)m×m; where
pij=
0 if f
i(x) is independent of xj
1 else i; j= 1; : : : ; m:
Let PFl= (plij) be the sparsity pattern of Fl; l= 1; : : : ; L. Using Schubert’s method instead of
Broy-den’s method for each subproblem in (1.2) leads to the following computational model: Given initial solution x= (xT
1; : : : ; xTL)T; and matrices Bl.
Process l(l= 1; : : : L) Initialization for x= (xT
1; : : : ; xTl; : : : ; xLT)T; Bl
Steps 1–8 and 10 are the same as the computational model of PABB method; step 9: if plij= 1 then uij←zj else uij←0; i; j= 1; : : : ; nl;
Bl←Bl+ nl
X
i=1
ui6=0
eieTi
(y−Blz)
uT
iui
uT
i;
where ui= (ui1; : : : ; uinl)
T; i= 1; : : : ; n
l; and ei∈Rnl is the unit vector of which the ith component is
1, i= 1; : : : ; nl.
Its mathematical model can be described similarly as PABB method.
5. Connection with the discrete Schwarz alternating procedure
In this section, suppose that there are overlaps among the blocks x1; : : : ; xL; and correspondingly
among F1; : : : ; FL. This may happen when discrete Schwarz alternating procedure (see, e.g., [18, 19,
25, 30]) is used to solve (1.1), where each block xl corresponds to the unknowns in a subdomain.
In this case, the computational models need not change, but we need to modify the mathematical model so that the previous analysis can be applied.
Suppose that PL
l= 1nl=m ¿ n; we dene a vector x= ( xT1; : : : ;xTL)T∈Rm from x∈Rn by setting
xl=xl; l= 1; : : : ; L: Denote this mapping from Rn to Rm by x=(x), similarly we obtain F=
( FT
1; : : : ;FTL)T=(F). We can thus rewrite (1.1) by repeating the overlapping unknowns and the
corresponding equations:
Fl( xl;Zl) = 0; l= 1; : : : ; L; (5.1)
where the vector Zl is a subset of
S
i6=l{xl}; since Fl depends only on n variables. The vector Zl
may not be unique, since the overlapping unknowns may have dierent choices of sub-indices. As before, we can update the approximate values of the lth block of unknowns by exploiting the lth block equations. Note that x∗=(x∗) is the solution of (5.1). On the other hand, we can recover the solution x∗ from x∗ by simply discarding the redundant overlapping parts in x∗. The previous analysis now can be applied to (5.1).
If F(x) is dierentiable and satises the Lipschitz condition, then F( x) will also be dierentiable and satisfy the Lipschitz condition with the same Lipschitz constant. Condition (1.5) for F does not mean a similar condition for F. When F′(x∗) is an H-matrix, however, we can show that F′( x∗)
is also an H-matrix.
Theorem 5.1. If F′(x∗) is an M-matrix; then F′( x∗) is also an M-matrix; where x∗=(x∗);
F=(F∗).
Proof. For simplicity, let A=F′(x∗); A= F′( x∗). The diagonal entries of A are composed of the
diagonal entries of (@Fl( x∗))=@xl; l= 1; : : : ; L. Since each (@Fl( x∗))=@xl is a diagonal block of A;the
diagonal entries of A are diagonal entries of A; thus positive. The nonzero o-diagonal entries of A
are also o-diagonal entries of A; thus nonpositive. Let A= D−B; A =D−B; where D; D are the diagonal parts of A and A; respectively. To show A is an M-matrix, we need only to show that A−1
is nonnegative. It is enough to show that ( D−1B)¡1; see, e.g., [23, 28]. Since A is an M-matrix,
(D−1B)¡1; and D−1B is nonnegative, there exists a positive vector v∈Rn such that D−1Bv ¡ v.
Let v=(v); then we have D−1Bv ¡ v. Thus ( D−1B)¡1; see, e.g., [4].
Corollary 5.2. If F′(x∗) is an H-matrix; then F′( x∗) is also an H-matrix; where x∗=(x∗);
F=(F∗).
Proof. Since F′(x∗) is an H-matrix, the comparison matrix of F′(x∗) is an M-matrix; see, e.g. [6,
16]. Thus by Theorem 5.1, the comparison matrix of F′( x∗) is also an M-matrix, i.e. F′( x∗) is an
Theorem 5.1 is more general than those similar theorems in [14, 22], since the above augmentation process allows components of x to belong to more than two subdomains.
Acknowledgements
The author would like to thank Professor Daniel B. Szyld for his careful reading of the manuscript which improved the presentation, and also thank the referees for the valuable comments.
References
[1] Z.-Z. Bai, On the convergence of parallel chaotic nonlinear multisplitting Newton-type methods, Comput. Appl. Math. 80 (1997) 317– 334.
[2] Z.-Z. Bai, V. Migallon, J. Penades, D.B. Szyld, Block and asynchronous two stage methods for mildly nonlinear systems, Research Report 97-31, Dept. Math., Temple Univ., 1997, in Numer. Math., to appear.
[3] G. Baudet, Asynchronous iterative methods for multiprocessors, J. ACM 25 (1978) 226 –344.
[4] D.P. Bertsekas, J.N. Tistisklis, Parallel and Distributed Computation, Prentice-Hall, Englewood Clis, NJ, 1989. [5] J. Bhai, J.C. Miellou, K. Rhor, Asynchronous multisplitting methods for nonlinear xed point problems, Numer.
Algorithms 15 (1997) 315 – 345.
[6] A. Bhaya, E. Kaszkurewicz, F. Mota, Asynchronous block-iterative method for almost linear equations, Linear Algebra Appl. 154 –156 (1991) 487– 508.
[7] A. Bojanczyk, Optimal asynchronous Newton method for the solution of nonlinear equations, J. ACM 31 (1984) 792 – 803.
[8] C.G. Broyden, A class of methods for solving nonlinear simultaneous equations, Math. Comp. 19 (1965) 577– 593. [9] D. Chazan, W. Miranker, Chaotic relaxation, Linear Algebra Appl. 2 (1969) 199 –222.
[10] J.E. Dennis, J.J. More, Quasi-Newton methods, motivation and theory, SIAM Rev. 19 (1977) 46 – 89.
[11] J.E. Dennis, B.S. Robert, Numerical methods for unconstrained optimization and nonlinear equations, Prentice-Hall, Englewood Clis, NJ, 1983.
[12] D. El Baz, M-functions and parallel asynchronous algorithms, SIAM J. Numer. Anal. 27 (1990) 136 –140. [13] M.N. El. Tarazi, Some Convergence results for asynchronous algorithms, Numer. Math. 39 (1982) 325 –340. [14] D.J. Evans, D.-R. Wang, An asynchronous algorithm for solving a class of a nonlinear simultaneous equations,
Parallel Comput. 17 (1991) 165 –180.
[15] A. Frommer, D.B. Szyld, Asynchronous two-stage iterative methods, Numer. Math. 69 (1994) 141–153.
[16] A. Frommer, Generalized nonlinear diagonal dominance and applications to asynchronous iterative methods, Comp. Appl. Math. 38 (1991) 105 –124.
[17] A. Frommer, H. Schwandt, D.B. Szyld, Asynchronous weighted additive Schwarz method, Electronic Trans. Numer. Anal. 5 (1997) 48 – 61.
[18] L.-S. Kang, L.L. Sun, Y.-P. Chen, Asynchronous Parallel Methods for Problems of Mathematical Physics (Chinese), Academic Press of China, 1985.
[19] S.-B. Kim, A. Hadjidimos, E.N. Houstis, J.R. Rice, Multi-parametrized Schwarz alterating methods for elliptic boundary value problems, Math. Comput. Simulation 42 (1996) 47–76.
[20] H.T. Kung, Synchronized and asynchronous parallel algorithms for multiprocessors. in: J.F. Traub, (Ed.), New Directions and Recent Results in Algorithms and Complexity, Academic Press, New York, 1976.
[21] E.S. Marwil, Convergence results for Schubert’s method for solving sparse nonlinear equations, SIAM J. Numer. Anal. 16 (1979) 588 – 604.
[22] J.C. Miellou, D. El Baz, P. Spiteri, A new class of asynchronous iterative algorithms with order intervals, Math. Comput. 67 (1998) 237– 225.
[24] L.M. Schubert, Modication of a quasi-Newton method for nonlinear equations with sparse Jacobian, Math. Comput. 24 (1970) 27– 30.
[25] B. Smith, P. Bjrstad, W. Gropp, Domain decomposition: Parallel multilevel methods for elliptic partial dierential equations, Cambridge University Press, Cambridge, 1996.
[26] D.B. Szyld, Dierent models of parallel asynchronous iterations with overlapping blocks, Comput. Appl. Math. 17 (1998) 100 –115.
[27] D.B. Szyld, J.-J. Xu, Asynchronous nonlinear multisplitting methods, in prepartion. [28] R.S. Varga, Matrix iterative analysis, Prentice-Hall, Englewood Clis, NJ, 1962.
[29] R.S. Varga, On recurring theorems on diagonal dominance, Linear Algebra Appl. 13 (1976) 1– 9. [30] W.P. Tang, Generalized Schwarz splittings, SIAM J. Sci. Statist. Comput. 13 (1992) 573 – 595.
[31] J.-J. Xu, An asynchronous parallel quasi-Newton methods for nonlinear equations (Chinese), in: L.-S. Kang et al. (Eds.), Proc. Nat. Conf. on Parallel Algorithms, Zhuo-Zhou, China, 1989, pp. 113 –117.
