and α2,b respectively, an explicit formula for the probability P02,b(0,t) in (4) is obtained, and hence the (net) survival probability,
Pb[X>t] =1−P02,b(0,t)
=1−q02
α1,b+ q10
λ1,bλ2,b
1−
1 2
(q01+q02+q10)α1,b−α2,b .(6) Neyman’s RBAN (regular best asymptotic normal) estimates (1949) were used for estimating the risks in Qawith the breast cancer data. This will be discussed in section “Neyman’s Method of Minimum Modifiedχ2”.
In complete parallel, Kaplan and Meier estimated Qc(t) with a sample of n independent right-censored survival times, where a right-censored survival time is defined previously (above Eq. (1)). The estimate of q02(t), ˆq02(t) obtained in the Qc(t) model is employed to construct an estimate of the survival probability in (3). One estimator could be exp(−0tqˆ02(v)dv). Another is the K-M product-limit estimator.
In the Kaplan-Meier formulation the failure rate q02(t)is unspecified. If q02(t) = λ0(t)eβz,then the Cox-regression (or proportional hazard rates) model with right censoring is a 3-state Markov chain with two absorbing states as specified by Qc(t).
There is no particular advantage of using the Markov model to obtain the K-M estimator of the survival probability, because Qc(t)and Qd(t)are very simple risk matrices. Reformulating the Kaplan-Meier model as a Markov process is to show that one could extend in the direction of the popular K-M model to include relapses and recovery events as in the Fix-Neyman model Qa, but with some time-dependent qi j(t). As the number of states increases and recurrences are allowed, the theory of Markov processes provides important analytical tools for survival analysis. For many diseases, recovery and relapse could be significant occurrences in the course of disease development and treatment. Breast cancer is one example as studied by Fix and Neyman. Other examples abound. A leukemia patient might recover from a bone marrow transplant and later experience a relapse leading to the need for further transplants. A patient with aplastic anemia (an auto-immune disease) is usually treated with an immune-suppressant (IST). Some patients will respond to IST and relapse several months later. The same patient may receive a 2nd IST and so on and so forth until death or loss to follow up. It is likely that a patient’s survival time could be affected by such recurring recovery – relapse events. It is desirable to include the available data on recovery and relapse in the survival analysis.
However that could pose mathematical challenges in solving a finite system of Kolmogorov equations of transition probabilities given below:
dPi j(s,t)
dt =
∑
l=j
Pil(s,t)ql,j(t) +Pi j(s,t)qj,j(t), for all states i,j (7)
with initial conditions Pi j(s,t) =1 if i=j, 0 otherwise.
The derivation can be found in Feller ([10], Vol. I, Chap. 17, 2nd edn.) under the conditions that for 0≤s<t, (i) Pii(s,t)→1 as t→s; (ii) for each j, there is a non negative continuous function−qj j(t)such that
h→0lim
1−Pj j(t,t+h)
h =−qj j(t);
(iii) for each pair of i,j with i=j there is a non negative continuous function qi j(t) such that
h→0lim
Pi j(t,t+h)
h =qi j(t).
Only special cases of (7) have been solved explicitly. Otherwise, we rely on numerical solutions. This is a system of forward equations. We shall not dwell on the details of the Eq. (7) and refer the reader to a standard reference, Feller ([10], Vol. I, 2nd edn.), and Feller [9] for the existence and uniqueness of the solution.
If qi jare independent of t, the transition probabilities Pi j(s,t)simplify to Pi j(t) = P[ξt=j|ξ0=i]with s setting equal to 0. The solution is given by
P(t) =eQt, for t≥0. (8)
where P(t)is a matrix with components Pi j(t)and initial condition P(0) =I, an identity matrix, and Q is the corresponding risk matrix with components qi j.
Several of Neyman’s students continued the work in this direction. In particular, B. Altshuler [3] and C. L. Chiang [8]. Altshuler considered time-dependent qi j(t) in the multiple decrement model and obtained nonparametric estimates of survival probabilities which were later studied by Aalen in a seminal paper [1] based on his PhD thesis (1975). Aalen used an entirely different approach based on counting processes and Le Cam’s LAN theory. Chiang (Sect. 7 of Chap. 11, [8]) proposed a staging (or illness-death) model for analyzing survival times of a patient with a chronic disease. It is assumed that the disease progresses from a mild stage S0to a severe stage through intermediate stages{S1,···,Sk−1} and the patient may enter the death state Skfrom each of these stages as shown in the transition paths in Fig.5.
The transition rates from Sito Sjare given in the matrix Qg.
This staging model has been used for studying HIV and other chronic diseases.
Chang et al. [7] developed a statistical test of goodness of fit for a 3-state (k=2) staging models. These authors formulated the problem in terms of counting processes and developed an asymptotic test of a Markov staging model versus a semi-Markov model with power calculations.
Fig. 5 Chiang’s staging model
In Chiang’s staging model the transition rates of the transient states are constant and one-directional where qi j =0 if i> j. Thus the components of Qg below the diagonal are zero. This special feature permits a closed-form solution to the corresponding Kolmogorov equations (7) with time-dependent qi j(t). There are no risks of recovery and relapses in the model. At the request of a reviewer, we shall provide the solution for the nonhomogeneous Qg. We shall assume that for j=0,···,k−1, qj,j+1(t)>0 and qjk(t)>0. Also 0∞qj,j+1(v)dv=∞ and ∞
0 qjk(v)dv=∞as required of a hazard function of a random variable.
We shall set the initial time s=0 and assume the initial state of a patient at time 0 isξ(0) =0. We write the transition rates as qi j(t)to state explicitly their dependence on time. The Kolmogorov equations (7) for model Qgare
dP0 j,g(0,t)
dt =P0,j−1,g(0,t)qj−1,j(t) +P0 j,g(0,t)qj j(t)
for 0≤ j≤k−1. (9)
with initial conditions
P0 j,g(0,0) =1 if j=0
=0 if j=0, and qj j(t) =−
qj,j+1(t) +qjk(t)
for j=0,···k−1.
For consistency we use in (9) the symbol Pi j,g(0,t) to denote the transition probability under model Qg.
A patient can enter the death state k from any one of the states{0,1,···,k−1}.
Therefore the survival time X of a patient is
X=inf{t>0 :ξ(t) =k}. (10) The patients survival probability is given by
Pg[X>t] =Pg[ξ(t)∈ {0,1,···k−1} |ξ(0) =0]
=k−1
∑
j=0
P0 j,g(0,t) for t>0. (11)
Use Eq. (9) to solve for P0 j,g(0,t). Starting with P00,g(0,t), P0 j,g(0,t)can be solved recursively for j=0,1,···k−1. For j=0,
P00,g(0,t) =e
t
0q00(u)du (12)
In what follows we put
μj(t) =e−0tqj j(u)du, for j=0,1,···k−1.
The equation for P01,g(0,t)is a first order linear equation, dP01,g(0,t)
dt =P00,g(0,t)q01(t) +P01,g(0,t)q11(t). (13) The solution of P01,g(0,t)is given by
P01,g(0,t) = t
0μ1(v)P00,g(v)q01(v)dv
μ1(t) . (14)
Substituting (12) for P00,g(0,v)in (14), we obtain an explicit solution for P01,g(0,t). By the same token, for j=1,···,k−1, the solution of P0 j,g(0,t)is given by
P0 j,g(0,t) = t
0μj(v)P0,j−1,g(0,v)qj−1,j(v)dv
μj(t) , (15)
with the initial conditions stated in (9).
Chiang ([8], Chap. 11.7) derives the survival probability for constant qi j. The above is a generalization to the nonhomogeneous case.
A different line of attack is to use product integrals. Aalen and Johansen [2]
express the transition probabilities for finite nonhomogeneous Markov processes in terms of product integrals. Andersen et al. ([4], p. 312) is one of the very few publications that put recovery – relapse of a disease in a nonhomogeneous Markov model. A three-state nonhomogeneous Markov model is used to study survival time of patients with liver cirrhosis where loss to follow up is not considered in the model (which may not be needed for this particular study). An individual is at any time t in one of the three states: having normal prothrombin level (0), having abnormal prothrombin level (1) and death (2). The transition rates q01(t), q10(t), q12(t), and q02(t)are assumed to be positive where q21(t)and q20(t)are of course zero.
The product integral representation facilitates the estimation of transition rates nonparametrically but the estimation requires the data on the exact time of the direct transition of each patient from one state to another. Many of these direct transitions are difficult to observe if possible at all. To overcome the data problem, Andersen et al. defined changes of states to take place at the time when patients are examined at follow-up visits to the hospital and obtained the estimates ˆqi j(t)
of qi j(t). These estimates are used to estimate the survival probability P[X >t]
of a patient which in their notation is equal to 1−P02(0,t). No explicit analytical solution for P02(0,t)is provided. Andersen et al. used numerical solutions to obtain an estimate ˆP02(0,t), known as the Aalen-Johansen (A-J) estimate. Figures IV.4.15 and IV.4.16 on pages 315–316 [4] show discrepancies in the estimated survival curves and standard deviations between the A-J estimate and the K-M estimate.
In our interpretation, the discrepancies could be attributed to the fact that these probabilities were estimated using two different models. Although both are Markov models, the K-M model Qc(t)allows no recovery – relapse while the model for A-J estimate does. It is interesting to note that in the treated group, the A-J estimate of the survival curve is larger up to the 4th year, then the K-M estimate is larger.
For the placebo group, the A-J estimate appears to be uniformly worse than that of the K-M estimate. The estimated standard deviations of the A-J estimates of the survival probabilities are nearly always smaller than that of the K-M estimates.
The recovery rate q10(t)seems to have played a role in the treatment effect. The statistical significance of the result is not known.
Yang and Chang [21] used recurrent events in a parametric analysis to study prevalence of hepatitis A antibodies. This will be discussed in the next section.