• Tidak ada hasil yang ditemukan

Risk Assessment and Evaluation of Predictions

N/A
N/A
Protected

Academic year: 2023

Membagikan "Risk Assessment and Evaluation of Predictions"

Copied!
447
0
0

Teks penuh

In October, the Biostatistics and Risk Assessment Center (BRAC) and the Department of Epidemiology and Biostatistics at the University of Maryland hosted an international conference entitled "Risk Assessment and Prognostic Evaluation." The conference was held in Silver Spring, Maryland. The Risk Assessment and Prediction Valuation Conference and this book would not have been possible without the help, support and hard work of many people.

Risk Assessment in Lifetime Data Analysis

Because proportionality rarely holds in practice, the standard analytical approach should allow relative hazards to depend on time, which can be easily accomplished with commonly available software.

Introduction

Compatibility of proportionality of sub-risks and cause-specific risks for the same type of event. Compatibility of proportionality of sub-risks for one type of event and cause-specific risks for the other type of event.

Simulation

The logarithms of the relative sub-risks for each type of event are presented at the bottom of Table 1. The results of this analysis are presented at the bottom of Table 2 and show a highly significant downward trend in the relative sub-risks.

Fig. 1 Sub-hazards (panels a and b) and relative sub-hazards (panels c and d) for the competing risks setting defined by the true model in Table 1
Fig. 1 Sub-hazards (panels a and b) and relative sub-hazards (panels c and d) for the competing risks setting defined by the true model in Table 1

Application

Panels c and d depict relative sub-hazards under proportionality (in red) and linear deviation from proportionality (in green). Panels c and d depict relative sub-hazards under proportionality (in red) and linear deviation from proportionality (in green).

Table 3 Number of observed events and relative sub-hazards (logarithmic scale) of ESRD and transplant for household annual income in children with CKD
Table 3 Number of observed events and relative sub-hazards (logarithmic scale) of ESRD and transplant for household annual income in children with CKD

Discussion

An example of this is the income data in our application, because the apparent proportionality of the subhazards implies that the cause-specific hazards were time-dependent. This is not a surprising result, since the relative subhazards in the proportionality framework are entirely determined by the frequency of the two types of events and not by their timing.

Zhang, X., Zhang, M.J., Fine, J.: A proportional hazard regression model for the subdistribution with right-censored and left-truncated competing risks data. Lau, B., Cole, S.R., Gange, S.J.: Parametric mixture models for evaluating and summarizing hazard ratios in the presence of competing risks with time-dependent hazards and delayed entry.

Reduction and the Restricted Mean Survival Difference

Under the piecewise Cox model with the distribution of 0–2, 2–5 and 5+ years (the distribution used in [ 16 ]), the hazard ratio has an upside-down U shape. On the other hand, under the piecewise Cox model using the partition of 0–3, 3–6 and 6+ years (a plausible partition since the maximum follow-up time was almost 9 years), the hazard ratio has a U-shape .

Fig. 1 Estimated hazard ratio for the WHI clinical trial stroke data: Solid line—Model (1); Dashed line—Piece-wise Cox model with cut points at 2 and 5 years; Dash-dotted lines—Piece-wise Cox model with cut points at 3 and 6 years
Fig. 1 Estimated hazard ratio for the WHI clinical trial stroke data: Solid line—Model (1); Dashed line—Piece-wise Cox model with cut points at 2 and 5 years; Dash-dotted lines—Piece-wise Cox model with cut points at 3 and 6 years

The Estimators and Their Asymptotic Properties

Under the model, point-by-point confidence intervals are established for the absolute risk reduction and the limited mean survival difference. Under model (1), the absolute risk reductionΦ(t) can therefore be estimated based on. 4) In appendix 1 we show that ˆΦ(t) is strongly consistent for Φ(t) under model (1).

Simultaneous Confidence Bands

Therefore, cα can be estimated empirically from a large number of realizations of the conditional distribution of supt∈I|Uˆ/w|. Thus ˜cα can be empirically approximated from a large number of realizations of the conditional distribution of supt∈[a,b]|Vˆ(t)/wn| given the data.

Simulation Studies

Consistency

From these results and condition 4, we obtain strong consistency of ˆβ, ˆΦ(t) and ˆΨ(t) and almost sure convergence of ˆΩ.

Weak Convergence

By checking the density condition and the convergence of the finite-dimensional distributions, it can be shown that ˆUn(s) given the data also converges weakly to U∗. ii) From the results in (i) the propositions about Vnand ˜Vn follow. Schaubel, D.E., Wei, G.: Double inverse-weighted estimation of cumulative treatment effects under non-proportional hazards and dependent censoring.

When the distribution of the underlying random term is known or specified, the AFT model is a parametric model. According to Lee and Whitmore [20], the first hit model (FHT) has two basic components, namely (1) parent stochastic process {Y(t),t ∈T,y∈Y} with initial value Y (0) =y0, where T is the time space and Y is the state space of the process;.

Connecting TR and AFT Models

Again, for simplicity, if the runtime function r(t|z) is taken as t exp(−zγ), then the survival function of the corresponding AFT model is given by . The multiplier version of the AFT model in (2) is a special case of the general formation in (3), as can be seen if we define r(t|z) = t/exp(zγ).

Illustrative Examples

Furthermore, compared to the TR model, the AFT models cannot capture the crossover pattern of the Kaplan-Meier survival function estimates. As shown in Figure 5, the TR model successfully illustrates the intersection of the estimated Kaplan-Meier survival curves (Figure 6).

Fig. 1 Estimated survival functions by the TR model for the kidney dialysis data
Fig. 1 Estimated survival functions by the TR model for the kidney dialysis data

Life Regression Models

Residuals in AFT Models

Functional Form for a Covariate

Alternatively, we can use Cox-Snell residuals ˆri to obtain smoothed estimates ˆλ(x) as in the section “Residuals in AFT Models”, in order to estimate the function H(x). Lindqvist, B.H., Aaserud, S., Kvaløy, J.T.: Residual plots for model checking and for discovering the functional form of covariates in accelerated lifetime regression models.

Fig. 1 Simulated Weibull distributed data. Circles are ( x i , ˆ γ x i + σ ˆ s ˆ i ) using the adjusted residuals;
Fig. 1 Simulated Weibull distributed data. Circles are ( x i , ˆ γ x i + σ ˆ s ˆ i ) using the adjusted residuals;

Analysis

We will compare the calculation of a patient's survival probability in the competing risks F-N model with that in the Kaplan-Meier (K-M) formulation [16]. What characterizes the F-N model is the introduction of relapse and recovery of a breast cancer patient in the calculation of her survival probability.

The Fix-Neyman Competing Risks Model

Further discussion of the transition probabilities is in section "Extending the Fix-Neyman Competitive Risk Model". A distinctive feature of the F-N model is the inclusion of the possibility of patient recovery and relapse in the calculation of a patient's survival probability.

Fig. 1 The Fix-Neyman model with transition paths
Fig. 1 The Fix-Neyman model with transition paths

Comparison of the Fix-Neyman Model with the Kaplan-Meier Formulation

Both Fix and Neyman and Kaplan and Meier were after eliminating loss to follow or other causes in estimating the probability of survival of a patient. It is desirable to include available data on recovery and relapse in the survival analysis.

Fig. 3 Elimination of loss to follow up in the Fix-Neyman model Fig. 4 Elimination of loss to
Fig. 3 Elimination of loss to follow up in the Fix-Neyman model Fig. 4 Elimination of loss to

Extension of the Fix-Neyman Competing Risks Model

A non-homogeneous three-state Markov model is used to study the survival time of patients with liver cirrhosis, with loss to follow-up not considered in the model (which may not be necessary for this particular study). It is interesting that the A-J score of the survival curve is higher in the treated group until the 4th year, then the K-M score is higher.

Fig. 5 Chiang’s staging model
Fig. 5 Chiang’s staging model

An Example of a Nonhomogeneous Competing Risks Model with Application to Cross-Sectional Surveys of Hepatitis

By choosing the minimum modifiedχ2 method, Fix and Neyman reformulated the definition of RBAN in the context of their competing risk model. Let ˆpn be the relative frequency of the number of successes in n iid Bernoulli trials.

Fig. 6 Age-specific prevalence model with loss and regain of immunity
Fig. 6 Age-specific prevalence model with loss and regain of immunity

Concluding Remarks

We first discuss the properties of the residual quantile function and its close relation to the hazard function. We first discuss properties of the residual quantile function, including its close relation to the hazard function.

Residual Survival Basic Properties

Using the generalized gamma (GG) family, which we have previously advocated as a platform for parametric survival analysis [4], we next discuss the estimation of the residual quantile function from a parametric perspective. Another interesting property of the residual pth quantile function concerns the comparison of two distributions.

Fig. 1 Definition of the residual pth percentile after w in terms of the percentile function of the underlying distribution
Fig. 1 Definition of the residual pth percentile after w in terms of the percentile function of the underlying distribution

Appendix

The MACS is funded by the National Institute of Allergy and Infectious Diseases, with supplemental funding from the National Cancer Institute. Funding is also provided by the National Center for Research Resources (UCSF-CTSI grant number UL1 RR024131).

Evaluation of Predictions

Introduction Background

In the section “Measuring the predictive performance of a single model”, we consider a single risk model and use X for the predictors in the model. In the “Measuring the prediction performance of a single model” section, we focus only on the predictor X, while in the “Comparison of two risk models” section, we consider both X and Y together as predictors.

Fig. 1 Joint and marginal distributions of X and Y among cases and controls
Fig. 1 Joint and marginal distributions of X and Y among cases and controls

Validity of the Risk Calculator

If the circles follow the predictive curve, we conclude that the estimated risks are close to the observed risks and that the model is well calibrated (in a weak sense) in the dataset. But it can serve as a descriptive supplement to the visual representation of calibration, which is reflected in the predictive value or calibration graphs.

Fig. 3 Calibration assessed using the calibration plot. The model is well calibrated if points lie on the 45 ◦ line shown
Fig. 3 Calibration assessed using the calibration plot. The model is well calibrated if points lie on the 45 ◦ line shown

Measuring Prediction Performance of a Single Model Context

Note that HRD(r) is the true positive rate (TPR) or sensitivity, and HRD¯(r) is the false positive rate (FPR) or 1 minus specificity of the risk model using risk threshold r. Recall that Result1 tells us that the use of the risk threshold rH implies C/B= rH/(1−rH).

Fig. 5 Case and control distributions of risk shown with 1 minus cdfs, HR D ( t ) = P ( risk ( X )
Fig. 5 Case and control distributions of risk shown with 1 minus cdfs, HR D ( t ) = P ( risk ( X )

Comparing Two Risk Models

To see this, note that the performance of the risk (X) model must be derived from the case and control risk (X) distributions. The choice of risk threshold(s) should be based on an assessment of the costs and benefits associated with determining high risk (or each risk category).

Fig. 10 Event rates for subjects in each decile of estimated risk align well with the risk values for subjects in each decile
Fig. 10 Event rates for subjects in each decile of estimated risk align well with the risk values for subjects in each decile

The odds ratio, P(DP=1|(D=1|X,YX=,Yy+1)/=y)/PP((DD=0|=0|XX,YY==yy+1) )) , characterizes not prediction performance or improvement in prediction performance obtained by including Y in the risk model over the use of X alone. In the section "Illustration with examination of renal artery stenosis" we illustrate our methodology in connection with renal artery stenosis.

Measures of Improvement in Prediction Performance

The reduction in the proportion of population required to identify pD of cases (ΔPNF) obtained by adding Y to the model is. The change in the area under the ROC curve by adding Y to the model, called ΔAUC, is the most commonly used measure in practice.

Estimation from Matched and Unmatched Designs

For estimating the distribution of risk(X,Y) in the controls, we propose nonparametric and semiparametric approaches. Here, ˆE{risk(X,Y)|D=0,W =c} are the stratum-specific sample means of risk(X,Y) for controls in the case-control study for the nonparametric estimator.

Table 3 Efficiency of M 2− stage in matched and unmatched designs relative to the nonparametric M ad j estimator from the unmatched design
Table 3 Efficiency of M 2− stage in matched and unmatched designs relative to the nonparametric M ad j estimator from the unmatched design

Bootstrap Method for Inference

A matched or unmatched case-control subsample∗is then constructed in the same way as before. We derived two-stage estimators valid in matched or unmatched nested case-control studies.

Table 7 Results from a matched and an unmatched two-phase study simulated from the renal artery stenosis dataset
Table 7 Results from a matched and an unmatched two-phase study simulated from the renal artery stenosis dataset

Tree-Based Classification

The research interest is the creation of criteria and tools for the assessment of predictive accuracy based on multivariate markers (M1,M2,...,Mk). The research interest is to extend the rules and tools from the single-variant marker to the multivariate marker setting for assessing the predictive accuracy of the markers.

Univariate Marker Case

In the section "Other types of ROC and WROC functions", for the multivariate marker model, a function parallel to ROC∗(q) will be introduced and some interesting relationships similar to or different from those of the case will be explored. biased creators.

Multivariate Markers: ROC, WROC and AUC

Thus, if the markers are non-predictive for disease, the ROC function coincides with the diagonal line joining points(0,0) and(1,1), which is similar to the ROC function for univariate marker. For multivariate markers, the ROC function defined in (5) can be used to compare the performance of true positive rate locally by conditioning on FP(M0) = q.

Nonparametric Estimation

I(m1∈D(m0))I(FP(m0)≤p,TP(m1)>q)dF1(m1)dF0(m0), which is a useful formula for constructing a U-statistic for estimating the concordance probability with two-sided constraints. Note that the ROC function in (5) is defined as the average of the true positive rate given a fixed value of the false positive rate, where the calculation of the conditional expectation is through the two one-dimensional variables TP(M0) and FP( M0 ).

Other Types of ROC and WROC Functions

For estimation of ROC∗, WROC∗ and CON∗(p,q), nonparametric estimates can be constructed using methods similar to those for the functions ROC, WROC and CON(p,q). In this case, each of the WROC functions coincides with their counterpart of ROC functions.

Simulation and Data Example Simulation

For evaluation based on partial area under curve, subject to either smaller FP (FP≤p) or larger TP (TP>q), choices of these weighted ROC functions should be WROC and WROC∗ so that area under curve maximization would make sense . Choices of these weighted ROC functions should only include WROC and WROC∗ so that maximization of area under curve makes sense.

Fig. 1 Simulation for classifier I(M 1 > m 1 , M 2 > m 2 , M 3 > m 3 ) with ρ 0 = ρ 1 = 0
Fig. 1 Simulation for classifier I(M 1 > m 1 , M 2 > m 2 , M 3 > m 3 ) with ρ 0 = ρ 1 = 0

Gambar

Fig. 1 Sub-hazards (panels a and b) and relative sub-hazards (panels c and d) for the competing risks setting defined by the true model in Table 1
Fig. 3 Effect of nephrotic proteinuria (uP/C > 2) on the competing events of end-stage renal disease (ESRD) and renal transplantation in the CKiD study
Fig. 1 Estimated hazard ratio for the WHI clinical trial stroke data: Solid line—Model (1); Dashed line—Piece-wise Cox model with cut points at 2 and 5 years; Dash-dotted lines—Piece-wise Cox model with cut points at 3 and 6 years
Fig. 2 Estimated absolute risk reduction for the WHI clinical trial stroke data: Solid line—Model (1); Dotted line: Kaplan-Meier; Dashed line—Piece-wise Cox model with cut points at 2 and 5 years; Dash-dotted lines—Piece-wise Cox model with cut points at 3
+7

Referensi

Dokumen terkait

Alhamdulillahirobbil ‘alamiin, Dengan menyebut asma Allah SWT yang Maha Pengasih lagi Maha Penyayang, penulis panjatkan puji syukur atas kehadirat Allah SWT yang