Chapter 1 Introduction
5.1 Introduction
Chapter 5
Survey Logistic Regression Model
nested within hth stratum. Then the survey logistic model is given by
log(πijh) =x0ijhβ, i= 1,2, . . . , mhj, j = 1,2, . . . , nh, h= 1,2, . . . , H (5.1) Where xijh is the row of the design matrix corresponding to the characteristics of the ith business in jth PSU, nested withinhth stratum; and β is the vector of unknown parameters of the model. Hence, the log likelihood function is given by
l(β;y) =
H
X
h=1 nh
X
j=1 mhj
X
i=1
yijhlog
πijh 1−πijh
−log
1 1−πijh
(5.2) Parameter estimation for the survey logistic regression model can be calculated through a number of methods. According to Heeringaet al. (2010), the likelihood function for a simple random sampling ofnobservation on a binary response variable y with possible values 0 and 1, is based on the binomial distribution
l(β|x) =
n
Y
i=1
π(x)yi[1−π(x)]1−yi (5.3)
where π(x) is linked to the regression model coefficients and evaluated through the logistic CDF
π(xi) = exp(xiβ)
1 +exp(xiβ) (5.4)
For this case, the logistic regression model parameters and standard errors can be estimated using the method of maximum likelihood discussed in Section 4.1. However, if the survey data is collected from a complex sample design, application of the maximum likelihood esti- mation (MLE) is no longer possible. This is because: 1. the probabilities of selection for the sample observationsi=1,2,. . . ,n are no longer equal, due to the stratification and clustering of the survey design. Thus, sampling weights are required to estimate the finite population values of the parameters for the logistic regression model; and 2. the stratification and the clustering of the complex sample observation violates the assumption of independence of the observations that are crucial to the standard maximum likelihood approach used for estimating the sampling variance of the model parameters as well as for choosing a reference
distribution for the likelihood ratio test statistics (Heeringaet al. 2010). Generally, there are several methods used to estimate the covariance matrix (variance estimation) of the parame- ter estimates for data from complex survey designs. Among these methods are: the Jacknife method; the Pseudo-maximum likelihood method; the Taylor series (linearization) Method;
the Balance Repeated Replication (BRR) Method; Fay’s BRR Method; and the Hadamard Matrix. The variance can be easily estimated by these methods using PROC SURVEYL- OGISTIC procedure in SAS (9.2). For our case, the default Taylor series (Linearization) method is used because it estimates variance from among the PSU.
When survey data is collected using a complex sample design with unequal cluster size, most of the statistics of interest will not be simple linear functions of the observed data (Heeringaet al. 2010). Thus, Taylor’s (Linearization) method is applied, in order to derive an approximate form of estimator that is linear in statistics and for which variances and covariances can be directly and easily estimated. Taylor’s method is the most commonly used method to estimate the covariance matrix of the regression coefficients for complex survey data. It is the default variance estimation method used by PROC SURVEYLOGIS- TIC in SAS (9.2). Generally, variance estimation can be estimated using the Taylor series (Linearization) method as follows; the estimated covariance matrix of the model parameter βˆ by the Taylor series method is given by
Vˆ( ˆβ) = ˆQ−1GˆQˆ−1 (5.5) where
Qˆ =
H
X
h=1 nh
X
j=1 mhj
X
i=1
whjiDˆhji(diag(ˆπhji−πˆhjiπˆ0hji)−1Dˆ0hji
Gˆ = n−1 n−p
H
X
h=1
nh(1−fh) nh−1
nh
X
j=1
(ehj −eh)(ehj −eh)0
ehj =
mhj
X
i=1
whjiDˆhji(diag(ˆπhji−πˆhjiπˆ0hji)−1(yhji−πˆhji)
eh = 1 nh
nh
X
j=1
ehj
• Dhji is the matrix of the partial derivatives of the link function g, with respect to β and ˆDhji, and the response probabilities ˆπhji evaluated at ˆβ.
• h = 1,2, . . . , H is the stratum index, j = 1,2, . . . , nh is the cluster index, i = 1,2, . . . , mhj is the observation index within clusterj of stratum h.
• n is the total sample size.
• yhji is a D-dimensional column vector whose elements are indicator variables for the first D categories for variable Y. If the response of the jth unit of the ith cluster in stratum h falls in category d, the dth element of the vector is one, and the remaining elements of the vector are zero, whered= 1,2. . . , D.
• whji is the sampling weight.
• πhji is the expected vector of the response variable.
• fh is the sampling rate for stratim h
• Finally, p is the number of covariates in the model.
Also see the work of Lehtonen and Pahkinen (1995), and Hosmer and Lemeshow (2000) for further discussion on fitting logistic regression models to data from complex survey designs.
Inference and hypothesis tests for the survey logistic model can be calculated using the likelihood ratio, the score statistic, and the Wald statistic, to test the null hypothesis that assumes that all the explanatory effects in the model are zero. As was the case with the ordinary logistic regression model, the decision on whether to reject or not reject the null hypothesis is based on the chi-square test and the p-value. Thus, the null hypothesis is rejected at a p-value of less than 0.05, otherwise it is not rejected. The Wald chi-square statistic can also be used to test the significance of the model parameters (Heeringa et al.
2010). If the sample size is large, then the sampling distribution of the parameter estimators is approximately normal. The Wald chi-square statistic used for testing the significance and construction of the parameters confidence interval for the survey logit model, is given by
βˆj±z1−α2
q σj2
where z1−α
2 is the 100(1− α2)th percentile of the standard normal distribution, and σj2 is the variance of ˆβj given by the diagonal elements of the variance covariance matrix of ˆβ.
It can be noted here that if the data is from a simple random sampling, then the logistic model (PROC LOGISTIC) and the survey logistic model (PROC SURVEYLOGISTIC) are identical. However, if the data is from a complex sample design, then the survey logistic model uses the Pseudo maximum likelihood estimation or the Taylor linearization approach to estimate the variance estimates.
Generally, model selection and evaluation is based on a subjective combination of theo- retical issues, inspection of parameter estimates, goodness of fit test, interpretability, and a comparison of the performances of competing models (Marsh and Balla, 1994). The process of model selection for the survey logistic regression model follows the same procedures as for the ordinary logistic regression model. The only difference is that the selection procedures (forward, backward, and stepwise) are not yet incorporated in SAS PROC SURVEYLO- GISTIC. Model selection can however, be done using the type 3 analysis of effects, where insignificant variables are excluded from the model one at a time, and the contribution of the remaining effects to the deviance reduction, are then observed. Generally, as in all regression modelling, identification of a best logistic regression model for survey data should follow a systematic and scientific governed process (Heeringa et al. 2010). Hosmer and Lemeshow (2000) reported that the process for model selection should start by specifying the initial model (saturated model), refining the set of predictors, and then determining the final form of the logistic regression model that explains the data well. Among many model selection procedures that exist, no single one is always the best. Thus one should accept that any model is a simplification of reality (Agresti, 2002). Other criteria for model selection include the Akaike’s Information Criteria (AIC) and the Schwarz Criteria (SC), which are used to impose penalties to the likelihood ratio statistic−2logL (Agresti, 2002). Generally, the de- cision on either whether the AIC or the SC criterion is the best, depends on the objectives of the study and the more appealing model. Thus if the interest is in the consistency of the
approximation and the model fit, a model based onAICis preferred. However, if the interest is in the order of the model, then a model based onSC is preferred (refer to Bucklandet al.
(1997), and Burnham and Anderson, (2002) for further discussion on model selection).
The goodness of fit test for logistic regression for data from complex sampling designs is not yet developed or implemented in many available softwares; usually simulation studies and results were compared to the ordinary goodness of fit statistics discussed in Section 4.1 (Archeraet al. 2006). For instance, the Hosmer and Lemeshow goodness of fit test statistic, the Pearson residual, and the deviance residual test discussed in Section 4.1, are not yet incorporated in SAS PROC SURVEYLOGISTIC. Thus for the assessment of our model goodness of fit, we used the Akaike Information Criterion (AIC), the Schwarz Criterion (SC), and the -2log likelihood statistic as approximations for the goodness of fit test. These three criteria are displayed using the SURVEYLOGISTIC procedure (SAS 9.2). Letsdenote the explanatory effect in the model, and let ˆπj be the probability of the observed response for the jth observation. Then using the SAS notation, the model evaluation of goodness of fit can be done through the three criteria as follows:
-2Log-likelihood
The -2Log-likelihood statistic test is given by
−2LogL=−2X
j
wjfjlog(ˆπj)
where wj and fj are the weights and the frequency values of the jth observation. For the binary response model, the -2Log-likelihood is an equivent to
−2LogL=−2X
j
wjfj[rjlog(ˆπj) + (nj−rj)log(1−πˆj)]
where rj is the number of events, nj is the number of trials, and πj is the estimated event probability. This can be displayed using the event/trial syntax in SAS (9.2).
Akaike Information Criterion
TheAIC measures the discrepancy or symmeteric distance between a full (saturated) model and a reduced (selected) model of interest. TheAIC is given by
AIC =−2LogL+ 2p
where p is the number of parameters in the model. The AIC strategy is used to select the best model among two or more competing nested models. Thus the model with the smallest AIC is the best.
Schwarz Criterion
The Schwarz Criterion is another criterion used to asses/evaluate model goodness of fit.
Like the AIC, this method adjusts the -2LogL statistic for the number of parameters. The Schwarz Criterion is given by
SC =−2LogL+plog(X
j
fj)
wherepis the number of parameters in the model. It can be noted here that for cumulative response models for both the SC and the AIC assessments of fit, p=k+s, where k is the total number of response levels minus one, and s is the number of explanatory effects. For the generalized logit model, p=k(s+ 1).
The assessment of model predictive accuracy power in the survey logistic model can be achieved through statistics such as the concordance index (c), Sommer’s D (SD), Goodman- Kruskal Gamma (GKG), and Kendall’s Tau-a (KT). Unlike in the logistic model, where we used the ROC curve to assess the predictive power of the model, in the survey logistic model we used the concordance index c, which is equal to the area under the receiver op- erating characteristic (ROC) curve, to assess the model predictive power. The concordance index, as discussed in Section 4.2, estimates the probability that the predictions and the out- comes are concordant (Agresti, 2002). The concordance index as displayed by SAS PROC SURVEYLOGISTIC is given by
c= [(nc+ 0.5)(t−nc−nd)t−1]
where n is the total number of observations in the data set, t is the total number of pairs given by n(n−1)/2, nc is the number of concordant pairs, nd is number of discordant and t−nc−nd is the number of tied pairs. According to Agresti (2002), a value c= 0.5 means that the predictions were no better than random guessing. However as c approaches 1, the better the model predictive power. Refer to Section 4.2 for decisions on the AU C for the ROC, which is equivalent to the concordance index c.