2.4 Effect Size in Mediation Analysis
2.4.5 Effect size
Lachowicz et al. (2018) proposed as a measure of effect size for mediation analysis. is a measure of explained variance, interpretable as the variance in an outcome explained by a predictor through a mediator that appropriately adjusts for variance due to spurious correlation unaccounted for in the Fairchild et al. (2009) Rmed2 formulation in Equation 2.40, defined as
2 2 2
( ).
ym x Ry mx ryx
= − − (2.44)
36
Lachowicz et al. (2018) also showed that for a three-variable mediation model, is equivalent to the squared standardized indirect effect ( mx2 ym x2 ). Because is a measure of explained
variance, Cohen’s (1988) benchmarks for interpreting small, medium, and large effect size are applicable1. For an indirect effect with a binary predictor, the appropriate benchmarks are for 2 (Cohen, 1988, pp. 285-287). For an indirect effect with a continuous predictor, the appropriate benchmarks are those for R2(Cohen, 1988, pp. 412-414)2.
Because sample analog estimators of population standardized effect sizes typically are biased (particularly variance estimators), it was expected that the sample analog estimator of (
ˆ) was also biased. This was confirmed by a Monte Carlo simulation study, showing ˆ was upwardly biased particularly for small sample sizes and for small indirect effects. Although the complete sampling distribution of ˆ is not known or easily derivable, Lachowicz et al. (2018) derived the bias in the expected value of ˆ, and proposed an adjusted estimator that adjusted for this bias. Because Bˆmx and Bˆym x are independent and normally distributed,3 the expected value of ˆ is
1 If an effect is completely transmitted to an outcome through a mediator, the standardized indirect effect is equivalent to the total effect, which for a single predictor is a correlation coefficient. The squared correlation coefficient is equivalent then to , and could theoretically be judged against Cohen’s benchmarks for explained variance.
2 It is important to note that, although Cohen’s benchmarks may be applied, is not bounded by 0 and 1, and is not considered a proportion. can be greater than 1 when suppression is evident (i.e., direct and indirect effects have opposite signs).
3 As is common practice when standardizing regression coefficients, the variable variances x2 and 2y are assumed fixed, known quantities, and do not vary across samples. Therefore, it is assumed the variance ratio
2 2
x / y
does not affect the sampling distribution of ˆ ˆ2 2
mx ym x
B B .
37
2 2 2 2
2 2 2 2
2 2 2 2 2 2
2 2 2 2 2 2 2 2
ˆ ˆ ˆ
[ ] [ ]( / )
ˆ ˆ
[ ] [ ]( / )
( )( )( / )
.
mx ym x x y
mx ym x x y
mx mx ym x ym x x y
mx ym x mx mx ym x ym x mx ym x
E E B B E B E B
B B
=
=
= + +
= + + +
(2.45)
This means that the expected value of ˆ yields the parameter of interest mx2 ym x2 plus bias
2 2 2 2 2 2
mx ym x ym x mx mx ym x
+ + . The bias results from the fact that the effect size is comprised of products of normally distributed coefficients, the properties of which have been detailed in several sources (Arnold, 1982; Bohrnstedt & Goldberger, 1969; Goodman, 1960). Because each term is a function of the sampling variances of the regression coefficients, the magnitude of bias is therefore a function of sample size. In addition, the finding that bias in the expected value of ˆ is equivalent to the asymptotic variance approximation for indirect effects (Equation 2.39) is consistent with prior methodological work on bias reduction showing bias is generally proportional to error variance (Box, 1971).
It follows that the bias of the expected value of ˆ can be adjusted by subtracting a bias term from the sample estimates. However, it is important to note that the bias in Equation 2.45 is the asymptotic bias, and must be estimated from the sample. This is addressed by substituting an unbiased estimator of the asymptotic bias (Bˆmx2ˆym x2 +Bˆym x2 ˆmx2 − ˆ ˆmx2 ym x2 ; Goodman, 1960). This approach to adjusting for this bias of ˆ is similar to Ezekiel’s (1930) adjustment for R2 in simple linear regression (ˆ2 when the predictor is binary). A Monte Carlo simulation study showed that bias of was negligible for the vast majority of simulation conditions. For conditions where bias was non-negligible (> 5% relative bias; Boomsma, 2013), relative bias was still relatively small (< 20%). Whereas the information conveyed by ˆ is redundant with
38
that of the standardized indirect effect for three-variable mediation models, the adjusted conveys unique information that incorporates imprecision in effect estimates.
As previously discussed with the unstandardized indirect effect, parametric CIs rely on knowledge or a reasonable approximation of the sampling distribution of the estimator. The sampling distribution of the unstandardized indirect effect is a complex function of normally distributed variables, and it is unknown how the distribution changes when the indirect effect is standardized. Therefore, it is difficult to propose even an approximation to the distribution of squared standardized indirect effect. Lachowicz et al. (2018) used nonparametric bootstrapping to construct 95% CIs based on the percentiles of the empirical sampling distribution. Although in many conditions satisfactory CI coverage was achieved according Bradley’s criteria (92.5% - 97.5%; Bradley, 1978), coverage tended to be too high (> 97.5%) for conditions with small sample sizes and small effects, suggesting the 95% CIs constructed using the percentile method were overly wide. In addition, even when satisfactory coverage was achieved, the proportion of true values below the lower CI limit and above the upper CI limit were imbalanced, suggesting a small but systematic bias in the interval estimation procedure.
has many desirable properties as an effect size measure. It is interpretable as a measure of explained variance and can be compared to existing benchmarks for small, medium, and large effects. It is standardized, so it is invariant under linear transformations of x, m, and y. It is not dependent on sample size in the population. It is a monotonic function in absolute value of the standardized indirect effect. Although more research is needed to develop an accurate interval estimator across a wider range of study conditions, CIs for ˆ and can be constructed using a nonparametric bootstrap procedure. Finally, although the sample analog estimator ˆ is biased, the adjusted estimator has good statistical properties (i.e., negligible bias, consistent) in many
39
conditions common in applied research. In other words, the development of was consistent with the development of effect sizes for traditional research designs.