Supplementary Appendix – Explanation of Difference-in-Differences Method
The difference-in-differences method attempts to estimate the impact of large-scale policies and programs by using a control group to isolate the underlying secular or temporal change from change observed in the intervention group. Table S1 illustrates the conceptual layout of this analysis. In this analysis, patient outcomes (in-hospital mortality and hospital length of stay) were compared pre- vs.
post-implementation for patients in collaborative hospitals (A1 – A2 = Difference 1) and in non- collaborative hospitals (B1 – B2 = Difference 2). The independent effect of participation in the collaborative is represented by the DID estimator, which is the difference-in-differences between collaborative and non-collaborative hospitals (Difference 1 – Difference 2 = DID estimator).
Table S1. Conceptual layout of Difference-in-Differences (DID) analysis comparing outcome
measures pre- vs. post-implementation for patients in collaborative and non-collaborative hospitals, and estimating the independent effect of the MHA Keystone Sepsis Collaborative
(i.e. the DID estimator).
Patient Cohort
Outcome Measures
Pre- vs. Post- Implementation
Comparison a
Independent Effect of MHA Keystone Sepsis b Pre-
Implementatio n
Post- Implementatio
n
Collaborative A1 A2 A2 – A1 = Difference 1 Difference 1 –
Difference 2
= DID estimator
Non-Collaborative B1 B2 B2 – B1 = Difference 2
a Statistical significance tests for Diff 1 and Diff 2 represent significant changes in outcomes following implementation
b Statistical significance tests of DID estimator indicates significant effect of the MHA Keystone Sepsis collaborative
In a statistical model, the DID estimator corresponds to the interaction term between intervention status (i.e. collaborative vs. non-collaborative) and pre- vs. post-intervention. This is illustrated in the following generalized linear model, where the terms for collaborative and post represent binary dummy variables for intervention status and pre- vs. post-intervention, respectively, and XBi represents a set of patient and/or hospital characteristics used in risk adjustment.
E
(
Yi)
=β0+β1∗Collaborativei+β2∗Posti+β3(
Collaborativei∗Posti)
+XBiThe Table below specifies the interpretation of the beta coefficients in the DID model. The pre vs. post-implementation changes are specified by the sum of β2+β3 for collaborative hospitals, and
simply β2 for the non-collaborative hospitals. The relative difference in pre- vs. post- implementation for collaborative and non-collaborative hospitals is ( β2+β3 ) - β2 = β3 . Hence, the DID estimator isolates the effect of the collaborative, and is estimated by the model
interaction coefficient ( β3 ). If the model is a logistic regression, the interaction term is represented by an odds ratio. The odds ratio is interpreted as the relative odds of the outcome (i.e. mortality) in patients treated in collaborative hospitals compared to non-collaborative hospitals. Conversely, if the model is a linear model, the interaction term is represented as the absolute change in the outcome (i.e.
length of stay) in patients treated in collaborative hospitals compare to non-collaborative hospitals. A statistically significant interaction term would indicate that the intervention had an additional impact (either positive or negative) on patient outcomes, beyond changes observed in the control group.
Table. Parameter interpretation for Difference-in-Differences model comparing collaborative
and non-collaborative hospitals pre and post collaborative implementation
Pre- Implementation
Post- Implementation
Pre/Post Difference
Difference-in- Differences
Collaborative β0+β1 β0+β1+β2+β3 β2+β3 β3
Non-Collaborative β0 β0+β2 β2 -
