arXiv:1711.06399v1 [math.ST] 17 Nov 2017
Average treatment effects in the presence of
unknown interference
∗
Fredrik Sävje
† fredrik.savje@yale.eduPeter M. Aronow
‡ peter.aronow@yale.eduMichael G. Hudgens
§ mhudgens@bios.unc.eduNovember 20, 2017
Abstract
We investigate large-sample properties of treatment effect estimators under unknown interference in randomized experiments. The inferential target is a generalization of the average treatment effect estimand that marginalizes over potential spillover effects. We show that estimators commonly used to estimate treatment effects under no-interference are consistent with respect to the generalized estimand for most
experimental designs under limited but otherwise arbitrary and unknown interference. The rates of
convergence are shown to depend on the rate at which the amount of interference grows and the degree to which it aligns with dependencies in treatment assignment. Importantly for practitioners, our results show that if one erroneously assumes that units do not interfere in a setting with limited, or even moderate, interference, standard estimators are nevertheless likely to be close to an average treatment effect if the sample is sufficiently large.
Keywords:Causal effects, causal inference, experiments, interference,sutva.
1
Introduction
Investigators of causality routinely assume that subjects under study do not interfere with each other. The no-interference assumption has been so ingrained in practice of causal inference that its application is often left implicit. Yet, interference appears to be at the heart of most of the social and medical sciences. Humans interact, and that is precisely the motivation for much of the research in these fields. Just like ideas spread from person to person, so do viruses. The prevalence of the assumption is the result of perceived necessity; it is assumed that our methods require isolated observations. The opening chapter of a recent and much-anticipated textbook on causal inference captures the sentiment well. The authors state that “causal inference is generally impossible without no-interference assumptions” and leave the issue largely uncommented thereafter (Imbens & Rubin, 2015, p. 10). This sentiment provides the motivation for the current study.
We investigate to what degree one can weaken the assumption of no-interference and still draw credible inferences about causal relationships. We find that, indeed, causal inferences are impossible without no-interference assumptions, but the prevailing view severely exaggerates the issue. One must ensure that the
∗We thank Alex D’Amour, Matt Blackwell, Peng Ding, Naoki Egami, Avi Feller, Lihua Lei and Jas Sekhon for helpful suggestions
and discussions. This work was partially supported by NIH grant R01 AI085073.
†Department of Political Science, Yale University.
interference is not all-encompassing but can otherwise allow the subjects to interfere in unknown and largely arbitrary ways.
We focus specifically on estimation of average treatment effects in randomized experiments. A random subset of a sample of units is exposed to some treatment, and we are interested in the average effect of the units’ treatments on their own outcomes. The no-interference assumption in this context is the restriction that no unit’s treatment affects other units’ outcomes. We consider the setting where suchspillover effects
exist, and in particular, when the form they may take is left unspecified.
The contribution is two-fold. We first introduce an estimand—theexpected average treatment effector
eate—that generalizes the conventionalaverage treatment effect(ate) to a setting with interference. The conventional inferential target is not well-defined when units interfere since a unit’s outcome may, in that case, be affected by more than one treatment. We resolve the issue by marginalizing the effects of interest over the assignment distribution of the incidental treatments. That is, for a given assignment, we may ask how a particular unit’s outcome is affected when we change only its own treatment. We may ask the same for each unit in the experiment, and we may average these effects. This defines an average treatment effect that remains unambiguous also under interference. However, if we were to repeat the exercise for another assignment, the results may be different; average effects under this definition depends on which assignment we use as a reference. We marginalize the effects over all possible reference assignments in order to capture the typical effect in the experiment, and this gives useate. The estimand is a generalization ofatein the sense that they coincide whenever the latter is well-defined.
The second contribution is to demonstrate thateatecan be estimated consistently under mild restrictions on the interference and without structural knowledge thereof. Our focus is on the standard Horvitz-Thompson and Hájek estimators. The results also pertain to the widely used difference-in-means and ordinary least squares estimators, as they are special cases of the Hájek estimator. We initially investigate the Bernoulli and complete randomization experimental designs, and show that the estimators are consistent foreateas long as the average amount of interference grows at a sufficiently slow rate (according to a measure we define shortly). Root-nconsistency is achieved whenever the average amount of interference is bounded. We continue with an investigation of the paired randomization design. The design introduces perfectly correlated treatment assignments, and we show that this can make the estimators unstable even when interference is sparse. We must restrict the degree to which the dependencies introduced by the experimental design align with the interference structure in order to achieve consistency. Information about the interference structure beyond these aggregated restrictions is, however, still not needed. The insights from the paired design extend to a more general setting, and we show that similar restrictions must be enforced for consistency under arbitrary experimental designs.
Our findings are of inherent theoretical interest as they move the limits for casual inference under interference. They are also of practical interest. The results pertain to standard estimators under standard experimental designs. As such, they apply to many previous studies where interference might have been present, but where it was assumed not to be. In particular, our results show that studies that mistakenly assume that units are isolated might not necessarily be invalidated. The reported estimates are likely to be close to the expected average treatment effects if the samples are sufficiently large.
2
Related work
(1958). The iteration that is most commonly used today was, however, formulated by Rubin (1980) as a part of thestable unit treatment variation assumption, orsutva. Early departures from this assumption are modes of analysis inspired by Fisher’s exact randomization test (Fisher, 1935). This approach employs “sharp” null hypotheses that stipulates the outcome of all units under all assignments. The most common such hypothesis is simply that treatment is inconsequential so the observed outcomes are constant over all assignments. As this subsumes that both primary and spillover effects are non-existent, the approach tests for the existence of both types of effects simultaneously. The test has recently been adapted and extended to study interference specifically (see, e.g., Rosenbaum, 2007; Luo et al., 2012; Aronow, 2012; Bowers et al., 2013, 2016; Athey et al., 2017; Basse et al., 2017; Choi, 2017).
We are, however, interested in estimation, and Fisher’s framework is not easily adopted to accommodate that. Early treatments of estimation restricted the interference process through parametric or semi-parametric models and thereby presumed that interactions took a particular form (Manski, 1993). The structural approach has been extended so to capture effects under weaker assumptions in a larger class of interference processes (Lee, 2007; Graham, 2008; Bramoullé et al., 2009). It still has been criticized for being too restrictive (Goldsmith-Pinkham & Imbens, 2013; Angrist, 2014).
A strand of the literature closer to the current exposition relaxes the parametric assumptions. Interference is allowed to take arbitrary forms as long as it is contained within known and disjoint groups of units (Sobel, 2006). The assumption is calledpartial interference. Hudgens & Halloran (2008) was the first in-depth treatment of the approach, and it has received considerable attention since (see, e.g., VanderWeele & Tchetgen Tchetgen, 2011; Tchetgen Tchetgen & VanderWeele, 2012; Liu & Hudgens, 2014; Rigdon & Hudgens, 2015; Kang & Imbens, 2016; Liu et al., 2016; Basse & Feller, 2017). While some progress can be made simply from the fact that interference is isolated to disjoint groups, the assumption is usually coupled withstratified interference. This additional assumption stipulates that the only relevant aspect of the interference is the proportion of treated units in one’s group (i.e., the identities of the treated units are inconsequential). Much like the structural approach, stratified interference severely restricts the form the interference can take.
Recent contributions have focused on relaxing the partial interference assumption. Interference is no longer restricted to disjoint groups, and units are allowed to interfere along more general structures such as social networks (see, e.g., Manski, 2013; Toulis & Kao, 2013; Ugander et al., 2013; Basse & Airoldi, 2016; Eckles et al., 2016; Forastiere et al., 2016; Aronow & Samii, 2017; Ogburn & VanderWeele, 2017; Sussman & Airoldi, 2017). This allows for quite general forms of interference, but the methods generally require detailed qualitative knowledge about the interference (e.g., that some pairs of units are known not to interfere).
3
Treatment effects under interference
Suppose we have a sample ofn units indexed by the set U = {1,2,· · ·,n}. We intervene on the world in ways that potentially affect the units. Our intervention is described by an-dimensional binary vector z=(z1,z2,· · ·,zn) ∈ {0,1}n. A particular value ofzcould, for example, denote that we give some drug to a
certain subset of the units inUand withhold it from the others. We are particularly interested in how uniti is affected when we alter theith dimension ofz. For short, we say that uniti’s treatment iszi.
The effects of different interventions are defined as comparisons between the outcomes they produce. Each unit has a function yi: {0,1}n → R denoting the observed outcome for the unit under a specific
(potentially counterfactual) intervention (Neyman, 1923; Holland, 1986). In particular,yi(z)is the response
ofithat we would have observed if we intervened on the world according toz. We refer to the elements of the image of this function aspotential outcomes. It will prove convenient to write the potential outcomes in a slightly different form. Letz−i =(z1,· · ·,zi−1,zi+1,· · ·,zn)denote the(n−1)-element vector constructed by
deleting theith element fromz. The potential outcomeyi(z)can then equivalently be written asyi(zi;z−i).
We will assume that the potential outcomes are well-defined throughout the paper. The assumption implies that the way we manipulatezis inconsequential; no matter how we intervene on the world to setzto a particular value, the outcome is the same. Well-defined potential outcomes also implies that no physical law or other circumstance prohibits us from intervening on the world so to setzto any value in{0,1}n. This ensures that all potential outcomes are, indeed, potential in the sense that there exists an intervention that can realize them. The assumption does not restrict the way we choose to intervene on the world, and we may design our experiments so that some interventions are never realized.
We conduct a randomized experiment, so we intervene on the world by setting zaccording to some random vectorZ =(Z1,· · ·,Zn). The probability distribution ofZis the design of the experiment. The
experimental design is the sole source of randomness we will consider. LetYi denote the observed outcome
of uniti. Yi is a random variable connected to the experimental design through the potential outcomes,
Yi =yi(Z). Similar to above,Z−i denotes the random vector we get when deleting theith element fromZ.
The observed outcome can, thus, be written asYi =yi(Zi;Z−i).
3.1
Expected average treatment effects
It is conventional to assume that the potential outcomes are restricted so a unit’s outcome is only affected by its own treatment. That is, for any two assignmentszandz′, if a unit’s treatment is the same for both assignments, then its outcome is the same;zi =zi′is assumed to implyyi(z)=yi(z
′). We say that a sample
satisfies no-interference when the potential outcomes are restricted in this way. The assumption allows us to define the treatment effect for unitias the contrast between its potential outcomes when we change its treatment:
τi =yi(1;z−i) −yi(0;z−i),
wherez−i is any value in{0,1}n−1. No-interference implies that the choice ofz−i is inconsequential for the
values ofyi(1;z−i)andyi(0;z−i). The average of the unit-level effects is the average treatment effect that is
commonly used by experimenters to summarize treatment effects.
Definition 1. Under no-interference, theaverage treatment effect(ate) is the average unit-level treatment effect:
τate= 1
n
n
X
The definition requires that no-interference holds. Potential outcomes will depend on more than one treatment when units interfere, andτi may then vary under permutations ofz−i in ways we cannot easily
characterize. As a consequence, we can no longer unambiguously talk abouttheeffect of a unit’s treatment. The ambiguity is contagious; our average treatment effect similarly becomes ill-defined.
To resolve the issue, we redefine the unit-level treatment effect for unitias the contrast between its potential outcomes when we change its treatment while holding all other treatments fixed at a given assignmentz−i.
We might call this quantity theassignment conditional unit-level treatment effect:
τi(z−i)=yi(1;z−i) −yi(0;z−i).
Our redefined effects still vary withz−i, but we have made the connection explicit. Averaging these effects,
we similarly get a version of the average treatment effect that is well-defined under interference.
Definition 2. Anassignment conditional average treatment effect(acate) is the average of the assignment conditional unit-level treatment effect under a given assignment:
τate(z)=1
n
n
X
i=1 τi(z−i).
The acates are unambiguous under interference, but they are unwieldy. An average effect exists for each assignment, so the number of effects grows exponentially in the sample size. We do not imagine that experimenters would find it useful to study individualacates. Similar to how we aggregate the unit-level effects to an average effect, a summary of theacates will often be more informative.
Definition 3. Theexpected average treatment effect(eate) is the expectedacate:
τeate=E[τate(Z)].
The expectation in the definition ofeateis taken over the distribution ofZinduced by the experimental design. Under no-interference,τate(z)does not depend onz, and the marginalization is inconsequential.
eateis, thus, a generalization ofatein the sense that the two estimands coincide whenever no-interference holds. When units interfere,eatemarginalizes the effects over all possible treatment assignments; it asks what the typical average treatment effect is under the experimental design we actually implement.
3.2
Previous definitions under interference
Theeateestimand builds on ideas previously proposed in the literature. An estimand introduced by Hudgens & Halloran (2008) resolves the ambiguity of treatment effects under interference in a similar fashion. The authors refer to the quantity as theaverage direct causal effect, but we opt for another name in an effort to highlight how it differs fromateandeate.
Definition 4. Theaverage distributional shift effect(adse) is the average difference between the conditional expected outcomes for the two treatment conditions:
τadse= 1
n
n
X
i=1
E[Yi |Zi =1] −E[Yi |Zi =0]
.
assignment distribution, whileadsemarginalizes each potential outcome separately over different conditional distributions. The difference becomes salient when we express the estimands in similar forms:
τeate= 1
n
n
X
i=1
Eyi(1;Z−i) −yi(0;Z−i)
,
τadse= 1
n
n
X
i=1
E[yi(1;Z−i) |Zi =1] −E[yi(0;Z−i) |Zi =0]
.
As Hudgens & Halloran (2008) note, similar effects are investigated elsewhere in the causal inference literature;eateis related toadsein a similar way as the median treatment effect is related to the difference between median potential outcomes (Lee, 2000).
The two estimands answer different causal questions. eatecaptures the expected effect of changing a random unit’s treatment in the current experiment. It is the expected average unit-level treatment effect.adse
is the expected effect of changing from an experimental design where we hold the average unit’s treatment fixed atZi =1to another design where its treatment is fixed atZi =0. That is, the estimand captures the
compound effect of changing a unit’s treatment and simultaneously changing the experimental design (see, e.g., the discussion in Sobel, 2006). As a result, the estimand may be non-zero even when all unit-level effects are exactly zero. Expressed in symbols,τadsemay be non-zero even whenτi(z−i)=0for alliandz−i.
It is possible to define average treatment effects as the contrast between the average outcome when all units are treated and the average outcome when no unit is treated:n−1Pi=1n [yi(1) −yi(0)], where1and0refer
to the unit and zero vectors.ateunder this definition coincides exactly with the estimand in Definition 1 (and thuseate) whenever no-interference holds. The definitions are, for this reason, often used interchangeably in the literature. However, average treatment effects under the alternative definition rarely coincides witheate
under interference, and we are forced to choose our inferential target in that case. The estimands provide qualitatively different perspectives of the causal setting. eatecaptures the typical treatment effect in the experiment actually implemented, while the alternative estimand captures the effect of completely scaling up or down treatment. Both effects could be of interest, but, as Basse & Airoldi (2017) have established, average treatment effects under the alternative definition can only rarely be investigated under arbitrary interference.
4
Quantifying interference
Our results do not require detailed structural knowledge on how the units interfere. No progress can, however, be made if we allow for completely unrestricted interference. We show this formally below; the intuition is simply that the change of a single unit’s treatment could lead to non-negligible changes in all units’ outcomes when the interference is unrestricted. We use the following definitions to quantify the amount of interference. We say that unitiinterferes with unitjif changingi’s treatment changesj’s outcome under at least one treatment assignment. We useIi jto indicate such interference. That is, we define ourinterference indicator
for any two unitsiandjas:
Ii j=
1 ifyj(z),yj(z′)for somez,z′∈ {0,1}nsuch thatz−i =z′−i,
1 ifi=j,
0 otherwise.
Our concept of interference is directed. Unitican interfere with unitjwithout the converse being true. We have defined the indicator so thatIii =1even when a unit’s treatment does not affect its outcome since this
The interference indicator is defined purely on the potential outcomes. Thus, the definition itself does not impose any restrictions on how the units may interfere; it is simply a description of the interference structure in a sample. The indicators may, for this reason, not necessarily align with social networks or other structures through which units are thought to interact. In fact, we rarely know enough about the interference to deduce or estimate the value of the indicators. Their role is to allow us to define an aggregated summary of the interference.
Definition 5(Average interference dependence).
davg= 1
n
n
X
i=1 n
X
j=1
di j, where di j=
(
1 ifIℓiIℓj =1for someℓ∈U,
0 otherwise.
The interference dependence indicator, di j, tells us whether unitsi and j are affected by a common
treatment. That is to say,iand j are interference dependent if they interfere directly with each other or if some third unit interferes with bothiandj. The sumPnj=1di jgives the number of interference dependencies
for uniti, so davgis the unit-average number of interference dependencies. The quantity acts as measure
of how close an experiment is to no-interference. In particular, no-interference is equivalent todavg =1,
which indicates that units are only interfering with themselves. At the other extreme,davg=nindicates that
interference is complete in the sense that all pairs of units are affected by a common treatment. If sufficiently many units are interference dependent—in the sense thatdavg takes on a large value—small perturbation
of the treatment assignments may be amplified by the interference and induce large changes in many units’ outcomes.
The interference dependence concept might appear overly complex. It can, however, easily be related to simpler quantities. Consider the following definitions:
Ii =
n
X
j=1
Ii j, Iavg=1n
n
X
i=1
Ii, Imsq=1
n
n
X
i=1
I2
i, and Imax=max
i∈U
Ii.
Ii indicates how many units i interferes with. That is, if changing unit i’s treatment would change the
outcome of five other units,iis said to be interfering with six units andIi =6(i.e., the five units and itself).
Information about these quantities is often useful (see, e.g., Aronow et al., 2016), but such knowledge is beyond our grasp in most experiments. The subsequent three quantities provide a more aggregated description of the interference. In particular, they are the average (Iavg), mean square (Imsq) and maximum (Imax) of the
unit-level variables. The quantities can be shown to bounddavgfrom below and above.
Lemma 1. Iavg2 ≤davg≤Imsq≤I2max.
The proof of Lemma 1 and all other proofs are given in appendices.
5
Large sample properties
Definition 6(Horvitz-Thompson,ht, and Hájek,há, estimators).
ˆ
τht= 1
n
n
X
i=1
ZiYi
pi
−1 n
n
X
i=1
(1−Zi)Yi
1−pi
, and τhሠ=
n
X
i=1
ZiYi
pi
,
n
X
i=1
Zi
pi
!
−
n
X
i=1
(1−Zi)Yi
1−pi
,
n
X
i=1
1−Zi
1−pi
!
,
wherepi=Pr(Zi =1)is the marginal probability that unitiis assigned to treatment conditionZi =1.
The estimators were first introduced in the sampling literature to estimate population means under unequal inclusion probabilities (Horvitz & Thompson, 1952; Hájek, 1971). They have since received wide-spread attention from the causal inference and policy evaluation literatures where they are often referred to as
inverse probability weighting estimators(Hahn, 1998; Hirano & Imbens, 2001; Hirano et al., 2003; Hernán & Robins, 2006). It can be shown that other estimators commonly used to analyze experiments—such as the difference-in-means and ordinary least squares estimators—are special cases of the Hájek estimator. As a consequence, our results apply directly to those estimators as well.
We assume throughout the paper that the experimental design and potential outcomes are well-behaved as formalized in the following assumption.
Assumption 1(Regularity conditions). There exist constantsk<∞,q≥2ands≥1such that for alli∈U:
1a (Probabilistic assignment). k−1 ≤Pr(Zi =1) ≤1−k−1,
1b (Bounded outcome moments).E|Yi|q
≤kq,
1c (Bounded potential outcome moments).E|yi(z;Z−i)|s
≤ksfor bothz∈ {0,1}.
Remark1. The exact values of qands are inconsequential for our results in Section 5.2. The regularity conditions can, in that case, be written simply withq=2ands=1. However, as we show in the subsequent section, the rate of convergence for an arbitrary (i.e., worst-case) experimental design depends on which moments can be bounded. In other words, the rate depends on the values ofqands. The ideal case is when the potential outcomes themselves are bounded, in which case the assumptions hold asq→ ∞ands→ ∞.
Remark2. Assumption 1b does not imply 1c since there may exists somezsuch thatPr(Zi =zi|Z−i =z−i)=0
whilePr(Z−i =z−i)>0. The opposite implication does not hold sincesmay be smaller thanq.
5.1
Limited interference
Under our asymptotic regime, a sequence ofdavgexists, and it captures the amount of interference in each
sample in our sequence. Our concept of limited interference is formalized as restrictions on this sequence.
Assumption 2(Restricted interference). davg=o(n).
Assumption 3(Bounded interference). davg=O(1).
In addition to restricting the amount of interference, the assumptions impose weak restrictions on the structure of the interference. In particular, they rule out that the interference is so unevenly distributed that a few units are interfering with most other units. If the interference is concentrated—in the sense that a few treatments are affecting many units—small perturbations of the treatment assignments could be amplified through those treatments. At the extreme, a single unit interferes with all other units, so all units’ outcome would change if we were to change its treatment. The estimators would not stabilize if the interference is structured in this way even if the interference otherwise was sparse.
Lemma 1 tells us that we can define our interference restrictions usingImsqorImaxrather thandavg. For
example, our results follow if we assumeImsq =o(n)orImax =o(n0.5). These assumptions are, however,
stronger than necessary. There are sequences for which the alternative restrictions do not hold butdavg=o(n) does. The connection is useful as it may be more intuitive to make assumptions aboutImsqandImax rather
than on interference dependencies. We opt to usedavgas it affords the most generality.
Assumption 2 is not sufficient for consistency. There exist sequences of experiments for which the assumption holds but the estimators are not converging toeatewith high probability. Restricted interference is, however, necessary for consistency of thehtandháestimators in the following sense.
Proposition 1(Necessity of restricted interference). For any sequence of experimental designs, if Assumption 2 does not hold, there exists a sequence of potential outcomes satisfying Assumptions 1b and 1c such that the
htandháestimators do not converge in probability toeate.
The proposition implies that it is not possible to design an experiment so to avoid issues introduced by interference if we are completely agnostic about its properties; we must somehow limit the interference in order to make progress. The proposition also implies that the weakest possible restriction we can impose ondavgfor consistency is Assumption 2. If any weaker restriction is imposed, e.g.,davg =O(n), potential
outcomes exist (for any experimental design) so that the relaxed interference restriction is satisfied but the
htandháestimators do not converge. Naturally, it might be possible to achieve consistency if one imposes stronger regularity conditions than we do or if one restricts the interference in some other way.
5.2
Common experimental designs
We start our investigation with three specific experimental designs. They are among the designs that are most commonly used by experimenters (Athey & Imbens, 2017). They are, thus, of interest in their own right. The designs also provide a good illustration of the issues that arise under unknown interference and act as a backdrop to our investigation of arbitrary designs in the subsequent section.
5.2.1 Bernoulli and complete randomization
The simplest experimental design assigns treatment independently; we flip a coin for each unit and administer treatment accordingly. We call this aBernoulli randomization design, and it satisfies:
Pr(Z=z)= n
Y
i=1
pzi
i (1−pi) 1−zi
for some set of assignment probabilitiesp1,p2,· · ·,pnbounded away from zero and one.
design, two outcomes are dependent only when the corresponding two units are affected by a common treatment. That is, when they are interference dependent according to Definition 5. Limiting this dependence grants consistency.
Proposition 2(Consistency under Bernoulli randomization). With a Bernoulli randomization design under our regularity conditions and restricted interference (Assumptions 1 and 2), thehtandháestimators are
consistent foreateand converge at the following rates:
(τhtˆ −τeate)=Op n−0.5davg0.5
, and (τˆhá−τeate)=Op n−0.5davg0.5
.
The Bernoulli design tends to be inefficient in small samples as the size of the treatment groups vary over assignments. Experimenters often opt to use designs that stabilize the treatment groups. A common such design introduces dependencies in assignments so to keep the number of treated units fixed but otherwise assigns treatment with equal probability:
Pr(Z=z)=
( n
m
−1
if Pn
i=1zi=m,
0 otherwise,
where m = ⌊pn⌋ for some fixed p strictly between zero and one. We refer to this design as complete randomization.
The dependencies introduced by complete randomization are not of concern under no-interference. The outcomes are, then, only affected by a single treatment, and the dependence between any two treatments is asymptotically negligible. This need not be the case when units interfere, and that complicates the asymptotic analysis. There are two issues to consider. First, the interference could interact with the experimental design so that two units’ outcomes are strongly dependent asymptotically even when they are not affected by a common treatment (i.e., whendi j =0). Consider, as an example, when one unit is affected by the first half
of the sample and another unit is affected by the second half. Complete randomization introduces a strong dependence between the two halves of the sample. In particular, the number of treated units in the first half is perfectly correlated with the number of treated in the second half. The outcomes of the two units may therefore be (perfectly) correlated even when no treatment affects them both. Our assumptions do not allow us to rule out that such dependencies exist. We can, however, show that they are rare whenever Assumption 2 holds.
The second issue is that the dependencies introduced by the design will distort our view of the potential outcomes. Complete randomization introduces a negative correlation between assignments. Whenever we observe a unit assigned to a certain treatment condition, units that interfere with the unit tend to be assigned to the other condition.eateweights the potential outcomes for a givenz−iequally, so the dependence tends
to bias our estimators since it induces a weighting that differs from the estimand’s. To illustrate the issue, consider when the potential outcomes are equal to the total number of treated units,yi(z)=Pi=1n zi, soeate
is equal to one. Under complete randomization, the number of treated units is fixed atm, so all revealed potential outcomes arem. Our estimators will, thus, be constant at zero and biased.
Proposition 3(Consistency under complete randomization). With a complete randomization design under our regularity conditions and restricted interference (Assumptions 1 and 2), thehtandháestimators are
consistent foreateand converge at the following rates:
(τhtˆ −τeate)=Op n−0.5davg0.5
, and (τhሠ−τeate)=Op n−0.5davg0.5
.
Recall that no-interference is equivalent todavg=1, and Propositions 2 and 3 reassuringly give us root-n
consistency in that case. The propositions, however, make clear that no-interference is not necessary for root-nconsistency. As captured in the following corollary, we can allow for non-trivial amounts of interference and still achieve efficient rates of convergence.
Corollary 1. With a Bernoulli or complete randomization design under our regularity conditions and bounded interference (Assumptions 1 and 3), thehtandháestimators are root-nconsistent foreate.
5.2.2 Paired randomization
Complete randomization restricted the assignment to ensure fixed-sized treatment groups. The paired randomization design imposes even greater restrictions. In particular, units are divided into pairs, and assignment is restricted so that exactly one unit in each pair is assigned to treatment. It is implicit that the sample size is even so that all units are paired. Paired randomization could be forced on the experimenter by external constraints or used to improve precision (see, e.g., Fogarty, 2017 and the references therein).
In symbols, letρ:U→Udescribe a pairing so thatρ(i)=jindicates that unitsiandjare paired. Since the pairing is symmetric, the self-composition ofρis the identity function. Thepaired randomization design
then satisfies:
Pr(Z=z)=
(
2−n/2 ifzi ,zρ(i)for alli∈U,
0 otherwise.
The paired design accentuates the issues we faced under complete randomization. The dependence within any set of finite number of treatments is asymptotically negligible under complete randomization. As a result, issues only arose when the number of treatments affecting a unit was of the same order as the sample size. Under paired randomization,Zi andZjare perfectly correlated also asymptotically wheneverρ(i)=j.
Dependencies may therefore remain in large samples even if the number of treatments affecting a unit is of a lower order than the sample size. Assumption 2 is no longer sufficient for consistency. We must consider to what degree the dependencies between treatment introduced by the design align with the structure of the interference. The following two definitions quantify the alignment.
Definition 7(Average pair-induced interference dependence).
bavg=1
n
n
X
i=1 n
X
j=1
bi j, where bi j=
(
1 if(1−di j)IℓiIρ(ℓ)j =1for someℓ∈U,
0 otherwise.
Definition 8(Within-pair interference). R=Pni=1Iρ(i)i.
Under complete randomization, the dependence between the outcomes of two unitsiandjnot affected by a common treatment remained asymptotically non-negligible only when the number of treatments affecting the units grew sufficiently fast. This is not necessary the case under paired randomization.Yi andYj could
The definition is similar in structure to Definition 5. Indeed, the upper bounds from Lemma 1 apply so that bavg≤Imsq≤I2max.
The second issue we discussed under complete randomization is affected in a similar fashion. No matter the number of units that are interfering with uniti, if one of those units is the unit paired withi, we cannot separate the effects ofZiandZρ(i). The design imposesZi =1−Zρ(i), so any effect ofZioni’s outcome could
just as well be attributed toZρ(i). Such dependencies will introduce bias—just as they did under complete randomization. However, unlike the previous design, restricted interference does not imply that the bias will vanish asymptotically. We must separately ensure that this type of alignment between the design and the interference is sufficiently rare. Definition 8 captures how common interference is between paired units. The two definitions allow us to formulate our restrictions on the degree to which the interference aligns with the pairs in the design.
Assumption 4(Restricted pair-induced interference). bavg=o(n).
Assumption 5(Pair separation). R=o(n).
Experimenters may find that the first assumption is quite tenable under restricted interference. As both davgandbavgare bounded byImsq, restricted pair-induced interference tends to hold in cases where restricted
interference can be assumed. It is, however, possible that the latter assumption holds even when the former does not if paired units are interfering with sufficiently disjoint sets of units.
Whether pair separation holds largely depend on how the pairs were formed. We often expect that units interfere exactly along the pairing. It is not uncommon that the pairs reflect some social structure (e.g., paired units may live in the same household). The interference will, in such cases, align with the pairing almost perfectly, and Assumption 5 is unlikely to hold. Pair separation is more reasonable when pairs are formed based on generic background characteristics. This is often the case when the experimenter uses the design to increase precision. The assumption could, however, still be violated if the background characteristics include detailed geographical data or other information likely to be associated with interference.
Proposition 4(Consistency under paired randomization). With a paired randomization design under our regularity conditions, restricted interference, restricted pair-induced interference and pair separation (As-sumptions 1, 2, 4 and 5), theht andhá estimators are consistent foreateand converge at the following
rates:
(τhtˆ −τeate)=On−1R+n−0.5davg0.5 +n−0.5b0avg.5
, and (τhሠ−τeate)=O n−1R+n−0.5davg0.5 +n−0.5b0avg.5
.
Remark3. Similar to the Bernoulli and complete designs, root-nconsistency follows if we assume bounded interference. We must, however, now bound the newly defined interference measures as well. That is, we must assumebavg=O(1)andR=O(n0.5)in addition to Assumption 3.
5.3
Arbitrary experimental designs
It will prove useful to collect all treatments affecting a particular unitiinto a vector:
ZI
i =(I1iZ1,I2iZ2,· · ·,IniZn).
The vector is defined so that its jth element isZj ifjis interfering withi, and zero otherwise. Similar to our
previous definitions, letZI
−i be the(n−1)-dimensional vector constructed by deleting theith element from ZI
i. These definitions have the following convenient properties:
Yi=yi(Z)=yi Z
I
i
, yi(1;Z−i)=yi 1;Z
I −i
, and yi(0;Z−i)=yi 0;Z
I −i
.
These properties allow us to redefine our interference conditions usingZI
i. We haveYi = yi Z
I
i
, so the outcomes of two unitsiand j will be independent wheneverZI
i andZ
I
j are independent. We should, thus,
be able to characterize the worst-case outcome dependence introduced by the experimental design by the dependence betweenZI
iandZ
I
j. Similarly, the dependence betweenZiandZ
I
−iwill govern how distorted our
view will be of the potential outcomes. If the dependence betweenZiandZI−iis asymptotically non-negligible,
we will not be able to separate the effect ofi’s own treatment from potential spillover effects.
We use the alpha-mixing coefficient introduced by Rosenblatt (1956) to measure the dependence between the assignment vectors. Specifically, for any two random variablesXandY, let:
α X,Y = sup
x∈σ(X)
y∈σ(Y)
Pr(x∩y) −Pr(x)Pr(y),
whereσ(X) andσ(Y) denote the sub-sigma-algebras generated by the respective random variable. The coefficientα(X,Y)is zero if and only ifXandY are independent, and increasing values indicate increasing dependence. The maximum isα(X,Y)=1/4. In this sense, the coefficient is similar to the familiar correlation
coefficient. The coefficients differ in that the alpha-mixing coefficient is not restricted to linear associations between two scalar random variables but can capture any type of dependence between any two sets of random variables. The alpha-mixing coefficient allows us to capture the average amount of dependence betweenZI
i
andZI
j and betweenZiandZ−i.
Definition 9 (External and internal mixing coefficients). Let q and s be the maximum values such that Assumptions 1b and 1c hold:
αext= 1
n
n
X
i=1 n
X
j=1
(1−di j)
α ZI i,Z
I
j
q−2
q , and αint
=
n
X
i=1
α Zi,ZI
−i
s−1 s .
Each term of the external mixing coefficient,α ZI
i,Z
I
j
, captures the dependence between the treatments affecting unitiand the treatments affecting unit j. Thus, if the dependence betweenZI
i andZ
I
j tends to be
weak or rare,αextwill be small compared ton. Similarly, if dependence betweenZi andZI−i tends to be
weak or rare,αintwill be small relative ton. In this sense, the external and internal mixing coefficients are direct generalizations of Definitions 7 and 8. Indeed, one can show thatαext ∝bavgandαint ∝ Runder
paired randomization where the proportionality constants are given byqands. We can use these definitions to generalize the assumptions we made under paired randomization to an arbitrary experimental design.
Assumption 6(Design mixing). αext=o(n).
Design mixing and separation stipulate that dependence between treatments are sufficiently rare or sufficiently weak (or some combination thereof). This encapsulates and extends the conditions in the previous sections. In particular, complete randomization under bounded interference constitutes a setting where dependence is weak: α ZI
i,Z
I
j
approaches zero for pairs of units withdi j =0 in that case. Paired
randomization under Assumptions 4 constitutes a setting where dependence is rare: α ZI
i,Z
I
j
is equal to 1/4for some pairs also asymptotically, but those pairs are rare. Complete randomization under Assumption 2 combines the two settings:α ZI
i,Z
I
j
might be non-negligible asymptotically for some pairs withdi j =0,
but such pairs are rare. For all other pairs withdi j =0, the pair-level mixing coefficient will approach zero
quickly. A similar comparison can be make for the design separation assumption.
Proposition 5(Consistency under arbitrary designs).Under our regularity conditions, restricted interference, design mixing and design separation (Assumptions 1, 2, 6 and 7), thehtandháestimators are consistent for eateand converge at the following rates:
(τhtˆ −τeate)=On−1αint
+n−0.5d0avg.5+n−0.5αext0.5
, and (τhሠ−τeate)=On−1αint
+n−0.5d0avg.5+n−0.5αext0.5
.
Remark4. Assumptions 2, 6 and 7 are incompatible unless Assumption 1 holds for someq >2ands>1. If the regularity conditions only hold forq=2ands=1, the mixing coefficients can be redefined as:
αext= 1
n
n
X
i=1 n
X
j=1
(1−di j)1
α ZI i,Z
I
j
>0, and αint=
n
X
i=1
1α Zi,ZI− i
>0,
where1[·]maps to one if the expression in brackets is true, and zero otherwise.
Remark5. The convergence results for Bernoulli and paired randomization presented in the previous section can be proven as consequences of Proposition 5. That is, however, not the case for complete randomization. The current proposition applied to that design would suggest slower rates of convergence than given by Proposition 3 (unless the potential outcomes are bounded). This highlights that Proposition 5 provides worst-case rates for all designs that satisfy Assumptions 1a, 6 and 7. Particular designs might be better behaved and, thus, ensure that the estimators converge at faster rates. For complete randomization, we can prove that restricted interference implies a stronger mixing condition than the conditions defined above. In particular, we can redefine the conditions using the∗- orψ-mixing coefficients as introduced by Blum et al. (1963) and Philipp (1969) rather than the alpha-mixing coefficient. This provides rates of convergence that are independent of which moment we can bound in our regularity conditions, and Proposition 3 would follow. We have not encountered other designs that satisfy this stronger mixing condition and have, for this reason, not explored that route further.
Remark6. If no units interfere,ZI
−iwill be constant at zero, and Assumption 7 is trivially satisfied. However,
no-interference does not imply that Assumption 6 holds. Consider a design that restricts all treatments to be equal, Z1 = Z2 = · · · = Zn. The external mixing coefficient would not be zero in this case; in fact, αext→n/4. This illustrates that we must limit the dependencies between treatment assignments even when units do not interfere. Proposition 5 can, in this sense, be seen as an extension of the restrictions imposed in Theorem 1 in Robinson (1982).
5.3.1 When design separation fails
such a design with the paired randomization. Another example is sequential randomization (see, e.g., Efron, 1971). Treatment is in this design assigned according to some sequence. The design tends to induce strong dependencies in treatment assignments between units that are close in the sequence, which typically coincide with the interference structure in these experiments.
It might be advisable to choose simple designs such as the Bernoulli or complete randomization if one wants to investigate treatment effects under unknown interference. These designs cannot align with the interference structure, and we need only to worry whether Assumption 2 holds. Another approach is to design the experiment so to ensure that design separation holds. For example, one might want avoid pairing units that are suspected to interfere in the paired randomization design.
It will, however, not always be possible to ensure that design separation holds, and we might want to ask what the consequences of such departures are. Without the assumption, we cannot separate the effect of units’ own treatments from spillover effects, and the estimators need not be consistent foreate. They could, however, converge to some other quantity, and indeed, they do. The average distributional shift effect (adse) is defined using the conditional distributions of the outcomes. This means thatadsedoes not attempt to separate the effect of units’ own treatments from spillover effects, and Assumption 7 is immaterial.
Proposition 6(Consistency foradse). Under our regularity conditions, restricted interference and design mixing (Assumptions 1, 2 and 6), theht andhá estimators are consistent foradse and converge at the
following rates:
(τhtˆ −τadse)=Op n−0.5davg0.5 +n−0.5α0ext.5
, and (τhሠ−τadse)=Op n−0.5davg0.5+n−0.5αext0.5
.
6
Concluding remarks
Experimenters worry about interference. The first line of attack tends to be to design experiments so to minimize the risk that units interfere. One could, for example, physically isolate the units for the duration of the study. The designs needed to rule out interference tend, however, to make their experiments so alien to the topics under study that the findings are no longer relevant; the results would not generalize to the real world where units do interfere. When design-based fixes are undesirable, infeasible or incomplete, the situation is mended by adjusting the analysis to account for any lingering interference. This, however, requires detailed information about the structure of the interference—information we rarely have. The modal experimenter neither averts all interference by design nor accounts for it in the analysis. He or she conducts and analyzes the experiment as if no units interfere, even when the no-interference assumption, at best, holds true only approximately or with some probability. The disconnect between assumptions and reality is reconciled by what appears to be a common intuition among experimenters that goes against the conventional view in the causal inference literature: unadjusted interference is not a fatal flaw so long as it is sufficiently limited. Our results provide rigorous justification for that intuition.
The eate estimand generalizes the average treatment effect to experiments with interference. All interpretations ofatedo, however, not apply to our estimand. In particular,eatecannot be interpreted as the difference between average outcomes when no unit and all units are treated. The estimand is the marginal effect of changing a single treatment under the current experimental design, and we cannot extrapolate the effect to other settings without additional assumptions. A consequence is thateatedepends on the experimental design. Such design-dependence might appear unusual, but it is not unheard of that estimands depend on properties of the experiment. For example, rather than studying the effect in some large, partially unobserved population, experimenters occasionally investigate average treatment effects in fixed, fully observed samples of units (commonly referred to as thesampleate, orsate).
Our focus has been on the effect of a unit’s own treatment, but the results are not restricted to “direct” treatment effects as typically defined. In particular, the pairing between units and treatments is completely arbitrary from the perspective of our causal model, and an experiment could have several reasonable pairings. Consider, for example, when our intervention is to give some drug to the units in the sample. The most natural pairing might be to let a unit’s treatment indicator denote whether we give the drug to the unit itself. We may, however, just as well let it denote whether we give the drug to, say, the unit’s spouse (who may be in the sample as well).eatewould then be the expected spillover effect between spouses. Our investigation applies, in this sense, both to usual treatment effects and rudimentary spillover effects. We conjecture that our argument can be extended to other definitions of treatment, and would in that case afford robustness to unknown and arbitrary interference also when investigating more intricate spillover effects. We leave it to future contributions to explore the details.
References
Angrist, J. D. (2014).The perils of peer effects. Labour Economics, 30, 98–108.
Aronow, P. M. (2012).A general method for detecting interference between units in randomized experiments.
Sociological Methods & Research, 41(1), 3–16.
Aronow, P. M., Crawford, F. W., & Zubizarreta, J. R. (2016).Confidence intervals for means under constrained dependence. arXiv:1602.00359.
Aronow, P. M. & Samii, C. (2017). Estimating average causal effects under general interference, with application to a social network experiment.Annals of Applied Statistics, in print.
Athey, S., Eckles, D., & Imbens, G. W. (2017). Exact p-values for network interference. Journal of the American Statistical Association, in print.
Athey, S. & Imbens, G. W. (2017).The econometrics of randomized experiments.Handbook of Economic Field Experiments, in print.
Basse, G. W. & Airoldi, E. M. (2016).Optimal model-assisted design of experiments for network correlated outcomes suggests new notions of network balance. arXiv:1507.00803v2.
Basse, G. W. & Airoldi, E. M. (2017). Limitations of design-based causal inference and A/B testing under arbitrary and network interference. arXiv:1705.05752.
Basse, G. W., Feller, A., & Toulis, P. (2017).Exact tests for two-stage randomized designs in the presence of interference. arXiv:1709.08036.
Blum, J. R., Hanson, D. L., & Koopmans, L. H. (1963). On the strong law of large numbers for a class of stochastic processes.Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 2(1), 1–11.
Bowers, J., Fredrickson, M. M., & Aronow, P. M. (2016).A more powerful test statistic for reasoning about interference between units.Political Analysis, 24(3), 395–403.
Bowers, J., Fredrickson, M. M., & Panagopoulos, C. (2013).Reasoning about interference between units: A general framework.Political Analysis, 21(1), 97–124.
Bramoullé, Y., Djebbari, H., & Fortin, B. (2009). Identification of peer effects through social networks.
Journal of Econometrics, 150(1), 41–55.
Choi, D. (2017).Estimation of monotone treatment effects in network experiments.Journal of the American Statistical Association, in print.
Cox, D. R. (1958).Planning of Experiments. New York: Wiley.
Davydov, Y. A. (1970). The invariance principle for stationary processes. Theory of Probability & Its Applications, 15(3), 487–498.
Eckles, D., Karrer, B., & Ugander, J. (2016). Design and analysis of experiments in networks: Reducing bias from interference.Journal of Causal Inference, 5(1).
Efron, B. (1971).Forcing a sequential experiment to be balanced.Biometrika, 58(3), 403–417.
Egami, N. (2017).Unbiased estimation and sensitivity analysis for network-specific spillover effects: Appli-cation to an online network experiment. arXiv:1708.08171v2.
Fisher, R. A. (1935).The Design of Experiments. London: Oliver & Boyd.
Fogarty, C. B. (2017). On mitigating the analytical limitations of finely stratified experiments. arXiv:1706.06469.
Forastiere, L., Airoldi, E. M., & Mealli, F. (2016).Identification and estimation of treatment and interference effects in observational studies on networks. arXiv:1609.06245.
Goldsmith-Pinkham, P. & Imbens, G. W. (2013). Social networks and the identification of peer effects.
Journal of Business & Economic Statistics, 31(3), 253–264.
Graham, B. S. (2008).Identifying social interactions through conditional variance restrictions.Econometrica, 76(3), 643–660.
Hahn, J. (1998). On the role of the propensity score in efficient semiparametric estimation of average treatment effects.Econometrica, 66(2), 315–331.
Halloran, M. E. & Hudgens, M. G. (2016).Dependent happenings: a recent methodological review.Current Epidemiology Reports, 3(4), 297–305.
Hernán, M. A. & Robins, J. M. (2006). Estimating causal effects from epidemiological data. Journal of Epidemiology & Community Health, 60(7), 578–586.
Hirano, K. & Imbens, G. W. (2001). Estimation of causal effects using propensity score weighting: An application to data on right heart catheterization. Health Services & Outcomes Research Methodology, 2(3), 259–278.
Hirano, K., Imbens, G. W., & Ridder, G. (2003).Efficient estimation of average treatment effects using the estimated propensity score.Econometrica, 71(4), 1161–1189.
Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960.
Horvitz, D. G. & Thompson, D. J. (1952).A generalization of sampling without replacement from a finite universe.Journal of the American Statistical Association, 47(260), 663–685.
Hudgens, M. G. & Halloran, M. E. (2008). Toward causal inference with interference. Journal of the American Statistical Association, 103(482), 832–842.
Imbens, G. W. & Rubin, D. B. (2015).Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. New York: Cambridge University Press.
Isaki, C. T. & Fuller, W. A. (1982).Survey design under the regression superpopulation model. Journal of the American Statistical Association, 77(377), 89–96.
Kang, H. & Imbens, G. W. (2016).Peer encouragement designs in causal inference with partial interference and identification of local average network effects. arXiv:1609.04464.
Lee, L. (2007). Identification and estimation of econometric models with group interactions, contextual factors and fixed effects.Journal of Econometrics, 140(2), 333–374.
Lee, M. (2000).Median treatment effect in randomized trials.Journal of the Royal Statistical Society. Series B (Statistical Methodology), 62(3), 595–604.
Liu, L. & Hudgens, M. G. (2014).Large sample randomization inference of causal effects in the presence of interference.Journal of the American Statistical Association, 109(505), 288–301.
Liu, L., Hudgens, M. G., & Becker-Dreps, S. (2016). On inverse probability-weighted estimators in the presence of interference.Biometrika, 103(4), 829–842.
Luo, X., Small, D. S., Li, C.-S. R., & Rosenbaum, P. R. (2012). Inference with interference between units in an fMRI experiment of motor inhibition. Journal of the American Statistical Association, 107(498), 530–541.
Manski, C. F. (1993). Identification of endogenous social effects: The reflection problem. The Review of Economic Studies, 60(3), 531.
Neyman, J. (1990/1923). On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statistical Science, 5(4), 465–472. (Original work published 1923).
Ogburn, E. L. & VanderWeele, T. J. (2017).Vaccines, contagion, and social networks.The Annals of Applied Statistics, 11(2), 919–948.
Philipp, W. (1969). The central limit problem for mixing sequences of random variables. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 12(2), 155–171.
Rigdon, J. & Hudgens, M. G. (2015). Exact confidence intervals in the presence of interference. Statistics & Probability Letters, 105, 130–135.
Robinson, P. M. (1982). On the convergence of the Horvitz-Thompson estimator. Australian Journal of Statistics, 24(2), 234–238.
Rosenbaum, P. R. (2007). Interference between units in randomized experiments. Journal of the American Statistical Association, 102(477), 191–200.
Rosenblatt, M. (1956).A central limit theorem and a strong mixing condition. Proceedings of the National Academy of Sciences, 42(1), 43–47.
Rubin, D. B. (1980).Randomization analysis of experimental data: The Fisher randomization test comment.
Journal of the American Statistical Association, 75(371), 591.
Sobel, M. E. (2006).What do randomized studies of housing mobility demonstrate?Journal of the American Statistical Association, 101(476), 1398–1407.
Sussman, D. L. & Airoldi, E. M. (2017).Elements of estimation theory for causal effects in the presence of network interference. arXiv:1702.03578.
Tchetgen Tchetgen, E. J. & VanderWeele, T. J. (2012). On causal inference in the presence of interference.
Statistical Methods in Medical Research, 21(1), 55–75.
Toulis, P. & Kao, E. (2013). Estimation of causal peer influence effects. In S. Dasgupta & D. McAllester (Eds.),Proceedings of the 30th International Conference on Machine Learning, volume 28 ofProceedings of Machine Learning Research(pp. 1489–1497). Atlanta: PMLR.
Ugander, J., Karrer, B., Backstrom, L., & Kleinberg, J. (2013). Graph cluster randomization: Network exposure to multiple universes. InProceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’13 (pp. 329–337). New York: ACM.
Appendices
A1
Miscellaneous definitions and lemmas
Definition A1.
Proof. Follows directly from Definitions 3, 4, 6 and A1.
Corollary A2.
(τhtˆ −τadse)=(µˆ1−µ˘1) − (µˆ0−µ˘0), (τhሠ−τadse)=(µˆ1/ˆn1−µ˘1) − (µˆ0/ˆn0−µ˘0).
Proof. Follows directly from Corollary A1.
Lemma A1. µ˘z=O(1).
Proof. For some sufficiently large constantk<∞:
|µz˘ |=
where the last inequality follows from Assumption 1b and Jensen’s inequality.
Lemma A4. There exists a constantk<∞such that for alli,j ∈Uandz∈ {0,1}:
Cov 1[Zi =z]Yi,1[Zj =z]Yj ≤k.
Proof. Cauchy–Schwarz’s inequality, Jensen’s inequality and Assumption 1b give us:
Cov 1[Zi =z]Yi,1[Zj =z]Yj ≤
Proof. For some sufficiently large constantc<∞, Assumption 1a gives us:
E|yi(z;Z−i)|
Letk=2c2, and consider the expression to be bounded:
where the last inequality follows from Jensen’s inequality and Assumptions 1c and 1b .
A2
Proof of Lemma 1
Lemma 1. Iavg2 ≤davg≤Imsq≤I2max.
di j =davg (Jensen’s inequality)
A3
Proof of Proposition 1
Proposition 1(Necessity of restricted interference). For any sequence of experimental designs, if Assumption 2 does not hold, there exists a sequence of potential outcomes satisfying Assumptions 1b and 1c such that the
htandháestimators do not converge in probability toeate.
Proof. If Assumption 2 does not hold, there exists a constant0<csuch thatcn≤davgfor sufficiently large
n. Consider the following potential outcomes:
yi(z)=
where⌈·⌉is the ceil function. This implies:
di j =
We first ensure that the stipulated potential outcomes satisfy our regularity conditions. Fori > ⌈c0.5n⌉, we have yi(z)= 0for allz, so the moment conditions in Assumption 1 trivially hold for these units. For
i ≤ ⌈c0.5n⌉, we can bound the potential outcomes for allzby:
|yi(z)| ≤n⌈c0.5n⌉−1
Pr(Zi =1)+Pr(Zi=0) ≤c−0.5,
so Assumptions 1b and 1c hold even whenq→ ∞ands→ ∞if setk=c−0.5. Consider theeateestimand:
τeate= 1
Then consider thehtestimator:
A4
Proof of Proposition 2
Lemma A6. Under Bernoulli randomization,(τadse−τeate)=0.
Proof. From Definitions 2, 3 and 4, we have:
τadse= 1
where the second equality follows from the Bernoulli design and its independent treatment assignments.
Lemma A7. Under Bernoulli randomization,Var(µˆz)=On−1davg. Proof. For some sufficiently large constantk<∞:
Var(µˆz)=Var
where the last inequality follows from Lemma A4.
The Bernoulli design implies that the elements ofZare independent. Whendi j =0, unitsiand jshare
no common treatments affecting them, soZI i andZ
Proof. Markov’s inequality and Lemma A2 gives us:
Pr
for any constantk>0. Lemma A7 implies that for sufficiently largek:
Var ˆµz
Thus, for sufficiently largen, the probability thatµˆz−µ˘zdeviate from zero with more than a fixed multiple
ofn−0.5davg0.5 can be bounded by anyε >0by settingk=1/ε.
Proof. Follows directly from Assumption 2 and Lemma A8.
Lemma A9. Under Bernoulli randomization, nˆz−1=Op n−0.5. Proof. Consider the variance ofnˆz:
Var(nˆz)=Var
1 n
n
X
i=1
1[Zi=z] Pr(Zi =z)
= 1
n2 n
X
i=1
Var 1[Zi =z]
Pr(Zi =z)
2 =
1 n2
n
X
i=1
1−Pr(Zi =z) Pr(Zi =z)
≤n−1k,
where the last inequality follows from Assumption 1a. The lemma then follows from Markov’s inequality
and Lemma A3 using the same approach as in the proof of Lemma A8.
Corollary A4. Under Bernoulli randomization, nˆz−1=op(1).
Proof. Follows directly from Lemma A9.
Lemma A10. Under Bernoulli randomization and Assumption 2, µˆz/nˆz−µ˘z=Op n−0.5davg0.5
.
Proof. Corollary A4 ensures us that we safely can disregard the eventnˆz=0, which gives us:
ˆ
µz
ˆ nz
−µ˘z= µˆz−µ˘z
ˆ nz
− µ˘z(nˆz−1) ˆ nz
.
Let f(a,b)=a/band consider the first order Taylor expansion of the two terms around(0,1):
ˆ
µz−µ˘z
ˆ
nz =
f µzˆ −µz,˘ nˆz =(µzˆ −µz˘ )+r1, µ˘z(nˆz−1)
ˆ
nz =
f µz˘ (nˆz−1),nˆz =µz˘ (ˆnz−1)+r2,
and, thus:
ˆ
µz
ˆ nz
−µ˘z=(µˆz−µ˘z) −µ˘z(nˆz−1)+r1−r2,
wherer1 =op (µˆz−µ˘z)+(nˆz−1)andr2 =op µ˘z(nˆz−1)+(nˆz−1)by Lemma A1 and Corollaries A3 and
A4. [Note that we cannot expand µˆz/nˆzdirectly sinceµˆzandµ˘z may not be convergent; our assumptions
only ensure thatµˆz−µ˘zconverges.] Lemmas A1, A8 and A9 finish the proof.
Proposition 2(Consistency under Bernoulli randomization). With a Bernoulli randomization design under our regularity conditions and restricted interference (Assumptions 1 and 2), thehtandháestimators are
consistent foreateand converge at the following rates:
(τhtˆ −τeate)=Op n−0.5davg0.5
, and (τhሠ−τeate)=Op n−0.5davg0.5
.
Proof. Decompose the errors of the estimators as such:
ˆ
τht−τeate=(τadse−τeate)+(τhtˆ −τadse), τhሠ−τeate=(τadse−τeate)+(τhሠ−τadse).
A5
Proof of Proposition 3
Definition A2. Letm=Pn
i=1Zi be the element sum ofZ(i.e., the number of treated units).
RemarkA1. Complete randomization implies thatm=⌊pn⌋almost surely for some fixedpstrictly between zero and one.
Definition A3. LetJi=Pn
j=1Ijibe the number of units interfering with uniti.
Lemma A11. Under complete randomization, for anyi ∈Usuch thatJi ≤min(m,n−m)and anyz∈ {0,1}:
−i(i.e., the number of treated units interfering withi, excludingiitself). Complete randomization gives
us thatZi andZI−iare independent conditional onT, and as a consequence:
Pr ZI
Recall the definition of the binomial coefficient:
so we have:
n under complete randomization. The
expression within brackets is strictly between−1and1, so it follows:
i (i.e., the number of treated
units interfering withiincludingiitself). We surely have0 ≤Ti +Tj ≤Ji+Jj ≤Tmin since4JiJj ≤Tmin
by assumption andJi ≥1 by definition. Fromdi j =0we have thatZIi andZ
I
j are disjoint, thus complete
Complete randomization implies thatTiis hypergeometric:
Recall the definition of the binomial coefficient:
where the third equality is a binomial expansion. It follows that:
We now bound the factors as follows:
"
and it follows that:
Pr(Ti =ti |Tj =tj)
Taken together, have we:
Proof. LetTmin=min(m,n−m)andhi=1[Ji ≤Tmin]. Corollary A2 gives us:
. (Assumptions 1a and 1c)
We can apply Lemma A11 sincehi =1impliesJi ≤Tmin.
where the second to last inequality follows from Assumption 1a and the last follows from Lemma 1.
Lemma A14. Under complete randomization,(µzˆ −µz˘ )=Op n−0.5davg0.5
.
Proof. For some sufficiently large constantk<∞, Assumption 1a gives us:
Lethi j =1[4JiJj ≤Tmin]and recalldi j anddavgfrom Definition 5:
