III.4 Results
III.4.2 The Causal Effect of Principal Turnover
Next, we estimate the causal effect of principal turnover using a difference-in-differences model with a matched comparison group. We begin by examining a binary turnover outcome.
Figure III.3 demonstrates our approach graphically. We plot marginal predictions from our difference-in-differences model for school achievement in math and reading (and teacher retention in Panel B) in our “treatment” schools (i.e., those who change principals between year 0 and year 1) and “comparison” schools. This plot shows that, after matching, these groups follow the same downward trajectory in school performance in the pre-treatment period, consistent with the idea that we have constructed a plausible counterfactual. The difference in outcomes between the treatment and comparison groups in years 1 through 5 isolates the effect of changing principals. Panel A shows that, in both Missouri and Tennessee, schools that changed principals had lower math and reading achievement in the first and second years after the transition. In both states, schools that keep the same principal between year 0 and 1 experience an immediate rebound in performance in year 1. Schools
that change principals also rebound, but only after one or more additional years of depressed performance. In Panel B, we find that there is no obvious trend in teacher retention prior to a principal turnover event. However, there is a clear drop in teacher retention for schools that change principals between year 0 and year 1 that is not experienced by schools that keep their principal.
Figure III.4 plots the effect estimates from our difference-in-differences models (these results are also shown in Table III.A1). We estimate separate models by state for four out- comes: school achievement in math and reading, teacher retention rate (i.e., the proportion of last year’s teachers who are still in the school in the current year), and the number of new-to-school teachers. Retention and new-to-school teachers are closely linked, but show- ing the count variable facilitates interpretation of the magnitude of the effect of principal turnover on teacher turnover. In each model, the omitted category is the final year of the departing principal. Statistically significant estimates in the pre-transition period (time <
0) would suggest a violation of the parallel trends assumption, which would undermine the credibility of the post-transition coefficients (time > 0) as causal estimates of the effect of principal turnover. Across the models, the pre-transition estimates are almost uniformly small in magnitude and not statistically significant, indicating that our matching procedure was successful.
The results in Figure III.4 show that principal turnover negatively affects math and reading achievement in both Missouri and Tennessee. The effect is largest in the first year after the transition. In Missouri (Tennessee), changing principals lowers math achievement by -0.031 (-0.030) school-level standard deviations in the first post-transition year. The first- year effects on reading achievement are -0.029 and -0.019 SD in Missouri and Tennessee, respectively. These estimates correspond to roughly 7% of the typical amount of variation in a school’s achievement scores (i.e., the within-school standard deviation) over the study period in both states—similar to the magnitudes found by Miller (2013).6 We can also
6Specifically, we compute the the within-school standard deviation in math/reading achievement across the study period, then compare the magnitude of the coefficients from Table III.A1 to the within-school
(a) All Turnover
(b) Turnover Types
Figure III.3: The Effect of Principal Turnover in Missouri and Tennessee
Notes: Each plot shows the estimated margins for treatment and comparison schools relative to year of a principal change (year 0). The model coefficients are shown in Table III.A1.
Figure III.4: The Effect of Principal Turnover in Missouri and Tennessee
Notes: Each plot shows the focal coefficients (Treat x Time) from difference-in-differences models shown in Table III.A1. Error bars show 95% confidence intervals.
roughly translate these effects to student-level standard deviations in Tennessee: a principal change lowers the average student’s math (reading) score by 0.012 (0.007) SD.7
How do the first-year effects of principal turnover compare to the effects of teacher turnover? Estimating models at the school-year-grade level, Ronfeldt et al. (2013) find that 100% turnover lowers achievement in math and reading by 0.082 and 0.049 SD for all students in the grade, relative to no teacher turnover. Although the negative effects of principal turnover estimated here are smaller for the individual student, they are felt by all students in the school. For instance, the average school in Missouri (Tennessee) has a total enrollment of 408 (620) students compared to 100 (111) in the average grade level.
Even small effects on individual students, then, produce a policy-relevant total impact when aggregated across an entire school.
After the first year, the estimated effects of principal turnover in Missouri and Tennessee differ somewhat. In Missouri, there is a consistent negative effect on math achievement up to four years after a principal transition. For reading, there is a significant negative effect in the second year, but while the third and fourth year estimates are negative, they are not statistically significant at conventional levels. In Tennessee, the negative effect of principal turnover is only significant in the first year. However, it is important to note that the composition of turnover in terms of types is different between these two states. In particular, a larger share of principal transitions in Tennessee are demotions, whereas promotions and transfers are more frequent in Missouri. To the extent that the effect of principal turnover may vary by type (which we address in the next section), we would expect the effect of the average turnover event in each state to differ.
Principal turnover also decreases teacher retention and, as a consequence, the number of teachers who are new to the school in the years after the principal transition. For example,
standard deviation of the median school in each state. These median within-school standard deviations for math achievement in Missouri and Tennessee are 0.44 and 0.41, respectively (0.39 and 0.25 for reading).
7We make this translation by multiplying the first-year coefficients by the ratio of the standard deviation of school-average scores to the standard deviation of individual student scores. These ratios are 0.40 and 0.38 for math and reading, respectively.
columns 3 and 7 in Table III.A1 show that the proportion of teachers retained from the previous year decreases by 1.4 percentage points and 2.6 percentage points in Missouri and Tennessee, respectively. These negative effects on retention continue for several years after a principal transition. In Tennessee, for example, schools that changed principals (between year t and t+ 1) have lower teacher retention than comparison schools even five years after the transition. How large are these negative effects on teacher retention? In columns 4 and 8, we show the effects of principal turnover on the number of new-to-school teachers. Overall, the magnitude of these effects are modest. For instance, the first-year impact on retention translates to roughly half of an additional new teacher. Summing across the five years after a transition, the average principal turnover event increases the number of new-to-school teachers by 1.1 in Missouri and 1.9 in Tennessee. Comparing Missouri and Tennessee, one interesting note is that while the effects on achievement were smaller in Tennessee, the effects on teacher turnover are larger. Again, these differences could in part reflect differences in the composition of turnover types between the two states.8