California's Bilingual Students after Proposition 227?
5.5 Before and After Proposition 227
tions and misses were correlated with students' test scores, the underlying selection processes must be taken into account, otherwise the estimates will be biased and inconsistent (Greene 1993, 709).
I use Heckman's (1979) selection model to account for the selection process. In this model, two equations are estimated. The first equation, the one of interest, explains test scores. Without a selection process, this equation could be estimated by standard ordinary least squares (OL8) techniques. The second equation, the selection equation, uses a discrete binary model to explain whether a score is observed. In Heckman's model, the coefficients and parameters in both equations are estimated simultaneously through maximizing the likelihood of observing the data (Greene 1993, 706-711; Heckman 1979; Maddala 1996, 258- 267). An important parameter that is estimated is the correlation, p, between the errors (the non-deterministic components) in the two equations. If the correlation is statistically different from zero, this implies the two processes, scores and test-taking, are interdependent and the selection model is appropriate (see Appendix A for more details on the model). For example, a positive p being positive implies that students more likely to take the tests are also more likely to have higher scores.
bilingual LEP students may have caught up to non-bilingual LEP students' performance after Proposition 227. However, these numbers do not include statistical controls for background, nor do they account for the fact that many bilingual students did not take the tests in 1998.
The next section addresses these issues.
[Table 2 about here].
5.5.1 The Baseline in 1998
I use the Heckman specification to compare the 1998 scores of bilingual LEPs and non- bilingual LEPs, taking into account background characteristics and test-exemption biases.
Table 3 presents the complete selection model results for 1998 reading and math scores in PUSD. Almost all of the coefficients in the scores and selection equations are statistically different from zero at the 95 percent levelY The parameter, p, is 0.87 and 0.79 for reading and math, respectively. These values are large and statistically significant at the 95% level, justifying the use of the selection model. The positive p implies that the better achieving students were being tested. Furthermore, the coefficients from the test-taking equation imply that being a bilingual LEP, LEP, Hispanic, black or belonging to a school with a large percentage of Hispanic teachers significantly decreases a student's chances of taking the tests.
[Table 3 about here].
Consider the coefficients of the background and school variables appearing only in the scores equation. These can be interpreted directly as in an OLS model. In general, all of these coefficients are in the expected direction. For example, all else being equal, students with low SES backgrounds have lower scores in reading (-3.25) and math (-3.38) than students from high SES backgrounds. Having both parents in the family, on the other hand, is associated with higher scores in reading (2.46) and math (3.15) compared to students living with foster parents or in an institution.
With regard to school variables, which policymakers may influence more directly than students' SES characteristics, I find that the percentage of full credentials has a positive
11 If only the scores equation for reading is estimated with OLS, the R2 is 0.35 and a pro bit estimate of the test-taking equation has a pseudo-R2 of 0.22. These suggest a reasonable fit of the models to the data.
and statistically significant effect on reading and math scores. Increasing the percentage of full credentials in a school from 65 percent, close to the district's average, to 100 percent increases the predicted scores in reading and math by 7 points. This is a large effect when we consider, for example, that the performance increases ascribed to the recently touted reductions in class size are only about 3 points (Los Angeles Times, 1999). The impact of the class size variable is small, but surprisingly it is positive, possibly due to the fact that schools with more students in their classrooms are more likely to exempt students from taking tests.12
Next, consider the impact of the LEP and bilingual LEP variables on 1998 test scores.
These variables appear in both the score and test-taking equations. Their total marginal effect equals their effect in the scores equation plus their effect in the selection equation, with a correction weighted by the correlation estimate,
p
(Appendix B). Table 4 summarizes these total impacts. The net effect is that a LEP student enrolled in bilingual instruction in 1998 scored 2.4 points less in reading than a non-bilingual LEP, and 0.5 points more in math. These effects are statistically significant at the 95 percent level and confirm our initial expectations. Bilingual students enrolled in 1998 had statistically lower scores than non-bilingual LEPs in subjects that stress English skills.[Table 4 about here].
This lag in reading between bilingual LEPs and non-bilingual LEPs in 1998 is meaningful in educational terms. Its size is comparable to the effect of California's class size reform. In terms of the implications of this lag, the fact that bilingual LEPs did worse in reading than non-bilingual LEP students while they did virtually the same in math suggests the lack of exposure to English may have impacted bilingual students' scores.
Among all the predictors, the LEP variable has the largest effect. As might be expected, LEP students on average have much lower scores than non-LEP students. When we combine the effects of the LEP variable from both the scores and the selection equations, a repre- sentative LEP student (Hispanic and non-bilingual) scores 13.1 points less than a non-LEP student in reading and 9.7 less in math. These gaps are statistically significant and substan- tively large. Furthermore, race has an impact on scores even after language and background
12The correlation between having a reading test score in 1998 and class size is 0.24, for example.
controls. All else equal, Hispanic students score close to 6.3 and 7.3 points less in reading and math, respectively, than white students. For black students, the gap with respect to white students is even larger: 9.8 points in reading and 11.9 points in math. My analysis does not include other controls, such as parent's education and at-home behavior, which might reduce the gaps among these racial groups. On the other hand, these large gaps are consistent with many findings in the literature.13