California's Bilingual Students after Proposition 227?
5.4 Data and Methods
504.1 The District
Pasadena Unified School District (PUSD) is in Los Angeles county. In 1998-99 it had a total population of approximately 22,000 students, of whom 18,300 were eligible to take California's yearly mandatory academic tests. As seen in Table 1, the percentage of LEPs in Pasadena (26.3 percent) is very close to the overall state figure of 24.6 percent. Pasadena's academic performance in reading, on the other hand, lagged California's in every grade. With its large Hispanic and LEP student body, PUSD is representative of California's urban school districts and a good candidate for a study of the effects of Proposition 227. However, the fact that it has a more disadvantaged and diverse student body, as seen by the variables percent Hispanic, percent black, percent in the California Work Opportunity and Responsibility to Kids (CalWORKs) program, formerly Aid for Families with Dependent Children (AFDC), and percent free lunch, may make it more difficult for any reform to succeed.
[Table 1 about here].
In 1998, there were approximately 5,400 LEP students in the PUSD, with 2,900 of these, or 16 percent of the student population, were enrolled in bilingual classes, primarily in grades K-4.5 Bilingual instruction in PUSD was a deliberate program. The average stay in bilingual classes for LEPs was four years while close to 20 percent of the teachers had bilingual accreditations. By comparison, Los Angeles county had only 15 percent of its teachers accredited in bilingual education despite having a higher percentage of LEPs (33 percent) than did PUSD (28 percent). But by 1999, after the passage of Proposition 227, PUSD had largely dismantled its bilingual programs. The majority of the bilingual students were placed in structurcd-English-immersion classes (SEI) where English was taught at the students' level, or in classrooms with some English support. Approximately 200 waivers, all from the most heavily Hispanic school, were requested by parents to keep their children in bilingual classes. The district went from roughly two-thirds of its 30 schools offering bilingual programs in 1998 to just one school in 1999.
50ver 97 percent of students in bilingual education had Spanish as their primary language.
5.4.2
The Data
To test the impact of Proposition 227, I use a multivariate linear specification with a Heckman selection process. Details of this model are in the next section and in Appendix A. The dependent variable measuring the performance of students is test score, and the explanatory variables reflect students' background and school information.6 The test scores are from 1998 and 1999 Stanford 9 tests (on a scale from 1 to 99) that all California students in grades 2-11 are required by law to take. I will focus the present analysis on total reading and math, which are tested at all grade levels.7
The set of students in the main analysis includes: 1) students who were exempted from test-taking in 1998, but not in 1999 and were in the district both years, and 2) students who took the tests both years and were in the district both years. That is, the students excluded arc those who left in 1999, were new in 1999, or were exempted from test-taking in both years. In PUSD, this latter excluded group consisted in great part of the 200 or so students who continued in bilingual classes after the reform and all attended the same school.
The independent variables incorporated in the analysis can be grouped into three cate- gories: individual, group, and school variables. The individual variables describe a student's English proficiency classification: LEP and bilingual LEP. A LEP student is a child from a non-English speaking family who scores low in an English assessment test. A bilingual LEP student is a LEP student enrolled in bilingual classes in 1998.8 LEP students tend to score significantly lower than non-LEPs in reading and math, and I expect to see this gap in the Pasadena district (National Research Council, 1998).
The group variables are race (Hispanic, black, white, other), socioeconomic level (high SES, mid-SES, low SES), family receipt of welfare (AFDC/CalWORKs), free lunch pro- gram (free lunch), and legal guardianship at home (both parents, mother, father, fos-
GThe data (proprietary to PUSD) was provided by the Testing, Research and Evaluation Center at PUSD.
7The test scores are normed curve equivalent (NCE) scores. They are obtained by first scaling the scores according to the difficulty of the questions. Next, these scaled scores are translated into a national percentile rank (NPR), which is the percentage of the national norming sample that scored equal to or less than the student. Finally, the NPR is re-expressed as a value from (1 to 99) in a normal curve with mean 50. The benefit from using NCE scores is that comparisons can be made across subjects and grades.
8S tan dardized evaluations of LEP students are problematic in that not only do districts have different criteria but, given a certain criteria, even native English-speaking students may not pass them (Rossell 2000).
ter/institution). Socioeconomic levels are derived from the real estate value of a student's res- idence address. AFDC/CalWORKs is welfare for families with children and the free/reduced lunch program is a need-based federally-funded program. Legal guardianship can be held by both parents, the mother, the father or other (foster, institution, step-parents, etc.). In general, Imver SES and welfare variables are expected to be associated with lower scores (Hallushek 1986; J\1urnane 1975), while relatively more stable households composed of both parents are expected to have a small positive effect on scores (McLanahan and Sandefur 1994). In regard to race, ample research has documented the gap in education test scores between African-American and Hispanic students with respect to white students (Jencks and Philli ps 1998).
The school variables are class size, percent full credentials, magnet school and percent teacher Hispanic. Class size is the average class size of a school, and percent full credentials is the percent of credentials held by school staff that are full (as opposed to emergency or interim) credentials. Magnet school is an indicator for the three magnet schools in the district. The evidence on class size has been mixed, with some scholars finding no effect (Hanushek 1999) and others finding a positive effect (Pate-Bain et a1. 1992). I expect higher percentages of full credentials to be associated with higher scores (Darling-Hammond 2000;
Fetler 1999). The percentage of teachers of Hispanic origin may have an impact on the probability of being exempted from test-taking, a problem discussed next.
5.4.3 Methods
To assess the independent impact of Proposition 227, we need to control for students' back- ground characteristics since the assignment into a bilingual class was not random.9 Bilingual LEPs were not only less proficient in English than non-bilingual LEPs, but they also tended to belong to more disadvantaged families. In addition, out of a total of the 14,000 students enrolled in the district in both years, more than 1,000 were exempted from taking the tests, and close to 1,000 other students simply skipped the reading and math tests.lO If the exemp-
9The assignment of LEP students into bilingual classes was the result of the district's assessment through tests and subsequent recommendations from the Bilingual Center at PUSD to the parents.
lOIn the data set the students who were exempted or missed are indistinguishable.
tions and misses were correlated with students' test scores, the underlying selection processes must be taken into account, otherwise the estimates will be biased and inconsistent (Greene 1993, 709).
I use Heckman's (1979) selection model to account for the selection process. In this model, two equations are estimated. The first equation, the one of interest, explains test scores. Without a selection process, this equation could be estimated by standard ordinary least squares (OL8) techniques. The second equation, the selection equation, uses a discrete binary model to explain whether a score is observed. In Heckman's model, the coefficients and parameters in both equations are estimated simultaneously through maximizing the likelihood of observing the data (Greene 1993, 706-711; Heckman 1979; Maddala 1996, 258- 267). An important parameter that is estimated is the correlation, p, between the errors (the non-deterministic components) in the two equations. If the correlation is statistically different from zero, this implies the two processes, scores and test-taking, are interdependent and the selection model is appropriate (see Appendix A for more details on the model). For example, a positive p being positive implies that students more likely to take the tests are also more likely to have higher scores.