Three Oregon elementary schools participated in that first phase of the STI pilot program (Adrian and Vale elementary schools in eastern Oregon, and Captain Robert Gray Elementary in Astoria on the north coast). The second (Pilot School Settings) and third (Program Implementation) sections provide additional context that frames the evaluation of the STI program. The SOI program (classroom modules and Learning Center activities) is designed as a "treatment" to be completed by students within 7 months, i.e. the time frame of a normal school year (Robert Meeker, personal communication, Vida, OR, November 19 , 1998).
The number of students in the laboratory decreased from last year to 10% of the school population.
Student Academic Performance
Group-wise Comparisons
That is, once the effects of SES and last year's performance are accounted for, the difference in mean scores for comparison schools versus STI schools is not statistically significant at the 5% level (p = .283). That is, once the effects of SES and last year's performance are accounted for, the difference in mean scores for comparison schools versus STI schools is not statistically significant at the 5% level (p = .948).
ANOVA TOTALSS
Sum of Mean
Sum of
Sum of Squares df
ANOVA MPS COMP
Pair-wise Comparisons
In 2000, 8 of 17 comparison schools outperformed their corresponding STI school on the state assessment for Grade 3 reading/literature. In 2000, 7 of 17 comparison schools outperformed their corresponding STI school on the state assessment for 5th grade reading/literacy.
Case Studies
The student was involved in the SOI program in the classroom and in the SOI lab. The classroom teacher (the teacher who teaches four days a week) reported that the student had a good self-concept. It could be concluded that the SOI program had a moderate impact on student progress.
The SOI specialist said that the student had come to the SOI lab in the spring of 1999. The SOI specialist reported that the student was progressing through the program at a good pace. In the fall of 1998, the student was referred to the SOI Lab for academic skills and behavior.
Last year, the student's mother said the SOI Program had helped her with reading and math. The SOI specialist noted that the student continued to progress in the SOI Lab. The classroom teacher felt that the SOI Program had a positive impact on students' academic achievement in written language.
Special Education Referrals
Simply put, the available trend data do not show a systematic or widespread effect of the SOI program in terms of reducing the number of elementary school students referred for special education evaluations in these pilot schools. In addition to the quantitative data on the number of special education evaluation referrals provided by schools, there are three sources of data relevant to the question at hand, including I) transcripts of focus group interviews with SOI experts and technicians conducted in the spring of 2000;. The SOI experts and technicians who participated in this year's focus group meetings provided only modest direct evidence addressing the possible specific educational effects of the SOI program.
However, it should be understood that specialists and technicians working within the framework of the SOI lab are probably not in the best position to comment on changes in special education referral rates or services to schools. Still, the SOI Lab staff offered the following: “We see the kids coming out of IEPs, special services and being in the classroom.
ANOVA
Behavior (Disciplinary) Referrals
The evaluation of the SOI program included a comparative analysis of the rates at which students were referred to the school office (eg, principal or assistant principal) for unacceptable behavior (this includes classroom, playground, bus, and general school behavior). This question is of particular interest because, based on the literature, SOI and IDS argue that the benefit of the program is to improve the number of disciplinary referrals throughout the school (IDS, 1997a). In Year 2 of the evaluation, both SOI schools and relevant comparison schools were asked to provide data on the number of pupils referred to the school office for unacceptable behavior in the two years prior to the SOI program and in 1998–99.
In addition to graphical analysis of available trend data (each school's referral rates per student for the most recent two school years), focus group transcripts, interview data collected from school staff during site visits, and surveys Teacher satisfaction also provides evidence on the potential effects of the SOI Program on improving school-wide behavior (ie, reducing the number of school-wide disciplinary referrals). In other words, the analysis of variance showed no effect of the SOI Program on rates of. As mentioned above, the comparative statistical analysis was complemented by graphical analysis of the 2-year trend data available for the 16 OS.
Five of the 13 comparison schools (38%) saw a decline in behavioral referral rates over the year, with one school seeing a significant decrease (CS 1). Seven of the 13 comparison schools (54%) recorded a year-on-year increase in behavioral referral rates, with one school reporting a significant increase (CS 17). SOI experts and technicians who participated in this year's focus group meetings provided some scattered anecdotal evidence addressing the potential effects of the SOI program on improving student behavior.
ANOVA Behavior Referral Rate (per student)
English Language Acquisition
The evaluation of the SOI program included a comparative analysis of the number of students receiving English as a Second Language (ESL) services in SOI and comparison schools and the time it takes for students to progress through ESL programs (an indicator of students' frequency of English language acquisition). Both SOI and matched comparison schools were asked to provide data on the number of students classified as ESL receiving ESL program services for the two years prior to full implementation of the SOI program and for the two school years in which the program has functioned. For this school year, 16 of 17 SOI schools and 15 of 17 comparison schools reported data on the number of students receiving ESL services at the beginning and end of the current year.
In addition to the school data shown in Table 4.16, a number of informal interviews with school principals during school visits provided the necessary insight into changes in the number of students classified as ESL over the course of the school year. Comparing the number of children enrolled in ESL programs in schools and the time required to transition the program is challenging. According to school principals, in many schools, changes in the number of students attending ESL school are due to the annual migration (primarily to and from Texas) of mainly Mexican farm workers.
The essential point here is that in many cases that show seemingly remarkable changes in the number of ESL students served, these changes resulted from the movement to and from school of students accompanying migrant or mobile parents. From the data collected and analysis performed to date, the evaluation found that the number of ESL students serving schools is part of each school's unique context, and can vary widely within each school over the course of a school year . Therefore, the evaluation of the SOI program does not address whether the program has a beneficial effect on language acquisition rates, as the numbers of students entering and/or exiting ESL services reflect mobility rather than language acquisition.
School Attendance (Average Daily Attendance)
N = 34 (17 SOI schools & 17 comparison schools); In some cases, the average is based on less than 3 years of data; C = comparison school; *school participating in the SOI program for a third year; no = not reported. In addition to the statistical analysis of attendance rates for the current school year as well as changes in attendance rates, graphical analysis of the 3- or 4-year trend data available for 13 SOI schools is shown. As shown, 5 SOI schools experienced an improvement in attendance for the current year over the previous year's rate.
Interestingly, both of the STI schools that participated for a third year experienced attendance rates very similar (within half a percentage point) to those recorded before the STI program was in place (Adrian and Grey). The results of the teacher satisfaction surveys indicate that of the 198 teachers by the middle of the year, and 143 by the end of the year, 74% and 64% respectively agreed or strongly agreed that the STI curriculum modules were enjoyed by their students. In addition, the STI school staff at the focus groups and during site visits generally agreed that their students really enjoyed attending the STI lab.
There is little anecdotal evidence of the effect of the SOI Program on student school attendance. Also, statistical analysis shows that there are no differences in current year attendance rates, or change in attendance rates, between schools experiencing the SOI Program and similar schools that did not participate. The supplementary graphical analysis also shows only one SOI school which can be considered in an improving trend in terms of student attendance.
Teacher Satisfaction
The teachers expressed their support for the STI program by describing gains they observed in their students. For that purpose, 17 comparison schools were carefully selected that matched the salient features of the STI schools. Is there a significant difference in the levels of Special Education referrals between schools experiencing the STI program and comparison schools not participating in the Program.
Is there a significant difference in the levels of behavioral referrals between schools experiencing the SOI program and comparison schools not participating in the program? Is there a significant difference in student attendance rates between schools participating in the SOI program and comparison schools not participating in the program? Is there a significant difference in the rate of behavioral referrals between schools experiencing the SOI program and comparison schools not participating in the program?
That said, the implementation and effectiveness of the SOI program depends to some extent on the perspective presented. In many schools, children who received special education services were also enrolled in SOI Learning Centers. School administrators and teachers generally support SOI school staff in this regard.
Student Achievement Improvement Adnan Elementary SchoolssatisfyingsatisfyingsatisfyingimprovedCIsatisfyingsatisfyingimproved Allen Dale Elementary SchoolstrongstrongexceedinglysatisfyingimprovedC2strong_strongsatisfactorysatisfyingimprovedBear Creek Elementary SchoolstrongstrongstrongstrongsatisfactoryimprovedC3satisfactorysatisfactorysatisfactoryimprovedEvergreen Elementary Schoolfull doingstrongstronglysatisfyingimprovedC4satisfactorysatisfactorysatisfyingimproved Fairview ElementarystrongstrongstrongstrongstronglysatisfyingimprovedC5satisfyingsatisfyingstronglysatisfyingimproved Fossil Elementary SchoolstronglysatisfyingsatisfyingimprovedC6satisfactorysatisfactorylowlysatisfyingabout the same Goshen ElementaryschoolstrongstronglysatisfyingfactoryimprovedC7strong_stronglysatisfying improved gray primary schoolssatisfyingsatisfyingsatisfyingimprovedC8satisfyingimprovedvery satisfactoryimproved McGovern Elementary SchoolstrongstronglysatisfactorysatisfactoryimprovedC9strongstrongsatisfyingsatisfactoryimproved Milner Cre;t Elementary SchoolsatisfactorystronglysatisfactoryimprovedC1 0strong_strongstrongsatisfactoryImproved Rhododendron Elementary Schoolssatisfactorystrongsufficientsatisfactoryfar improvedC11strong_strongstronglysatisfyingimproved Riddle Primary SchoolsatisfactorysatisfactorysatisfyingdecreasedC12satisfactorysatisfyingstronglysatisfyingimproved Stella Mayfield ElementarysatisfyingstronglysatisfyingimprovedC13strongstrongstronglysatisfyingimproved Sweetoriar ElementaryschoolstrongstrongverysatisfyingimprovedC14strongstrongexceedinglysatisfyingImproved Thurston Elementary SchoolstrongstrongstrongstrongstrongsatisfactoryimprovedC15strongstrongstronglysatisfactoryImproved Warrenton Grade SchoolsatisfactorysatisfactorystronglysatisfactoryimprovedC16strongstronglysatisfactoryimprovedimprovedWhitworth Elementary Schoolsatisfactorysatisfactorystronglysatisfactoryimproved edC17satisfyingstronglysatisfactoryimproved
SOI THIRD PARTY EVALUATION YEAR 3 FOCUS GROUP PROTOCOL
What grade do you teach?
How long have you been teaching?
How many of your students go to the Bridges/SOI Lab?
Do you use the Bridges/SOI classroom modules in your classroom?
In your view how do the classroom modules benefit your students?
Please describe any negative aspect, for your students, in using the classroom modules
In your view, what contribution do the modules make in preparing your students to meet the Oregon Benchmarks?
For all of your students, what effects, in the following areas, have you directly observed that you feel can be attributed to the classroom modules
Please indicate your current feeling with regard to keeping the SOI program (modules and lab) next school year
What other comments do you have about the Bridges/SOI classroom modules, or the SOI model schools program in your school?
Thanks for your help in collecting the data for your school that will be used in our continuing evaluation of the SOI Program
Re: Collection of school data for the 1999-2000 Structure of Intelligence (SOI) model schools pilot program. Thank you for your help in collecting data about your school that will be used in our ongoing evaluation of the SOI Program.
As was the case last year, we will be sending you a check for $200.00 to help offset some of your copying, postage, and other expenses associated with collecting these data
Attached please find the data forms that you will need to complete (one each for special education referrals, behavior referrals, English as a second language, average daily
The teacher satisfaction form should be copied and distributed to all of your teachers twice this year, once at around Christmas break (mid-December), and once toward the
The other data forms should be completed on an ongoing basis, and returned to us at the end of the school year
Thanks again for your continued help in conducting this evaluation. We wish you a great school year!
