system receive notification in the fall of their admission to university. In the months between their noti- fication of admission and their graduation from high school in March, many of these students spend less time studying. By contrast, students admitted based on their written examination have to study hard until close to their high school graduation; on average, these students have higher academic skills than those admitted through the recommendation system.
Kochi University of Technology (KUT) is a mid-sized public university with an engineering division that consists of three schools: the School of Systems Engineering, School of Information, and School of Environmental Science Engineering. The academic aptitude level of the students is average among Japanese universities. In the first quarter of 2011, students (mostly first-year students) in an introductory mechanics course were divided into three classes. Two of the three classes offered 14 traditional lectures (duration of each lecture: 90 minutes) with problem-solving recitations and laboratory work. One of the three classes offered 14 lectures with 7 tutorial re- citations using IE instruction. The IE class, in which students studied a workbook by Knight and Andrews (2006), offered peer instruction lectures and tutorial recitations in accordance with the lecture tutorial method. Both the FCI and FMCE were administered to all three classes on two different occasions—before the first class (pre- test) and before the final examination (posttest). The scores were analyzed in terms of (1) students’ university mathematics placement test scores, (2) the type of high school physics courses they had taken, (3) the type of university entrance examinations they had taken (recommendation or written systems), (4) their NCT scores, and (5) the type of instruction strategies employed in the introductory mechanics class.
Methods
Students Surveyed
Most students in the School of Systems Engineering had taken advanced high school physics, whereas most students in the School of Information and School of Environmental Science Engineering had not.
All students in the School of Systems Engineering and School of Environmental Engineering had taken a university-level introductory mechanics class, whereas less than 20% of students in the School of Informa- tion had done so. We divided the students into three classes of around 100 students each. The number of students who completed both the pretest and posttest was 291. Because most of them were first-year students in the first quarter of their studies, the pretest results represent what they learned in high school.
The percentage scores for both the pretest and posttest and the average normalized gains were calcula- ted according to the type of class (Table 1). Of the students in the two traditional classes (TR), 80% were enrolled in the School of Systems Engineering and 20% in the School of Information. All students in the IE class were enrolled in the School of Environmental Science and Engineering. Over 70% of students in the TR classes had taken advanced high school physics, whereas less than 50% of those in the IE class had.
Concept Inventory
Two types of concept inventory tests were used to assess students’ conceptual understanding of Newto- nian mechanics concepts. The pretest and posttest scores on the FCI and the FMCE were used to assess instructional effectiveness. Half the students in each class took the FCI, and the other half took the FMCE during the same period. The average normalized gains <g> shown in Table 1 are about 0.2 for the TR clas- ses and about 0.4 for the IE class, which is higher than the lower boundary of the IE class value (0.3) that was previously reported in the United States (Hake, 1998).
The FCI consists of 30 multiple-choice questions with a broader domain of topics, including two-dimension and a wider application of forces. Students who have taken advanced high school physics may be able to choose Newtonian answers more easily for some questions, namely, those that resemble problems on the university entrance examination. The reliability indexes (KR-20) were 0.87 and 0.88 for the pretest and post- test, respectively. TR classes consisting of more students with advanced high school physics had intrinsically higher scores on the pretest. However, the posttest scores showed little difference among the classes.
The FMCE, which consists of 47 questions in total, is designed to measure student understanding of one- dimensional forces and motion as well as energy conservation. This study used single-number scores (Thornton, Kuhl, Cummings, & Marx, 2009) with 33 full points, which exclude the portion of the energy conservation. Compared to the FCI, the FMCE has a sharper focus to assess student understanding of
Newton’s laws. The reliability indexes (KR-20) were 0.92 and 0.95 for the pretest and posttest, respectively.
The FMCE pretest scores were about the same in the three classes. However, the posttest scores were higher in the IE class.
The class distribution of the pretest and posttest scores for the FCI and FMCE are shown in Figures 1(a) and 1(b), respectively. The distribution of TR1 scores shows little change between the pretest and posttest. The distribution of TR2 scores shows a shift to a higher score on the FCI and FMCE. The distribution of the IE class scores shows a whole shift with a large gain, indicating the effectiveness of instruction regardless of the pretest score.
Data and Findings
Mathematics Placement Test
At KUT, a mathematics placement test consisting of 20 multiple-choice questions is administered to first- year students before classes start. Japanese university faculty members expect students with a high mat- hematics aptitude to perform better in physics. This study investigated the relationship between the mat- hematics placement test score and the FCI and FMCE scores. As shown in Table 2, students were classified into a low, middle, or high mathematics aptitude group based on their mathematics placement score. In the traditional classes, students with higher mathematics scores had clearly higher scores on both the pretest and posttest. However, in the IE class, no difference was detected in students’ posttest scores on the FMCE, in which some questions included graphs. Students with low mathematics aptitude scores in the IE class earned FMCE posttest scores as high as students with high mathematics aptitude scores in the TR class. In the traditional classes, higher mathematics aptitude scores coincided with higher <g>, as most faculty members expected. In the IE class, high <g> values prevailed in all three groups, and the value was higher than that of any groups of students in the TR classes.
High School Physics Curriculum
Students’ self-reported answers to the added question on the FCI test and FMCE test about their high scho- ol physics background are summarized in Table 3. Students who had completed an advanced high school physics curriculum had intrinsically higher pretest scores than those who had not done so. However, their FCI scores were mostly below 18 points (60% mark), which is regarded as being the “entry threshold” to Newtonian physics (Hestenes & Halloun, 1995), and their FMCE scores were below 13 points (40%). Their scores indicate that studying advanced high school physics did not substantially contribute to conceptual understanding. In the TR1 class, the normalized gains were low for both categories, indicating the ineffective- ness of instruction. In the TR2 class, students who had studied advanced high school physics had larger gains on the FCI, indicating that the instruction was better suited for them. However, students without advanced high school physics had no gain on the FMCE. In the IE class, high <g> indicate that the instruction was very effective in promoting student learning, regardless of the students’ high school physics background.
Types of Admissions Systems
KUT has two kinds of admissions systems for applicants: the high school recommendation and written examination systems. The process of applying to KUT through the recommendation system occurs in late fall.
Students’ admission through the recommendation system depends on their high school recommendation and short thesis examination results. The process of applying through the written examination system occurs in late winter, just prior to high school graduation. Students’ admission through the written examination system depends on their written examination and NCT scores. In general, the level of academic aptitude is lower among students who apply through the recommendation system. These students receive notification of their admission to university a few months before high school graduation. During the months leading up to graduation, they tend to study less and are less likely to write the NCT, which suggests that they have a soft attitude toward studying. As shown in Table 4, their mathematics placement test scores and pretest scores were lower than for students applying through the written examination system. However, their <g> in the IE class were as high as those of the students who took the written university entrance examination.