(ACS Symposium Series 1228) Luker, Christopher S. Muzyka, Jennifer L - The flipped classroom v2-American Chemical Society (2016)

(1)

The Flipped Classroom Volume 2:

Results from Practice

Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.fw001

(2)

(3)

ACS SYMPOSIUM SERIES 1228 The Flipped Classroom

Volume 2:

Results from Practice

Jennifer L. Muzyka, Editor

Centre College Danville, Kentucky

Christopher S. Luker, Editor

Highland Local Schools Medina, Ohio

PRINTED IN THE UNITED STATES OF AMERICA

(5)

Foreword

The ACS Symposium Series was first published in 1974 to provide a mechanism for publishing symposia quickly in book form. The purpose of the series is to publish timely, comprehensive books developed from the ACS sponsored symposia based on current scientific research. Occasionally, books are developed from symposia sponsored by other organizations when the topic is of keen interest to the chemistry audience.

Before agreeing to publish a book, the proposed table of contents is reviewed for appropriate and comprehensive coverage and for interest to the audience. Some papers may be excluded to better focus the book; others may be added to provide comprehensiveness. When appropriate, overview or introductory chapters are added. Drafts of chapters are peer-reviewed prior to final acceptance or rejection, and manuscripts are prepared in camera-ready format.

As a rule, only original research papers and original review papers are included in the volumes. Verbatim reproductions of previous published papers are not accepted.

ACS Books Department

(6)

Preface

A full introduction to the flipped classroom and its history can be found in Chapter 1 of the first volume of this collection. Below is the description of the content in both Volume 1 and Volume 2 of this book, which also appears in Volume 1.

Volume 1 of this collection starts by demonstrating how faculty members generate buy-in for novel pedagogical methods. Swearingen describes how she flipped the syllabus in her general chemistry course at John Brown University, introducing students to the novel approach and generating buy-in among students for the method. Next, the reader is introduced to logistics of implementing the flipped classroom. Storer describes his implementation of the flipped classroom in a general chemistry course at a community college in rural Ohio. An important characteristic of this course is that it served as a dual enrollment course for high school students in the region, many of whom did not have Internet access in their homes. His creative approach demonstrates logistics that make flipping possible even in challenging circumstances.

The next few chapters describe different methods used in flipped courses, transitioning into the educational theory behind the flipped course. Although most flipping of chemistry courses happens in general chemistry, the following two chapters both focus on physical chemistry courses. Goss describes the use of Just- in-Time Teaching combined with screencast videos that demonstrate the use of a symbolic math program like Mathematica to flip her physical chemistry courses at Idaho State University. Hagen describes the use of team-based learning (TBL) to flip his thermodynamics course.

Morsch’s organic chemistry course is atypical, as each student is required to bring his or her own iPad to participate in the course. Morsch’s students access pre- class videos on iTunes U and read assigned text on the ChemWiki. Students use a variety of apps on their iPad devices to respond to questions that Morsch poses.

Morsch introduces the cognitive load theory to explain and interpret enhanced grades and student responses to surveys about the teaching method.

Lekhi’s general chemistry students at the University of British Columbia are being challenged to develop skills that will enable them to productively participate in research projects. She explains how the flipped classroom promotes in these students a more sophisticated epistemology as they develop these research-ready process skills.

Of the chemists who are aware of the flipped classroom, many believe that the approach can only work in small classes. Several authors in this collection (Stoltzfus, Link, Soult, and Yestrebsky) dispel that notion, describing their successful implementations in courses that have over two hundred

ix

Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.pr001

(7)

students. Stoltzfus teaches general chemistry at The Ohio State University.

Link teaches organic chemistry at University of California, Irvine. Soult teaches general-organic-biochemistry for nurses at the University of Kentucky.

Yestrebsky teaches general chemistry at University of Central Florida. Yestrebsky presents data demonstrating that average students benefit from the flipped teaching, with larger percentages of A’s and B’s in the flipped course than in a matched lecture course.

The chapters in Volume 2 of this collection provide further data about how flipping influenced their students’ learning. Most authors found enhanced learning (Yestrebsky, Casadonte, Haak, Read, Houseknecht, Esson, and Muth); one reports similar grades (Maloney) in a course that previously included significant amounts of active learning. Casadonte flipped his honors general chemistry course at Texas Tech University. Haak describes a hybrid course with reduced face-to-face time for a general chemistry course at Oregon State University. Read describes partial flipping at University of Southampton. Houseknecht implemented Just-in-Time Teaching in organic chemistry at Wittenberg University, having students generate iPad screencasts in groups. Maloney teaches organic chemistry courses for classes of biology majors with up to 100 students. Esson flipped both general and analytical chemistry at Otterbein University. Finally, Muth describes his flipped biochemistry course at St. Olaf College.

Jennifer L. Muzyka

Department of Chemistry, Centre College 600 W. Walnut St.

Danville, Kentucky 40422

[email protected] (e-mail)

Christopher S. Luker Highland Local Schools 4150 Ridge Rd.

Medina, Ohio 44256

[email protected] (e-mail)

x

Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.pr001

(8)

Chapter 1

Direct Comparison of Flipping in the Large Lecture Environment

Cherie L. Yestrebsky^*

Chemistry Department, University of Central Florida, Orlando, Florida 32816, United States

*E-mail: [email protected]

Very large lecture-based classes are a commonly used teaching mode at high-population universities. To ascertain the effectiveness of ‘flipping the classroom’ in these classes, a study focused on the change in the presentation mode: in-person lectures versus recorded lectures posted online with problem solving during class time. The study involved two very large classes (320 and 415 students) of second-semester general chemistry students taught by the same instructor. One class was taught in the traditional lecture format normally used within the department with example problems posted online.

The other class was taught using a flipped protocol and those students accessed all lectures online with class time devoted to instructor-led examples and small group problem solving.

Final grades were compared between the two groups and results showed that students in the flipped class had a greater percentage of high grades (‘A’ and ‘B’ grades) compared to the control group. The control group had more ‘C’ or average grades but the two groups had almost identical percentages of low grades (‘D’ and ‘F’). This suggests that the average performing students were aided by this teaching method compared to the traditional teaching format. Surveys that were administered to each class at the end of the semester revealed that students in the flipped class found the online instruction valuable; 86% watched at least some recorded lectures more than once and 68% responded that they would take another class using this teaching method. The control class expressed a high evaluation of the in-class instruction but did not express

Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch001

(9)

a high evaluation of the example problems and slides (without recorded lecture) provided online.

Introduction

The concept of ‘flipping the classroom’ or ‘flipping’ has received considerable interest in recent years (1–4). The basic concept refers to a classroom where students reverse the normal lecture-class routine of listening and observing an instructor during class time with homework and practice outside of class. In a flipped classroom, students listen to and watch the videotaped lecture or other instruction on their own, often via some form of access to the internet, and class time is used for discussion, independent work with teacher guidance, group work, peer instruction, teacher led examples, etc. Much of the published literature on the topic focuses on examples in relatively small classrooms of less than 50 students while far fewer publications focus on college-level, large lecture course studies of this mode of teaching.

The passive learning environment of a large science lecture presents fertile ground for testing better methods of engaging students. Motivated instructors can certainly engage many students but the interaction with students in this environment is limited. Therefore, if a student has a question, he/she is likely too intimidated to interrupt the lecture and relatively few will reach out to the instructor during office hours. Flipping is an effort to engage students in active learning, which requires learners to take some responsibility for their own learning experience. College-level studies have shown reductions in DFW grades (5–7) and benefits in final grades of students in courses that involved varying levels of a flipped classroom environment for moderate- and small-sized chemistry courses; however, literature is lacking for flipping the larger classes of over 300 students. Schneider (2015) (8) showed that students liked the flipped classroom environment but there was no improvement in their grades. Other studies have shown little or no benefit as measured in student performance or student opinion of flipping (9) and not all subject areas may benefit from this change in teaching.

This study seeks to evaluate the basic concept of flipping in a large chemistry classroom by using a side-by-side comparison of two very large classes, one with 320 students and the other with 415. The intent was to compare the final grades of the two classes, keeping all materials and actions the same with the exception of an in-class lecture versus recorded lectures available through the university’s Webcourses (Learning Management System) site. The goal in this study was to evaluate the effectiveness of flipping to improve the DFW rate for this course.

Methods

Description of the Classes

This study took place at the University of Central Florida (UCF) Chemistry Department. UCF is a large public institution with over 63,000 students, 86% of whom are undergraduates. Many of our undergraduate students transfer to UCF from regional state colleges.

2

(10)

Fundamentals of Chemistry II is the second-semester course in a two-semester sequence of the pre-requisite chemistry courses for most science, health, and many engineering majors. The level of college experience of the students in this class varies as shown in Table 1 with sophomore, junior, and senior level students comprising approximately equal populations in the class and with freshman students making up only 8-10% of the class. Fundamentals of Chemistry I is a prerequisite course that introduces students to the theories of chemistry and some simple calculation problems, but is not as mathematics-dependent as the second semester course. Based on past student perception of instruction survey comments, students find the math in Fundamentals of Chemistry II challenging and believe that more examples and help with problem solving would improve their grades.

It is not uncommon to have completely full classes with as many as 450 students enrolled at the beginning of the semester. The environment is not ideal for significant interaction with the professor, particularly during lecture. It is taught in a large stadium-seating auditorium using a computer projection onto one or more very large screens, depending on which auditorium is used, with the instructor using a wireless microphone for communication. This does allow for some instructor movement about the classroom, but clarity of voice can diminish due to limited microphone range. There are opportunities for questions from students during class; however, the interaction is limited. Because the auditorium is large, the distance between the instructor and many of the students can cause those students to feel dissociated from interaction with the class. Homework problems are suggested and examples are worked in class by the instructor.

Further examples are often uploaded to the class website on the university Webcourses learning management system, as are copies of the lecture slides.

The classes are 50 or 75 minutes, depending on the scheduled days (Monday/Wednesday/Friday or Tuesday/ Thursday) that the classes are taught.

Grades are determined by four multiple choice exams, the best 10 of 14 quizzes, a final exam (ACS two-semester general chemistry 2011 version), and up to 3% attendance credit. The course is known for having a high DFW rate, so outside the classroom, help is available to students including supplemental instruction, group tutoring through the Student Academic Resource Center, and a department-supported Chemistry Tutoring Center.

In order to understand the effect of flipping, the researchers changed only one aspect of the flipped class and kept all other variables constant. Therefore, only the lecture delivery mode was changed for the test class and problem solving periods were used to replace lectures during class time. The specific problems addressed in the flipped class were uploaded to the class website for the traditional class to access so that both classes had the ability to review and study the same worked examples. The traditional and flipped classes had 320 and 415 enrolled students, respectively. Students registered for the classes prior to knowledge of the study and were comprised of overwhelmingly science and engineering majors. The efficacy of using the flipped instructional method was evaluated using two classes of very high enrollment, taught by the same instructor, with only one variable changed, and comparing 1) quiz and exam grades, 2) distribution of final grades, and 3) responses from end-of-semester surveys. Final grades were assigned based on a

3

(11)

standard 10-point grading scale of 90-100% = A; 80-89.9% = B; 70-79.9% = C;

60-69.9% = D; and below 60% = F. Percentages are calculated based on total points earned divided by total possible class points (800) multiplied by 100.

The slides used for the flipped class were identical to those used in the traditional class with the exception of the voice-recording (using a plug-in microphone headset) over PowerPoint slides. One could argue both for and against video recording, but the time and location flexibility provided in preparing slides with voice recording was an important benefit for the instructor. The slides and time periods spent on each chapter were the same. The problems worked out during class time for the flipped class were made available to the students in the control class. The quizzes and exams were of equal difficulty, covering the same topics from the chapters with the same number of applied and conceptual problems. Based on the idea that long modules would lead to bored listeners who might procrastinate listening to lectures, all recorded modules were 18 minutes or less. This equated to one 50-minute lecture for the traditional class and three to four recorded modules for the flipped class for each class period. The recorded modules were then uploaded to the course website. Only the flipped class could access the recorded lectures but both classes could access the slides without the recorded lecture. Each chapter was covered in seven to twelve recorded modules. Dates were assigned for students to complete specific modules and the course calendar was used to communicate these dates. During class time, either the instructor or the students (in small groups or individual) in the flipped class worked on end-of-chapter problems from the course text that corresponded to the material covered in the appropriate lectures. Of the time spent on problem-solving in class, approximately 40% was instructor-led, 40% small-group, and 20%

individual work.

Surveys

During a two-week period near the end of the semester, both classes completed Student Perception of Instruction (SPOI) surveys, administered online and mandated by the university. The SPOI surveys include general questions regarding professionalism of the instructor, timeliness of assignments and grading, respectfulness of the instructor towards students, and open-ended questions for the students to express their likes and dislikes of various aspects of the course.

A second survey was developed specifically for this study and was administered to both classes during class time at the end of the semester. This survey queried students on instructional components that were specific to these courses, including the usage of online materials (both recorded slides for the flipped class and the materials posted for the traditional class), students’

anticipated grade for the class, satisfaction with the course format, and other general likes and dislikes of the course and/or its format. There was no extra credit or incentive offered to students for completing the survey and no penalties for those who did not participate. Participation was voluntary and students’

survey data were aggregated into the data for the study as a whole.

4

(12)

Student demographics and academic ability levels (as indicated by aptitude test scores) for each of the sections were compared from data obtained from the student information system (SIS).

Results

Student Demographics

Academic experience of the students enrolled in these classes is distributed mostly across the sophomore, junior, and senior level with 8-10 percent at the freshman level. Table 1 illustrates the breakdown of student academic level.

Table 1. Academic level of classes (%) Traditional

(N=320)

Flipped (N=415)

Non-degree seeking student 0 2

Freshman 8 10

Sophomore 32 36

Junior 32 28

Senior 27 21

Table 2 illustrates the proportion of males and females in each section. Both sections of chemistry had a higher proportion of females, but were similar overall.

Table 2. Gender distribution of classes (%).

Gender Traditional

(N=320)

Flipped (N=415)

Male 46 40

Female 54 60

Table 3 lists the distribution of ethnicity, which varied slightly for each of the courses, with the flipped class enrolling more Asians, while the traditional section had slightly more Black/African American, Hispanic/Latino, and White/

Caucasian students.

5

(13)

Table 3. Ethnicity distribution of both classes (%).

Ethnicity Traditional

(N=320)

Flipped (N=415)

Asian 8 18

Black/African American 12 9

Hispanic/Latino 23 20

Native Hawaiian/Other Pacific Islander 0 0.2

White/Caucasian 52 48

Multiracial 4 4

Other 1 1

This research did not examine differences in demographics. Students were unaware that they were registering for a flipped or traditional course and it is possible that the disparity in ethnicity is due to day or time of each class and how they fit with the particular student’s schedule. This, however, is outside the scope of this research.

Students’ Prior Academic Ability Measures

Table 4 illustrates the differences across the two classes in prior academic ability measures—namely, college entrance exam scores (SAT and ACT) and high school grade point average (GPA). Independent t-test analyses comparing these averages across the two classes found no significant differences (p<.05).

Grades

Final grades in the course were calculated based on the assessments listed in Table 5. Fourteen quizzes were administered over the course of the semester, each with five questions with the 10 highest scores used for calculating the final grade.

Four exams with 20 questions each were administered approximately every three weeks over the semester. Values listed in tables for quizzes and exams are the mean of the raw score for the number of correct answers.

The grading rubric shown in Table 6 illustrates the calculation of student grades. This was identical for both traditional and flipped sections.

6

(14)

Table 4. SAT, ACT, HS GPA Traditional

(N=320)

Flipped (N=414)

Mean SD Mean SD t df

SAT Total 986.82 412.21 990.77 407.75 -0.13 687

SAT Verb. 491.14 208.67 484.74 202.94 0.41 687

SAT Math 495.69 209.79 506.03 210.97 -0.64 687

ACT Total 17.84 11.22 19.04 10.82 -1.42 687

ACT Engl. 17.41 11.17 18.69 10.97 -1.51 687

ACT Math 17.78 11.25 19.19 10.94 -1.66 687

ACT Sci. 17.22 17.94 18.53 10.59 -1.58 687

HS GPA 3.34 1.19 3.47 1.17 -1.50 720

Table 5. Assessment tools used for both classes.

Assessment Frequency How it was used

Quizzes 14 Used best 10 for final grade

Exams 4 Replaced lowest with final exam % if higher

Final Exam 1

Attendance Quizzes 3 Added 1% to final grade % for each one taken

Table 6. Grading rubric for courses.

Assessment Value Each Total Value (pts.)

Notes

Quizzes 20 pts. 200 14 total, drop lowest 4

grades

Exams 100 pts. 400 4 total, lowest score replaced

with final exam % if higher

Final Exam 200 pts. 200

Total possible points 800

Extra Credit Attendance Quizzes

1% 3% added to final grade percent

7

(15)

Scores on all graded work, quizzes, and exams were compared for each section. Students in both classes were given 14 quizzes through the semester, of equal difficulty and question type, with five questions each. The mean raw score for each of the quizzes along with statistical analysis of the differences in the scores between classes is listed below in Table 7. Independent samples t-tests were used to examine whether there were significant differences between the various assessment metrics. Of all graded materials, the quizzes showed the most variability with the flipped class having significantly higher scores for seven of the quizzes, the traditional class having one quiz with significantly higher scores, and the other quizzes having no significant difference between the two classes.

Table 7. Analyses of flipped vs. traditional quiz scores.

Traditional (N=269)

Flipped (N=369)

Quiz 1 2.68 .87 2.69 .88 -0.21 636

Quiz 2 2.46 1.43 2.85 1.31 -3.52^a 636

Quiz 3 2.55 1.32 2.70 1.29 -1.42 636

Quiz 4 2.05 1.36 2.24 1.58 -1.60 636

Quiz 5 3.13 1.43 3.86 1.36 -6.54^a 636

Quiz 6 2.94 1.61 3.24 1.52 -2.41^b 636

Quiz 7 2.85 1.45 3.50 1.63 -5.17^a 636

Quiz 8 3.29 1.53 3.85 1.60 -4.40^a 636

Quiz 9 4.08 1.40 4.17 1.43 -0.82 636

Quiz 10 2.08 1.42 1.83 1.36 2.24^b 636

Quiz 11 3.86 1.69 3.86 1.84 -0.05 636

Quiz 12 2.72 1.71 3.45 1.82 -5.12^a 636

Quiz 13 2.39 1.65 2.51 1.71 -0.92 636

Quiz 14 3.52 1.78 3.34 1.97 1.21 636

ap<.01. ^bp<.05.

Exam scores shown in Table 8 present no consistent increase in achievement for either method of instruction with the traditional class scoring higher on two exams and the flipped class scoring higher on one. The scores for the final exam, the American Chemical Society two-semester general chemistry exam (2011) are similar for both classes. The overall (composite) scores for all quizzes and exams were not statistically different for the two classes.

8

(16)

Table 8. Analyses of flipped vs. traditional exam scores.

Traditional (N=269)

Flipped (N=369)

Exam 1 10.90 3.70 12.96 3.25 -7.31^a 531

Exam 2 14.45 3.96 13.80 4.11 2.02^b 636

Exam 3 13.70 4.31 12.15 3.92 4.72^a 636

Exam 4 11.60 4.67 11.78 4.74 -0.46 636

Final exam 40.24 12.28 41.47 13.15 -1.21 636

Exams + quizzes 68.90 15.27 70.64 15.04 -1.43 636

ap<.01. ^bp<.05.

Discussion and Conclusion

To encourage attendance for class time, attendance quizzes were given three times during the semester, unannounced, and each counted for one percent added to the final calculated percentage grade for the class. This provided an opportunity to increase the final grade by up to three percent in both classes. Figure 1 shows the distribution of final gradeswithoutattendance credit for both classes involved in this experiment. This data represents solely student performance on assessments and allows for direct comparison of the different teaching modes used. The percentages of D and F grades (and withdrawals) remained almost the same in the flipped class as in the traditional class. This outcome was disappointing since part of the motivation for testing the flipped modality was to ascertain if this teaching style could reduce the number of lower-performing students. Also, this outcome is not in agreement with other studies that found a significant decrease in DFW grades using the flipped teaching method (5–7). One reason for this could be due to the fact that great care was taken in this study to only evaluate the effect of moving the lecture content to online access. Further efforts to add more student-involved experiences during the problem solving session held during class time may have improved the success of the students who earned a DFW final grade. The percentage of C grades in the traditional lecture class was almost six points higher than the flipped class, while the number of A and B grades are three percent higher in the flipped class. One might conclude, therefore, that those students who would otherwise be performing at an average level were aided in improving their grades with the flipped teaching mode. It is also logical that the more highly motivated students who might otherwise earn a B in the class were helped to improve their grade to an A. The option of listening to lectures multiple times through the online lectures and having more instructor and peer-led practice in class most likely helped those who had improved grades;

however, data and the lack of survey questions directly associated with the students’ grades cannot substantiate the validity of that speculation.

9

(17)

Figure 1. Comparison of final grades for both classes without points for attendance included.

The results of the final grade comparison changes slightly with the addition of the attendance credit (Figure 2). Students in the traditional class had an average of 2.1 points for attendance and the flipped class had an average of 2.2 points for attendance. With the exception of the percentage of students earning a C grade, the distribution appears to have moved towards higher grades with a decrease in D and F grades and an increase in A and B grades. The comparison of B grades for the traditional class is higher than that of the flipped class but A grades still are higher for the flipped class. The differences between the two sets of data are mostly greater than three percent which suggests that many students were close to a higher grade without the added point(s).

Furthermore, the consistency of improvement in grades with attendance credit could be viewed as a valuable tool for the course given that students across grades A through F were still coming to class.

10

(18)

Figure 2. Comparison of final grades for both classes with points for attendance.

Student Surveys

Student surveys provided valuable information on their perspectives and experiences given the particular instructional method they experienced. One survey was the standard Student Perception of Instruction survey (SPOI) offered to all UCF students enrolled in a class and administered automatically by the university within a two week period near the end of the semester. Students are reminded that the survey is open when they enter the university portal. Response rates for this survey were 58% in both the traditional (N=185) and flipped (N=239) sections. The second survey was administered in each class and was developed specifically for this research. Response rates were high for both courses, with 253 students (61%) completing the survey for the flipped section and 320 students (68%) who responded to the survey for the traditional course.

11

(19)

Student Perception of Instruction (SPOI)

Based on positive anecdotal feedback from students in class, it was expected that the overall SPOI ratings would be similar for the two sections. However, those who completed the SPOIs for the traditional class responded with higher overall satisfaction and seemed to have more of a connection with the instructor than the flipped class (Table 9). Possibly, this might have been influenced by the comfort and experience of the instructor with the traditional lecture teaching format. Using the 50-minute class period for problem-solving/small-group or individual work was a new teaching mode and the instructor may not have been as approachable for student interaction. The flipped class had the opportunity and flexibility to watch and learn from the online lectures when it was convenient for them. They could also review the lecture more than once, if desired, which led to high favorability for the online component.

Table 9. Selected Survey Results SPOI Survey Questions Traditional Class

Response

Flipped Class Response Overall evaluation of the

course

72% very good or excellent

66% very good or excellent

What did you like most about the course (open ended question)

Most common response:

78% mentioned professor characteristics

Most common response:

54% mentioned online component

Class-Specific Surveys

Several questions on the research-specific surveys revealed remarkably similar attitudes among students from both classes. Students were asked what grade they expected to make in the course and the percentage responses were almost identical across the two sections (Table 10). If students were accurate regarding their grade, then respondents’ grades deviate widely from the actual overall class final grades, possibly reflecting a potential bias in who completed the survey. Better-performing students often have better attendance and involvement, and are likely to be a larger proportion of the respondents. If so, the survey will not capture feedback from students who may not be as successful in the class and have either dropped out or rarely attend. As the survey was anonymous, researchers could not corroborate expected grade with students’ actual grades to determine if their expectations were accurate.

12

(20)

Table 10. Students’ expected grade in the course (%) Traditional

(N=215)

Flipped (N=250)

A/A- 23 25

B+/B/B- 38 37

C+/C/C- 36 36

D+/D/D- 3 2

F 1 0

When asked to rate the quality of the course, the responses from both classes, shown in Table 11, were also very similar, χ²(4,469) = 0.55, p=.97. Overall, the majority of students rated both sections as good or better.

Table 11. Students’ perceptions of overall course quality (%) Traditional

(N=217)

Flipped (N=253)

Poor 2 2

Fair 14 19

Good 35 30

Very Good 33 35

Excellent 16 15

The results in Table 12 show that both the traditional class and the flipped class overwhelmingly chose the flipped mode when asked which teaching mode would be more effective in helping them learn.

Table 12. Format students indicated helped them learn more (%) Traditional

(N=215)

Flipped (N=250)

Flipped 70 80

Traditional 30 20

13

(21)

However, if a department depends heavily on SPOI surveys to analyze teaching for promotion, tenure and awards, switching to this method may be preferred, and possibly benefit some students but could harm the instructor if students rate non-tenured faculty lower in a flipped mode class. Certainly, research is warranted to determine how best to utilize class time when using the flipped method in order to maximize the benefit for students while minimizing any burden or adverse effect on the instructor.

The flipped teaching mode requires students to be self-disciplined regarding listening and watching modules each week prior to the live session in order to prepare for in-class problem solving. Table 13 illustrates that 74% of students indicated that they watched the recorded lectures in the same week it was covered in class. This is certainly an important aspect of student success for the flipped teaching mode. Another important advantage of the flipped class is being able to watch and listen to lectures more than once. Table 13 shows that 86% of students watched at least some lectures more than once.

Table 13. Students’ interaction with videos in flipped course (%).

Nev=Never, R=Rarely, S=Sometimes, M=Most of the time, A=Always

N Nev R S M A

Watch video in same week material

was covered 251 1 4 21 46 28

Wait to watch the videos close to

exam 249 14 30 34 19 3

Watch videos more than once 248 14 17 33 23 13

Table 14 illustrates a significant difference in the amount of homework that students in the two sections indicated they completed: χ²(5,467) = 12.50, p=.03.

In the flipped section, 27% of students indicated that they completed 75% or more of their homework, while in the traditional class 34% of students completed that amount. This is a concern based on the difficulty of the class and the importance of the homework toward successfully completing the class. Perhaps the students in the flipped class believe they have mastered the concepts because they are either watching the instructor work out problems or were involved in peer-problem solving during class time.

14

(22)

Table 14. Homework completed by students (%) Traditional

(N=216)

Flipped (N=251)

0-None 19 13

1-24% 12 14

25-49% 13 22

50-74% 22 25

75-99% 20 19

100%-All 14 8

Responses to students’ opinions of various class aspects and teaching formats are shown in Table 15. At least 80% of respondents in both classes found the lectures helpful to their understanding of chemistry with 10% more students in the flipped classroom strongly agreeing to this. Chi square analysis indicated a difference between the two classes in this regard χ²(4,469) = 12.67, p=.01.

Regarding homework, 66% (traditional class) and 74% (flipped class) found it helpful. This 8% difference in attitude towards homework may help to explain the higher grade distribution in the flipped class. However, the respondents from the flipped class may consider the class time problem solving as equivalent to their solving homework problems, and therefore spent less time on their own working problems. Chi square analysis indicated no difference between the two classes on their perceptions of the usefulness of homework χ²(4,470) = 3.63, p=.46.

The next two questions queried students from the traditional class on the use of online lectures; 53% of those students thought they would benefit from online lectures, which is remarkably similar to the responses from the flipped class for the open-ended question ‘what did you like most about the course?’ from the SPOI (see Table 9). Many students asked if the in class lectures could be posted online. The concern of course is whether or not students would still come to class if they know they can listen to lectures online. This professor has considerable doubt that students would attend in-person lectures unless the class-time is very different from the online component or specific incentives, such as extra credit or quizzes, compel them to come to class. The final question in Table 15 presents an important summation of student attitude toward the teaching style of the class in which they participated and Chi square analysis illustrates the difference in the courses χ²(4,464) = 25.12, p=.00. Students in the flipped class (68%) agreed or strongly agreed that they would take another class using that format of teaching and 74% of students in the traditional class were comfortable with the standard lecture teaching mode. The traditional response was predictable given it is the teaching mode with which they are familiar. While not quite as positive, the 68%

positive response for the flipped class indicates students were comfortable with this style of teaching and found value in it after experiencing it.

15

(23)

Table 15. Student opinion of the following statements (%). SD=Strongly Disagree, D=Disagree, U=Unsure, A=Agree, SA=Strongly Agree^a

SD D U A SA

T F T F T F T F T F

The lectures helped my understanding of chemistry

0 2 7 6 13 8 56 50 24 34

The homework problems helped my understanding of chemistry

1 1 7 4 26 22 42 46 24 28

It would be helpful if the lectures were videotaped and made available online

1 - 6 - 10 - 31 - 53 -

I would still come to class if lectures were videotaped and made available online

3 - 6 - 21 - 36 - 34 -

I would take a class with this format again

1 6 9 7 17 19 51 31 23 37

a Note: a dash (-) indicates the question was only available to the traditional class.

T=traditional section; F=flipped section.

Challenges, Benefits, and Helpful Hints: Advice from the Author

This section is largely directed towards faculty who would try to flip a class for the first time. The biggest challenge for me was to convey the same information to the flipped class using slides and voice that I provide in person using physical movements to the traditional class to make my point understandable. I quickly learned to hear my own words from a different perspective and realized this teaching mode creates student vulnerability. When words are recorded, using incorrect terminology, omission, skipping a phrase, etc. could handicap some students, especially those whose major mode of learning is listening to lectures.

I recorded many slides, multiple times, before I was satisfied with the clarity of the message. I also incorporated some links for students to view videos of demonstrations from the internet and made them available to both classes. The recording process was definitely more difficult than I had imagined but it was also a very useful exercise in self-awareness of my communication style. Increased planning time for the flipped lecture is absolutely necessary.

Voice control when recording also involves practice: too monotone and your students will fall asleep; too animated eliminates effectiveness when changing tone to stress a point.

Early in the semester, students in both classes thought the other class had the advantage. I expected this response from both classes. This would not be an issue unless a professor chose to use both modes of teaching in the same semester or frequently alternates teaching modes over semesters.

I strongly recommend weekly quizzes for using the flipped mode for a chemistry course as encouragement for staying up to date with the lectures. For

16

(24)

this particular course, if a student gets behind by even one week it will likely alter the level of his/her success. I kept an online calendar noting dates when each lecture should be finished (for the flipped class) and which lectures or chapter sections (for the traditional class) would be covered on the weekly quiz. Many students commented on how much they appreciated the calendar.

I focused problem solving in class on material covered in the most recently assigned lectures to encourage students to maintain the lecture schedule. This allowed more time for students to work problems and help each other master the challenging problems. While this peer instruction can create chaos in a large class, students asked questions and engaged in discussion more after peer instruction than when I worked out the problem.

Conclusions

This study was an attempt to investigate one change in the traditionally taught general chemistry class at UCF. Based on the final grades of the two classes, the flipped mode may be of use to some students in the class as more students in that class made an A or B. Some motivated students can use this change in teaching style to improve their grades, but from this limited experiment it would appear that students who are not performing at a high level may not be helped by this method. The material covered in the course is difficult for some students and simply flipping the lecture content may not have been enough intervention to help these students. More improvement in grades may be achieved by using the online component for other materials, possibly recorded problem-solving sessions posted online, or having some lectures in class and others online. Using the flipped mode for teaching does not have to be all or nothing and may be most beneficial to large classes with a hybrid approach. Student perceptions of the flipped method were largely positive. The majority of those surveyed expressing willingness to take another flipped class indicates a reason to attempt further efforts.

Using the flipped method is more time-consuming for the instructor, at least for the first semester during the change. However, once the recorded lectures are made, improving/updating the lectures for future use will require much less time. The process of making the recordings is an excellent self-evaluation of one’s teaching and lecturing styles.

Certainly, more research to examine the nuances of which students might best be helped by the flipped method is needed. Given the recent advances in analytics for lecture-capture platforms, this may also be a new source of data to allow for precise documentation of how students interact with the videos, rather than relying on student anecdote and memory.

References

1. Alvarez, B. Robert Townsend. NEA Today2012, 27–29.

2. Bergmann, J.; Waddell, D. To flip or not to flip? Learn. Leading Technol.

2012, 6–7.

17

(25)

3. Abeysekera, L.; Dawson, P. Motivation and cognitive load in the flipped classroom: definition,rationale and a call for research. Higher Educ. Res.

Dev. 2015,34, 1–14.

4. Saitta E.; Waldrop J.; Bowdon M. Joining the Flipped Classroom Conversation. In Best Practices for Flipping the College Classroom;

Waldrop, J., Bowdon, M., Eds.; Routledge: New York, NY, 2015; pp 1−16 5. Flynn, A. B. Structure and evaluation of flipped chemistry courses: organic &

spectroscopy, large and small, first to third year, English and French.Chem.

Educ. Res. Pract2015,16, 198–211.

6. Weaver, G. C.; Sturtevant, H. G. Design, Implementation, and Evaluation of a Flipped Format General Chemistry Course. J. Chem. Educ. 2015,92, 1437–1448.

7. Ryan, M. D.; Reid, S. A. Impact of the Flipped Classroom on Student Performance and Retention: A Parallel Controlled Study in General Chemistry. J. Chem. Educ. 2016,93, 13–23.

8. Schneider, J.; Munro, I.; Krishnan, S. Flipping the Classroom for Pharmacokinetics.Am. J. Educ. Res. 2014,2, 1225–1229.

9. Waldrop, J. Flipping the Graduate Course in Nursing: Application to Solve Patients’ Health Problems. In Best Practices for Flipping the College Classroom; Waldrop, J., Bowdon, M., Eds.; Routledge: New York, NY, 2015; pp 44−54.

18

(26)

Chapter 2

The Effectiveness of Course Flipping in General Chemistry – Does It Work?

Dominick Casadonte^*

Department of Chemistry and Biochemistry, Texas Tech University, 1 Memorial Circle, Lubbock, Texas 79409

*E-mail: [email protected]

The author has been involved in flipping classes in both on-line and face-to-face formats since 2008. In this study, I have flipped the Honors General Chemistry course sequence at Texas Tech University from the fall of 2010 through the fall of 2015. All of the pre-class lectures were recorded using the Mediasite platform and placed on Blackboard for students to watch in advance of class time. Online web learning homework assignments were used to determine if students had watched the lecture. Class time was used 1) in a discussion format to summarize lectures and clear up muddy conceptual points, and 2) to work advanced problems using a variety of active learning modalities. The efficacy of the method was determined by giving exams that had been given to other honors classes 5 years previously as a baseline and comparing exam results, as well as through standardized ACS content exams. I was especially interested in the pre-post differential percentile rankings as an indication of improvement in student learning outcomes over time. A 40-question Likert assessment and a 40-question free-response assessment were also given to the students in a pre-post format. Results of the various assessments, as well as the effectiveness of the method for different student cohorts, are discussed.

(27)

Introduction

The Concept of Course Flipping

The beginning of the terminology associated with “course flipping” can be traced to Lage’s discussion of the “inverted classroom” in the early 2000’s (1).

In its most general concept, the inverted classroom allows for the movement of learning activities which have historically occurred inside the classroom and during class time to be recontextualized as activities to be conducted in an extramural or asynchronous setting, freeing up classroom time for alternatives to the traditional lecture. The inverted classroom was designed to provide flexibility with regard to students’ differing learning modalities as well as to accommodate the variety of teaching styles used by faculty. The idea of using technology to move lecture content outside of the physical classroom in order to allow for greater discussion and more engaged and active learning strategies in the classroom environment has been dubbed, in different settings, “time-shifted (a term that has been co-opted from the video industry during the court battles between Universal and Sony in 1984) instruction” (2), “reverse instruction”

(3), and “naked teaching” (4), and has been particularly effective in terms of increasing the amount of time in the classroom that can be devoted to discussion in the arts, humanities, and professional schools (5–10).

Although the concept of moving the traditional lecture outside of the classroom in order to provide learning space during class time for active learning strategies has been around for more than a decade, the application and coining of the term “course flipping” to a chemistry classroom environment is credited to the pioneering work of two high school chemistry teachers, Jonathan Bergmann and Aaron Sams, in 2012 (11). They began by asking the question, “how can we use our in-class time more effectively to teach our students chemistry?”. This query led them to the use of technology to flip the traditional lecture-homework-test paradigm so common in the traditional chemistry course at the high school level.

Shortly thereafter the term “flipped classroom” was applied at the university level. Since the advent of course flipping in particular in the university chemistry course environment, the number of cases where flipping has been examined as an alternative and perhaps superior pedagogy has substantially increased. The flipped chemistry classroom has been tried and evaluated in general chemistry classes (12–18), in organic chemistry classes (19–23), in physical chemistry lecture courses (24, 25), in analytical chemistry classes (26), in biochemistry classrooms (27), and in chemistry laboratory classes (28–30). The flipping of chemistry classes both at the K-12 and university levels has been the subject of reviews (31,32) and at least two ACS symposia (33,34).

Advantages and Disadvantages of Course Flipping

Many potential advantages, both with regard to student learning outcomes as well as for the instructor, exist in the flipped pedagogy compared to the traditional classroom. Some of these include:

20

(28)

For the Student:

1. Students are more engaged in their own learning 2. There is the real potential for more engaged time on task

3. Students can review the lecture as often as they would like or need 4. Students can watch the lecture when most convenient between real-time

interaction with the instructor

5. Better scheduling of class; students know what to study 6. It allows the students to learn at their own pace

7. Flexibility of the platform: Students can use computers or portable devices (smart phones, etc) to watch the videos, and are hence not tied to the classroom or technology class setting

8. Potentially less time required for exam preparation

For the Instructor:

1. The instructor can produce specific, targeted lecture topics and materials 2. It allows the instructor greater design and control of the classroom setting 3. It provides more flexibility in designing classroom interaction

4. It provides for increased interaction with students in the classroom 5. The instructor has time to use guided inquiry in the classroom if desired 6. A better understanding of student’s thinking often emerges

7. It puts the responsibility for learning significantly into the hands of the students

8. It can (depending upon the design of the course) allow more time on task for the student

9. It allows for the identification of groups of students during the classroom session in need of remediation and the consequent development of peer or teacher-led mini-tutorials.

10. It potentially saves time by not having to repeat lectures or topics from year to year

11. Since the assignments are placed on the internet, various educational platforms (e.g., Blackboard) allow the instructor to determine whether or not students have watched the videos

12. Assessments can be built in prior to class attendance

13. Chat rooms can be set up and out-of-class discussion with the instructor or teaching assistant can occur to facilitate understanding during the pre- class experience

14. It can potentially improve exam and class performance 15. Asymmetric learning situations are possible

16. Can be done in a purely online format, if there is real-time instructor interaction online

17. Flipping has value added relative to online classes, in that there is face- to-face in-class active learning to complement video presentations. The instructor is a necessary component to the success of student learning and student learning outcomes (SLOs).

21

(29)

There are, to be fair, some disadvantages that have been identified to flipping a course, especially the first time that the course is taught (31, 32,35). These include:

1. Time-intensive to set up

2. Can lead to poorer class attendance 3. Can cause students to disengage 4. Takes more time for the student

5. Potentially more difficult for ESL students?

6. Requires responsible student

7. Works well for majors and honors students, but may not be as effective for lower-performing students

8. May not be as effective for lower socioeconomic groups who do may not have access to technology

Items (4)–(8) require some additional commentary. The existing literature concerning ESL students tend to indicate that, contrary to anecdotal belief, the use of a flipped classroom setting actually increases the verbal fluency and use of language of ESL students (5). With regard to the level of engagement, prior knowledge, or performance level of the students within a class, the data are mixed.

Some studies have indicated that there is statistically little difference in learning outcomes between upper and lower performing students (12, 13), while others have observed that the learning improvements occurred for the higher-performing students. With regard to weaker students in the class, some studies have shown little or no improvement (15), while others have shown substantive learning gains by previously bottom performing students (36). It has been suggested by a number of practitioners that the improvements that occur in the flipped setting depend, to a certain extent, on the nature of the active learning strategies involved (12).

In studies at both open-enrollment colleges (37) as well as for primarily HBCUs (38), significant learning gains have been observed by the use of course flipping techniques.

The Flipping Components

As Bergmann and Sams have pointed out (11), there is no specific way to flip a class, and, in fact, the ability to flip a classroom in a variety of ways is one of the strengths of the pedagogy. It is useful, however, to consider the flipping process in terms of five possible components that may be blended to provide the actual flipped environment:

•Pre-Class Instruction:This can take the form of instructor-prepared online videos , textual, or multimedia presentations as well as those that can be found online (such as those produced by the Khan Academy) (39). Depending upon the context, students can either access the online information separately or work together in groups using various collaboration software.

• Pre-Class Assessment: This is often done using a web-based assessment package from either professional content managers or through learning

22

(30)

management systems. It could also include active learning assessments such as Just-In-Time Teaching (40), POGIL, or hands-on activities.

•In-Class Discussion: This methodology is often used to help the students recap the material from the pre-class instruction. It is also useful to clear up muddy points or misconceptions. Case studies have also been used as a vehicle for in-class discussion of topics (41).

• In-Class Active Learning Activities: This, again, can take many forms, including POGIL, Think-Pair-Share, Clickers, Active Response Systems, etc.

One of the hallmarks of the flipped classroom is that the active learning strategies implemented are dependent upon the needs of the students and the ability and imagination of the instructor.

•In-Class Assessment:Many possibilities exist here, including short quizzes, authentic activity assessements, online quizzing, etc.

These components within the pre-class and in-class activities may, to a certain extent, be mixed and matched according the needs of the students and instructor and the effectiveness of the pedagogies that one employs. As new technologies and strategies for active learning unfold, the opportunities for even richer flipped environments will appear.

Definition of Course Flipping

For the purposes of the rest of this chapter, I will define course flipping as the process by which the typical lecture-homework-lecture-homework-test-lecture- homework paradigm is altered so that the lecture content is delivered outside of class, typically in an online or multimedia setting. The “in-class” time (whether face-to-face or in real-time online interaction, usually through video) can be spent having in-depth discussion for mastery or by engaging the students in any number of active learning strategies. This is the working definition employed in this study of the efficacy of the flipped class.

The Study

Concurrent with many of the studies cited, I have recently completed a five-and-a-half year longitudinal study concerning the effectiveness of course flipping in a moderately-sized honors general chemistry class. My fundamental research question was whether or not course flipping would provide significant improvements in learning outcomes in a general chemistry classroom setting.

The following will discuss the manner of the study as well as the outcomes. This study was approved by the Texas Tech University Institutional Review Board.

The Course of Instruction

The author began course flipping in the spring semester of 2009 in an on-line graduate conceptual chemistry class taught at Texas Tech University as part of a multidisciplinary master’s science degree. The methodology was borne from much the same motivation that Bergmann and Sams had for flipping at the high

23

(31)

school level, i.e., to be able to maximize teaching effectiveness during the limited time available with the students. By 2010 it was clear to the author that “course flipping” (as the term had been coined) could potentially provide a more effective means of improving SLOs in the general chemistry classroom than through the normal lecture-homework-exam paradigm. A study was begun to evaluate the efficacy of the pedagogy. The first classes involved were the Honors General Chemistry courses for F 2011 - S 2012 at Texas Tech University (CHEM 1307, Principles of Chemistry I (Fall) and CHEM 1308, Principles of Chemistry II (Spring)). The courses contained 75 and 70 students, respectively, and met T Th 9:30 AM – 10:50 AM. In 2015 the class size was increased to 96 students to cover an increasing population of honors students.

All of the lectures for each course were pre-recorded in the summer of 2011 (for CHEM 1307) and the fall of 2011 (for CHEM 1308) using a Mediasite®

recorder, a document camera, and a video camera. The Mediasite® recorder had picture-in-picture capabilities. All of the lectures were composed of class notes with strategic blanks for examples to be worked. The notes were provided to the students, who could then fill in the blanks while watching the lectures.

This allowed for a tactile component to the learning process as well as the visual and audio representations in the recordings. The lectures were subsequently re-recorded in the summer of 2015 in high definition. Table 1 shows the characteristics of the lectures. Although the average lecture time was over 30 minutes, in a free-response survey given at the end of the first year of flipped instruction in CHEM 1308, 67% of the students thought that the videos were not too long. The author has polled the students in the CHEM 1308 classes every year from 2012-2015 concerning the length of the videos and has found a similar response. The main comment was that the students could pause the videos if they wanted to parse the time spent in viewing. There have been limited studies in the STEM disciplines concerning video length as it relates to improved SLOs (42).

The author is currently performing a study of video length as it relates to course flipping in general chemistry.

Table 1. Lecture Characteristics Lecture Times

No. of Videos Shortest Longest Average

CHEM 1307 26 3:40 64:35 33:20

CHEM 1308 27 19:27 47:17 31:25

The syllabus carefully listed the lecture number and the topic for each class.

Each lecture was correlated to a folder on the Learning Management System (LMS; here, Blackboard) that contained the videotaped lecture, a set of notes to be filled in, and a link to the Cengage online learning platform OWL for post-video quizzing. After watching each lecture (homework), the students then worked 6-10 homework questions using the OWL format (Mastery question bank). The

24

(32)

scores were then recorded, and counted for 150 points out of a total of 800 points allotted for the course. This allowed for a determination of who had watched the videos each week.

Class time was divided into two parts:

- First half: Review of the lecture material. During this time, the instructor checked (in a discussion format) for main ideas and was able to clear up any misconceptions. In this way the instructor could determine what the class had learned by watching the videos, and could provide additional information and insight, as well as prevent any misconceptions or muddiness from propagating through the curriculum. This review often involved a variety of techniques, including having the students “act out”

molecular-level processes.

- Second half: This involved problem solving, using problems from the textbook (Oxtoby, Gillis, and Campion/Butler, Principles of Modern Chemistry. 7^thand 8^thEds., Cengage Learning, 2012 and 2015) (43,44) which had been previously indicated in the syllabus, so that the students could try the problems before coming to class, if so desired. Problems were worked in a variety of formats, depending upon the material and the class including group work, going to the board, modeling the answers, think-pair-share, etc.)

In addition to class time, the class was roughly divided in half and attended one of two zero-credit hour 1.5-hour discussion sections. The discussion section had additional interaction with the course material as well as preparation for a quiz given during each of the sections (one quiz per week per student). Three exams were given during the semester, as well as a final exam (cumulative). An ACS End-of-Term exam was administered as a pre- and post- test. As a way on incentivizing the exam, students were told that if they scored at or above the 90^th percentile in the post-test, they did not have to take the class-based final exam.

An additional item that is often discussed is the number of contact hours in the flipped model compared to course credit hours, so that the students are not engaged for longer than the number of credit hours mandate. This is not really an issue, as in a traditional lecture-homework format, the out of class homework can take a variable number of hours, depending on the number and level of difficulty of questions asked. Care is often taken in the development of flipped classes so that if there is out-of-class assessment, the number of questions is relatively small (here 6-10 low to moderate-level questions, to assess initial understanding of the lecture material only). Given the in-class time constraints, the number of advanced problems worked is usually small (in this study, typically 3-5, after an initial discussion).

Evaluation of the Model: Methods of Assessment

Four methods of assessment were involved to determine whether or not course flipping in the method described above would be able to improve learning outcomes. These include:

25

(33)

• Method I: Class-based Exam Score Comparisons

• Method II: ACS End-of-Term Exam Score Comparisons

• Method III: 40 –Question Likert Scale Questionnaire

• Method IV: Free-Response Questionnaire (Spring, 2012)

Evaluation of the Model: Results and Outcomes

The author has been the only teacher of the Honors sections of CHEM 1307 and 1308 since 1998, and, as such, has access to data for relatively homogeneous student populations over time (the average SAT scores (verbal + math; pre-2012 scale) over the period of study was 1350^±50). Consequently, a historical approach has been used for comparison. The demographics of the study group are shown in Table 2.

Table 2. Demographics of Study Group

Fall 2006

Fall 2011

Fall 2012

Fall 2013

Fall 2014

Fall 2015

Spring 2007

Spring 2012

Spring 2014

Spring 2015 Male

% 44 58 45 58 49 43 35 53 43 38

Female

% 56 42 55 42 51 57 65 47 57 62

White

% 93 84 88 80 78 79 88 77 69 72

Hispanic

% 4 5 7 7 7 12 7 7 19 7

Asian/

Other

%

2 11 5 12 14 9 2 14 11 20

Black

% 0 0 0 1 1 0 2 1 1 1

Table 3 provides the average scores for the three exams and final exam that were given in the fall of 2006 in CHEM 1307 (pre-flipped) as well as the fall exam periods from 2011-2013, the years in which the study was conducted. For each comparative data set, a 1-tailed heteroscedasticttest was performed to determine the statistical significance.

26

(34)