Partial Flipping in Practice: Case Studies
2. Enhancing the Flip: Adding Interactivity to Pre-Lecture Videos Using Simple Web Software
in future years, and can also be very persuasive in encouraging colleagues to experiment with similar innovations, as evidenced in the second case study. One particularly eloquent student quote sums up the success of this trial, and is included in its entirety below as a compelling piece of evidence that the partial flipping approach used in this case did indeed achieve its objectives.
“I have found the flipped lectures implemented into the syllabus to be an incredibly valuable resource over the last several months.
“They are an ingenious way to convey the information prior to a main lecture, giving the students relevant background and understanding in order to constructively contribute in class.
“Utilising technology for this purpose allows me time to pause, comprehend and think about how the information being conveyed fits into what has been learnt previously."
2. Enhancing the Flip: Adding Interactivity to Pre-Lecture Videos Using
Table 2. Evidence extracted from student responses to the questions“What are the advantages and disadvantages of using the flipped videos at home as opposed to being taught the material in class?”and“Do you have any other comments on flipped teaching and its effectiveness in helping you to learn?”
Benefit Evidence extracted from student comments
Reduced cognitive load
“…you arent as overwhelmed as perhaps you would have been without them.”
“…its nice to not feel rushed or get information overload during the lecture.”
“I can pause (when my brain has an overload moment)…”
“[it is better than]having to continue with no hope of comprehending the material as you have not grasped a central concept of the topic…”
“It feels good to come into class and starting off by feeling content instead of puzzled.”
“…Id had an insight into the topic so felt more comfortable in the lecture.”
Better preparation for scheduled
lectures
“I can rewind[and]look up in textbook for deeper understanding and I feel very well prepared...”
“It allows you to put in some extra research into points…”
“…it gave us chance to be ready and also study further…”
Confidence “…flipped lectures have changed my confidence regarding clicker questions and intellectually grasping what is going on…”
More time available in
lectures
“The availability of time to ask questions is key I think…”
“…gave us more time[to]answer and get immediate feedback on questions relating to the topic.”
“Using the time freed up in the lectures to do more[clicker]questions was really helpful…”
“…has created more time to explain harder content/work through more examples etc.”
“…more actual lecture time to learn the harder bits.”
“…more time in chemistry lectures to go into more detail or for better explanations.”
Enjoyment
“…this tool is very useful for me and I really enjoy[it]…”
“I have loved it, it was a revelation to me and a huge help.”
“I really like the idea of teaching via flipped lectures.”
This innovation involved a first semester course in introductory Organic Chemistry which is compulsory for all first year students (~180 in 2014/15).
A number of distinct concepts are covered during the course, all of which are underpinned by the concept of electron flow. Although students encounter curly arrows in their pre-university chemistry studies in the UK, our experience is that many lack confidence in using the concept of electron flow to describe and explain mechanistic processes, and often rely instead on rote-memorization (45). A key aim of this module is to provide students with a common foundation of knowledge and skills from which to progress in their future studies, by developing the skills required to derive mechanisms from first principles rather than attempting to rote-learn mechanistic processes.
67
Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch004
A Role for Learning Analytics in Enhancing Lecture Delivery and Feedback An educator who is able to interact with their students and collect data regarding what they are (and are not) learning can adjust their teaching style and provide improved feedback to students, as discussed by Toto and Nguyen (46). The principle of Just-in-Time Teaching, as defined by Novak et al. (47), can be supported by the use of learning analytics, helping an educator to adjust their teaching to give more attention to areas in which students demonstrate weaker understanding. The key consideration, of utmost relevance for organic chemistry, is that if students misunderstand important pieces of knowledge or earlier learning outcomes, they may fail to progress in grasping higher-order concepts. The use of learning analytics can help to identify misconceptions and gaps in knowledge which can be immediately addressed before progressing to more advanced material, and this was instrumental in informing the design of the interactive pre-lecture videos.
The reduction in the amount of feedback students receive when they progress to university has already been discussed as one of the barriers that hinders a smooth transition to university (19). As such an additional aim of this work was to use learning analytics to enhance the feedback provided to students, and this was achieved in a number of ways as outlined in the implementation section below.
Improved feedback can empower students to manage their own thought processes (48) and generate feedback for themselves or their peers (49), while meaningful group discussion and reflection may also be encouraged (50). The preceding points were important in ensuring that students were well-prepared to engage with the in-class activities being introduced as part of this project. Enhanced feedback can also help students to become more aware of their own learning, helping them to develop skills of metacognition (51), and supporting the key objective that this work would assist students in becoming the effective independent learners they need to be in order to succeed at university.
In the example outlined in this case study, interactive online pre-lectures were created which were based on existing material and did not require extensive preparation time. Usage data and the responses to Zaption questions posed during pre-lectures formed the basis of the analytics which were collected and analyzed by the instructor to support teaching and learning as outlined below.
Methodology
Pre-lecture videos were again prepared using Panopto (40) on a tablet PC.
The instructor annotated PowerPoint slides using a stylus to add structures and mechanisms while explaining his actions verbally, as illustrated in Figure 3.
68
Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch004
Figure 3. An example of a Zaption pre-lecture video based on annotation of PowerPoint slide. (Zaption images reproduced with permission from reference
(44). Copyright 2016 Zaption.)
The videos were uploaded to YouTube for online hosting, after which Zaption (44), a piece of web-based software running in the browser, was used to add interactivity to the videos. Videos augmented with Zaption can be made to pause at any point in order that questions can be posed to the viewer. In these examples, multiple choice questions (MCQs) were used to probe students’ understanding of key concepts, with Zaption providing instant feedback to students on their answers. Open response questions were also employed to gain an understanding of students’ thought processes, while giving them opportunities to reflect on the reasoning behind their responses. This meant students could evaluate and refine their understanding prior to the face-to-face session. In some cases, explanations of answers were included as part of the flipped lecture so as to provide additional feedback. Students’ responses to all questions, along with viewing statistics, were subsequently available for download as .csv files.
69
Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch004
Implementation of Zaption Pre-Lecture Videos in Practice
The first pre-lecture video that was shared with students introduced the fundamental concepts that govern the strength of organic acids, which came at the beginning of a four-lecture unit on this particular topic. During the video, students were taught the definitions of acids and bases in the context of organic chemistry, and were also shown the convention for drawing acid-base reaction mechanisms using curly arrows. At the end of the video, students were asked three questions.
In the first (Figure 4a), they were shown a curly arrow mechanism and asked if it was correct. In the second question (Figure 4b), they were asked to draw a curly arrow mechanism themselves. Since there was no straightforward way for them to input molecular structures and curly arrows into the online platform, the video was automatically paused at this point so students could draw the mechanism. The students then un-paused the video to view the instructor drawing the mechanism so they could check their work. They could then select a response to a multiple choice question which indicated how close they had been to the correct answer.
Figure 4. a and b: examples of questions presented to students during a Zaption pre-lecture. (Zaption images reproduced with permission from reference (44).
Copyright 2016 Zaption.)
70
Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch004
Finally, to assist them in moving to higher-order thinking, students were asked to consider two molecules, ethanol and acetic acid. These are similar molecules chemically, each containing two carbon atoms, one of which is attached to oxygen, but they have different acid-base properties. The information that ethanol is a weaker acid than acetic acid was provided to the students, and they were then asked to consider why this is the case. Students gave their responses to this prompt, which were then reviewed to inform the preparation of the scheduled lecture, with some correct and incorrect example answers incorporated into lecture slides for review purposes. The overall result was that students received immediate feedback on their grasp of basic concepts and skills (curly arrow mechanism), and also then had some time to consider a deep learning level question on the application of this concept to explain a physical phenomenon with which they are all familiar, i.e. the different properties, including taste, of alcohol and vinegar. This was then followed by further feedback in the scheduled lecture slot, which was particularly timely in view of the fact that most students watched pre-lectures videos in the 24 hours preceding the scheduled lecture.
The remaining videos were produced in a similar format, with the rich data collected being analyzed by the instructor prior to each scheduled session.
Another approach used was to follow up a multiple choice question (e.g. “Which of these compounds will react fastest?”) with an open response question in which students were invited to explain their answer. This helped to ensure that students were thinking on a deep level rather than simply ticking an answer, and also provided insight regarding whether or not they were using the correct reasoning.
The process of skimming through the students’ answers could be completed surprisingly quickly, and it soon became clear whether or not students were on the right track and what the predominant points of confusion were among the cohort.
Towards the end of the semester, an unexpected outcome illustrated the value of this approach. During one of the pre-lectures, the students were asked three questions. Almost all of the students got the correct answers to the first and last questions, but very few answered the middle question correctly. This was very surprising, since the question was not expected to be more challenging than the others. Moreover, the most popular answer selected was the most incorrect of the options available, indicating a fundamental misunderstanding of the way electrons are shared within molecules. Having access to these learning analytics allowed the adjustment of teaching to address this fundamental misunderstanding during the scheduled lecture. Without analysis of the data, the misconception would have gone undetected, potentially for some time thereafter, with consequences for understanding of more complex concepts.
As well as providing further opportunities for feedback, the scrutiny of analytics also allowed the instructor to moderate the pace of delivery to meet the needs of students, with higher-order concepts only being covered once students had grasped the underpinning material. Additionally, the lecture time freed up by partial flipping also allowed the implementation of in-class self- and peer-assessment activities, and clicker quizzes. These activities were designed to build on concepts covered in the pre-lecture videos, and again were adapted to take account of the misconceptions and misunderstandings uncovered through scrutiny of the learning analytics.
71
Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch004
Student Engagement with Flipped Lectures and Zaption Questions
Analytics indicated that approximately 50 % of the cohort students made use of the flipped lectures throughout the module, which was perhaps a reflection of the fact that these were short pre-lecture videos which some students evidently felt they were able to miss. This contrasts with case study 1, where engagement was typically > 90%, which is perhaps a result of the smaller class size in that case, meaning that the instructor was better placed to incentivize individuals who may otherwise have attempted to hide in the shadows. Fortunately, the nature of the formative in-class activities meant that students who missed pre-lectures would be able to pick up some of what they missed, even if they didn’t gain as much benefit as their more engaged peers.
Interestingly, students who answered all of the open answer questions throughout the sequence tended to provide more detailed responses, marking these out as a more engaged group. Analysis of these students’ responses showed that the quality (in terms of correctness) was variable, indicating that it wasn’t necessarily the highest attaining students who were most engaged. The active approaches employed in-class were evidently well received, with the instructor reporting excellent engagement during scheduled lectures which provided further valuable insight regarding students’ progress.
Enhanced Provision of Feedback
As discussed in the introduction, it has been reported previously that one of the impediments to a successful transition to university learning is the reduction in the amount of feedback students receive in comparison to their experience at school (19). The challenges associated with providing such feedback are clear, particularly in the large-class lecture setting, where it is difficult for a single instructor to provide personalized feedback to individual students based on knowledge of their strengths and weaknesses. As described above, the use of interactive pre-lectures of the type outlined above can go some way towards addressing the problem, as students receive instant feedback on their answers to closed-response questions through Zaption, as well as further feedback during the scheduled lecture. It may be the case that such feedback results in more effective metacognitive processes in students, thus helping them to understand what they need to focus on in their private study. This could represent a real breakthrough in terms of supporting students in making a smooth and effective transition from school-to-university, but thorough research would be required to confirm whether or not this is the case.
Perhaps the most valuable impact on feedback provision is that fact that data collected regarding student’s responses to Zaption questions (both open and closed) can be viewed by the instructor at any time. As discussed above, the use of such data enabled the pace and structure of individual lectures to be adjusted in a just-in-time manner to suit student needs. Time was explicitly allocated in the
72
Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch004
lectures to provide feedback on students’ responses to Zaption questions, and this information was also used to design clicker questions, providing further feedback opportunities to promote increased metacognition.
There were also positive impacts on the instructor, who was in effect receiving feedback from students via the mechanisms outlined above. Importantly, having identified common misconceptions by scrutinizing responses to Zaption questions, he was able to directly address these before moving onto more advanced material.
On the other hand, where most students had responded correctly to questions, this resulted in a feeling of security about their level of understanding, allowing the instructor to confidently introduce more stretch and challenge where appropriate.
Evaluation Data
As with the first case study, the evaluation data presented is relatively limited, but does provide valuable insight, and has informed research which is currently being undertaken to ascertain the true impacts of partial flipping with interactive pre-lecture videos on student learning and metacognition. There were some positive outcomes which can be reported here, including a 10% increase in the average mark achieved on a mid-term exam. The prior attainment of students in the two cohorts (2013/14 and 2014/15) was broadly similar, and both tests targeted the same material. The module was taught to both cohorts by the same instructor, and the only material change to delivery was the use of the partial flipping approach, accompanied by enhanced interactivity in the scheduled sessions. This provides evidence that the novel approach was indeed beneficial to student learning, although the usual caveats apply when considering such data as evidence of impact.
In order to evaluate student perceptions of the value of the partial flipping approach, an in-class clicker survey, which was not validated, was used in the final lecture of the module to investigate students’ usage of the pre-lecture videos, and also their opinions regarding the value of different elements of the teaching and learning associated with the module, as documented in Table 3.
It is particularly noteworthy that students were almost as positive in their view of the impact of pre-lectures on their understanding as they were about the lectures themselves. The data relating to the active learning elements introduced into timetabled lectures are a little less positive, but are still indicative of a favorable response. The same clicker survey also probed students’ attitudes regarding the value of the more traditional teaching resources which supported the module, with interesting results. Large numbers of students didn’t use these resources, with 52% reporting that they didn’t do the recommend reading from the textbook and 72% reporting that they didn’t complete any of the problems from the textbook. Surprisingly, 42% of students did not make use of practice worksheets on Blackboard despite the fact that these represented a good opportunity to become familiar with exam-style questions.
73
Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch004
Table 3. Students’ views regarding the value of pre-lecture and in-class elements of teaching (n = 110)a
VH H N NH DNU
How helpful were pre-lectures in improving your understanding of material?
37% 43% 9% 1% 10%
How helpful were lectures in improving
your understanding of material? 47% 34% 9% 1% 9%
How helpful were clicker questions in improving your understanding of material?
28% 40% 18% 7% 7%
How helpful were in-class peer-assessment tasks in improving your understanding of material?
16% 36% 23% 12% 14%
aVH = very helpful; H = helpful; N = neutral; NH = not helpful; DNU = did not use
In addition to this, students were surveyed using a Zaption video containing text response questions which probed how they felt the videos had impacted their practice. Although this provided richer data, the response rate (approximately 40 students) was lower than in the case of the clicker survey. Evidence has been extracted from students’ comments and is linked to the benefits of the approach as inferred from analysis of the data (Table 4).
These comments provide evidence that, in some cases at least, students’
perceived improvement in understanding is related to reduced cognitive load during the scheduled lecture, in accordance with Seery and Donnelly’s earlier findings (31). The data also show that students felt that they were better prepared for lectures as a result of the approach, and there is evidence that this provided a structure within which students could study more effectively outside class.
If the flipped lecture structure is reducing cognitive load, we might also expect students to feel more comfortable with the amount of material covered in a lecture course. Gratifyingly, in the final course evaluation questionnaire for this module, there was a significant jump in the score for responses to the statement “I was comfortable with the amount of material covered” (3.9/5 in the previous year, rising to 4.3/5 when the flipped structure was introduced), despite the fact that there was actually a small increase in the amount of content covered in the module.
This effect may be explained by reduced cognitive load, with students comfortably able to assimilate more information when their minds are suitably prepared prior to scheduled lectures. Overall, this data is very encouraging indeed, seems to point to positive impacts similar to those reported by others such as Eichler and Peeples (36).
74
Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch004