9.4 Systems to Support Co-creative Collaboration in Mixed-Reality Environments
9.4.1 The Teaching Context — The Need for Blended Pedagogy Systems
Based on the initial research and the encouraging feedback from the use of MiRTLE at the University of Essex, we felt that it was important to carry out a more in-depth assessment of this approach in the classroom. We also hoped to better understand the wider pedagogical and behavioral aspects affected by the use of this technology. In this section we review the results of a field trial conducted over six years involving roughly 600 students in which the MiRTLE platform was utilized to deliver a standardizedfirst course in computer science. During this trial the course syllabus, learning objectives, assessments, instructor and assignments were held constant.
During the 2008–2009 academic year the computer science department at Saint Paul College, MN, USA, implemented a MiRTLE installation in order to begin distance delivery of coursework.
MiRTLE was selected as the primary platform for distance learning for the following reasons:
Fig. 9.1 MiRTLE classroom
1. MiRTLE brings the classroom to the distance learner and the distance learner to the classroom in a very engaging manner.
2. MiRTLE is a mixed realty platform that extends the virtual world that supports it. When this virtual world is not being used for lecture activities it can be easily utilized as a virtual world platform on which students and faculty can com- municate and collaborate. See Gardner and O’Driscoll (2011).
3. Based on the Open Wonderland platform, MiRTLE easily supported the typical teaching tools used in afirst course in computer science.
4. With a properly configured client computer MiRTLE is easily accessible and simple to use.
5. A MiRTLE installation can support alternative platforms such as Second Life or OpenSim with very little effort.
6. A MiRTLE facility is not disruptive, it does not require the faculty or students to significantly change their educational routine.
A general-purpose classroom was remodeled into a MiRTLE classroom in 2008.
The general arrangement of this classroom is shown in Fig.9.2. Life size avatars are projected on the rear wall of this facility and lectures are captured and dis- tributed using an inexpensive security camera mounted on the ceiling of the classroom. For a wider discussion on the general infrastructure for virtual and augmented reality in the classroom see Richards (Chap.6).
Thefield trials began following the construction of the MiRTLE facility.
Fig. 9.2 MiRTLE classroom layout
Two measurements were considered reliable indicators of student success and data was collected on these:
1. Student achievement—measured by the improvement in the score on a stan- dardized examination administered during the first class meeting and subse- quently used as thefinal examination in the course.
2. Student retention—measured as the percentage of students attending the course relative to the initial enrolment in the course.
The first field trial began in September 2008 and ended in May 2010 that involved two class-groups per term for four terms (8 cohorts of 30 students) for a total of 240 students. Data from the eight class-groups was aggregated.
Based on the aggregated data,first meeting scores on the standardized exami- nation were around 30% and end of term scores were around 75%. For those attending physically in the classroom the ending scores were 75.3% and the MiRTLE attendees scored about 74.4%. This seemed to indicate that there was essentially no difference in achievement between students learning on the MiRTLE platform and students who attended lectures.
The overall retention of students was about 61%. Weekly attendance trends are shown in Fig.9.3. As can be seen, retention was slightly better over the course of the term for the students attending via MiRTLE.
During this trial the classes began with about 15% of the total enrollment indicating that they would be attending via MiRTLE. Over the course of the term students who planned to attend the classes in person began to migrate to the virtual platform. The attendance by the end of the term was typically 40% attending virtually and 60% attending in person.
We feel the results show that MiRTLE when used in this manner provides distance students with similar learning outcomes to those in the physical classroom.
This is considered a successful trial since MiRTLE met all of its design expecta- tions, with students performing as well on the MiRTLE platform as in the lectures
Fig. 9.3 MiRTLE retention
and it appeared that as students became more comfortable with the MiRTLE platform they migrated to it.
Due to the initial success of MiRTLE the computer science faculty elected to run a secondfield trial beginning in September of 2010 utilizing the MiRTLE platform and a learning management system to offer the course in a blended delivery format.
The idea was to move the static curriculum content to the asynchronous learning management system and utilize weekly meetings to answer questions and conduct lecture reviews of the material for the week. Students were also encouraged to attend the weekly meetings remotely using the MiRTLE facility and to meet vir- tually on it to collaborate on problems together.
The second field trial was terminated in May of 2011 due to lower student retention and measureable decreases in learning achievement. The frequency and nature of student complaints and interventions by the college administration also played a role in electing to end the trial. The data gathered from this trial involved 4 class groups over two academic terms with 4 cohorts of 30 students each for a total of 120 students. Data from the four groups was aggregated.
Based on the aggregated data the first meeting scores on the standardized examination were again around 30%. However the final examination scores dropped from 75% as observed in thefirst trial to 63%. Overall student retention dropped from 61 to 52% of initial enrollment. Students made little or no use of MiRTLE and did not utilize the virtual world meeting rooms available to students in these sections as in the earlier trials with MiRTLE. The retention experience is shown in Fig.9.4.
This outcome was obviously worse than the previous computer science courses that used MiRTLE for synchronous instruction. A lengthy retrospective review was done which included students who had dropped the blended courses, participants from earlierfield trials and students and faculty who participated in a similar trial within the mathematics department. The following are considered keyfindings from this:
1. There appeared to be a critical coupling between pedagogical design and the technological platforms used in the course.
Fig. 9.4 Blended delivery lecture review
2. The weekly review sessions in the blended sections were essentially useless if as few as 10% of the attendees had not prepared for them.
3. The failed in-person sessions were the primary source of student complaints and were very disruptive to course conduct.
4. Students abandoned the use of MiRTLE and collaboration within the virtual environments due to their feeling of a loss of course structure.
Since MiRTLE had been successfully used in a traditional course during thefirst trial the group decided to re-examine the pedagogy in light of the technological approach being used. This led to the development of an alternate model of course design and a modification of the instructional pedagogy. This model envisions a course design of sequential learning modules that utilize a mixture of behaviorist and constructivist-based pedagogies. Figure9.5illustrates this general design.
This pedagogical approach utilizes active/collaborative learning laboratories for in-class instruction and divides online learning into behaviorist and constructivist learning activities. Behaviorist activities are primarily used for instruction in the lower order cognitive skills as identified by Bloom et al. (1956) they include knowledge, comprehension and application. The remainder of the online coursework which includes online discussions, online interactive laboratories and collaborative problem solving in the MiRTLE virtual environment, follow the constructivist paradigm.
Constructivist activities are primarily used for instruction in the higher order cognitive skills as identified by Bloom et al. (1956) that includecreate,evaluateandanalyze.
They represent approximately 80% of all learning activities.
The computer science department began a third field trial in September 2012 which lasted through May 2014. This trial utilized the modified pedagogical model with the option of in-person or MiRTLE attendance. The trial involved two course
Fig. 9.5 Revised pedagogical model for a module
groups per term for four terms (8 cohorts of 30 students) for a total 240 students.
Data from the eight groups was aggregated.
Based on the aggregated data,first meeting scores on the standardized exami- nation were around 30% and end of term scores were around 81%, which was an improvement over the traditional lecture led classes. For those attending physically in the classroom the ending scores were 80.3% and the MiRTLE attendees scored about 82%, again both higher than in the traditional delivery format. This seemed to confirm the results of thefirstfield trial that there is essentially no difference in achievement between students learning on the MiRTLE platform and students who attended lectures.
The overall retention of students was about 74%. This was a dramatic improvement over the second trial that utilized blended delivery and also an improvement over thefirst trial which used a traditional delivery format. Weekly attendance trends are shown in Fig.9.6. As can be seen, retention was slightly better over the course of the term for the students attending in person rather than with MiRTLE.
Table9.1summarizes the student achievement based on the increase in their raw scores on a standard exam given at the beginning and conclusion of each course section, and the end of term retention rates of those students who were initially enrolled in the course.
Based on these experiments, we believe the following observations are relevant.
Fig. 9.6 Blended delivery— active/collaborative
Table 9.1 Comparison of student achievement Student retention and achievement
Configuration Trial period Student achievement (%) Student retention (%)
Traditional 2008–2010 45.3 60.4
MiRTLE 2008–2010 44.4 62.8
Blended—lecture 2010–2011 33 51.8
Traditional—active 2012–2014 50.3 75.8
MiRTLE—active 2012–2014 52 72.1
Traditional or blended coursework utilizing the MiRTLE platform for remote students had learning outcomes similar to those who attended the course in the physical classroom. From the anecdotal evidence collected, we believe the effec- tiveness of MiRTLE lies in the level of presence the learner has within the virtual classroom environment, the ease with which the participants adopt the technology and the relative ease that learners using this platform have in interacting with other people in the lecture hall.
A simple unifying observation can be made for the tests conducted at Saint Paul to date. When implementing any system of technologically supported learning, care should be taken to consider the overall course delivery mechanisms, systematically recognizing as key system components the nature of the platform, the structure of the pedagogy and the capacity of both students and instructors to adjust their respective approaches to learning and instruction in the new environment.