4.2 Technologies Fostering Social Immersion .1 Transformed Social Interaction
4.2.2 Animated Pedagogical Agents
Besides transformed social interaction which targets human-human-interaction in immersive environments, other technology tries to incorporate autonomously acting entities in immersive settings that are based on artificial intelligence. These so-called pedagogical agents autonomously interact with and support the learner as tutor or peer. Even if so far no respective applications exist, these agents are conceivable to enhance social learning in Virtual Reality and Augmented Reality environments (for example in EcoMOBILE,http://ecomobile.gse.harvard.edu).
Fig. 4.2 Non zero sum gaze (Beall et al.,2003)
Pedagogical agents have long been suggested as emerging technology in order to enhance students’learning and motivation. Future systems are expected to enhance e-learning programs for individual learning with a human-like motivator or to support teachers in the classroom by attending to small groups of students. As one major advantage of these embodied automatic tutors, their advocates claim that pedagogical agents can, or will be able in the future, communicate via verbal and nonverbal means, thus facilitating and personalizing the interaction with an e-learning program (see Fig.4.3). Furthermore, increased motivation is expected:
Baylor and Ryu (2003) suggest that the key advantage is that human-likeness creates more positive learning experiences and provides a strong motivating effect.
In line with these assumptions, research on pedagogical agents suggests that even the mere social presence of an embodied autonomous tutor might be capable of fostering interest, attention, and subsequently learning.
Several overviews of pedagogical agent research (Baylor, 2001; Heidig &
Clarebout, 2011; Moreno, 2004) outline various developments, theoretical assumptions on effects and results of evaluation studies. With regard to develop- ments, systems from both cognitive and educational psychology and from thefield of computer science have been presented. For example, based on discourse analyses of face-to-face tutoring lessons, Graesser, Wiemer-Hastings, Wiemer-Hastings, Kreuz, and The Tutoring Research Group (1999) constructed the AutoTutor as a dialog partner, which by asking questions and giving feedback helps the student to actively construct subjective explanations and engage in deep reasoning. Here, the agent utilizes a hint–prompt–elaboration circle until the learner utters the correct answer. A talking head is used in order to ground the conversation between the tutor and the learner by means of nonverbal feedback cues (nodding or shaking the head, facial expressions). Similarly, also Baylor and colleagues provide a sophis- ticated learning program that is amended by simple embodiment techniques and basic nonverbal capabilities (Baylor & Ryu, 2003). Here, the focus is clearly on social interaction and social support for the learner (Kim & Baylor,2016). One of
Fig. 4.3 Virtual agent
the oldest, but nevertheless ground-breaking systems has been presented by Rickel and Johnson (2000) who locate their agents in a virtual learning environment.
Agent Steve leads learners through a US navy ship and is capable of reacting to changes in the virtual environment as well as to learners’ behavior. Based on information on the environment and the learner, he asks the student appropriate questions or gives explanations. The best known pedagogical agents are probably the systems by Lester et al. (2000) who developed various agents that are supposed to motivate children to learn within desktop-based learning environments. Some of these agents are able to use gestures and movements to highlight objects, but are also capable of displaying a wide array of emotions.
4.2.2.1 Results on the Effects of Pedagogical Agents
Several reviews and meta-analyses on pedagogical agents report that there is empirical support for the notion that pedagogical agents motivate the learner and lead to increased learning (Baylor 2001; Moreno 2004; Schroeder, Adesope, &
Gilbert,2013). However, results are not consistent and mostly show small effect sizes. For example, Graesser, Jackson, and McDaniel (2007) concluded that AutoTutor improves learning by nearly one letter grade compared with reading a textbook for an equivalent amount of time or in comparison with a pretest.
However, Rajan et al. (2001) demonstrated that it isfirst and foremost the voice that is responsible for these effects. Baylor and colleagues provided more evidence for an impact of pedagogical agents on learners’ subjective experiences, but not on their performance and learning outcome (Baylor & Ryu, 2003). Moreno (2003) further summarized that—in line with results that especially the voice is decisive— there is no evidence for the social cue hypothesis as it has not been shown that the mere presence of social aspects such as a human-like body leads to distinct effects.
However, the cognitive guiding functions provided by vocalizations and a pro- gram’s didactic concept did prove to be influential. Moreno concluded that the main strength of pedagogical agents resides in the specific instructional method embedded in the agent rather than in the visual presence of the agent itself. Also, recent research (Carlotto & Jaques, 2016) as well as a recent meta-analysis (Schroeder & Adesope,2014) has supported the notion that voice is more important than nonverbal expressiveness.
However, these results have to be considered with caution given the fact that the systems that had been evaluated did not (yet) include very sophisticated nonverbal behavior. Nonverbal behavior as it is used in face-to-face interaction includes facial displays and all kinds of kinesics in the sense of body movement. It needs to be acknowledged that nonverbal behavior is very complex: The dynamics of the movements are important, very subtle movements have distinct effects (e.g., head movements such as a head tilt) and the effects are context-dependent (e.g., a smile leads to a different effect when accompanied by a head tilt). This complexity, however, is mostly not considered and implemented in pedagogical agents. For example, as Graesser himself acknowledges, the nonverbal implementations as well
as their theoretical foundations for AutoTutor are rather shallow and fall short of the sophisticated dialogue model, while Rickel and Johnson (2000) focused more on multimodal input (e.g., tracking the student’s behavior) than multimodal output (e.g., deictic gestures and further nonverbal behaviors). Also, Lester’s agents do not show gestures and movements which represent human nonverbal behavior— especially as the agent is only partly anthropomorphic. So far, only very few pedagogical agent systems (even more recent ones) have achieved realistic and sufficiently subtle nonverbal behavior in order to administer a fair test. And indeed, when employing technology that provides realistic, dynamic nonverbal behavior, results show that nonverbal rapport behavior leads to an increase in effort and performance (Krämer et al.,2016). Therefore, the conclusion that embodiment and nonverbal behavior is less decisive compared to voice is premature. On the other hand, there are studies which have demonstrated considerable effects of nonverbal behavior even though the cues displayed were very basic (eye gaze of a comic-style agent, realized by eye-direction of the eyeball only, Lee, Kanakogi & Hiraki,2015).
Similarly, Mayer and DaPra (2012) present evidence for the“embodiment effect”in the sense of the question whether nonverbal behavior will yield better learning results. They demonstrate both increased values regarding a learning transfer test and more positive ratings of the social attributes of the agent. They explain the result with social agency theory (Mayer, 2005): social cues from a social agent prime a social stance in learners that causes them to work harder by activating deep cognitive processing in order to make sense of the presented material.
4.2.2.2 The Importance of Social Processes When Building Pedagogical Agents
Recent developments increasingly take the fact that learning is a social process into account. While there are still studies focusing on cognitive and metacognitive aspects, more and more priority is set on the social relationship with the user.
Instead of merely providing expert guidance, the agents are considered as tools to support learners by social and affective capabilities (Kim & Baylor,2016; Krämer
& Bente,2010; Veletsianos & Russell, 2014). Kim and Baylor (2006) argue that learning environments should provide situated social interaction since it is well documented that the cognitive functioning of learners is framed by social contexts and that teaching and learning are highly social activities.
Situated social interaction, as Kim and Baylor argue, can be realized by peda- gogical agents that simulate human instructional roles such as teacher or peer.
Similarly, Graesser (2006) states that social psychological aspects have to be considered in pedagogical agent research since cognitive representations might be social. He specifies conditions under which more or less social context has to be provided and concludes that the social context of knowledge construction is par- ticularly important when knowledge is vague, open-ended, underspecified, and fragmentary. As a consequence, especially when building computer systems that can conduct effective conversation, the system has to possess basic social abilities
in terms of having internal representations of the knowledge, beliefs, goals, intentions, and values of the human user (see Krämer2008, who argues in favor of implementing a theory of mind).
Some systems already explicitly consider social aspects: For example, in several developments agents take the role of a (fellow) student instead that of a teacher and foster learning by the fact that the student has to engage in explaining the learning matter to the (artificial) peer (see, for example, the teachable agents paradigm, Schwartz, Blair, Biswas, & Leelawong 2007). According to Veletsianos and Russell (2014) these agents are expected to lower learner anxiety (by seeming less threatening than instructors), act as role models and“teach”via giving the students the chance to detect mistakes that the agent makes during the learning process.
Another possibility to engage the learner socially has been presented as social instructional dialog (Ogan, Aleven, Jones, & Kim,2011) by which the agent tries to improve learner-agent interpersonal relations. Agents who used conversational strategies assumed to produce positive interpersonal effects, lead to positive effects on learners’entitativity (feeling of working together in a team), shared perspective, and trust with the agent.
Therefore, with a view to social psychological aspects the distinction between different roles of the agent as either a mentor, a peer or a protégé is important.
Against the background that by means of pedagogical agents developers attempt to replicate useful human characteristics in the virtual world, future research needs to gain insights on what the desirable and beneficial attributes for each role in face-to-face interaction are. These attributes can be transferred to the virtual agents in order to test whether these attributes also have beneficial effects in VR.
At least with regard to the agent’s role as mentor, future research can build on Klauer’s (1985) taxonomy of teaching functions that have been proposed by Heidig and Clarebout (2011) as useful for the realm of pedagogical agents: motivation, information, information processing, storing and retrieving, transfer of information, monitoring and directing.
What is additionally important to note, is that social effects of agents are to be expected anyway: early evaluation studies of conventional computers characterized by human-like attributes (Reeves & Nass, 1996) as well as with embodied con- versational agents provided evidence that machines and agents are readily perceived as social entities: even minimal cues and similarity with humans suffice to lead users to show behavior that would be expected in human–human interaction. It is therefore plausible to assume that learners will also interact socially with peda- gogical agents. This will also enhance chances that social mechanisms of learning will also transfer to the virtual world.
Given the open questions, e.g. with regard to the effects of the agents’nonverbal behavior, future research is needed. This is best conducted against the background of a coherent theoretical framework that is able to explain the mechanisms. There have been suggestions for frameworks for future studies (Heidig & Clarebout, 2011) as well as suggestions on the necessities for methodology. With regard to the latter, it will be important to test agents in naturalistic settings and in longterm studies (Veletsianos & Russell,2014; Krämer & Bente,2010).