MULTIMODALITY: WHAT, WHY, AND HOW Dr. Nur Arifah Drajati
3 TOPICS FOR DISCUSSION
What is multimodality?
Why should we use multimodality for our teaching and learning?
How to implement multimodality in our ELT
classroom?
WHAT IS MULTIMODALITY?
What is Multimodal literacy?
Multimodal literacy is defined as the study of language combining two or more modes of meaning originated in social semiotics (Mills, 2010, 2015; Mills & Unsworth, 2017).
The processing of modes, such as image, words, sound and movement within texts can be synchronous and integrated.
The term indicates the process of literacy and meaning-making, and the process occurs differently when interpreting, viewing, understanding, interacting with new communication media and multimodal texts (Jewitt & Kress, 2003; Walsh, 2008).
It may include listening, talking and creating as well as writing, designing and producing (Walsh, 2009).
4
Multimodal Literacy
WHY SHOULD WE CONSIDER MULTIMODALITY FOR OUR TEACHING
AND LEARNING?
THE COMBINE USE OF
8
Texts/genres/purp oses
Images/visual (photos and
drawings)
Sounds/audio (oral and music)
Videos
Gestures
Costume, etc
(Yi, 2017)
Semiotic resources for meaning making
Metalanguage for teaching multimodal literacy
WHY MULTIMODALITY?
10
Naturalistic? That’s just the way?
Authentic (lived experience, authors, audience) Intentional (Self-motivated, regulated)
Collaborative/Interactional Active (anywhere, anytime) Constructive (iterative)
Meaningful learning
(Jonassen, 2000)
LEARNING
Learning from
Technology Learning with Technology
Emerging Web 2.0 technologies Technology as a tool for participation and
creation
Learners as active participants –
“Knowledge Building”
Has been dominant in our classroom Mostly drill & practice
Technology as a delivery tool –
“Knowledge Transfer”
HOW TO IMPLEMENT MULTIMODALITY IN
ELT CLASSROOMS?
WHAT IS TPACK?
Teachers’ knowledge base PCK: Pedagogical Content Knowledge (Shulman, 1985)
WHAT IS TPACK?
What kinds of knowledge do teachers need to have to teach with technology?
Teaching with technology is complex.
14
Figure 1. TPACK framework (From:http://tpack.org/).
Language Input
Language Output
Comprehending &
Using language
Active Language Learning
CONCEPTUAL INSTRUCTIONAL DESIGN
Blended learning
Task Based Language
Learning
Discovery
Learning Experiential Learning Digital
Storytelling Literature
Circles Collaborative
Learning Cooperative Learning Problem
Based Learning
Action
Learning Project Based
Learning Peer Learning
INSTRUCTIONAL DESIGN
Knowledge & Beliefs Skills
Attitudes Values
Experience/Behavior
GENERIC PBL ESSENTIALS (NEWMAN, 2005)
Students must have responsibility for their own learning. .
The problem simulations used in Problem Based Learning must be ill structured and allow for free inquiry.
Learning should be integrated from a wide range of disciplines or subjects.
Collaboration is essential.
What students learn during their self-directed learning must be applied back to the problem with reanalysis and resolution.
A closing analysis of what has been learned from work with the problem and a discussion of what concepts and principles have been learned is essential. .
Self- and peer assessment should be carried out at the completion of each problem and at the end of every curricular unit. .
The activities carried out in Problem Based Learning must be those valued in the real world. .
Student examinations must measure student progress towards the goals of Problem Based Learning. .
Problem Based Learning must be the pedagogical base in the curriculum
PROBLEM BASED
LEARNING
PASSIVE LANGUAGE VS.
ACTIVE LANGUAGE
Passive language: Can comprehend /understand language input (listening, speaking); but cannot use the language (language output)
Active language: Can do both!
Also: passive vocabulary vs. active vocabulary
Decoding vs. Encoding
20
TRADITIONAL LANGUAGE LEARNING
A language is a fixed structural system
Teacher transfers the knowledge of language structure to learners learners imitate and practice repeatedly
Learning goal: practice until “automated”, becoming a “habit”
Examples: Drill & Practice, Direct method
Focusing on explicitly learning and practice fragmented linguistic knowledge
But usually no attempt to link such knowledge with communicative
skills
Language input and language output activities are equally important!
interaction!
Language development is achieved through social interactions
Learning by “doing” (i.e., interactions)
Learning and “doing” are interwoven
Communicative Approach: Focus on “meaning” (fluency) before “form”
(accuracy)
22
Contemporary Language Acquisition/Learning Approaches
Q 4. You will see one of a short movie. Make comics based on this movie. Create at least in 4 cuts but maximum in 6 cuts.
https://youtu.be/uh4dTLJ9q9o
① ②
③ ④
Sample
Q 4. You will see one of a short movie. Make comics based on this movie. Create at least in 4 cuts but maximum in 6 cuts.
24
https://youtu.be/uh4dTLJ9q9o
① ②
③ ④
Sample
Design elements Evaluation questions
Linguistic design
• Was the linguistic content comprehensible without major grammatical errors?
• Was the linguistic content structured in a logical and organized manner?
• How did the linguistic design represented in the multimodal text enable or limit the author’s communicati on of meaning?
Visual design
• Did the author adopt a visual theme?
• Did the author carefully design the use of color and typology to reflect the selected visual theme?
• If chosen to use, did the author make meaningful use of available visual elements, such as graphics, to c onstruct meaning in a cohesive manner?
• How did the visual design represented in the multimodal text enable or limit the author’s communication of meaning?
Auditory design
• Did the author make use of any auditory elements, such as music, sound effect or narration?
• If chosen to use, were the auditory elements used purposefully and meaningfully to complement or supp lement the other design modes for meaning construction in a cohesive manner?
• How did the auditory design represented in the multimodal text enable or limit the author’s communicati Table 1: The theory-driven design rubric developed in the study
Multimodal Assessment Rubric
TRY TO TEACH IN
DIFFERENT WAY
AND FIND THE
HAPPINESS
INSIDE
THANK YOU
