5.3 Method
5.3.2 Study 2: Preservice Teachers Designing a Lesson Plan Through Knowledge Building
thinking is usually represented by idea synthesis activity. It was therefore expected that progressively more idea-synthesizing activities (rather than just focusing on idea diversification and clarification) could be found towards the end of the course.
T-tests were employed to compare between the two phases and see if there were any changes in terms of how ideas evolved over time.
Analysis of Technological Products
Second, the quality of the technology products designed by the participants was peer-assessed using Besemer’s (1998) “Creative Product Analysis Matrix (CPAM).” The original scale has an internal consistency reliability level of Cronbach’s α¼.83. One thing to note is that due to time constraints, students were only required to design technology products in concept form during the course. No actual products were made. The instructional goal was to engage students in design thinking by means of a collaborative process of working with ideas, rather than producing real products. So, of the three dimensions of CPAM, only the first dimension (i.e., novelty) was employed in the final assessment. In this dimension, there are two assessment criteria, including originality and surprisingness. The assessment adopted a seven-point Likert scale. Using a four- point average as a baseline rating for comparing another commonly seen product (e.g., a typical toilet), students were asked to judge the novelty of a designed product. As there were ten groups in this course, there were ten products/artifacts to be assessed. A one-sample t-test was conducted to see if there were any significant differences between the designed products and the baseline products.
5.3.2 Study 2: Preservice Teachers Designing a Lesson Plan
a tutorial lesson on how to use Knowledge Forum™was provided. Students learned some of the basic features and functions such as how to create a note in a “view”
(i.e., a collective problem-solving space in Knowledge™Forum) or “build on” a previous note. The main course requirement was for each participant to practice their teaching twice, to design a complete lesson plan before each practice, and to provide peer feedback for teaching improvement in Knowledge Forum™after each practice. Each participant took turns to practice teaching for the first time during the first half of the semester, and the same cycle was repeated during the second half of the semester (i.e., the second teaching practice). The main class activities were thus participants’ teaching practices, and the main online activities were their idea generation activity (i.e., posting initial teaching ideas for their design of lesson plans) and their idea improvement activity in the form of peer feedback through sustained online collaboration.
5.3.2.2 Data Source and Analysis
Data sources include ideas recorded in the database, students’lesson plans and their videotaped teaching practices. Details regarding data analysis are explained as follows.
Analysis of the Quality and Types of Ideas
To understand how collaborative knowledge building affected the quality and types of feedback for ideas contributed online, content analysis was conducted. The main feedback activity was focused on improving one another’s teaching ideas and practices. This is important in this study as the capacity to give constructive and productive feedback to help one another design lessons and improve teaching practices represents a critical factor in developing one’s critical-thinking and articulation skills (Shute2008), which constitutes an important part of one’s design thinking. To assess the quality of feedback ideas, a coding scheme based on the TPACK conceptual framework originally proposed by Mishra and Koehler (2006) was adopted. Table5.2shows the coding scheme, with a description of each form of teaching knowledge and coding example being provided. The seven forms of TPACK were further categorized into two levels: (1) basic, lower-level TPACK, which includes only one form of teaching knowledge (i.e., TK, PK, and CK), and (2) integrated, higher-level TPACK, which includes two or three forms of teaching knowledge (i.e., TCK, PCK, TPK, and TPACK). Using each specific idea as a unit of analysis, each suggested improvement idea was categorized into one of the two levels mentioned above. Furthermore, all online ideas were categorized into two additional types. One type focuses on ideas contributed to address teaching weak- nesses, another on ideas contributed to reinforce teaching strengths. This was used as an indicator to tell how preservice teachers’teaching practices were improving over time. A paired-sample t-test was performed to see whether there were any
changes between the first and the second teaching practices in terms of the quality of ideas (as reflected in the use of TPACK) and types of ideas (as reflected in whether teaching strengths or weaknesses were addressed by the contributed ideas).
Analysis of Design Focus
Rubrics for assessing participants’design focus were employed in this study. The rubrics were developed based on Collins’(1996) conceptualization about designing for different kinds of learning activities. He argues that depending on one’s design focus, there are trade-offs in designing any learning environments. For example, if a teacher intends to design a more didactic-oriented learning environment, he or she might highlight memorization (i.e., memorizing things in order to do tasks fast and easily) rather than thoughtfulness. Table5.3shows eight design dimensions.
Table 5.2 A coding scheme developed based on TPACK framework
Knowledge Description Example
Content knowl- edge (CK)
Understanding of the subject matter taught
There is too much emphasis on test items in terms of teaching contents.
(S3) Pedagogical
knowledge (PK)
Use of teaching approaches and strategies to promote learning, man- age classroom, and create proper learning atmosphere
You lecture too much, but I like that you summarized what you taught at the end of the class. (S4) Technology
knowledge (TK)
Knowledge to utilize various instruc- tional media, including books, black- boards, PowerPoint, video clips, internet information, etc.
It is exciting to use an interactive whiteboard to teach as it can be used both as a projector and a blackboard. (S6)
Pedagogical con- tent knowledge (PCK)
Understand how to utilize various pedagogical approaches for different teaching content, and select appro- priate ones for different classes/
teaching purposes
My teaching strategy is to let stu- dents solve the problem first by themselves, instead of feeding them the correct knowledge/
answers directly. (S2) Technological
content knowl- edge (TCK)
Effectively combine the use of instructional media and teaching content
Before, I seldom used (technology- based) teaching aids, but I just found out that with the help of them, I can make a huge difference in helping students understand the content that I want them to learn.
(S8) Technological
pedagogical knowledge (TPK)
Understand how to use instructional media with various instructional approaches; select proper media for various instructional strategies
Using a learning sheet as a media to promote discussion among stu- dents can make class learning more joyful; also students can learn from one another that there is usually more than one way to solve a math problem. (S1)
76 5 Design Thinking and Preservice Teachers
In it, the first four dimensions indicate a didactic-oriented design focus, whereas the later four dimensions indicate a more constructivist-oriented design focus. Using a five-point Likert scale, students were instructed to peer-assess their two lesson plans designed for implementation of their first and second teaching practice. A paired-sample t-test was conducted to see if there was any change in terms of students’design focus as reflected in their lesson plans.
Analysis of the Two Videotaped Teaching Practices
Accordingly, to understand how participants performed their teaching practices, analysis of the videotaping of students’ two teaching practices was conducted.
Using each instructional activity as unit of analysis, the two teaching practices were parsed from the video and categorized into various types of teaching activity (e.g., teacher lecture or student discussion). Then, the activities were content- analyzed following the same coding categories mentioned above, namely, the two main types of instructional activity: didactic-oriented and constructivist-oriented activities. The former mode highlighted conventional instructional activities such as lecture, demonstration, asking true-false questions, direct instruction, and the like. The latter mode highlighted self-initiated or self-directed student learning activities, such as problem solving, teamwork, open-ended questioning, group debate and discussion, and the like. The percentage of time actually allocated to each mode of teaching activity (as shown in the video) in the two teaching practices was compared, using a paired-samplet-test for analysis.
Table 5.3 Rubrics for assessing two courses of design focus
Rubrics Description
Didactic-oriented design focus
Memorization Memorizing things in order to do tasks fast and easily (e.g., memorizing a multiplication table)
Component skills tasks
Having students perform simplified tasks that focus on specific subskills (e.g., having students practice sounding out different phonic patterns) Breadth of
knowledge
Having students learn a little about a lot of things
Uniform expertise Having all students learn the same thing (perhaps at the same time) Constructivist-oriented design focus
Thoughtfulness Learning to do things thoughtfully (e.g., solving complex problems) Whole tasks Having students perform whole tasks that involve integration of a variety of
skills (e.g., having students read a whole book for enjoyment) Depth of
knowledge
Having students understand a few topics in a deep way
Diverse expertise Allowing students to gain diverse expertise by learning different things at the same time