CHAPTER II RESEARCH METHOD RESEARCH METHOD
2.3 Research Instruments
In the present study, some research instruments were employed to collect the data. They are writing prompt used to elicit data on students‘ argumentative writing on student initiated topic (see Appendix 1A) and on teacher initiated topic (see Appendix 1B), rubric for assessing topic familiarity through mind map (see Appendix 2A), rubric for assessing writing performance (see Appendix 2B) and rubric for assessing critical thinking skills (see Appendix 2C). Each instrument is explained below.
2.3.1 Writing Prompts
There are two writing prompts used as the instrument to collect the data on students‘ argumentative writing. Since good prompt should go together with the instructional objective (Weigle, 2007), the writing prompts in this study were made on the basis of the instructional objective of the critical writing course. The prompt on writing a composition is chosen as it requires students to communicate by putting together what they know about a topic into a piece of connected discourse (Jacobs et al., 1981).
The first prompt is the instruction on writing a composition of at least 400 words presenting student‘s argument on free topic or the topic which interests him/her (see Appendix 1A) while the second one is on one of the given topics. In other words, the first prompt concerns with the argumentative writing on student initiated topic, whereas the second one involves the writing on teacher initiated topic (see Appendix 1B). The different prompt is needed to see whether there is
different performance as a good assessment should call upon a broader construct than is usually tested in assessments that focus on relatively simple, on-demand writing tasks (Deane et al., 2008; Martin, 2006). The prompts also requires mind map writing related to the given topic. Accordingly, the writing prompts were used to measure the three variables at the same time, namely topic familiarity, writing performance and critical thinking skills.
2.3.2 Rubric for Assessing Topic Familiarity
The assessment on topic familiarity is done through mind map which is accomplished before the students wrote their essays as stipulated in the writing prompt (see Appendix 2A). It is used to gain the information on the students‘
knowledge on the topic in the writing prompt. The rubric is adapted from Franker (2011) to identify the student‘s familiarity on certain topic seen from the
arrangement of concepts, links and linking lines, content, and text. The
arrangement of concept identified through the division of the main idea and sub-concepts. The links used clarify the connection among the concept presented. The content and the text deal with the logical clarity and the readability of the
information given. Each category is rated as follows: 5 representing
unsatisfactory, 10 representing proficient, and 15 representing exemplary. The total score is categorized into very good (60-53); good (52-45); fair (44-37); poor (36-29) and very poor (28-20).
2.3.3 Rubric for Assessing Writing Performance
The assessment of writing performance is done based on the essay writing.
This is also applied in assessing critical thinking skills. Although the assessments are based on the same essay writing, the measures for writing performance and
critical thinking skills have different components. The components are presented in the form of rubrics.
The rubric for assessing writing performance is taken from the result of considerable and careful research conducted on ESL Composition Profile (Jacobs et al., 1981; Hartfiel et. al, 1985). The rubric is chosen as it fulfills the criteria of good assessment tool as it uses specific and appropriate language to describe the data gathered and the patterns that are observed (Connors, 2008; Crook, 2006;
Dappen et al., 2008; Peha, 2003). It assesses the content, organization,
vocabulary, language use and mechanics. The content is assessed through some descriptors such as knowledgeable, substantive, thorough development of thesis and relevant to assigned topic. The organization refers to the fluent expression, ideas supported, logical sequencing and other descriptors such as brief, well-organized and cohesive. The vocabulary is examined in terms of the sophisticated range, effective word choice, word form mastery and appropriate register. The language use concerns with the use of effective complex construction, agreement, tense, number, and word order. The mechanics deals with the attention on the use of spelling, punctuation, capitalization, and paragraphing (see Appendix 2B).
The total score of writing performance is interpreted into some categories such as excellent to very good (100-88); good to average (87-75); fair (74-64);
poor (63-49) and very poor (48-34). Using the rubric the focus of the assessment deals with the way the writers perform their academic writing skills.
2.3.4 Rubric for Assessing Critical Thinking Skills
The rubric for assessing critical thinking skills assesses the five elements:
arguments, evidence, recognition of opposition, refutation, and conclusion (see
Appendix 2C). The argument is assessed by the way the students construct their claim supported by reasons. The evidence deals with how the statements or assertions strengthen the argument. The way the students identify the statements that run counter their claim belongs to the recognition of opposition. Refutation occurs when the students make the statement that the opposing viewpoints are inadequate in some ways. Students‘ conclusion is examined on how they set out what they want the reader to believe. The rubric is adapted from Stapleton (2001) and used to enable the raters to assess the students‘ essays more effectively (Rezaei & Lovorn, 2010).
Based on the quality of each of the critical thinking skill, the scale given is from 1 to 5 (see Appendix 2C). Score 1 means that the elements of critical
thinking is not existing, score 2 means the elements are reflected implicitly, score 3 means the elements are identifiable; score 4 means the elements are reflected explicitly and adequately, and score 5 means the elements are reflected and
developed well. The total score is categorized as very good (25-21); good (20-17);
fair (16-13); poor (12-9) and very poor (8-5) in reflecting the elements of critical thinking. By employing the rubric the focus of the assessment concerns the rhetorical strategies employed in presenting the argument.