CHAPTER 3 RESEARCH METHODOLOGY
3.5 Data Collection Method
3.5.1 Writing Test
Test was used to measure the tenth grade students‘ writing achievement which covered content, organization, vocabulary, grammar, and mechanics. The achievement test is used to know how successful individual students, group of students or course themselves in achieving the learning objectives. (Weigle, 2002:116).
In the writing test every student in both classes would select one of the topics to write a descriptive text consisting of two paragraphs (60 - 80 words). Students have to complete the test in 90 minutes.
A good writing test needs to be valid and reliable. Heaton (1990:159). Hughes (2003:26) says that a test is considered to be valid if it can measure what is intended to measure. To meet the content validity of the test, the researcher constructs the test based on the Curriculum 2013.
Concerning the scoring method, the researcher would use analytical scoring method. According to Weigle (2002:114) in analytical scoring, scripts are rated on several aspects of writing criteria rather than given a single score. It means that the writing will be scored separately based on the certain aspects. In this research, there were five aspects that will be evaluated covering content, organization, vocabulary, language use and mechanics. The following is the analytical scoring rubric that will be used adapted from Weigle (2002:116). Because the researcher wants to adjust the student needs, the theories in chapter two, and easily to scoring their writing.
24 The original scoring rubric
SCORE LEVEL CRITERIA
C O N T E N T
30 – 27 EXCELLENT TO VERY GOOD: knowledgeable – substantive – thorough development of thesis – relevant to assigned topic.
26 – 22 GOOD TO AVERAGE: some knowledge of subject – adequate range – limited development of thesis – mostly relevant to topic, but lacks detail.
21 – 17 FAIR TO POOR: limited knowledge of subject – little substance – inadequate development of topic.
16 – 13 VERY POOR: does not show knowledge of subject – non-substantive – not pertinent – OR not enough to evaluate
O R G A N I Z A T I O N
20 – 18 EXCELLENT TO VERY GOOD: fluent expression – ideas clearly stated/supported – succinct – well organized – logical sequencing – cohesive.
17 – 14 GOOD TO AVERAGE: somewhat choppy – loosely organized but main ideas stand out – limited support – logical but incomplete sequencing.
13 – 10 FAIR TO POOR: non-fluent – ideas confused or disconnected – lacks logical sequencing and development.
9 – 7 VERY POOR: does not communicate – no organization – OR not enough to evaluate.
25 V
O C A B U L A R Y
20 – 18 EXCELLENT TO VERY GOOD: sophisticated range – effective word/idiom choice and usage – word form mastery – appropriate register.
17 – 14 GOOD TO AVERAGE: adequate range – occasional errors of word/idiom form, choice, usage but meaning not obscured.
13 – 10 FAIR TO POOR: limited range – frequent errors of word/idiom form, choice, usage – meaning confused or obscured.
9 – 7 VERY POOR: essentially translation – little knowledge of English vocabulary, idioms, word form – OR not enough to evaluate.
L A N G U A G E
U S E
25 – 22 EXCELLENT TO VERY GOOD: effective complex construction – few errors of agreement, tense, number word order/function, articles, pronouns, prepositions.
21 – 18 GOOD TO AVERAGE: effective but simple constructions – minor problems in complex construction – several errors of agreement, tense, number, word order/function, articles, pronouns, preposition but meaning seldom obscured.
17 – 11 FAIR TO POOR: major problems in simple/complex construction – frequent errors of negation, agreement, tense, number, word order/function, articles, pronoun, preposition and/or fragments, run- ons, deletions – meaning confused or obscured.
10 – 5 VERY POOR: virtually no mastery of sentence construction rules – dominated by errors – does not communicate – OR not enough to evaluate.
26 M
E C H A N I C S
5 EXCELLENT TO VERY GOOD: demonstrates mastery of conventions – few errors of spelling, punctuation, capitalization, paragraphing.
4 GOOD TO AVERAGE: occasional errors of spelling, punctuation, capitalization, paragraphing but meaning not obscured.
3 FAIR TO POOR: frequent errors of spelling, punctuation, capitalization, paragraphing – poor handwriting – meaning confused or obscured.
2 VERY POOR: no mastery of convention – dominated by errors of spelling, punctuation, capitalization, paragraphing – handwriting illegible – OR not enough to evaluate.
Total Score =
27 Table 3.5.2 Scoring rubric that has been adapted.
SCORE LEVEL CRITERIA
C O N T E N T
5 EXCELLENT: very relevant to topic - very relevant to content.
4 GOOD: knowledgeable – substantive – relevant to assigned topic.
3 AVERAGE: some knowledge of subject – adequate range – mostly relevant to topic, but lacks detail.
2 POOR: limited knowledge of subject – little substance – inadequate development of topic.
1 VERY POOR: does not show knowledge of subject – non-substantive – not pertinent – OR not enough to evaluate
O R G A N I Z A T I O N
5 EXCELLENT: very fluent expression - very clearly ideas - very well organized.
4 GOOD: fluent expression – ideas clearly stated/supported – succinct – well organized – logical sequencing – cohesive.
3 AVERAGE: loosely organized but main ideas stand out – limited support – logical but incomplete sequencing.
2 POOR: non-fluent – ideas confused or disconnected – lacks logical sequencing and development.
1 VERY POOR: does not communicate – no organization – OR not enough to evaluate.
28 V
O C A B U L A R Y
5 EXCELLENT: very effective word - very appropriate vocabulary.
4 GOOD: sophisticated range – effective word/idiom choice and usage – word form mastery – appropriate register.
3 AVERAGE: adequate range – occasional errors of word/idiom form, choice, usage but meaning not obscured.
2 POOR: limited range – frequent errors of word/idiom form, choice, usage – meaning confused or obscured.
1 VERY POOR: essentially translation – little knowledge of English vocabulary, idioms, word form – OR not enough to evaluate.
L A N G U A G E
U S E
5 EXCELLENT: very effective complex construction – no errors of agreement, tense, number word order/function, articles, pronouns, prepositions.
4 GOOD: effective complex construction – few errors of agreement, tense, number word order/function, articles, pronouns, prepositions.
3 AVERAGE: effective but simple constructions – minor problems in complex construction – several errors of agreement, tense, number, word order/function, articles, pronouns, preposition but meaning seldom obscured.
2 POOR: major problems in simple/complex construction – frequent errors of negation, agreement, tense, number, word order/function, articles, pronoun, preposition and/or fragments, run-ons, deletions – meaning confused or obscured.
1 VERY POOR: virtually no mastery of sentence construction rules – dominated by errors – does not communicate – OR not enough to evaluate.
29 M
E C H A N I C S
5 EXCELLENT: very good demonstrates mastery of conventions – no errors of spelling, punctuation, capitalization.
4 GOOD: demonstrates mastery of conventions – few errors of spelling, punctuation, capitalization.
3 AVERAGE: occasional errors of spelling, punctuation, capitalization.
2 POOR: frequent errors of spelling, punctuation, capitalization, poor handwriting.
1 VERY POOR: no mastery of convention – dominated by errors of spelling, punctuation, capitalization, handwriting illegible.
Total Score =
(Adapted from Weigle, 2002:116) Based on the kinds of scoring, test is divided into two: objective test and subjective test. Objective test is a kind of test that does not need any judgment from the scorer. Meanwhile, subjective test is a kind of test that needs judgment/
opinion/impression from the scorer. The answer of subjective test is usually in the form of sentences, paragraph, or text/description including the essay type of test. In this research, the test that will be used belongs to subjective test which is constructed in the form of essay.
Due to the high subjectivity of subjective test, it can be reduced by using inter-rater or intra-rater reliability. According to Weigle (2002:135) intra-rater reliability refers to the tendency of a rater to give same score to the same script on different occasion, while inter-rater reliability refers to the tendency of different raters to give the same scores to the same script. In this research, the researcher will apply inter-rater scoring method. There will be two raters in this research, the first rater is the researcher and the second rater is the English teacher.
30