(A Case Study of the First Grade Students of 3 State Junior High School of Tangerang Selatan)
Written By:
Dwi Ciptaningrum
109014000051
DEPARTMENT OF ENGLISH EDUCATION
FACULTY OF TARBIYAH AND TEACHERS’ TRAINING
SYARIF HIDAYATULLAH STATE ISLAMIC UNIVERSITY JAKARTA
of Junior High School of 3 Tangerang Selatan), Skripsi, English Education Department, Faculty of Tarbiyah and Teachers’ Training, Syarif Hidayatullah State Islamic University Jakarta.
Key words: Item Difficulty Level, Item Discriminating Power, Summative Test This study is purposed to measure the difficulty level and the discriminating power of the English summative test for the first grade of SMPN 3 Tangerang Selatan at odd semester 2013/2014 academic year. Furthermore, this study is conducted to analyze the test items whether it is included in easy, moderate, or difficult items and whether the items are able to discriminate the upper and the lower group of students.
This study is categorized as a qualitative research by using the descriptive analysis because the writer describes the difficulty level and discriminating power by analyzing the test items of the English summative test. This research is also supported by some numerical data which are analyzed statistically by using Anates Program
and manual counting. The writer takes 92 students’ answer sheets as the data which are divided the students into three groups: upper, middle, and lower group by
arranging the students’ score from the highest till the lowest scores. But the writer
only takes 27% (25 students) from the upper and the lower group of students to be analyzed.
of Junior High School of 3 Tangerang Selatan), Skripsi, Jurusan Pendidikan Bahasa Inggris, Fakultas Ilmu Tarbiyah dan Keguruan, Universitas Islam Negeri Syarif Hidayatullah Jakarta.
Kata kunci: Tingkat Kesukaran Butir Soal, Daya Pembeda, Tes Sumatif
Penelitian ini bertujuan untuk mengukur tingkat kesukaran soal dan daya pembeda pada tes sumatif bahasa inggris untuk tingkat pertama di SMPN 3 Tangerang Selatan pada semester ganjil tahun akademik 2013/2014. Terlebih lagi, penelitian ini dilakukan untuk menganalisa butir soal apakah tergolong dalam butir soal mudah, sedang, atau sulit dan apakah butir soal tersebut dapat membedakan siswa yang berkemampuan tinggi dan berkemampuan rendah.
Penelitian ini dikategorikan sebagai penelitian kualitatif dengan menggunakan analisis deskriptif, penulis mendeskripsikan tingkat kesukaran soal dan daya pembeda dengan menganalisa tiap soal pada tes summatif bahasa inggris. Penelitian ini juga didukung oleh beberapa data numerik yang dianalisa secara statistik. Dalam penelitian ini, penulis hanya menggunakan 92 lembar jawaban siswa sebagai data yang dibagi menjadi tiga kelompok siswa: kelompok pandai, sedang, dan kurang pandai dengan menyusun skor siswa mulai dari yang paling tinggi hingga yang paling rendah. Tetapi penulis hanya menggunakan 27% (25 siswa) dari kelompok siswa yang pandai dan kurang pandai untuk di analisa.
First of all, praise and gratitude be to Allah, Lord of the world Who has given the Blessing to the writer, so that this “Skripsi” could be finished completely. Then, Peace and Salutation be upon our prophet Muhammad, his families, composing, and his followers.
Grateful thanks also to her beloved families, special for her parents Mr. Tirsan and Mrs. Semirah, then her two brothers Eko Cahyo Prihartono, S. Sos. and Tri Ahmad Nur who have prayed and supported to the writer in finishing this “Skripsi”. Also the writer would like to express her greatest thanks and gratitude to:
1. Mrs. Nida Husna, M.Pd., M.A. TESOL as the first advisor and Mr. Dadan Nugraha, M.Pd. as the second advisor who had kindly spent their time and knowledge to give valuable advice, guidance, correction, and suggestion in finishing this “Skripsi”.
2. Mr. Drs. Syauki, M.Pd. and Mr. Zaharil Anasy, M. Hum. as the Head and the Secretary of English Education Department
3. Mr. H. Maryono, S.E, M.MPd. as the Headmaster and all of the teachers in SMPN 3 Tangerang Selatan who had given the writer allowed in taking the sample and supports to finishing her “Skripsi”.
4. Mrs. Dr. Farida Hamid, M.Pd. as an academic advisor for class B of English Education Department 2009/2010 academic year
their time for sharing and supports in accomplishing this “Skripsi”.
8. Finally, the writer must thank to staffs of the Main Library of State Islamic University Syarif Hidayatulah Jakarta and Library of the Faculty of Tarbiyah and Teachers’ Training. Thanks for providing the sources to fulfill the references of the writer’s writing.
May Allah the Almighty bless the all, So Be It.
Finally, the writer realizes that this “Skripsi” is still far from being perfect. So, the constructive criticism and suggestion would be welcomed to make it better.
Jakarta, September 30th, 2014
ABSTRAK ... ii
ACKNOWLEDGEMENT ... iii
TABLE OF CONTENTS ... v
LIST OF TABLES ... viii
LIST OF APPENDIXES ... ix
CHAPTER I: INTRODUCTION ... 1
A. Background of Study ... 1
B. Identification of Problem ... 4
C. Limitation of Problem ... 4
D. Formulation of Problem ... 5
E. Objective of Study ... 5
F. Significance of Study ... 5
CHAPTER II: THEORETICAL FRAMEWORK ... 6
A. Summative Test ... 6
1. Definition of Summative Test ... 6
2. Criteria of Good Test in Summative Test ... 8
a. Validity ... 8
b. Reliability ... 8
c. Practicality... 9
3. Types of Test Item in Summative Test ... 9
a. Objective Test ... 9
1) True-False ... 10
b) Types of the Essay Test... 15
1. Extended Response Type ... 15
2. Restricted Response Type ... 15
B. Item Analysis ... 16
1. Definition of Item Analysis ... 16
2. Kinds of Item Analysis ... 17
a. Level of Difficulty... 17
b. Discriminating Power... 20
c. The Effectiveness of Distractor ... 22
3. The Difficulty Level and Discriminating Power ... 23
4. The Importance of Item Analysis ... 25
C. Previous Studies ... 27
D. Thinking Framework ... 29
CHAPTER III: RESEARCH METHODOLOGY ... 30
1. Place and Time of the Research ... 30
2. Method of the Research ... 30
3. Population and Sample ... 30
4. Technique of Data Collecting ... 31
5. The Instrument of the study ... 31
6. Technique of Data Analysis ... 31
CHAPTER IV: RESEARCH FINDINGS ... 34
A. Data Description... 34
B. Data Analysis ... 34
Table 2.2 The classifications of the index of Discriminating Power ... 22
Table 3.1 The rank scale of Level of Difficulty ... 33
Table 3.2 The classifications of the index of Discriminating Power ... 33
Table 4.1 The Group Position Based on the Test Result ... 44
Table 4.2 Format of Item Analysis of the English Summative Test ... 47
Table 4.3 Classification of Items Based on the Proportion of Difficulty Level... 51
Appendix 2: Format of Item Analysis of the English Summative Test ... 47
Appendix 3: Classification of Items Based on the Proportion of Difficulty Level ... 51
Appendix 4: Classification of Items Based on the Proportion of Discriminating Power ... 52
Appendix 5: The English Summative Test Paper ... 53
Appendix 6: The Answer Key of the English Summative Test ... 60
Appendix 7: Table of Students’ Answer Sheet of the Upper Group ... 61
A. Background of the Study
Evaluation is very important in the process of teaching and learning to know the condition and the result of those in the continual time. Based on“Pasal 58 ayat
(1) UU RI No. 20 Tahun 2003 tentang Sisdiknas, yang menyatakan bahwa evaluasi
hasil belajar peserta didik dilakukan oleh pendidik untuk memantau proses,
kemajuan dan perbaikan hasil belajar peserta didik secara berkesinambungan.”1
Based on the section of 58, verse (1) The Law of Republic of Indonesia Number 20, years of 2003 about Sisdiknas (System of National Education) “The evaluation of the result of student learning which is conducted by the teacher to observe the process, the progress and revision the result of the student learning continually”.
In line with the statement above, evaluation is conducted by the teacher in periods of time within the process of teaching and learning. Its purpose is to give information whether the teaching and learning have succeeded or not. On the other hand, the teacher will know the weakness of the teaching in the classroom so the teacher will revise his or her teaching, arranging the lesson planning, giving assignment, and controlling the classroom well. As Paulina Rea and Kevin Germaine, they state, “Evaluation is an intrinsic part of teaching and learning. It is important for the teacher because it can provide a wealth of information to use for the future direction of classroom practice, for the planning of courses, and for the management of learning tasks and students.”2 So, the teacher can get the information about the success of the teaching and learning process because test is one of the ways of
1
M. Sukardi, Evaluasi Pendidikan Prinsip & Operasionalnya, (Jakarta: PT Bumi Aksara, 2011), p. 12.
2
evaluation. It can be known from the amount of the correct answer of the test, if many test items can be answered by the students correctly, it can be concluded that the teacher has succeeded in the teaching and learning process.
According to Rebecca M. Valette states, “Through tests the teacher can evaluate the effectiveness of a new teaching method, of a different approach to a difficult pattern, or of new materials.”3Therefore, “the classroom test is concerned
with evaluation for the purpose of enabling teachers to increase their own effectiveness by making adjustments in their teaching to enable certain groups of students or individuals in the class to benefit more.”4
Talking about test, there are some tests which can determine the student’s competence grades in the past of the learning activities in the classroom, one of the tests is achievement test. There are four types of achievement test which are very commonly used by teachers in the classroom: placement, formative, diagnostic, and summative test.5 The type of the achievement test which often used by the teacher to evaluate the successfulness his or her teaching and learning in the classroom is summative test.
The summative test is used in the end of the courses of instruction to know the students’ performance grade whether they have already mastered all of the materials which they have reached while the teaching and learning process. Absolutely, in the summative test has to represent all of the topics which have taught by the teacher.
3
Rebecca M. Valette, Modern Language Testing, (New York: Harcourt Brace Jovanovich, Inc., 1977), p. 5.
4
JB. Heaton, Writing English Language Tests, (London: Longman Group UK Limited, 1988), p. 6.
5
Therefore, in making a test the teacher should have some criteria of good test. It means that, all good tests must have three qualities: validity, reliability, and practicality.6
Besides the three qualities, the test must also has a good difficulty level and an effective discriminating power because the difficulty level will give the information about the percentages of easy, moderate, and difficult items whereas discriminating power also will give the information about the effectiveness of each item whether the test item is able to differentiate the high students performance and the low students performance. Therefore, both of them can be analyzed by using item analysis.
According to J. Stanley Ahmann and Marvin D. Glock in their book about item analysis:
“Item analysis usually concentrates on two vital features; level of difficulty and discriminating power. The former means the percentage of pupils who answer correctly each item; the latter the ability of the test item to differentiate between pupils who have done well and those who have done poorly.”7
Based on the statements above, the writer would like to explain her problem while she was following the teaching practice/PPKT in the first grade of SMP Negeri 3 Tangerang Selatan she found that there were some test items which had not been taught by the teacher in the English summative of the final test which had been given on Wednesday, December 11th, 2013 at odd semester 2013/2014 academic year.
According to one of the English teachers at SMP Negeri 3 Tangerang Selatan, it is not important that in making test items the teacher tested the same materials which is given to the students, because he believed that it was similar to English National Examination test in term giving the test items.
At one side, the materials taught by the teacher cannot be answered by the students well especially for the materials that have not been taught, it will be more difficult.
Based on the fact above, the writer would like to analyze the test by using the item analysis which is focused only on the difficulty level and the discriminating power of the English summative test of the first grade in SMP Negeri 3 Tangerang Selatan. So, the writer will conduct the study under the title “AN ITEM ANALYSIS
OF ENGLISH SUMMATIVE TEST ON DIFFICULTY LEVEL AND DISCRIMINATING POWER (A Case Study of the First Grade Students of 3 State Junior High School of Tangerang Selatan)”.
B. Identification of the Problem
Based on the background of the study, the writer identifies in some problems: 1. The teacher of the first grade at SMPN 3 Tangerang Selatan believed
that it is not important in making test items used the same materials given to the students.
2. There are some test items which have not been taught by the teacher in the English summative test at the final test of odd semester 2013/2014 academic year.
3. The upper group of students cannot answer some items well because they had not learned the material and the lower group of students can answer some items only by guessing the answers.
C. Limitation of the Problem
difficult item of the English summative test and the discriminating power to analyze whether the test items can differentiate the upper and lower of students.
D. Formulation of the Problem
Based on identification and limitation of the problems, the writer conducts an item analysis to find out the percentages of difficulty level then whether the test items can differentiate the upper and lower of students. So, the writer formulates the problem as follow: “Does the English summative test for the first grade students at SMP Negeri 3 Tangerang Selatan have a good quality of the difficulty level and the discriminating power?”
E. Objective of the Study
The objective of the study is to find out the difficulty level from easy, moderate, and difficult items and the discriminating power of the English summative test whether each item can differentiate the upper and lower group of students which is tested of the first grade at SMPN 3 Tangerang Selatan in the first semester 2013/2014 academic year.
F. Significance of the Study
The first of all, the result of the study to give clear description to the reader about the quality of the English summative test of the first grade at SMPN 3 Tangerang Selatan, especially in the percentages the easy, moderate, and difficult test items then to analyze whether the test items can differentiate the upper and the lower of students.
The second, the writer hopes that the result of the item analysis can be benefit for English teachers or the test makers to improve their competences to make a good test items and also to improve their teaching and learning in the classroom.
In this chapter, the writer tries to give the clear description of theoretical framework which explains the definition of summative test, categories of good summative test, types of the test item in summative test, the definition of item analysis, kinds of the item analysis, and the importance of the item analysis.
A. Summative Test
1. The Definition of Summative Test
Before talking about summative test, the writer wants to elaborate first the meaning of the test. Many experts have stated about some definitions of the test, according to Antony J. Nitko in his book, Educational Tests and Measurement: An Introduction, he writes test is defined as “a systematic procedure for observing and
describing one or more characteristics of a person with the aid of either a numerical scale or category system.”1
Another opinion, test is a technique or way consisting of some questions, statements, or tasks that are delivered to students in term of measuring their performance or behavior.2 To support, Wilmar Tinambunan, said that “a test is a set of questions, each of which has a correct answer, that examinees usually answer orally or in writing.”3
Based on the definitions above it can be concluded that, test is a utility to collect the information about students’ performances in term of a set of some items
1
Antony J. Nitko, Educational Tests and Measurement: An Introduction, (New York: Marcourt Brace Jovanovich, Inc, 1983), p.6.
2
Zainal Arifin, Evaluasi Pembelajaran, (Bandung: PT. Remaja Rosda karya, 2009), p. 118.
3
such as questions, statements in orally or writing by using category system such as a scoring.
Talking about the summative test, actually it is one of the types of the achievement test. The achievement test itself including in the test which can determine the student’s competent grades in the past of the learning activities in the classroom. The type of the achievement test which often used by the teacher to evaluate the successfulness his or her teaching and learning in the classroom is summative test.
According to WilmarTinambunan in his book, “the summative test is intended to show the standard which the students have now reached in relation to other students at the same stage. Therefore it typically comes at the end of a course or unit of instruction.”4
To support the statement above, the summative test is given at the end of a marking period and measures the “sum” total of the material covered. On this type of a test, students are usually ranked and graded.5The summative test is given in the end because it comes at the end of a course (or unit) of instruction. It is designed to determine the extent to which the instructional objectives have been achieved and is used primarily for assigning course grades or certifying pupil mastery of the intended learning outcomes.6
It means that summative test is tested in the end of a course to know the students’ competence from all the materials which have been taught by the teacher.
4
Ibid, p. 9.
5
Rebecca M. Valette, Modern Language Testing, (New York: Harcourt Brace Jovanovich, Inc., 1977), p. 11.
6
2. Categories of Good Test in Summative Test
Actually, the summative test has to represent all the materials which have been taught by the teacher. So, the teacher should have some criteria of good summative test. Those are validity, reliability, and practicality.
a. Validity
According to Wilmar Tinambunan, “Validity refers to the extent to which the results of an evaluation procedure serve the particular uses for which they are intended. Thus, the validity of the test is the extent to which the test measure what is intended to measure.”7
It can be concluded that the test will be useful if the test is able to measure what is intended to measure to know the quality of the test. It means that the test items in the summative test, the test maker must give the materials which are covered as a category of good test.
b. Reliability
The second criteria of good test are reliability. It is measured by a correlation between the scores of the same set of students on two consecutive administrations of the test.8 It can be supported by Herbert and William that, “reliability refers to the degree to which the measurements yielded by a test are consistent or stable.”9
It means that, if the test has been tested more than once in the same student in the different time but the score does not really different or change drastically with the score before it means the test can be called reliable. Another opinion, result of the score of the test not only stable but also dependable means show of the readiness of a test, and predictable that is the test is able to predict the next the result.
Heaton states that there are five factors affecting reliability of the test. The
Robert Lado, Language Testing, (New York: McGraw-Hill Book Company, 1964), p.31.
9
administration of the test, the third is the instruction, the fourth is personal factors such as motivation and illness, the last is about the scoring the test.10
c. Practicality
Practicality, the third criteria of a good test can be called as usability. In this case the teacher or the test maker in making a test should be consideration some practical such as economy, scoring, and interpretation. As Douglas Brown said in his book, “a good test is practical. It is within the means of financial limitations, time constraints, ease of administration, and scoring and interpretation.”11 It means that there are some factors which are considered before make a test.
According to Wilmar said, “before administering a test, some factors about the administration and the test itself must be carefully considered.”12 It can be concluded that before the test is used, the test maker has think of the some consequences of the usability the test itself, such as scoring procedure in order to easier the teacher to calculate the result of the test, in giving the test item in a test, and so on.
3. Types of Test Item in Summative Test
Besides the categories of the summative test, in that test there are some kinds of the types of the test items. Those are:
a. Objective Test
This test can be called objective if the answers of the test have only one the correct answer as a key of the test item. In this test the students have choose one correct answer from some choices which provided by the teacher. In giving the scoring of objective items requires much less time than the scoring of essay items.
10
J.B.Heaton, Writing English Language Tests, (New York: Longman, 1988), p. 162-163.
11
H. Douglas Brown, Teaching by Principles, (New York: Addison Wesley Longman, Inc., 2001), p. 386.
12
With its objectivity, objective items can be accurately scored with little if any dispute about the correctness of response.
According to Zainal Arifin’s book, “There are many varieties of there new test, but four kinds are in most common use, true-false, multiple-choice, completion, matching.”13
1) True-False
Based on Jum C. Nunnally, “The popularity of the true-false item is probably due to the ease with which such items can be composed. It is usually easy to make up many such items in a relatively short period of time.”14
Besides that, James Dean Brown and Thom Hudson, in their book “requires student to respond to the language by selecting one of two choices, for instance, between and true and false or between correct and incorrect.”15
To sum up, in true-false the students are able to answer the statement with true or false by short of time. Then the function of this test is to measure the competence of the student to differentiate between the fact with the opinion. In addition, the teacher not only provides question or statement in this test but also possible to give the picture, diagram, or table.
2) Multiple-Choice
The multiple-choice is the most popular in types of test because of the multiple-choice often is used in many kinds of objective test. To support the statement above, William states in his book, that “by far the most popular type of
13
Zainal Arifin, op. cit., p. 135.
14
Jum C. Nunnally, Educational Measurement and Evaluation, (New York: McGraw-Hill, Inc., 1964), p. 160.
15
James Dean Brown and Thom Hudson, Criterion-Referenced Language Testing,
objective item is that in which the student is required to choose one alternative response to a problem or question.”16
An additional, “A multiple-choice item is an item that presents a statement (called the stem) and the student is required to select one of two or more (usually more) options that correctly completes the statement or correctly answers the problem posed in the statement.”17
Similarly, a multiple-choice item consists of one or more introductory sentences followed by a list of two or more suggested responses from which the examinee chooses one as the correct answer.18
Above all, it can be concluded that in multiple-choice item consist of two parts, the question or statement it can be called as a stem and some choices which included as the correct answer, the most correct answer and the distractors, or it can be called as option. The form of the option is possible consist of words, numbers, or statements.
For instance: There is a stem:
Who is the boy in the text above?
and a number of options – one of which is correct, the others being distracters:
A. John B. Peter C. Smith D. George Education, (California: Addison-Wesley Publishing Company, 1976), p. 169.
17
Ibid, p. 200.
18
Actually, this item effectively to test the students’ knowledge such as the definition, names of country, and so on.19To support, “Usually, completion items require the testees to supply a word or a short phrase.”20
An example, in reading text of the summative test the teacher provides incomplete sentence or statement then the students have to fill the correct answer in the blank of that sentence or statement. For instance:
The author was surprised to meet Dr Short ………..
That item consists of blank to complete the sentence or statement which based on the text with one words or short phrase.
4) Matching
The matching item commonly appears in a two-column-format although variations on this format can be used. The two columns of a matching item are commonly called the premises and responses. Matching items lend themselves well to testing a knowledge of relationships or definitions.21
It means, there are many kinds of matching form, it begins from the premises it can be list of definitions, antonym, or synonym then the responses consist of the list of choices of the best or the appropriate answer. Usually, the answers consist more than the questions.
Besides that, Jum C. Nunnally in his book, he states that, “Students are asked to write in the blank space the letter corresponding to the option on the right which matches the item on the left. The major advantage of the matching item is that a considerable amount of material can be presented in a short space.”
19
Wilmar Tinambunan, op. cit., p. 61.
20
J.B. Heaton, op. cit., p. 124.
21
b. Subjective Test
In subjective test, the students have to answer the question based on their knowledge which have they got using their words to their writing. Talking about giving score of the subjective test, the teacher will score the answer based on the students’ answering, whether it is simple or complex answering and of course it depends on the teacher’s subjectivity. According to Arthur, in his book, “If judgment is called for, the scoring is said to be subjective.”22
Also, the students’ answering not only focuses of true or false answer but also it depends of complete or incomplete answer. At this point, the teacher also will know the competence of their students from as far as they mastered the materials which have given by the teacher.
a) Essay Test
The type of essay item the students supply their answer rather than choose the correct answer. To support, “the essay-type question requires the examinee to read the question, formulate his response and express the response in his own words.”23 It means that the students are given a freedom to express their idea to answer the question.
In addition, J. Stanley Ahmann and Marvin D. Glock, “an essay test item demands response composed by the pupil, usually in one or more sentences, of a nature that no single response or pattern of responses can be judged subjectively only by one skilled and informed in the subject, customarily the classroom teacher.”24
22
Arthur Hughes, Testing for Language Teachers…, p.19.
23
Ibid, p. 56.
24
Furthermore, the essay test usually consists of questions beginning with or including such directions as discuss, explain, outline, evaluate, define, compare, contrast, and describe.25
For instance:
Explain the differences between narration and recount text!
Therefore, there are some the advantages of the essay questions, those are: 1. The essay item is the most effective in assessing complex learning outcomes. 2. Constructing essay questions is relatively easy.
3. The possibility of guessing is minimized. 4. Constructing essay questions require less time.26
Besides that, essay test have some limitations, they are:
1. It would probably be discarded entirely as a measuring instrument did it not measure significant learning outcomes that cannot be measured by other means.
2. Unreliability of the scoring.
3. The amount of time required for scoring the answers. 4. The limited sampling they provide.27
25
Victor H. Noll, Introduction to Educational Measurement, (Boston: Houghton Mifflin Company, 1965), p. 131.
26
Ibid, p. 87-88.
27
Norman E. Gronlund & Robert L. Linn, Measurement and Evaluation in Teaching 6th Ed.,
b) Types of the Essay Test
Based on the amount of freedom of response, the essay test can be divided into two forms, those are:28
1. Extended Response Type
In this type, the students have a freedom to express their argument which based on their competence, so they can begin from the definition first then the example and the opposite, on the other hand, it can be called deductive and inductive text. But the teacher also has the criteria of the scoring of the students’ answering which based on the question.
According to Wilmar, “In the Extended Response Type Test, the pupil is given almost complete freedom in making his response.”
For example:
“Describe what you think should be the contribution of „Psycholinguistics’ to the improvement of language testing. Include those examples discussed in class”.29
2. Restricted Response Type
In the Restricted Response Type, the test item provides the specific problems and more structure. Although, they have a freedom to supply their ideas into their writing but the students have to answer systematically based on the question.
In addition, “The restricted response question usually limits both the content and the response. The content is usually restricted by the scope of the topic to be discussed. Limitations on the form of response are generally indicated in the question”.30
28
Wilmar Tinambunan, op. cit., p. 56.
29
Ibid.
30
B. Item Analysis
1. Definition of Item Analysis
According to Anthony J. Nitko in his book states, “Item analysis refers to the process of collecting, summarizing and using information about individual test items, especially information about pupils’ responses to items.”31
It can be concluded that analyzing the test item is important because it will give information for teachers, students, on how to process the teaching and learning well. Besides that, the teacher will know the quality of the test item and the effectiveness of the teaching instruction.
To support, “the feedback on individual items can help the instructor to identify points or concepts that are in need of review and further instruction.”32 It means that, the teacher will get the feedback of the progressing and revising of the teaching and learning also the test items itself in order to the teacher can provide a good test items in the next exam.
At one site, “Item analysis usually concentrates three vital features: level of difficulty, discriminating power, and the effectiveness of each alternatives. Thus, item analysis information can tell us if an item was too difficult or too easy, how well it discriminated between high and low scores on the test, and whether all the alternatives functioned as intended.”33
In the same way, item analysis tells us basically three things: 1. how difficult each item is,
2. whether or not the question “discriminates” or tells the difference between high and low students, and
31
Anthony J. Nitko, op. cit., p.284.
32
Kenneth D. Hopkins, Educational and Psychological Measurement and Evaluation (eight edition), (University of Colorado, Boulder: Allyn & Bacon, 1998), p. 254.
33
3. which distracters are working as they should.34
An additional, Lyle F. Bachman, states that, “to conduct an item analysis (IA) by hand, we first arrange the scored test papers or answer sheets in order from the highest score to the lowest score. Next, we separate the papers into upper and lower groups, according to their total test scores. In order to optimize these two objectives, for large group (i.e. 100 or larger) we would choose the upper and lower 27 percent, while for small groups, we would typically choose the upper and lower one-third.”35
2. Kinds of Item Analysis a. Level of Difficulty
In this kind of item analysis, level of difficulty have many other names, those are: item facility, item difficulty, item easiness, p-value, or it can be marked by IF. According to Heaton, “The index of difficulty (or facility value) of an item simply shows how easy or difficult the particular item proved in the test.”36
In the level of difficulty, there are three kinds of difficulties, it from easy, moderate, and difficult item. Actually, a good test item in summative test should have a certain level of difficulty it is not too difficult or too easy. If the test item is
Lyle F. Bachman, Statistical Analyses for Language Assessment, (Cambridge: Cambridge University Press, 2004), p. 123.
36
According to Kathleen M. Bailey, difficulty level is “an index of how easy an individual item was for the people who took it. I.F. is a number, typically printed as a decimal, ranging from 0.0 to 1.0. It represents the proportion of people who got the item right (out of all the people who took the test).”37
The formula of the difficulty level of each item in large group is stated below:38
In which:
FV : The index of difficulty
R : The number of correct answers
N : The number of students taking the test U : Upper half
L : Lower half
n : Number of candidates in one group
To calculate the difficulty level in short-answer item, ZaenalArifin states as follows:39
37
Kathleen M. Bailey, Learning about Language Assessment: Dilemmas, Decisions, and Directions, (New York: Heinle & Heinle Publishers, 1998), p. 132.
38
J.B. Heaton, loc. cit.
39
Zainal Arifin, loc. cit.
FV
=
The next step, the writer tried to find out the difficulty level of all items in the English Summative Test for the first grade of SMPN 3 Tangerang Selatan at odd semester academic year 2013/2014 by using the following formula:40
In which:
P : The difficulty level of all items
B : The total number of difficulty level of each item ∑ : Sigma (Total)
N : The total number of test items
The scale of the difficulty level of all test items ranged from 0.00 to 1.00. It can be interpreted in the rank scale of difficulty level, as follow:
40
Zaenal Arifin, loc. cit., p. 272.
’
Table 2.1
The rank scale of Level of Difficulty41
P Interpretation
< 0.30 Difficult
0.30 – 0.70 Moderate
> 0.70 Easy
From the rank scale above, it shows the easiness and the difficultness of test items. So, the teacher will know the difficulty level of each test item in the summative test.
b. Discriminating Power
According to J. Stanley Ahmann and Marvin D. Glock, “the discriminating power of a test item is its ability to differentiate between pupils who have achieved well (the upper group) and those who have achieved poorly (the lower group).”42
Normally, the upper group will be able to answer the question well rather than the low group. The index of discriminating power is recognized by proportion if the proportion is high it means that the test item is included in good test item because it can discriminate the upper and lower group.
As stated by Kenneth D. Hopkins, items that yield a discrimination index of .30 or more are relatively good in distinguishing between knowledgeable and less knowledgeable examinees.43.
The maximum size of the index is + 1.00 and the minimum size is – 1.00. Any negative value means that the test item discriminates-to some degree-in the wrong direction. Hence, the discriminating power of the test item is unsatisfactory. Positive
41
Suharsimi Arikunto, Dasar-dasar Evaluasi Pendidikan, (Jakarta: PT. Bumi Aksara, 2006), p. 208op. cit., p. 210.
42
J. Stanley Ahmann and Marvin D. Glock, op. cit., p. 187.
43
values show that the test item discriminates in the desired direction, even though it may not be completely satisfactory.44
At one side, a good test item is it can be answered by upper group correctly and incorrectly by lower group.
To analyze the index of discriminating power of an item, Wilmar Tinambunan states the formula as follows:45
In which:
D : The index of item discriminating power
U : The number of pupils in the upper group who answered the item correctly L : The number of pupils in the lower group who answered the item correctly N : Number of pupils in each of the groups
Then, to calculate the discriminating power in short-answer item, it can be used by the formula, as follows:46
44
J. Stanley Ahmann and Marvin D. Glock, op. cit., p. 189.
45
Wilmar Tinambunan, op. cit., p. 139.
46
Zainal Arifin, op. cit., p. 278.
D =
T =
X1 X2√
2 1
In which:
1 : The total quadrate of individual deviation from upper group 2
2 : The total quadrate of individual deviation from lower group n : 27% x N (upper and lower group)
Table 2.2
The classifications of the index of Discriminating Power (D) are:47 Index of Discriminating
Negative value on D Very poor
c. The Effectiveness of Distracter
Based on, Mozaffer and Farhan Jaleel’s article that, “another important technique is analysis of distractors, that provides information regarding the individual distractors and the key of a test item.”48
It means that, the teacher will know the ability of the students from the answer which they have chosen.
47
Anas Sudijono, Pengantar Evaluasi Pendidikan, (Jakarta: PT. Raja Grafindo Persada, 2006) p. 389.
48
Mozaffer Rahim Hingorjo & Farhan Jaleel, Analysis of One-Best MCQs: the
Actually, the test item which can be called good quality that is the distractor will be chosen by students who answer incorrect equally. On the contrary, a poor test item is the distractor will be chosen unequally. Besides that, the test item will be called good quality too if many upper group can answer correctly and only a little lower group can answer correctly.
3. The Difficulty Level and the Discriminating Power
According to J. Stanley Ahmann and Marvin D. Glock, “It has been long known that the discriminating power of an item is influenced by its difficulty.”49
To support, it also tells how difficult or easy the questions were, the difficulty index, and whether the questions were able to discriminate between students who performed well on the test, from those who did not, the discrimination index.50
Logically, after analyzing the level of difficulty the teacher will get information that it can be known the level of difficulties of each item whether the test items are included in easy, moderate, or difficult level. Then, from that level of difficulties the teacher can discriminate the students whether they are included in upper or lower group.
As an example, based on Zainal Arifin’s statement, the upper group will be able to answer the question rather than the lower group.51 So, it can be concluded that the difficult level only can be answered by the upper group of students and it cannot be answered by the lower group of students.
So, after analyzing the test items based on the difficulty level it will be better if the teacher analyzing the discriminating power in the same times and procedures. Because, in analyzing the test items the teacher not only know the level of difficulties of those items but also know how well the test items discriminate the upper and lower group students.
49
J. Stanley Ahmann and Marvin D. Glock, loc. cit. p. 189.
50
Mozaffer Rahim Hingorjo& Farhan Jaleel, loc. cit.
51
According to Heaton, “Facility values and the discrimination indices are usually recorded together in tabular form and calculated by similar procedures.”52
Above all, by analyzing the test items in term difficulty level and discriminating power, the teacher will know the qualities of the test items whether the test items have an easy, moderate, or difficult level and a high or low discriminating power.
To calculate them, it can use the formula as follows:53
In which:
FV : The index of difficulty
R : The number of correct answers
N : The number of students taking the test D : Discriminating index
U : Upper half L : Lower half
n : Number of candidates in one group
52
J.B Heaton, op. cit., p. 182.
53
Ibid.
FV
=
or
FV =
An additional, difficulty has a bearing on the discrimination value of a test – that a test which is too hard or too easy will not discriminate between individuals of different levels of achievement as well as one which is more appropriate for the range of abilities in the group.54
4. The Importance of Item Analysis
Talking about the importance of the item analysis, it is very important for the teacher to get much information from the result of the analyzing the items. By the analyzing, the teacher will know about the achievement of his or her teaching and learning process in the classroom. Not only the process itself but also the teacher knows the students’ performances. It means that, there are so many benefits of the item analysis.
Based on Anthony J. Nitko, there are six points of the importance of item analysis, those are:55
1. “Determining whether an item functions as the teacher intends.” In this book, there are five points about deciding the functioning items which teacher needs to consider: (a) whether it seems to be testing the intended instructional objective, (b) whether it is of the appropriate level of difficulty, (c) whether it is able to distinguish those who have command of the learning objectives from those who do not, (d) whether the keyed answer is correct; and (e) (for response-choice items) whether the distractors are functioning. 2. “Feedback to students about their performance and as a basis for class discussion.” The teacher gets information about the performance of their students. For example in reviewing the test, the teacher will know the students’ errors.
54
Victor H. Noll, Introduction to Educational Measurement, (Boston: Houghton Mifflin Company, 1965), p. 180.
55
3. “Feedback to the teacher about pupil’s difficulties.” In item analysis the teacher will know the students’ problem, it can be seen from the students’ errors of their answering and the teacher is suggested to take the score more than one item such as group and individual test in all the topics which has taught.
4. “Areas for curriculum improvement.” From this statement, item analysis will give information to conduct the revision of the curriculum. 5. “Revising the items.” By doing the item analysis, teacher can revise the item from students’ responses of each item if there are some items which are revised it just need a few time to revise than make the new item and the items can be used for the next testing.
6. “Improving item-writing skills.” The teacher should have item-writing skill in making items in the test and the way to improve the item-writing skill is to analyze the item from students’ responses.
In addition, Widdowson said in his book, Language Testing about the item analysis. It usually provides two kinds of information on items:56
1. Item facility, which helps us decide if the test items are at the right level for the target group, and
2. Item discrimination, which allows us to see if the individual items are providing information on candidates; abilities consistent with that provided by the other items on the test.
56
C. Previous Studies
There are some studies about the difficulty level and the discriminating power which had conducted by several researchers.
The first, Andrian Dwi Prayoga did an analysis about the difficulty level of English summative test for the second grade of Junior High School at odd semester 2010/2011 at SMPN 13 South Tangerang. This study was included in quantitative research because the researcher used some numerical data which were analyzed statistically. Also, this study was categorized as descriptive analysis because it was intended to describe the objective condition about the difficulty level of the English summative test. In this study, the researcher only took 93 students as a sample. The findings of this study were moderate items had the highest percentages with 66,7 % followed by difficult items with 20 % and easy items with 13.3 %. Overall, the difficulty level of the test was moderate level with 0.50 index of difficulty it means that the test had a good difficulty level.57
The second, Hikmah Lestari did an item analysis about the discriminating power of English summative test at the second year of SMPN 87 Pondok Pinang. This study was categorized as a descriptive analysis because it was intended to describe the objective condition about the discriminating power by analyzing the quality of English summative test items in discriminating students’ achievement. This study was considered as a quantitative research because the researcher used some numerical data which was analyzed statistically. The researcher only took 60 students as an ordinal sampling in her study. The findings of this study was the English summative test which was tested at second grade of SMPN 87 Pondok Pinang had
57
good discriminating power because there were 35 items ranging from 0.25 until 0.75 (70%) of the test items had fulfilled the criteria of a positive discriminating power.58
The last, Taufan Maulana Firdaus did an analysis about the difficulty level of English Try out Test of National Examination for Junior High School level at MTs Pembangunan UIN Jakarta. This study was categorized as a descriptive analysis because it was intended to describe the difficulty level of the English National Examination Try out test and it was also considered as quantitative research because the writer used some numerical data which was analyzed statistically. Then, he only took 110 students as a sample. The findings of the study that the test had 24 items (48%) were classified as moderate items, 22 items (44%) were concluded in easy items, and 4 items (8%) including in difficult items. The index of difficulty of the whole test items were 0.696. So, the level of difficulty of English Try out Test of National Examination test for the third grade students of MTs Pembangunan UIN Jakarta was moderate level.59
Above all, the writer compares those studies to her research that there is a similarity to the instrument that is in using the summative test. Almost all of the studies are categorized as a descriptive analysis and considered as quantitative research because of using the numerical data. But it is different from the writer that she only uses the qualitative research in her study by using the descriptive analysis and supported by some numerical data which are analyzed statistically. Then, in taking sample almost all of the previous studies above using 60 till 110 students’ answer sheet as a sample. The findings of both of them, Andrian and Topan’ s study in term the difficulty level have the same result of the writer’s research that is
58
Hikmah Lestari, An Item Analysis on Discriminating Power of English Summative Test (A Case Study of second year of SMPN 87 Pondok Pinang ), Research Paper at State Islamic University Syarif Hidayatullah Jakarta. Jakarta, 2011, p. i, unpublished.
59
moderate level and in the Hikmah’s study that the discriminating power also has the same result to the writer’s research result that is good discriminating power.
D. Thinking framework
Evaluation is important in teaching and learning process to know the teacher’s successfulness, in term his or her teaching, arranging the lesson planning, giving assignment, and controlling the classroom. One of the ways to get the information it can be obtained from the test.
Talking about the test, the teacher often uses the summative test because it is tested in the end of period of a course to know the students’ competence whether they have already mastered all of the materials which have been taught by the teacher or not. To know all above, it can be conducted by analyzing the test items.
In analyzing the test, it can be conducted by the teacher or the test maker after tested the test to the students. By analyzing the test items, it will be better if the teacher analysis the test based on the difficulty level and the discriminating power in the same times and procedures. Because, in analyzing the test items the teacher not only know the level of difficulties in term easy, moderate, and difficult items but also know how well the test items discriminate the upper and lower group students and whether the test items are able to be used for the next exam.
1. Place and Time of the Research
This research was conducted at SMPN 3 Tangerang Selatan which is located on Jl. Ir. H. Juanda Ciputat 15412, Tangerang Selatan. The writer conducted this research by taking the English Summative Test and students’ answer sheets of odd semester 2013/2014 academic year.
2. Method of the Research
This study is categorized as a qualitative research by using the descriptive analysis because the writer described the difficulty level and discriminating power by analyzing the test items of the English summative test of the first grade students of SMPN 3 Tangerang Selatan. This research is also supported by some numerical data which are analyzed statistically by using Anates Program and manual counting.
3. Population and Sample
The writer took the data from the first grade students of SMPN 3 Tangerang Selatan. She only took 92 students’ answer sheets as a purposive sampling.1 Then, the writer divided the students into three groups: upper, middle, and lower by arranging the students’ score from the highest till the lowest scores. But the writer only took 27% of the upper and lower group of students to be analyzed because the writer used largegroups (more than 40 students).2
1
Suharsimi Arikunto, Manajemen Penelitian, (Jakarta: Rineka Cipta, 2007), p. 97.
2
4. Technique of Data Collecting
The writer used the documentary study in this research by collecting the English summative test paper of first grade students of SMPN 3 Tangerang Selatan 2013/2014 academic year which had provided the answer key and students’ answer sheets to be analyzed.
5. The Instrument of the Study a. English summative test paper
The writer used the English summative test paper which had been tested on Wednesday, December 11th, 2013 at odd semester 2013/2014 academic year.
b. Students’ answer sheets
Students’ answer sheets which is used by the students to answer the questions.
6. Technique of Data Analysis
This research, the writer used qualitative and quantitative analysis technique. In qualitative, the writer analyzed the difficulty level and discriminating power of each item whereas in quantitative, she calculated the data by using some formulas. Also, the analyzing had been conducted in the same times and procedures.3
The step had been conducted with:4
a) Taking the English summative test paper and students’ answer sheets while she was following the practical teaching or PPKT
b) Arranged or ranked the score from the highest till the lowest
c) Divided the students into three groups: Upper, middle, and lower groups by arranging the students’ score from the highest till the lowest scores.
3
J.B. Heaton, Writing English Language Tests, (New York: Longman Inc., 1988), p. 182.
4
d) Decided the total of students become: 27% Upper and lower group students because the writer used large groups (more than 40 students).5 The writer took 27% from the upper group (25 students’ answer sheet) and 27% from the lower group (25 students’ answer sheet) from 92 students’ answer sheet so the upper group is taken from number 1 till 25 and lower group from number 68 till 92.
e) Analyzing the test based on the difficulty level and discriminating power f) Categorized and concluded the result of that analysis of the English
Summative Test for first grade students at SMPN 3 Tangerang Selatan 2013/2014 academic year
To calculate the level of difficulty and the discriminating power, it can use the formula as follows:6
5
Loc. cit. 6
J.B. Heaton , loc. cit.
FV
=
or
FV =
Level of difficulty for each item must be interpreted in the rank scale of the difficulty level, as follow:
Table 3.1
The rank scale of Level of Difficulty7
P Interpretation
< 0.30 Difficult
0.30 –0.70 Moderate
> 0.70 Easy
Table 3.2
The classifications of the index of Discriminating Power (D) are:8 Index of Discriminating
Power
Classifications
0.70 – 1.00 Excellent
0.40 – 0.70 Good
0.20 – 0.40 Satisfactory
≤ 0.20 Poor
Negative value on D Very poor
7
Suharsimi Arikunto, Dasar-dasar Evaluasi Pendidikan, (Jakarta: PT. Bumi Aksara, 2006), p. 210.
8
A. Data Description
In this research, the writer used the English summative test of the first grade students of SMPN 3 Tangerang Selatan at odd semester 2013/2014 academic year and the students’ answer sheets as the data sources. The total numbers of the test items are 45 items those are 40 multiple-choice items and 5 short items.
The writer used 92 students’ answer sheet then she divided into three groups which are upper, middle, and lower groups but the writer only used the upper and the lower group to analyze the data. To get these groups the writer divided 27 % of 92 students’ answer sheets. They are 27% (25 students) from upper group and 27% (25 students) from lower group.
Based on the data in the appendix of the table 4.1, the score in the upper group is begun from the highest score that is 95.5 till the lowest score of this group that is 85.5 which have the total of 25 students of this group from number 1 to 25 then the lower group is begun from the highest score of this group that is 58.5 till the lowest score that is 31.5 which have the total of 25 students this group from number 68 to 92 and the rest group is the middle group is begun from the highest score of this group that is 85.5 till the lowest score of this group that is 60 which have the total of 42 students of this group from number 26 to 67.
B. Data Analysis
First of all, the writer calculated the difficulty level and the discriminating power by using the Anates Program then she matched the result by using manual counting based on the formula. The way of calculating the difficulty level is the amount of the correct answers of the upper group are added by the correct answers of the lower group then divided with the amount of the students. Then, from the result can be interpreted based on the classification whether the range less than 0.30 it is concluded in easy level, the range 0.30 to 0.70 as moderate level, or the range more than 0.70 it is included in difficult level.
The way of calculating the discriminating power is the amount of the correct answers of the upper group are lessened with the correct answers of the lower group then divided with a half of the amount of the students. Then, from the result can be interpreted based on the classification whether it is included in excellent with the range 0.70 to 1.00, good with the range 0.40 to 0.70, satisfactory with the range 0.20 to 0.40, poor with the range less than 0.20, or very poor (negative value on discriminating power).
The results of the each item, the writer combined two classification of each item whether the item is included in easy level and poor quality or difficult level and good quality. To make clear, the writer made the description of the analysis of each item based on the appendix of the table 4.2 which is conducted by the writer that she found ten classifications of the test items:
1. There are 6 items which are included in easy level and satisfactory quality,
those are items number 1, 9, 11, 25, 39, and 43.
There are the differences of index of each item in this level but still included in the same range that is more than 0.70 as easy level in term difficulty level and the range 0.20 to 0.40 as satisfactory quality in term discriminating power.
There are the differences of index of each item in this level but still included in the same range that is more than 0.70 as easy level in term difficulty level and the range less than 0.20 as poor quality in term discriminating power. 3. There are 8 items which are included in easy level and good quality, those are
items number 3, 14, 20, 22, 23, 27, 29, and 36.
There are the differences of index of each item in this level but still included in the same range that is more than 0.70 as easy level in term difficulty level and the range 0.40 to 0.70 as good quality in term discriminating power. 4. There are 5 items which are included in moderate level and poor quality,
those are items number 4, 12, 16, 33, and 35.
There are the differences of index of each item in this level but still included in the same range that is 0.30 to 0.70 as moderate level in term difficulty level and the range less than 0.20 as poor quality in term discriminating power. 5. There are 4 items which are included in moderate level and excellent quality,
those are items number 13, 41, 42, and 45.
There are the differences of index of each item in this level but still included in the same range that is 0.30 to 0.70 as moderate level in term difficulty level and the range 0.70 to 1.00 as excellent quality in term discriminating power. 6. There are 3 items which are included in difficult level and poor quality, those
are items number 18, 30, and 32.
There are the differences of index of each item in this level but still included in the same range that is less than 0.30 as difficult level in term difficulty level and the range less than 0.20 as poor quality in term discriminating power . 7. There are 6 items which are included in moderate level and good quality,
those items are 19, 21, 24, 26, 28, and 40.
8. There are 1 item which is included in difficult level and good quality, that is item number 31.
There are the differences of index of each item in this level but still included in the same range that is less than 0.30 as difficult level and the range 0.40 to 0.70 as good quality in term discriminating power.
9. There are 1 item which is included in difficult level and very poor quality, that is item number 34.
There are the differences of index of each item in this level but still included in the same range that is less than 0.30 as difficult level and the range is negative value, so this item should be distracted in term discriminating power. 10.There are 2 items which are included in difficult level and satisfactory quality,
those are item numbers 37 and 38.
There are the differences of index of each item in this level but still included in the same range that is less than 0.30 as difficult level in term difficulty level and the range 0.20 to 0.40 as satisfactory quality in term discriminating power.
It can be concluded from the description of analyzing above that although in the same level of difficulties and the qualities of discriminating power but the range of each item is variation because each item has the different amount of the correct answer of the upper and lower groups. Besides that, it depends on the difficulties of each item which can be answered by the student correctly.
Number 8 has 1.00 index of difficulty so it belongs to easy level also this item has 0.00 index of discriminating power it can be called as poor qualification. This item is included in looking for the detail information of the text, so all of the groups are able to answer the question.
Number 13 has 0.54 index of difficulty so it is included in moderate level and this item has 0.76 index of discriminating power it means that the item has excellent
qualification. The types of this item is completion, so the students feel quite difficult although the student only looking for the appropriate word of the sentence but the distractors are good because all of the options are verb.
I have a friend. Her name is Sarah. She is a new student in my class. She lives at number 27 Hang Tuah raya street, Plumbungan Indah, Sragen, Central Java. She is slim and tall. She has short and straight hair. She is twelve years old. Her nose is pointed. Her eyes are brown. She is charming and smart. Everybody likes her.
8. Where does she live?
a. Bandung c. Medan
b. Solo d. Sragen
It is Thursday. Fazar is in the school ____11____. He wants to find a book. The book is about flora and fauna. First, he goes to the catalogue to find the title and the name of ____12____. Then, he ____13____ one piece of paper. He meets the librarian and tells that he wants to _____14_____ book.
13. a. writes c. speaks
Number 31 has 0.6 index of difficulty so it can be called as difficult item also this item has 0.52 index of discriminating power it is included in good qualification items. To answer this item the students have to answer based on their knowledge so only the students who have high competence who are able to answer the question.
Number 34 has 0.12 index of difficulty so it can be called as difficult item also this item has -0.16 index of discriminating power it is included in very poor
item. Because the students have to conclude the purpose of the text in the letter and to answer this item the students have to know the clue of the text.
31. When we celebrate Heroes’ day? a. October 1st
b. August 17th c. April 21st
d. November 10th
Hello Felix, How’s going?
We’re waiting for your coming … Please, go home soon.
Miss you so much,
Hana, your beloved’s sister. 34. What is the purpose of the text?
a. to greet Felix b. to warn Felix
c. to give an advice
