THESIS
SITI FATIMAH NIM.TE 141045
ENGLISH EDUCATION PROGRAM
FACULTY OF EDUCATION AND TEACHER TRAINING THE STATE UNIVERSITY FOR ISLAMIC STUDIES
SULTHAN THAHA SAIFUDDIN JAMBI 2018
i THESIS
Submitted as Partial Fulfillment of Requirements to Obtain Undergraduate (S1) Degree at English Education Program Faculty of Tarbiyah and Teacher Training
SITI FATIMAH NIM.TE 141045
ENGLISH EDUCATION PROGRAM
FACULTY OF EDUCATION AND TEACHER TRAINING THE STATE UNIVERSITY FOR ISLAMIC STUDIES
SULTHAN THAHA SAIFUDDINJAMBI 2018
vi
(S1) In English Department of Tarbiyah faculty of State Islamic University Sulthan Thaha Saifuddin Jambi by presenting a thesis entitled “An Analysis of English Final Examination Instrument Designed by Teachet at the First Grade Students Islamic Senior High School Mahdaliyah Jambi”.
The writer would not have been completed without supports, guidance, advice and special recognition for their invaluable help in accomplishing this thesis. Therefore, the writer would like to express deep appreciation to:
1. Dr. H. Hadri Hasan, MA as a rector of the state Islamic University Sulthan Thaha Saifuddin Jambi.
2. The vice of rector I, II, III of the state Islamic University Sulthan Thaha Saifuddin Jambi.
3. Dr. Hj. Armida, M.Pd as the dean of tarbiyah faculty and teacher training the state Islamic University Sulthan Thaha Saifuddin Jambi.
4. Dr. Lukman Hakim, M.Pd, Dr.Zawaki M.Pd, Dr.Kemas Imran Rasadi M.Pd as the vice dean I, II, and III tarbiyah faculty and teacher training the state Islamic Sulthan Thaha Saifuddin Jambi
5. Amalia Nurhasanah, S.Pd, M.Hum as the chief of English study program in tarbiyah faculty of state Islamic Sulthan Thaha Saifuddin Jambi
6. Dr. Faurina Anastasya, M.Hum as the first advisor who has given a lot of input such as as correction, idea, and contribution of thought and beneficial idea toward the progress of this thesis
7. Dr. Muhammad Taridi, M.Pd as second advisor who has given a lot of input such as correction, idea, and, contribution of thought and beneficial idea toward the progress of this thesis
8. All of lecturers in tarbiyah faculty of state Islamic University Sulthan Thaha Saifuddin Jambi who has given knowledge to the writer
iv
for the blessing and leading me.
I could finally finish writing this thesis.
Sholawat to the Prophet Muhammad SAW.
His coming really changed the world Special thanks to:
My beloved parents, Mustamin and Nurma who love and encourage me and never stop praying for my success now and in the future with endless love,always
being with and guiding me, your prayers are really powerful.
My beloved Brother ( Septamin, Muhammad Zakaria and Muhammad Rahmad Ramadhan) and All of my family, thanks for your support, love, and never stop giving spirits to finish this thesis, make me strong and wish your hope will come
true.
My best friends( Ratumas jumratul aina, Pranita Rahmadhani, Rizky Widia Wulandari, Robiatul Adawiyah Harahap) I thank Their support, motivation, and
quality time to share happiness with me. For all students of English education program batch 2014 especially class B, and also all of my Kukerta’s friends, and my PPL’s friends who can not say your name one by one, thank you for being my
best friends in giving support every day, thanks for help, suggestions and spirit in arranging this thesis, without all of you I don’t mean.
May Allah SWT always bless us.
Aamiin .
viii
Title : An Analysis of English Final Examination Instrument Designed by Teacher at the First Grade Students Islamic Senior High School Mahdaliyah Jambi
The aim of this study was to analyze the face validity and the content validity of the English Final Examination Instrument Design by Teacher at the First Grade students Islamic Senior High School Mahdaliyah Jambi. This research is qualitative research. The data of the Research are the analysis of face validity and content validity of the test. The data sources of this research are taken from the test sheet used test instrument of Islamic Senior High School Mahdaliyah Jambi.
From the finding of this research, the face validity of the test is unfair. It is because the test only fulfills 6 criteria from 15 criteria of face validity. It means that in term of face validity the test only cover 40% of the criteria of face validit.This percentage obviously falls into the level 40-50% which means less good. Then, in term of content validity of test sheet from 30 items that have been tested, 20 items among have been declared as valid items whereas 10 items are an invalid items. So, the English final examination intrument designed by teacher at the first grade students islamic senior high school mahdaliyah jambi is considered as invalid test because having the unfair face validity and the bad content validity.
Keywords: face validity and Content validity
ix
Judul : Analisis soal ujian akhir semester bahasa inggris buatan guru untuk siswa kelas sepuluh di Madrasah Aliyah Mahdaliyah Jambi
Tujuan dari penelitian ini adalah untuk menganalisa face validity dan validitas isi dari soal bahasa inggris yang di ujikan di kelas X MA Mahdaliyah Jambi. Penelitian ini merupakan penelitian kualitatif. Data penelitian ini adalah analisis dari face validity dan validitas isi dari soal. Sumber data dari penelitian ini diambil dari lembar soal yang di ujikan di kelas X MA Mahdaliyah jambi.
Dari penemuan penelitian ini, validitas perwajahan dari test tersebut adalah kurang baik. Ini karena test tersebut hanya memenuhi 6 kriteria dari face validity.
Ini berarti dalam hal validitas perwajahan test hanya meliputi 40% dari kriteria face validity. Persentase ini termasuk kedalam level 40-55% yang berarti kurang baik. Kemudian, dalam hal validitas isi soal tes dari 30 yang telah di uji, 20 di antaranya telah di nyatakan sebagai item yang valid sedangkan 10 soal lainnya di nyatakan tidak valid. Jadi test ujian akhir bahasa inggris buatan guru untuk siswa kelas sepuluh di madrasah aliyah mahdaliyah jambi di anggap sebagai invalid tes karena face validity nya kurang baik dan validitas isinya buruk.
Kata kunci : face validity dan validitas isi
x
OFFICIAL NOTE. ... ii
STATEMENT OF ORIGINALITY. ... iii
DEDICATION... iv
MOTTO. ... v
ACKNOWLEDGMENT. ... vi
ABSTRACT. ... vii
ABSTRAK. ... viii
TABLE OF CONTENT. ... ix
CHAPTER 1 : INTRODUCTION. ... 1
A. The Background of theStudy. ... 1
B. Focus of the the study. ... 3
C. The formulation of the Study. ... 4
D. The Objective ofthe Research. ... 4
E. The Significances of the Research. ... 4
CHAPTER II : REVIEW OF RELATED LITERATURE... 5
A. Definition of test. ... 5
B. Kinds of Test. ... 6
C. Types of Test Item. ... 7
D. The Principle of Test Item. ... 10
E. Face Validity. ... 10
F. Content Validity. ... 12
G. Characteristics of a Good Test. ... 13
H. Technique Analyzing Learning outcomes Test Items... .... 18
I. Previous Study. ... 19
CHAPTER III : RESEARCH METHOD... 21
A. Method and Design of the Research. ... 21
B. Setting and Subject of the Research ... 21
C. Sources of Data. ... 21
D. Technique of Collecting the Data. ... 21
E. Technique of Analyzing the Data. ... 23
F. Checking Finding Validity. ... 27
G. Schedule of the Research. ... 28
xi
CHAPTER V : CONCLUSIONS AND SUGGESTION. ... 53
A. Conclusions. ... 53
B. Suggestions. ... 53
REFERENCE. ... 53 APPENDIX. .... ...
1
A. Background of the Study
English as an international language already learned in most of countries in the world include indonesia. Nowdays, English has become a very important subject in most of schools in indonesia especially at high school. Moreover, it already becomes one of subject which is tested in national examination. It means that english has a very important position in indonesian education. All of students should be able to use english whether it is in written or spoken. So that‟s why english should be taught as good as possible by the teacher. In the process of learning, to know the students understanding about the material, the teacher should hold an evaluation. Evaluation is important part of a teaching – learning process which allows teacher to evaluation their students during and at the end of an educational course. Evaluation of a teaching learning process can be administrated in the form of tests. Because of it. The test should really measure the students‟ achievement.
Test is an instrument to measure students in certain skill or ability.
Through test, teacher can identify how far students understand the materials and how far students are able to achieve certain skills, it means that a test is a part of assessment, one of them is multiple choice. Test also has a role to measure the students‟ ability in a subject because the result of the test can describe the students‟ ability in the subject itself. Brown (2004) states “test is a method of measuring a person‟s ability, knowledge, or performance in a given domain”
(2004:3).
At a school there are several tests given to the student in one semester, one of them is final examination. Final examination is a test given to students at the end of a course of study or the end of semester, final examination is known as summative test. According to Brown (2004:6) summative test is aims to measure,or summarize, what a student has grasped, and typically occurs at the end
of a course or unit of instruction (2004:6).At a school, test is used as an instrument to measure students‟ understanding about their lesson. Students will get the result of what they have done the test.
In English final examination, most of teachers designed the test instrument by themselves, start from the questions until the scoring system of test, the question are designed based on the materials that have been taught. But not all teachers design the test instrument perfectly. At least there are several weakness of the test instruments that designed by the teacher. Sometime, the validity of the test is questionable because most of teacher design the test instrument without following the rules of making a test instrument.
Test that is made by teacher called teacher made test. It is still to be questioned whether the test is valid or not because the teacher rarely trying out and revising the test first before giving it to the students. Brown (2004:22-27) mentions five types of test validity. They are: content-related evidence, criterion-related evidence, construct – related evidence, consequential validity, face validity.
Between the five test validity, the most basic and the most important are face validity and content-related evidence or content validity. Face validity is the way the test looks to the testiest, teachers, moderators and administrator. Instead of content validity is concern with the material that the students have learned.
Sometime, in arranging a test for students, the teacher ignore these test validity.
Whereas, the way the test instrument looks like can make the students more comfort in answering the question; and if the question that the teacher gave to the students are appropriate with the standard competences, of course the test that the teacher hold will be effective.
When the researcher did practice teaching at First Grade Mahdaliyah Jambi at the time, the researcher found several weaknesses of the english final examination test. Firstly, there is no score written in the test instrument for each question; secondly there is no evaluation procedure written in the test; thirdly, the teacher did not give picture in the test and then most of them got low score it is about 54% from 120 students got low score. Based on that the researcher did
interview with some students.They said that the test instrument was too difficult for them and caused they cannot do the test well.
The problems that mentioned above are some problems related to the test instrument that designed by English teacher in the school. This study conducted to analyze the test-instrument after being for evaluation, to know whether the test- instrument was good for assessing the student‟s mastery. Actually, the quality of test instrument has a contribution to the score of test. In order to make the students get a good score in English final examination test, the teacher should design a final examination test instrument correctly.
In this study, the researcher analyzed test instrument to know whether the test is matched with characteristics of a good test and suitable with the curriculum and syllabus or not. If the test not a good instrument, it will not measure the students achievement well.
Based on the elaboration above, the researcher would like to plan qualitative research entitled: “AN ANALYSIS OF ENGLISH FINAL EXAMINATION INSTRUMENT DESIGNED BY TEACHER AT THE FIRST GRADE STUDENTS OF ISLAMIC SENIOR HIGH SCHOOL MAHDALIYAH JAMBI
‟‟.
B. Focus of the Study
In this study, the researcher will focused on analyze the validity ( face validity and content validity) of the English final examination used at the First Grade Students of Islamic Senior High School Mahdaliyah Jambi Academic year 2017/2018.
C. Formulation of the Problem
The researcher has two major questions as follows:
1. How is the face validity of English final examination instrument designed by teacher at the First Gade Students of Islamic Senior High School Mahdaliyah Jambi?
2. How is the content validity of English final examination instrument designed by teacher at the First Gade Students of Islamic Senior High School Mahdaliyah Jambi?
D. Purpose of the Study
The purpose of this research is to know these following matters:
1. To know the face validity of English final examination instrument designed by teacher at the First Gade Students of Islamic Senior High School Mahdaliyah Jambi?
2. To know the content validity of English final examination instrument designed by teacher at the First Gade Students of Islamic Senior High School Mahdaliyah Jambi?
E. Significances of the Research
The results of this research is expected to be able to give the following benefits:
a.
This research can provide feedback to teachers in order to improve the quality of test instrument.b. Hopefully this research can used as a reference when will design a good test instrument.
c. This researcher can provide input for other researchers to develop other similar studies.
d. Hopefully this research can used as reference to analyze the test instrument more better.
5
A. Definition of test
Test is one of assessment that can be applied to see students‟ mastery in what their have learned and also the teachers can be use test to measure students‟
knowledge and achievement in teaching learning process. Before doing a test the students should mastery their lesson first. So,when they will have a test, they are already and just keeping in mind about the lesson that has mastered before.
According to Brown (2004:3), a test is a method measuring a persons’
ability, knowledge, or performance in a given domain (p.3). A test is first method.
It is an instrument, a set of techniques, procedures, or items that requires performance on the part of test taker. To qualify as a test, the method must be explicit and structured; multiple-choice with drescribed correct answer; a writing prompt with a scoring rubric; an oral interview based on a questions script and checklist of expected responses to be filled in by the administrator.
Furthermore, test is given to measure students‟ understanding about the material that has been taught before. According to Hughes (2003:13), test is intended to measure students‟ achievement and the degree of success of teaching and learning process. It means that through testing, the teacher can measure students‟ knowledge or ability and also by giving a test the student can see their progress in learning process whether they understand or not about the lesson and the teacher can make adjustment to their instruction in teaching process to make students understand.
B. Kind of test
Richard (2009:299) says that different type of test can be used to measure changes in learning at the end (or at intermediate stages) of a course. Sanjaya ( 2013:239- 240) divides test into several groups. Based on the number of test, test can be divided into group test and individuals test. Grouping test is a test which done together; whereas individual test is a test given to student individually.
Seen from the test arrangement, test can be divided into teacher made tests and standardized test. Teacher-made test is arranged to produce the information required by teacher itself. For example, to collect the information about the level of material comprehension that already done. Standardized test is a test used can predict the students‟ ability so that based on that ability, standardized test usually used for selection. For example, the selection for new students, selection for employees, etc. As a test that has a function to measure ability, a standardized test should have degree of validity and reliability by series of trials, or degree of difficulty, and have a high index of differentiated.
Based on the implementation of the test, test can be divided into written test, oral test, and behavior test. Written test is a test that done by the student answer the number of questions in writing. There are two kinds of test that included into written test. They are essay test and objective test.
Furthermore, Harmer(2001:321) mention four types of test, they are:
a. Placement test
Placing new students in the right class in a school is facilitated with the use of placement test. Usually based on the syllabuses and materials students will follow the use once their level has been decided on, these test grammar and vocabulary knowledge and assess students‟ productive and receptive skills. Some school ask students to assess themselves as part of the placement process, adding this self-analysis into final placing decision.
b. Diagnostic test
While placement tests are designed to show how good a student‟s English in relation to previously agreed system of level, diagnostic tests can be used to expose learner difficulties, gaps in their knowledge, and skill
deficiencies during courses. Thus, when we know what the problem are, we can do something about them. Brown (2004) says that diagnostic test is designed to diagnose the phonological features of English that are difficult for learners and should therefore become part of time a curriculum (p.46).
c. Progress or achievement tests
These tests are designed to measure learner‟s language and skill progress and relation to the syllabus the have been following.Achievement tests at the end of a term (like progress test the end of a unit, a fortnight,etc) should reflect progress, not failure. They should reinforce the learning that has taken place, not go out of their way to expose weakness. They can also help us to decide on changes to future teaching programs where students do significantly worse in (parts of) the test than we might have expected.
d. Proficiency tests
Proficiency test give a general picture of a student‟s knowledge and ability (rather than measure progress). They are frequently used as stages people have to reach if they want to be admitted to foreign university, get a job, of obtain some kind of certificate. Proficiency test have a profound backwash effect since, where they are external exams, students obviously want to pass them, and teachers‟ reputation sometimes depend (probably unfairly) upon how many of them succeed.
C. Types of Test item
An item is the basic unit of language testing. James Dean Brown( 1996) stated that item is the smallest unit that produces distictive and meaningful information on a test or rating scale. The test used in classroom tests are commonly divided into two broad categories: a) The objective item and b) Essay test.
a. Objective test
Objective test items can be used to measure a variety of knowledge outcomes.
The most generally useful is yhe multiple choice item, but other item types also have a place. Following simple but important rules for construction can
improve the quality of objective test items. In constructing an achievement test, the test maker may choose from a variety of item types. One of them is referred to as objective item. This kind of items test can be scored objectivelty. Furthermore, equally competent scorers can score them independently and obtain the results.
The objective items can be classified in two types, which are selection- type test item and supply-type test item. There are many kinds of selection- type test items. They are multiple choice items, true-false items and matching items. Then, the researcher just focuses on the multiple choice items.
1. Multiple Choice items
Multiple choice items are made up of an items, which present a problem situation, and several alternatives, which provide possible solution to the problem. The options usually of, a, b, c, d and e. That will be counted correct, and the distractors, which are those choices that will be counted as incorrect.The multiple choice items plays such an important role in the objective testing of knowledge outcomes that it wil be treated first and in considerable detail. Designing multiple choice items test, the test maker should be consider in some ways. There are 18 basic rules in designing multiple choice item test. They are.
1. Design each item to measure an important learning outcome.
2. Present a single clearly formulated problem in the stem of the item.
3. Put the alternatives at the end of the question, not in the middle.
4. Put as much of the wording as possible in the stem.
5. Eliminate unnecessary wordiness
6. Avoid negatively worded stems. “Which of the following is not...”
7. Avoid requiring personal opinion. Other item types are more suitable for this,
8. Avoid textbook wording
9. Do not have linked or clued items.
10. All options should be homogenous 11. All options should be plausible.
12. Put repeated words in the stem, not in the opinions 13. Punctuation should be consistent
14. Make all options gramatically consistent with the stem of the item.
15. List of option pravtically 16. Other option logically
17. Use the option”all the above “sparingly.
18. Use the option”none of the above”sparingly.
b. Essay Test
Essay question give student the greatest oppurtunity to supply and construct their own response, making them the most useful for assessing higher-level thingking processess such as analyzing, synthesizingand evaluating. The essay question is also primary means by which teachers assess students‟ ability to organize, express and defend ideas. The main limitations of essay are that theyare time-consuming to answer and score, and they place a premium on writing ability. Essay questions require the students to create an explicit answer that the scorer can rate without describing the basis of any rating scale or without showing his or her own version of an ideal answer. On the other hand, essay tests are the best measure of students‟
skills in higher-order thingking and written expression.
There are some basic rules in designing essay item test. They are:
1. Define the behaviour the student is expected to exhibit before writing the prompt
2. Ask questions that are relatively spesific and focused and which will elicit relatively brief responses.
3. If you are using many essay questions in a test, ensure reasonable coverage of the course objectives.
4. Formulate question that present a clear task to be performed.
5. Provide ample time for answering and suggest a time limit for each question.
D. The Principle of Test Design
Anas Sudijono(2008) state that there are six principles in designing the test, they are :
1. The test should measure objectively instructional purposes which have been stated before.
2. The test item should be sample which represent of teaching material.
population in one unit or course in teaching learning process.
3. The test should be various.
4. The test should be design to measure an important learning outcomes.
5. The test should have high reliability.
6. The test can be used as gaining useful informstion in revising the teaching learning method.
E. Face Validity
Face validity means the way the test looks to the testiest, teacher, moderator, and administrators. Therefore it is useful to show a test to colleagues or friends order to discover absurdities and ambiguities of a test. Grounlund in Brown (2004:26) states “an importante faced on consequential validity is the extent to which “student view the assessment as fair, relevant, and useful for improving learning” or what is popularly known face validity.
Face validity refers to the degree to which a test look right, and appears to measure the knowledge or abilities it claims to measure, based on subjetive jugment of the examinees who take it, the administrative personnel who decide on its, and other psychometrically unsophisticated observers. (Mousavi in Brown,2004:266). Sometimes students dont‟t know what is being tested when they tackle a test. They may feel,for a variety of reasons,that a test isn‟t testing what it is. “supposed” to test. Face validity to means that the students perceive the test to be valid. Face validity asks the questions “does the test, on the “face” of it, appear from the learner‟s perspective to test what it is designed to test?”
Face validity will likely be high if learners enconter:
a. A well-constructed, expected format will familiar tasks.
The criteria of well-constructed in face validity:
1. Grammar 2. Capitalization 3. Speeling 4. Font 5. Space
6. Test regulation 7. School identify 8. Time allocation
9. The score for each question 10. Printing quality
b. A test that is clearly doabe within the alloated time limit c. Items that are clear and uncomplicated test item
d. Directions that are crystal clear direction
e. Task that relate to their work course work ( content validity) f. A difficulty level that present a reasonable challenge
The analysis, the criteria adopted from Arkunto‟s opinion:
Table 2.1
Criteria of result percentage
Achievement Criteria
76%-100% Good
56%-75% Sufficient
40%-55% Less Good
>40% Bad
F. Content validity
Further and Didson (2007:6) said that content validity is a defined as any attempt to show that the content of the test is a representative samples from the domain that is to be tested.Content validity testing also can do by asking the expert judgment. The people who have competences in an area can be asked their opinion to judge the accuracy of the content of the test items. And the last, the content validity testing can be done by asking the inter rater judgment. It can be done by giving respond on the suitability of test item with the lattice planning.
Then, theresponds will be content in certain way.Content validity is concerned with the material that the student have learned. The test should cover samples of the teaching materials given. To fulfill this, the teacher should refer his consideration to the teaching.Content validity is a test that questioned whether the content of the test items which is related reflects the content of the curriculum should be measured or not.
In content validity will be easier to see similiarity between aim and material to measurement. A test has content validity when indicator aim with subjectthat given measurable. To measure questions validity, in this research used the formula of point biserial correlation.
:
Korelation point biserial:
Mean score items : Mean score total : Standard deviationP : Proportion of cases answering the item right q : Proportion of cases answering the item wrong
G. Characteristics of A Good Test
There are some characteristics of a good test. Brown (2004:19) state that to know the test effectivity, there are five cardinal criteria for “testing a test”:
practicality, reliability, validity, aunthenticity, and washback. Brown (2004:27) said that validity is complex concept, yet it is indispensable to the teacher understanding of what make a good test. If in your language teaching you can attend to the practicality, reliability and validity of tests language, whether those tests are classroom test related to a part of a lesson, final exams, or proficiency tests, then you are well on the way to making accurate judgements about the competence of the learners with whom you are working.
1. Practicality
An effective test is practical. This means that it a. Is not excessively expensive,
b. Stays within appropriate time constraints, c. Is relatively easy to administer, and
d. Has a scoring/evaluation procedure that is specific and time-efficient.
A test that is prohibitively expensive is impratical. A test of language proficiency that take a student five hours to complete is impratical – it consumes more time (and money) than necessary to accomplish its objective. A test that requires individual one-on-one proctoring is impratical for a group of several hundred test- taker and only a handful of examiners. A test takes a few minutes for a student to take and several hours for and examiner to evaluate is impractical for most classroom situations.A test that can be scored only by computer is impractical if the test take place a thousand miles a way from the nearest computer. The value and quality of a test sometimes hinge on such nitty-gritty, practical consideration.
2. Reliability
Brown (2004:20-22) A reliable test is consistent and dependable. If you give the same test to the same student or matched students on two different occasions, the test should yield similar results. The issue of relaibility of a test my best be addressed by by considering a number of factors that may constribute to
the unreliability of a test. Consider following possibilities: fluctuations in the student, in scoring, in test administration, in test administration, and in the test itself.
a. Student-Related reliability
The most common learner-related issue in reliability is caused by temporary illness fatigue, a “bad day”, and other physical or physichological factors, which may make an “observed” score deviate from one‟s “true”
score. Also include inthis category are such a test-takers “test wiseness”, or strategies for efficient test taking (Mosavi in Brown:2004-21)
b. Rater (scorer)Reliability
Human error, subjectivity, and bias may enter into the scoring process. Inter-rater reliability occurs when two or more scorers yield inconsistent scores of the same test, possibly for lack of attention to scoring criteria, inexperience, inattention, or even preconceived biases. Intra reliability is a common occurrence for classroom teachers because of unclear scoring criteria, fatigue, bias toward particular “good” and “bad” students, or simple carelessness.
c. Test administration reliability
Unreliability may also result from the conditions in which the test is administreted. Once witnessed the administration of a test of aural comprehension, but because of street noise outside the building, student sitting next to windows could not hear the tape accurately.
d. Test reliability
Sometimes the nature of the test nature of the test itself can cause measurement errors. If a test is too long, test takers may become fatigued by the time the reach the later items and hastily respond incorrectly.
Test and test administration reliability can be achieved by making sure that all students received the same quality of input. Part of achieving test reliability depends on the physical context-making sure, for example, that every students has a cleanly photocopied test sheet,sound amplification is clearly audible to everyone in the room,video imput is equally visible to
all,lightning,temperature,and other classroom condition are equal ( and optimal) for all students.
3. Validity
The first and foremost question to be asked with respect in any testing procedure is: how valid is it? When this question is asked, it inquires whether the test measure what the people want to measure, all of what to people to measure, and nothing but what the people want to measure. The validity of the score is not self-evident but is something that must be established on the basis of adequate evidence. A test may be thought of as corresponding to some aspect of human behavior in any one of the three senses. The terms that have been adopted to designate these senses are content validity, criterion validity, and construct validity.
a) Content Validity
If the measurement of English competency will be made, the first thing to do is reading some agreement as to the skill and knowledge that comprise correct and effective use of English, and that have been the objectives of language instructions. The next step is examining the test to see what skills, knowledge, and understanding it calls for. The final step is matching the analysis of test content against the analysis of course content and instructional objectives and seeing how well the former represents the latter. It should be clear that relationship between teaching and testing is typically intimate. Test is drawn from what has been taught, or what is the purpose to be taught. The instructional program is the original source of test materials.
b) Criterion-Related Validity
Frequently, a test is used in correlation with decision that implies predicting some specific future outcome.Evaluation of a test as predicting is primarily an empirical and statistical evaluation, and this aspect of validity has some time been spoken as empirical orstatistical validity. The basic procedure is to give the test to a group who are entering some job or training
program, to follow them up later to get for each one specified criterion measure of success on the job or training program, and then to complete the correlation between test scores and the criterion measure of success. The higher the correlation, the more effective the test as predictor. If a test is analyzed using criterion-related validity, and the result is high, it means the test is effective as a predictor.
c) Construct Validity
Some questions that are asked in psychology tests are neither “how well does this test predict job success?” nor “How well does this test represent our curriculum?”, but rather “What do score on this test mean or sight?” “What does the score tell about an individual?” “Does it correspond to some meaningful trait or construct that will help in understanding him?”
For this question of whether the test tells something meaningful about people, the term construct validity has been used.
Some educational tests are intended to measure general traits or qualities of an individual. Verbal reasoning, special visualizing, sociability, introversion, mechanical interests are all designation of traits and constructs.
Tests of these functions are valid in so far as they behave in the way that such a trait should reasonably be expected to behave.
d) Face Validity
Grownland in Brown (2004) said an important face of consequencial validity is the extent to which ”students view the assessment as fair, relevant and useful for improving learning” (p.26). Face validity refers to the degree to which a test looks right, and appears to measure the knowledge or abilities it claims measure, based on the subjective judgement of the examinees who take it, the administrative personnel who decide on its use, and other psychometrically unshopisticated observers” (Mousavi in Brown:
2004, p.26). Face validity means that the students perceive the test to be valid. Face validity will likely be high if learners encounter:
A well-construct, expected format with familiar task, A test that is clearly doable whitin the allotted time limit, Items that are clear and uncomplicated,
Direction are crystal clear,
Task that relate to their course work (content validity), and A difficulty level that presents a reasonable challenge 4. Authenticity
Bachman and Palmer said in Brown (2004) define authenticity as “the degree of correspondence of the characteristic of a given language test task to the features of a target language task”, and then suggest an agenda for identifying those target language tasks and for transforming them into valid test items (p.28).
According to Brown (2004, p.28) in a test, authenticity may be present in the following ways:
The language in the test is as natural as possible Items are contextualized rather than isolated
Topics are meaningful (relevant, interesting) for the learner
Some thematic organization to items is provided, such as through a story line or episode
Task represent, or closely approximate, real word tasks 5. Washback
Washback is to comment generously and specifically on test performance.
Many overworked teachers return tests to students with a single letter grade an numerical score and consider their job done. In reality, letter grades and numerical score give absolutely no information of intrinsic interest to the students. A little
bit of washback may also help students through a specification of the numerical scores on the various subsections of the test.
H. Techniques for Analyzing Items Test
Analyze the items test of learning outcomes can be done in three aspects: (1) from analyze difficulty index, (2) analyzediscriminatory power item, (3) analyze function of the distractor.
1. Analyze difficulty index
Sudijono (2008:371-372)The quality items test of learning outcomes firstly can be known from difficulty index or the level of difficulty that each item has. Learning outcomes test items can be expressed as good items if the item test is not too difficult and not too easy. In other words, the difficulty index is should be medium or sufficient. The number of difficulty index items can be obtained using a formula by Du Bois :
Where :
P : Proportion difficulty Index
B : The number of testees who can answer correctly item test JS : The number of testees who take the learning test
P Interprestation
Less than 0,30 Difficult
0,30 – 0,70 Medium
More than o,70 Easy
P
=2. Analyze discriminatory power item
Sudijono (2008:387-389) Discriminatory power item can be seen through by looking at the size of the item discrimination index. Discriminatory power item basically calculated testee division into two groups, there are the higher group which is classified as clever and the lower group which is classified as stupid.
Where : Index discrimination
item
Classification Interprestation
Less than 0,20 Poor The item in question has a bad discriminatory power 0,20 - 0,40 Satisfactory The item in question has a medium
discriminatory power 0,40 – 0,70 Good The item in question has a good
discriminatory power
0,70 – 1,00 Excellent The item in question has a very good discriminatory power
Negative (-) - The item in question has a very bad discriminatory power
I. Previous Study
Feni Andri Astuti (student number: 200932073). On 2013 She has also a student of English Education Department. Teacher Training and Education Faculty. Muria Kudus University. The title of her research is “ The Analysis of English Final Examination test for twelve Grade Students of the odd semester 2012/2013 of SMK N 1 Kudus” This research aims to know how is the face validity of the teacher-made english final examination test, how is the content validity of the teacher made english final examination test, and how is the descrimination power of teacher made english final examination test of SMK N 1 Kudus use by english teacher of SMK N 1 Kudus. This research is descriptive Qualitatif research of the
English final examination test items for twelve grade in 2012/2013 and students‟
score od office administration class as a sample.
Sri Rahayu Ningsih (2016) from Universitas Muria Kudus conducted a research entitled ”The Item Analysis of Teacher-Made Test of English Final Examination Test at the Second Semester of the Eleventh Grade Students of SMA 2 Kudus in 2013/2014 Academic Year. The purposes of this research are divided into three parts. firstly is to find out of the face validity item test English final examination of the eleventh grade students of SMA 2 Kudus in academic year 2013/2014. The second is to discuss the content validity of test design for the eleventh grade students of SMA 2 Kudus in academic year 2013/2014. And the last is to know item diffuculty of test design for the eleventh grade students of SMA 2 Kudus in academic year 2013/2014. The research used qualitative as the research method.
The object of the research is the item in final examination design by teacher for the eleventh garde students of SMA 2 Kudus in academic year 2013/2014. In order to collect the data, the researcher analyzed the test items,which consist of 40 multiple choice items where option A, B, C, and D being applied and the focus of this research is about face validity and content validity.
From two theses above, the similarity of this research with those theses is on the subject of the research. They are analyzing the English final examination test. Then, the difference of this research with those theses above is on the research focus. In this research the writer only focus on analyzing the validity of the test; especially, face validity and content validity of the test.
21
A. Method and Design of the Research 1.
Method of the ResearchThe method of the research is qualitative research . It is based on the research focus; to analyze the face validity and content validity of a test designed by teacher at the first grade students of islamic senior high school mahdaliyah jambi. According to Marshall(1999:2:3) qualitative research is an approach to the study of social phenomena; its various genres are naturalistic and interpretative, and they draw on multiple methods of inquiry p (2-3). Some characteristics of qualitative research are: take places in the natural word, use multiple methods that are interactive and humanistic, it is emergent rather than prefigured and fundamentally interpretative.
According to Sugiyono(2009:15) Qualitative research is research method that is based on the philosophy postpositivisme, used to examine the condition of natural objects in which the researcher is a key instrument, sampling is done by purpose data sources, gathering with triangulation techniques, data analysis is inductive/qualitative, and outcomes research emphasizes qualitative meaning rather than generalization.Qulaititave research relies on natural background holistically, human position as a research tool, inductive data analysis, more concerned with process than outcomes and results of research conducted by researchers agreed upon and subject of reserach.
2.
Design of the researchThe study will be conducted by using a descriptive qualitative analysis method. Bodgan and Taylor (2002:3) state that descriptive qualitative analysis method is a research produce which produces
descriptive data in form of written and spoken of peoples or behaviour that observed Whereas. Best in Sukardi (2003:157) declares that descriptive research is a method of research that attempt to describe and interpret the object naturally.The descriptive qualitative analysis will be used to describe the face validity and the content validity of the test. The researcher will analyze whether the test instrument designed by teacher already full fill all the criteria af face validity test also analyze whether the test or the instrument items cover all the standar competences.
B.
Setting and Subject of the Research
1. Setting of the research
The research took place at the Islamic Senior High School Mahdaliyah jambi in academic year 2017/2018.
2. Subject of the research
The subject of this research was the English final examination instrument designed by teacher at the first grade students of Islamic Senior High School Mahdaliyah Jambi.
C.
Source of the Data
The source of data was the subject where the data can be obtained in detailed, those data were the documents and they are
1. The English final examination sheet designed by English teacher at the first grade student of islamic senior high school mahdaliyah jambi.
2. The syllabus designed by English teacher at the first grade student of islamic senior high school mahdaliyah jambi.
D.
Technique of Collecting the Data
According to Lofland (1984) in meleong ( 2014:157) main source data in qualitative research are words and actions. Moleong (2014) stated that the source data in qualitative research can be collected by observation, interview, the
documentation, field note, photo, recording, and picture. In this research, the researcher collected the data by using methods as follow:
1. Documentation
According to Sukardi(2003:81) states that in this technique, the researcher is able to get information from many kinds of written sources or document on the respondents or a place where the respondents are live or do their daily activity. In this case the researcher will take the primary documents from the teacher; they are the test instrument (test sheet) designed by teacher and also the syllabus that the teacher used to teach from the beginning of the odd years until in the final examination test. Test items will be analyzed whether it is already full fill all the criteria of face validity. Then, syllabus will be used by the researcher to analyze the content validity of the test.
2.
InterviewKahn and Cannel in Marshal ( 1999:108) described interview as conversation with a purpose; it may the overall strategy or one of several methods employed in a study. Interviewing varies in terms of a priopruty structure and in the latitude the interviewee has in responding to the questions. An interviewee is useful way to get large amounts of data quickly. In this research the researcher was conduct the interview to get information with English teacher as an informant of Islamic Senior High School Mahdaliyah Jambi.The researcher did the interview on Saturday, August 10 2018 for data about English final examination instrument designeed by teacher at the first grade students islamic senior high school mahdaliyah jambi.
E. Technique of Analyzing the Data
According to Sugiyono (2008:240) defines data analysis as the process of bringing the order, structure, and interpretation to the mass of collected data.
There are three activities on data analysis; data reduction, data display.
1. Reduction of the data
Reduction of data is the beginning step which has to do in analysis of the data. Reduction refers to resume the data, choosing main things, focusing on the important things, look for the theme. Therefore, data that was reduced will give clearer drawing and make the researcher easier to collect the next data (sugiyono, 2008:92).Next steps from these activities are selecting, focusing, and make simpler data, abstracting, and transforming raw data that were written on the field note that together with the record.
Reduction steps in this research are:
a. Resume the result of documentation
b. Classify the result of documentation and observation the students about face validity and content validity of the test
c. The result of observation and documentation changed to be good language, then it is transforming into the note.
2. Display the data
After reduction the data, next step is display the data. Display the data is the collection of information which arranged and gives possibility to get the conclusion and take the action.
Some activities of display the data of this research are
1. Display the documentation and observation data that was done and has been arranged into written text.
2. Display the data from the observation about the condition in the process of teaching and learning.
From display the data, researcher can get the conclusion in order to answer all about the research question in this research. In this step, the researcher
will display the data. In this research, the researcher will use checklist. In term of face validity, the researcher will analyze several aspects. The aspects will bw showed in the table below.
Table 3.1
The Aspects of Face validity Developed by the Researcher Based on to Brown
No Aspects Criteria
Official statement
Yes No
1 A well-constructed
Grammar of the test is correct Capitalization of the test items are correct
Spelling of the words in the test are correct
Font is good in type and size (readable)
Test identify is written in the test The score for each of question are written in the test
Printing quality is clear 2. A test that is
clearly doable within the allotted time limit
The time alocation is enough for students to finish the test
3. Items that are clear and uncomplicated
The questions are clear and uncomplicated
4. Directions that are crystal clear
The questions of test are easy understand
5. Tasks that relate to their work course (content validity)
The test is match with the material which already taught by the teacher 6. A level difficulty
level that presents a reasonable
challenge
All the questions are able to done by students
Total score
For analyzing the face validity of the test, the researcher use checklist with YES and NO Aanswer. Then, the researcher gives the score for the answer of the questions based on Guttman scale. Guttman scale is a scale which want a distinct answer, for example true or false. Yes or no, and so on. For positive answer (yes) the score is 1, and for the negative answer (no) the score is 0. In term of content validity, the researcher will analyze the data based on the competence standard, basic competence, and indicators of the syllabus which is used by the teacher.
The data will be done by using checklist and the result of the checklist will be counted by using simole percentage formula, this formula is used to know haw many percent aspects that achieved by the test instrument, it is the formula:
Where :
P : percentage f : frequency n : number/amount
For more detailed information, the writer gives the explanation about the formula above
P : percentage, is the final value of the total number od analysis that has been carried out it is used to see how many peercen the tests covers the indicators in the English syllabus
f : frequency, is the tota number of indicators that are sampled in determining the appropriateness of the test
n : number/amount, is the whole total indicators available in english syllabus.
In comparing the percentage for the analysis, the writer uses the criteria adopted from Arikunto‟s opinion:
P =
Table 3.2
Criteria of test result percentage
Achievement Criteria
76-100% Good
56%-75% Sufficient
40%-50% Less good
>40% Bad
3. Conclusion drawing / verification
The next step after displaying data is conclusion drawing/verification.
Verification refers to the process which is able to answer research question and research objective.In this research, drawing the conclusion was conducting by comparing the analysis of documentation and observation data. Thus, the researcher will get the conclusion about the quality of English Final Examination Instrument Designed by Teacher at the First Grade Students of Islamic Senior High School Mahdaliyah Jambi.
F. Checking finding validity
In this research, the researcher was conducted triangulation sources and theory technique. In this case, the data that has been collected was linked with the theory and implementation of English Final Examination Instrument Designed by Teacher at the First Grade Students of Islamic Senior High School Mahdaliyah Jambi. By using this technique, the researcher believes that the facts,data, and the information obtained can be acconted for and full fill the requirements of validity.
1. Triangulation of Data
Triangulation is qualitative cross validation, it assessing the suffiency of the data according to the convergence of multiple data souce multiple data collecting procedure (Iskandar, 2009:157). In this case the writer used triangulation, it compare the data from observation to interview finding when researching, the general statement to the personal statement about the finding of observation, compare the interview finding of the data document that related to the object of research also was compare the
people statement to the time during the observation at the place of researching ( Latif, 2004:84). By this technique the writer hope the data was perfect and valid in using, and the data originally represent the real condition of the object that the writer obtained in research.
G. Schedule of the research
Schedule of the research is a frame work to allocate the time to conduct this thesis. The schedule is related to the preparation of proposal untik final report. It can be changed so it‟s not permanent time.
Table 3.3
No Acrivities
Months
March April Mei June July Agustus September
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
1 Preparation
2 Arranging Proposal
3 Preparation of seminar
4 Improvement of seminar
5 Making
research permission
6 Collecting data
7 Arranging research list
8 Writing final report
29
discussions. For this purpose, two points of discussion are presented. They are, Research finding and discussion.
A. RESEARCH FINDINGS
The results of the analysis showed the face validity and content validity of English Final Examination Instrument Designed by Teacher at the First Grade Students of Islamic Senior High School Mahdaliyah Jambi. The researcher put the test sheet on the appendix.
1. Face validity
For analyzing the face validity of the test, the researcher used the theory adapted from Brown (2001:27) that the face validity will likely be high if it encounters:
a) Well constructed, expected format with familiar tasks.
b) A test that is clearly doable within the alloated time limit.
c) Items that are clear and uncomplicated.
d) Directions that are crystal clear.
e) Task that relate to their students course work (content validity).
f) A difficulty level that present a reasonable challenge.
a) Well-constructed (1) Grammar
The researcher found incorrect grammar in the test. It can be seen from the item number 25, as follows:
25. What the things are offered in the new library ? A. Books and DVDs
B. Old and new books C. New books and novels
D. New DVDs and non-fiction books E. New Situation
The sentence number 25 above is incorrect. The sentence is
“What the things are offered in the new library....”. The position of the word “are” in the sentence is incorrect. The correct sentence is “What are the things offered in the new library....”.
2. Mona is a new member of Tapulh club. She introduce herself.
Mona : Ladies and Gentlemen
To complete the dialogue, the suitable expression is ...
A. Allow me to introduce my self B. I‟m happy to meet you
C. Let me introduce you to the audience D. May I introduce you to the introduce E. I will tell you my name
In the sentence, in test item number 2 above is incorrect.
The sentence“Mona is a new member of Tapulh club. She introduce herself”. The sentence did not put to be after subject, it is
” she introduce herself”. Thesentence it shoud be put to be (is),”she is introduce herself”. So, the correct sentence it should be Mona is a new member of Tapulh club. She is introduce herself”.
(2) Capitalization
In theEnglish Final Examination Instrument Designed by Teacher at the First Grade Students of Islamic Senior High School Mahdaliyah Jambi, there are five letters in the test item which are incorrect in term of capitalization. It can be seen fromthe data below.
(a) The test items number 9.
(9). Kate : We will have a long holiday next month. what are you going to do?
Yani : ...
Complete the dialogue above ...
A. I am thinking of going to bali Sorry B. Sorry, I can‟t tell you
C. It‟s not your business D. I have nothing to do E. I don‟t know what to do
In the sentence, in test item number 9. There is no capital letter after full stop. The sentence is “We will have a long holiday next month. what are you going to do?. The letter what should be written with capital letter, because it is the beginning of the sentence after fullstop. The sentence should be “We will have a long holiday next month. What are you going to do? ” .
(b) The test item number 11.
(11). michelle .... begin medical school next year A. Is going to
B. Go to C. Will go to D. Going to E. Is gone
The sentence in the item number 11 there is no use capital letter in letter michelle. Whereas, it is the beginning of the quotation. The sentence is “michelle .... begin medical school next year”. The correct sentence
“Michelle .... begin medical school next year” because it is the beginning of the sentence.
(c). The last, the researcher found three mistakes of capitalization in the tes sheet. The sentence in test item number 4, 7 and 9 there is no use capital letter when she used subject “I” in the middle of a sentence. See the following data:
(4). Lisa :”Don‟t mention it. i .... (4)
(7). Ryan : Can you tell me how to get the nearest restaurant?
Hani : may be you can ask the policement over there.
A. Yes, there is one across the street B. Next to library
C. Behind the hotel
D. I‟m sorry. i don‟t leve here.
E. No thank you
In the three test item above, the sentences there is no capital letter when use subject “I” in the middle of the test item. The sentence is test item number 4 “Lisa : Don‟t mention it. i ... “. The correct sentence is
“Lisa :Don‟t mention it. I ...”.
The sentence is test item number 7 “D : I‟m sorry. i don‟t leave here. From the sentence we know that after fullstop it must be use capital letter, so the correct sentence must be “I‟m sorry. I don‟t leave here.
(3) Spelling
In the English final examination instrument designed by teacher at the first grade students of islamic senior high school mahdaliyah jambi, the researcher found five word that have misspelling. The misspelling word is on the test item number 7, 11, 14, 17, and 26. See the following data:
7. Ryan : Can you tell me how to get the nearest restaurant?
Hani : may be you can ask the poliement over there.
A. Yes, there is one across the street B. Next to library
C. Behind the hotel
D. I‟m sorry. I don‟t leve here.
E. No thank you
In the sentence, in test item number 7 of the test wrong letter in word “poliement. So, in the test the word “poliement” is written
“policeman”. The sentences is “Hani :may be you can ask the poliement over there ”. The correct sentence is “Hani : may be you can ask the policeman over there “Nita : what are you planing to do with all this sculpture?
Rachmat : I .... them in cultural fair next month.
A. To be exhibiting B. To exhibit C. Will exhibit D. Be exhibiting E. To be
In the sentence, in test item number 14 of the test missed one letter in word
“planing”. The misspelling letter is “n”. So, in the test the word “planing” is written “planning”. The sentences is “Nita : what are you planing to do with all this sculpture? “. The correct sentence is “Nita : what are you planning to do with all this sculpture? “.
(4) Font
Font is about how is face of the letter. The size and also the style of the letter itself. In the test, the test use Times New Roman as the letter type with 12 as the size of the letter. The combination between the letter types and the size of the letter is good. Because the test and also the question in test are readable and very clear for student.
(5) Space
For the space between lines the teacher used 1.5 lines. It is the most popular line spacing that conducted by many people in the world. This line spacing makes the test item not dofficult to read. Sometime, when the space between lines in the test instrument is too near, it will make the students difficult to read. Of course, this situation will make thestudents waste the time just reading the test. In the case of this test, the text and also the questions is on proportional line spacing.
(6) Test regulation
Ideally, the test instrument should be provide test regulation in the test sheet. Test regulation is very important in the test instrument.
Because from the test regulation the students know what is forbidden to do and what is allowed to do during the test. In the case of English test, the test regulation could be about the porhibition of opening dictionary, cheating, noisy in the class, and so on. In the English final examination instrument design by teacher at the first grade students of islamic senior high school mahdaliyah jambi, the
researcher found that the instrument did not put the test regulation in the test sheet. The test without test regulation enables the students to open dictionary, open their English book, or moreever they cheat during the test. This situation will make the test not effective to measure the students ability.
(7) School identify
In the English final examinationinstrument designed by teacher at the first grade students of islamic senior high school mahdaliyah jambi. The school identity can be about the school address, the district, phone number, and of course the name of the school. The school identity can show to the students that the test is really designed by teacher who teaches in the school. It can also make the test instrument are clear for the students that the test is really designed for them.
(8) Time allocation
Time allocation is also very important in a test instrument. Time allocation is the time that the teacher give to the students to finished all the questions in the test. In the English final examination instrument designed by teacher at the first grade students islamic senior high school mahdaliyah jambi, the teacher did not put the time allocation in the test sheet. It made the students confused about how long the time that they had to finish the questions. In this case, the teacher had to tell the students directly in class about the time allocation for finishing the test. Telling the students directly in the class will waste the time. The researcher knew that the time allocation to finish the test is 90 minutes.
(9) Printing quality
The printing quality of the test sheet that designed by teacher is good. Because all the words and sentences in the test are clear.
The test is not blueprint. The text, and also the questions are easy to read (readable)
b) A test that is clearly doable within the alloated time limit
Based on the interview with the teacher, the researcher get the information that the time allocation thet the teachers give to finish the question in the test is about 90 minutes. Moreever, based on the teacher explanation there are some students who finish the questions before the time is over. It showed tahat the test that is clearly doable within the time limit.
c) Items that are clear and uncomplicated
There are 30 items in the test, the test items were multiple choice.
All items were not clear and there was item complicated. It is because the misspelling of sentence that made the test unclear and uncomplicated there are question number 1, 2, 7, 11, 17, and 26.
d) Directions that are crystal clear direction
The direction on the test very important. The directions can guide the test taker to answer the question correctly. The test instrument can not ignore this term. In the English final examination instrument designed by teacher at the first grade students of islamic senior high school mahdaliyah jambi, there are one directions in the test. All the directions in the test are very clear for the students. See the following data :
CHOOSE THE ANSWER BY CROSSING A, B, C, D. OR E !
e) Task that relate to their work course work ( content validity)
Based on the interview with English teacher, all question that the teacher give to students are match with the material that the students learn in the
