STUDENT TEACHERS’ ABILITY IN DESIGNING
ASSESSMENT INSTRUMENT AT ENGLISH
TEACHER EDUCATION DEPARTMENT UIN
SUNAN AMPEL SURABAYA
THESIS
Submitted in Partial Fulfillment of the Requirement for the degree of
SarjanaPendidikan (S.Pd) in Teaching English
By
:
NurAmaliaFadhila
NIM D05211018
ENGLISH TEACHER EDUCATION DEPARTMENT
FACULTY OF EDUCATION AND TEACHER TRAINING
SUNAN AMPEL STATE ISLAMIC UNIVERSITY
SURABAYA
ABSTRACT
Fadhila, NurAmalia. (2015).Student Teachers’ Ability in Designing Assessment Instrument at English Teacher and Education Department UIN SunanAmpel Surabaya.A Thesis.English Teacher and Education Department, Faculty of Education and Teacher Training, SunanAmpel State Islamic University Surabaya.Advisor :Dra. Irma Soraya, M.Pd and HernikFarisiaM.Pd.I.
Key Words: Assessment instrument, Standardization assessment, Assessment weakness
D. Significance of The Study ... 9
E. Scope and Limit ... 9
F. Definition of Key Term ... 10
CHAPTER II : REVIEW OF RELATED LITERATURE A. Review of Related Literature ... 14
1. Cognitive and Knowledge Assessment ... 14
2. Cognitive and Knowledge Assessment Techniques ... 17
3. Assessment Instrument ... 18
a. Assessment Instrument in Indonesian curriculum ... 18
b. English Assessment Instrument ... 31
1) Assessment and Testing ... 32
2) TestingReading Skill ... 32
c. English Assessment Instrument in Indonesian Curriculum ... 36
4. Testing the Assessment Instrument ... 37
B. Review of Previous Study ... 39
CHAPTER II I : RESEARCH METHODS A. Approach and Research Design ... 44
B. Research Presence ... 46
D. Data and Source of Data ... 47
E. Data Collection Technique ... 49
F. Research Instruments ... 50
G. Data Analysis Technique ... 51
H. Checking Validity ... 55
I. Research Stages ... 55
CHAPTER IV : FINDINGS AND DISCUSSION A. Research Findings ... 58
1. Student Teacher’ Assessment Instrument Based on Standardization Test ... 58
2. The causing factor of the weakness indicator that could not be fulfilled by student teacher ... 69
B. Discussion ... 84
CHAPTER V : CONCLUSION A. Conclusion ... 98
B. Suggestion ... 100 REFERENCES
LIST OF TABLES
2.1. Types of Assessment Instrument ... 19
2.2. Technique in Assessing Reading ... 36
3.1. The Schedule of Practice Teaching Class... 47
3.2 Technique for Collecting Data ... 49
4.1. Students Assessment Technique and Their Weakness Point ... 69
4.2. Weakness Point in Students Assessment and Its Causing Factors... 81
CHAPTER I
INTRODUCTION
This chapter provides the background of the research that describes the reason why the researchers intends to conduct the research toward student teachers’ assessment instrument. Then, the problems are formulated in the research questions along with the objectives of the study. It is continued with the significance of the study to inform how the result of the research is useful. The limitation about what are included in this research is given in scope and limits of the study. Finally, the last part is definition of key terms relates with this study.
A. Research Background
Education department is intended to educate professional teacher. As a teacher, there are some competencies which need to be achieved; pedagogical, professional, social and personal competence.1 In line with it, English Teacher Education Department at Sunan Ampel State Islamic University (UIN Sunan Ampel Surabaya) has vision to give the contribution through the development of educational quality in education institute or society.2 To achieve that vision, English Teacher Education Department includes PPL (Praktek Pembelajaran Lapangan) or known as practice teaching class in its curriculum.3 In practice
1
Fakultas Tarbiyah dan Keguruan UIN Sunan Ampel Surabaya, Pedoman Praktik Pengalaman Lapangan (PPL) (Sidoarjo: Dwi Putra Pustaka Jaya, 2014), 3.
2
Visi Misi Jurusan Pendidikan Bahasa Inggris, (https://pbisa.wordpress.com/about/, accessed on May 19th, 2015)
3 IAIN Sunan Ampel Surabaya, Panduan Penyelenggaraan Pendidikan Program Strata Satu (S1)
2
teaching class, students are trained to develop their skills in teaching such as designing the lesson plan, developing the material, managing the class and designing proper assessment.
Practice teaching is divided into two parts, practice teaching 1 (PPL 1) and internship program (PPL 2).4 Practice teaching is intended for the students to apply their ability in teaching with their friends as the students, and they analyzed or observed by both of their lecturer and their friends. Here, the student who takes practice teaching 1 course is called student-teacher. While, internship program is the students real teaching practice; they practice their teaching at school with the real students and do the whole teacher’s duty. Furthermore, it is essential for the students to pass practice teaching 1, since it is prerequisite course for the following course (internship program). Besides, the teaching practice which is done in practice teaching 1 be the background experience for the students in facing their internship program. Based on it, this study is focused on practice teaching 1.
In practice teaching 1, student teachers deal with the main duty of a teacher. There are 5 main duties of a teacher; making lesson plan, doing the teaching and learning process, assessing the students, doing the learning analysis result and designing follow up program.5 Assessing the students becomes the focus of this research, since assessment becomes an indicator of how well the
4IAIN Sunan Ampel Surabaya, Panduan Penyelenggaraan Pendidikan..., 121. 5
3
students comprehend the lesson. Assessment is used as the way to collect an important variable of teaching and learning process which is used by the teacher to determine the process and the learner learning result.6 It becomes the evidence of the result in teaching and learning process. The evidence is the achievement of students’ competence which shows whether the objective of the lesson can be fulfilled or not.
Concerning to the assessment design, it must be based on the applied curriculum. In Indonesia, there are two curriculums which are applied by the government; there are KTSP (Kurikulum Tingkat Satuan Pendidikan) and K-13 (Kurikulum 2013). In line with it, in this semester, in practice teaching 1 the lecturers apply both of curriculums, since some of schools still use KTSP as the curriculum and some of them have implemented K-13.7 It is aimed to let the student teachers experience both of it, so it can be their background knowledge in facing their internship program. Related with the assessment design, the concept of the assessment is similar. In KTSP, the domain or the competence which is assessed is cognitive domain, affective domain, and psychomotor.8 While in K-13 there are attitude competence, theologies competence, knowledge competence and skill competence.9 Both of the curriculums have the same concept of the domain, whereas affective domain is similar with attitude competence, then
6
Popham–W. Janes, Classroom Assessment: What Teacher Need to Know (Los Angeles: Allyn and Bacon, 1995), 6.
7
Kementerian Pendidikan dan Kebudayaan,Surat Keputusan Penetapan Sekolah Pelaksana Uji Coba Kurikulum 2013 No 233/C/KR/2015 (Jakarta: Kementerian Pendidikan dan Kebudayaan, 2015) 8Sudaryono, Dasar-dasar Evaluasi Pembelajaran (Yogyakarta: Graha Ilmu, 2012), 43. 9
4
cognitive domain is related with knowledge competence,10 and psychomotor domain with skill competence, except for theologies aspect, it is additional competence in K-13.11
In the case of assessing attainment of the students with the lesson, this research is focused on the assessment instrument of cognitive domain for KTSP and knowledge competence for K-13. It is becauseboth of these domains have the same characteristics. It deals with how well the students reflect their knowledge that they have gotten during the lesson theoretically. Besides that, the score of daily test, midterm test and final test is gotten from cognitive or knowledge competence.12
There are three techniques for assessing cognitive and knowledge competence; written test, oral test and assignment.13 Here, the researcher focuses on written test or task. This technique is chosen since it is the most possible assessment technique that student teachers do in their practice teaching. Moreover, written test is used as one of common technique to assess the students in their daily test, mid-term test and final test. In this research, by focusing on one of assessment techniques, the research is expected to be focus and deep.
10
Kunandar, Penilaian Autentik..., 165. 11
Sudaryono, Dasar-dasar Evaluasi Pembelajaran..., 47. 12
Kementerian Pendidikan dan Kebudayaan Direktorat Jenderal Pendidikan Menengah Direktorat Pembinaan SMA, Model Penilaian Hasil Belajar Peserta Didik (Jakarta: Kementerian Pendidikan dan Kebudayaan, 2013), 38.
13Kementerian Pendidikan dan Kebudayaan dan Kebudayaan Direktorat Jenderal Pendidikan
5
As stated before, since the researcher does the research toward the student teacher of English Teacher Education Department, the course that they teach is English. They teach based on four English skills; reading, listening, speaking and writing. Each skill has its own assessment technique. Moreover, the researcher focus on written test technique, the most possible skill is listening and reading. In this case, the researcher focuses on reading skill because it is the common skill which is tested at school, even at national examination.Reading is receptive skill whereas people understand what they listen and read, then decode the meaning to understand the message.14
Regarding with it, the proper design of assessment for both of cognitive domain and knowledge competence is needed. Therefore, designing English assessment might have some differences with designing any other subject. Instead of designing the assessment based on the Indonesian curriculum, they have to pay attention on the English assessment technique itself. This problem leads the confusion of student teachers in designing their assessment. Dealing assessment design, there are some criteria which are considered to test the assessment instrument itself, starting from the material which is used, the construction of the instrument and the language. Material is related with the substance which represent the competence, construction is about the use of the assessment instrument technique, and language is about the use of the language
14
6
which is suitable with the student’s level.15 Those criteria help the teacher in designing the assessment which is able to reflect the objective of the lesson.
Since the assessment design also influences how well it reflects the objectives of the lesson and the students result, it can be the way to figure out the weakness and the strength of the instrument. Then from the strength and weakness, there are causing factors which influence it, especially for the weakness factor which needs to be found. Here, the researcher attempts to have the research dealing with the causing factor that make student teachers unable to fulfill the indicator of the standardization assessment, in other words, it is the weakness factor in their assessment instrument. By finding the weakness factor, it becomes reference for the student teachers who are going to face their internship program to be aware of their previous fault.
Related with this research, there was similar research which had straight relationship with this research; First, the research which has been done by Abidatul Khoiro entitled “An Analysis of Teacher Made English Try Out Test from National Examination 2010-2011 for the Third Graders of MAN Sidoarjo in 2012”.16 Here, Abida analyzes the validity, reliability and some other criteria of the test which is designed by the teacher. The second one is the newest research done by Aris Bahari Rizki entitled “A Study of Formative Assessment Under the
15
Kunandar, Penilaian Autentik..., 53. 16
Abidatul Khoiro, Undergraduate Thesis: An Analysis of Teacher Made English Try Out Test From National Examination 2010-2011 for The third Graders of MAN Sidoarjo in 2012
7
2013 Curriculum at SMP Wachid Hasyim 7 Surabaya in 2014.”17 He attempts to find the implementation of formative assessment by the teacher. Then, related with this research, although the object is similar, the researcher here is focusing on the assessment instrument design, the design and how it fulfills the requirement as assessment instrument. Besides, by choosing practice teaching 1 as the subject, not a real teacher like those previous studies, the research is expected to give significant impact as the background knowledge for student-teachers to face their internship program, and be the real teacher someday. Therefore, students confusion in designing assessment could be anticipated.
The effectiveness of the assessment is responsive to the strength, needs and clearly articulated learning destination of students. Consequently, finding whether the assessment instrument fulfill the standardization is important. Besides, the causing factors of the weakness point in the assessment instrument can be the reflection or background knowledge to design the assessment instrument better. Based on it, the researcher do the research dealing with an analysis of how the student teachers’ assessment instrument fulfill the requirement of the standardization test and the causing factor of the weakness point in their assessment instrument analysis. Therefore, in the following practice teaching, it can become the reference for both the following students in practice teaching 1 and the students who face their internship program.
17
8
B. Research Questions
Based on the background above, the problems could be formulated as follows; 1. How do student teachers’ assessment instruments fulfill the requirement of the
standardization of cognitive or knowledge competence assessment?
2. What are the causing factors that make student teacher unable to fulfill the indicator of the standardization assessment?
C. Objectives of the Study
According to the statement of the problem, the researcher purposes are; 1. To describe the analysis of the assessment instrument toward the requirement
of the standardization of cognitive or knowledge competence assessment 2. To explain the causing factor that make student teacher unable to fulfill the
indicator of the standardization assessment or the weakness point in their assessment instrument.
D. Significance of the Study
By conducting this research, the researcher hopes that the finding gives many beneficial for both of the students and the lecturer.
1. For Student teacher
9
2. For Lecturer
This research provides information about how well the student understands about assessment instrument through the analysis result of their instrument and the causing factor of their weakness point. In that case, it provides an overview and idea what lecturer should do to help the students in designing better assessment.
3. For Future Researcher
It can be a reference for any further research which focuses on the similar subject with this research.
E. Scope and Limits of the Study
1. Scope of the Study
The scope of the study is focused on the assessment instrument. There are 3 assessment domains in KTSP; cognitive, psychomotor and affective, while in K-13 it has theology competence, attitude competence, knowledge competence, and skill competence. Both of the curriculums have the same concept of the domain,18 whereas cognitive domain is related with knowledge competence and psychomotor domain with skill competence. For attitude competence, actually, it is similar with affective domain. The difference is, in K-13, attitude competence is divided into two categories; theologies aspects and social aspects. In this case, the researcher focus on the cognitive domain and knowledge competence.
18
10
In practice teaching class, the student teachers do teaching practice based on English skill which is decided by them or class agreement, there are four skills; speaking, writing, listening and reading. In this research, the researcher focuses on reading skill. To assess the reading skill, test is used as the assessment instrument. There are two types of test; written and oral test. Written test become the focus of the research since it is the most possible way to assess reading skill. In short, the research deals with reading test which is designed by student-teachers during their practical teaching.
2. Limits of the Study
This research is limited on practice teaching 1 course at English Teacher Education Department, UIN Sunan Ampel Surabaya. There are 7 classes in practice teaching 1; A, B, C, D, E, F and G with three different lecturers.
F. Definition of Key Terms
1. Ability
Ability is competence of an activity or occupation because of one’s skill, training or other qualifications.19 In this research ability means, the ability of student teacher in designing their assessment instrument which fulfill the requirement of standardization test. The ability is reflected from student teachers’ final score in their assessment.
19
11
2. Cognitive and knowledge competence
Cognitive and knowledge competence assessment is an assessment which reflects the knowledge concept that must be mastered by the students during the teaching and learning process.20 Cognitive is the domain in KTSP while knowledge is the domain in K-13. Both of cognitive or knowledge competence have the same concept of the assessment model, the difference is on its name of categorization.
3. Assessment instrument
It includes the checklists (or other ‘instruments’) and instructions needed to conduct one part of a competence-based assessment (e.g. Written test with answer key, observation checklist, verbal questioning instrument, log book, etc.). Each instrument outlines the evidence the candidates need to supply, and describe the evidence criteria used to judge the quality of performance.21 In this research, assessment instrument is a task or test which is designed by student-teacher to assess their students understanding toward the lesson during their practice teaching class.
20
Kunandar, Penilaian Autentik..., 165.
12
4. Designing Assessment Instrument
Design defines the specification, plans, parameters, costs, activities, processes, and how and what to do within legal, political, social and environmental, safety and economic constraints in achieving that objectives.22 In this research, design means, the student’s process or the steps in designing the assessment instrument.
5. Standardization
Standardization is collective activity to establish solution to a repetitive situation.23 In this research, standardization means the presence of all requirements that can be fulfilled by the assessment instrument which is designed by student-teachers.
6. Practice Teaching 1
Practice teaching is a period of guided, supervised teaching. The college student is gradually introduced into the teaching role for a particular class by a mentor or cooperating teacher.24 In this research, practice teaching means a class which is intended as practical teaching for the students in English Teacher Education Department UIN Sunan Ampel Surabaya.
22
Don Kumaragamage Y, Design Manual Vol 1,(http://www.creative-rights.org/design-law-2/,accessed on March 30th, 2015)
23Standardization,(
13
7. Student Teacher
CHAPTER II
REVIEW OF RELATED LITERATURE
This chapter provides two sub-sections, first is the review of related literature that describes the basis theory of assessment which is used in this research. The second one is review of previous study that explains the differences of this research with other research which is done by another researcher.
A. Review of Related Literatures
1.Cognitive and Knowledge Assessment
Assessment is defined as the process of gathering information which is used to propose decision regarding the students, curriculum program, educational policy, and method or education instrument.1 It is an integral part of learning, whereas it is used to check the students’ understanding related to the material that has been given by the teacher. The indicator of the achievement of learning outcomes can be reflected from it.
In Indonesia, there are two curriculums which are applied by the government; there are KTSP (Kurikulum Tingkat SatuanPendidikan) and K-13 (Kurikulum 20K-13). In line with it, in this semester, in practice teaching 1 (PPL 1) the lecturer applies both of curriculums, since some of school still use KTSP as the curriculum and some of them have implemented K-13.2 It is
1
Anthony J - Nitko, Educational Assessment of Students(Ohio: Merrill, an Imprit of Prentice Hall, 1996), 4.
2Kementerian Pendidikan dan Kebudayaan,Surat Keputusan Penetapan Sekolah Pelaksana Uji Coba
15
aimed to let the student teachers experience both of it, so it can be their background knowledge in facing their internship program (PPL 2).
Curriculum regulates the process of teaching and learning process, including designing the lesson plans, developing the material, designing assessment and designing follow up program.3 Assessment as part of the curriculum plays role as the way to collect an important variable of teaching and learning process. It becomes the evidence of the result in teaching and learning process. The evidence is the achievement of students’ competence which shows whether the objective of the lesson can be fulfilled or not. Related with the assessment design, in KTSP and K-13, the concept of the assessment is similar. In KTSP the domain or the competency which is assessed is cognitive domain, affective domain, and psychomotor.4 While in K-13 there are attitude competency, theologies competency, knowledge competency and skill competency.5 Both of the curriculums have the same concept of the domain,6 whereas cognitive domain is related with knowledge competency and psychomotor domain with skill competency. For attitude competency, actually, it is similar with affective domain. The difference is, in K-13, attitude competency is divided into two categories; theologies aspects and social aspects.
3
Kunandar, Penilaian Autentik (Jakarta: Raja Grafindo Persada, 2014), 31. 4
Sudaryono, Dasar-dasar Evaluasi Pembelajaran (Yogyakarta: Graha Ilmu, 2012), 43. 5Kunandar, Penilaian Autentik..., 103.
6
16
In the case of assessing attainment of the students with the lesson, this research is focused on the assessment instrument of cognitive domain for KTSP and knowledge competency for K-13. In KTSP, cognitive is the step whereas the students understand something, and apply it in psychomotor domain.7 While in K-13, knowledge becomes the main competency (KI 3), whereas the process of transferring knowledge or material happens. Both of these domains have the same characteristics whereas it deals with how well the students reflect their knowledge that they have gotten during the lesson theoretically. There are 6 domains of knowledge competency8 and so does cognitive.9
a. Knowledge: it is the stage whereas the students are able to remember the learning material in the previous meeting.
a. Comprehension: the ability of grasping the meaning of the material. b. Application: the student’s ability in applying general idea, theory,
formula, principal and every material in a new and concrete situation. c. Analysis: the student’s ability in explaining and elaborating detail part of
the lesson and understanding the connection from part to part.
d. Synthesis: it is the stage whereas students are able to fuse every part of the lesson logically and structure it all.
7
Sudaryono, Dasar-dasar Evaluasi Pembelajaran..., 255. 8Kunandar, Penilaian Autentik...,168.
9
17
e. Evaluation: the student’s ability in considering idea, situation, value and method of guideline and the certain criteria.
So it can be seen, both of cognitive and knowledge domain has the same concept and indicator to assess the students. It is used to reflect the achievement of the students learning result that they got through teaching and learning process.
2. Cognitive and Knowledge Assessment Techniques
There are three techniques for assessing cognitive and knowledge assessment; written test, oral test and assignment.10 Written test can be in the form of multiple choice, descriptive test, short answer, true-false, matching and so on. While oral test is in the form of question lists, and assignment is in the form of homework which is done individually or group.
Furthermore, the subject that student teachers’ teach is English. They teach based on four English skills; listening, speaking, reading, and writing. As stated in the background the researcher focused on reading.The only assessment techniques which suit with reading is written test. Written test as the technique for assessing reading has its own instrument which is regulated in Indonesian curriculum and in English assessment instrument. Thus, considering and combining both of instrument are possible to be done in case of getting reliable reading assessment.
18
3. Assessment Instrument
a. Assessment Instrument in Indonesian Curriculum
Teacher assesses the knowledge competency through the instrument that has been designed. The assessment instrument here is called test. It is a task or set of tasks that elicits observable behavior from the test taker.11 Because test is a form of assessment, test also answers the question of how well the individual perform either in comparison with others or in comparison with a domain of performance task. Test is commonly used to upgrade the teaching and learning process. It is because the students’ answer becomes the best sources for knowing the achievement of the teaching aim, the students’ comprehension of the lesson and the effectiveness of the teaching process.
Both of cognitive and knowledge competency has its own test instrument. In cognitive the test is called objective test and subjective test,12 while in knowledge competency it is called written test.13 The detail of the test as the assessment instrument can be seen below.
11
Ered Genesee- John A Upshur, Classroom-Based Evaluation Second Language Education, (New York: Cambridge University Press,2007), 141.
12Hamzah B Uno - Satria Koni, Assessmen Pembelajaran (Jakarta: Bumi Aksara, 2013), 112. 13
19
Table 2.1 Types of the Assessment Instrument
Cognitive Domain (KTSP) Knowledge Competency (K-13)
Objective Test Subjective Test Written Test
Multiple choice Short answer Multiple choice True-false Descriptive test True-false make it clear, the descriptions of each technique is described below.
1) Multiple choice
Multiple choices are a form of assessment in which a question needs to be answered by selecting the best possible answers out of the choices from a list.14 Every multiple choice item consist of stem (the beginning of the item) and either three, four, or five answer options (with four options being the most common format), one and only one of the options are correct, and that is called the key, the incorrect options are called distractors.15 Here is the example.16
14
Kunandar, Penilaian Autentik..., 120. 15
Kathleen Bailey, Learning about Language Assessment (America: Heinle Publishers, 1998), 131. 16Steven J. Burton, et all, How to Prepare Better Multiple-Choice Test Items: Guidelines for University
20
The advantages of creating multiple choice tests is the amount of the question are expected to represent the entire competency that are going to be assessed. Besides, it will ease both of the teacher and the test-taker, for the teacher, they will be easily grading and analyzing the question.17 While for the test taker, it will ease them since they only need to respond by making a mark on the paper.18 Instead of the advantages, the difficulties of creating multiple choices are also existed. For example, guessing might be done by the test taker whereas it will impact on test scores.19 Since the answer choice is existed, the students tend to choose it randomly when they do not get the right answer. Moreover, it might facilitate the students to cheat. As a result it can be difficult to analyze the question and the result of the student’s test.
Designing multiple choices as the test or task to measure the student’s competence has to follow some certain criteria. It will assess
17
Kunandar, Penilaian Autentik (Jakarta: Raja Grafindo Persada, 2014), 187.
18Arthur Hughes, Testing for Language Teacher(New york: Cambridge University Press, 2007), 76. 19
21
the assessment instrument whether it will be able to measure what is going to measure or not. There are three categories that need to be considered when designing multiple choice tests; there are material, construction and language.20 Each category have its own indicator, the detail of the indicator is below.
a) Material
(1) The questions are appropriate with the indicator. (2) The limitation of the question and the answer are clear. (3) The material is measured based on the competence.
(4) The question is suitable with the students’ level or class level. (5) The answer choices are logic, the distractor does not looktotally
wrong. b) Construction
(1) The main question (stem) isproposed clearly.
(2) The question and the answer choices are formulated in brief and clear way.
(3) The questions do not show the key answer.
(4) The questions do not use double negative statement.
(5) If the negative statement is used, it must be in italic form or underline.
(6) The answer choices are homogenous.
20
22
(7) The answer choices do not use the statement of “all of answer are correct” and “all of answer are wrong”.
(8) The length of the answer choice is relatively similar.
(9) The answer choice in time and numeric form is formed based on the amount and chronological events.
(10)Picture, graph, table and diagram are clear and useful. (11)The questions do not depend on the previous questions. c) Language
(1) It use communicative language.
(2) The statement use good and correct English.
(3) The questions do not cause double interpretation or misunderstanding.
(4) Use common language, not local language or taboo.
(5) The question do not consist word that can insult others’ feeling. Each of the indicators above has to be fulfilled, so the test or task will be able to meet the objective of the test and measure the student’s competence.
2) True-false
23
with only two options.21The true word will be symbolized by (T) and (F) for false. Here is the example.
Surabaya is the capital city of east java T/F
Albert Einstein is the discoverer of lamp T/F
The students’ duty is to decide whether the statement is true or false. The characteristic of this test is, it is easily to arrange and grade. In contrast, for the students, it might facilitate them in guessing the answer, so it will be difficult to know their understanding of the material through that test. To overcome it, sometimes this test is modified by adding the reason of the students in stating the true and false of the statement.
Using true and false as the technique for testing the students is not only about creating the questions and asking the students to answer it. Nevertheless, the test design has to meet the requirement of standardization test. There are 12 criteria that need to be aware when designing this technique.22 Here are the criteria.
a) The questions are appropriate with the indicator. b) The questions do not reflect the answer.
c) The question is in positive sentences, if it use negative sentence, it is not more than two sentences.
21Kunandar, Penilaian Autentik..., 79. 22
24
d) The question is not similar with the book sentences.
e) Do not use the question that use to distract the students’ answer f) Do not use the words that are able to disturb the answer, (usually,
never, no one, only will disturb true answer), (actually, maybe and sometimes will disturb the false answer).
g) Use brief sentences for the question. h) Use recognizable word for the students.
i) The amount of true answer and the false answer are balanced. j) Do not use ambiguous statement “some of them are correct and
some of them are false”.
k) Do not use qualitative or quantitative language which is easily to guess.
l) The arrangement of true and false statement is arranged randomly. By paying attention on the criteria or indicator above, it will help the teacher to meet the need of the assessment objectives, so the assessment will be able to assess the students’ ability well.
3) Matching test
Matching test is written test which consist of question and answer category. The category group is written as the questions and the left side will be the answer category.23 The question can be in the
23
25
form of interrogative sentence, or statement. Then, the answer will be in the form of word, number, picture or symbol. Here is the example.
A. Narrative B. Procedure C. Recount
1. To tell the past experience 2. To tell how to make something 3. To amuse the reader
Here the students are intended to match the word or the statement in right group to the statement in left group. Dealing with the use of matching test, it can be useful for assessing student’s competence in simple ways, like identifying information based on simple relationship, or identifying the correlation between two things.
In addition, when a teacher set a test or task for their students, it is not about deigning the question and answer. They must pay attention on their instrument whether it is appropriate or not for their students. In this case they have to follow the indicator of the assessment technique that they use. In matching technique, there are nine categories or indicator which needs to be aware.24 Here they are. a) The questions are appropriate with the indicator.
b) The problem in the question is in the same genre, for example; time with time, place with place (connect each other).
c) The words in the question are not more than 15 words.
d) The words of the answer are more than the words in the question.
24
26
e) The question is proposed in the brief sentences.
f) The question is placed on the left with number, while the answer is placed on the right with alphabet.
g) The question and answer are arranged in systematic, if it is used number, it is arranged from the biggest number, and then if it is a word, it is arranged based on alphabet.
h) The statement is written in a page.
i) The score is calculated from the right answer.
The indicator above will help the teacher in designing the right test, so the test objectives are able to be fulfilled.
4) Completion test (fill in the blank)
Completion test requires the student to answer a question or to finish an incomplete statement by filling in a blank with the correct word or phrase.25 Here is the example.
Declaration of independence was the first to be _______ by the
United States26
From that example, it can be seen that the answer might be based on the reading that the students had been read, or it based on the student’s knowledge. The limited context provided by the sentence
25
Nuhad Yazbik Dumit, Writing Effective Questions: True & False, Matching, & Completion Test Items (USA: American University of Beirut, 2011), 29.
27
allows the teacher to focus the item on the one thing she wishes to assess.27 Then related to the implementation of fill in the blank test, it is essential that the statement is clearly stated and students have already told very clearly and firmly that only one word can be the answer and put it in the blank.
Using fill in the blank as the test technique might look simple, since the teacher only need to remove some of words from the text that has been given to the students. In contrast, it is not about remove the words from the text, there are some indicator that need to be aware related with the design of this technique. There are nine categories or indicators that will lead this technique to meet the objectives of the assessment.28
a) The questions are appropriate with the indicator. b) The question is not taken directly from book. c) The question is clear, it does not cause ambiguity.
d) Use the words that do not have important meaning regarding to the answer.
e) The answer consists of one or two words.
f) Put the blank part of the answer in the middle of the question.
27
Kathleen M.Bailey, Learning about Language Assessment (America: Heinle & Heinle Publishers, 1998), 59.
28
28
g) The length of the full stop for the blank answer is similar from one question to another.
h) There is possible key answer under the question list.
i) The score of the answer is counted based on the amount of the blank.
5) Short answer
In this test, the teacher gives the question to the students which need a short answer. It is commonly used in listening and reading test. Moreover, it is also suitable in assessing knowledge aspects especially in memory domain. Here is the example.29
What does “it” in the last sentence refer to?
How old was Hannibal when he started eating human being?
Here, the focus of the question is the word it in the sentence, so there will be one possible answer. Instead of it, it might be difficult for the students who have difficulties in using the language, since they need to produce language in order to respond. Then for the teacher by having short answer test, it will minimize the possibility of guessing and cheating. So the grading will show the real result of the students’ understanding.
As stated above, using short answer as the technique for assessing the students might help the teacher to minimize the
29
29
possibility of cheating, since the students need to answer it by producing the language. However, designing the question for short answer technique might become a challenge for the teacher. They have to pay attention whether the questions have already met the assessment objectives or not. Therefore, they need to fulfill the indicator of short answer technique. There are six indicators that need to be aware when using short answer techniques.30
a) The questions are appropriate with the indicator. b) Use interrogative sentence.
c) The answer consists of one or two words.
d) The answer sheet is separated from the question list. e) The question is not similar with the book sentences. f) There is only one correct answer.
6) Descriptive test
Descriptive test is an assessment instrument which demands the students in remembering, understanding, and organizing idea that have been learned by stating it with their own words. Descriptive test is divided into two categories:
a) Open-ended test
The question is given based on main problem which is discussed without giving any answer instruction. For example
30
30
“Why does Indonesia support Palestine to achieve their
independence?”
b) Closed test
The question is directed into certain problem, so the answer of the students will be based on the question claim. For example.“mention three reasons of Michael living his family!” In designing descriptive question, the teachers have to follow the indicator of each categories in this technique; material, construction and language.31 The indicator will assess the test before it is used to test the students. It will analyze whether the test is able to measure that the teacher is going to measure, so that the test objectives can be fulfilled. The categories and indicator is divided as follows.
(1) Material
(a) The questions are appropriate with the indicator. (b) The limitation of question and answer is clear.
(c) The contain of the material is suitable with the assessment purpose.
(d) It is suitable with the student’s level or class level. (2) Construction
(a) The question is in the form of interrogative sentence or command which demands descriptive answer.
31
31
(b) There is clear instruction about how to answer the question.
(c) The question does not depend on the previous question. (3) Language
(a) Use communicative language.
(b) Use the language which is appropriate in English form. (c) The question forms do not cause double interpretation or
misunderstanding.
(d) Use common language (English form) not local language or taboo.
(e) The question do not consist word that can insult others feeling.
b. English Assessment Instrument
1) Assessment and testing
Assessment is ongoing process which cover wider domain.32 Whenever a teacher asks their students, offer their comment, or try a new word together, at that moment, the teacher assesses their students’ performance. When the teacher tries to assess the students’ performance, it is not always incidental or intended. They might assess it while teach them.
32
32
Testing is one of form of assessment.33It helps the teacher to discover information related with the students’ achievement toward the lesson. Moreover, it can be the way for a teacher to analyze students’ strength and weakness point.Test is designed based on its purpose, especially test which is used to test various abilities; reading, listening, speaking and writing. Each of those abilities has its own test techniques. Since the researcher focuse on reading, so the further explanation of reading test techniques will be described below.
2) Testing Reading Skill
Teaching is the primary activity in teaching and learning process, while testing will inform the result of the teaching process. It will be needed to provide information about the student’s achievement.34 There will be some techniques to test the students learning result. The difference of each technique is caused by the purpose of the test. In case of assessing reading, there will be some of certain techniques which are able to ease the teacher to test their students. The technique below is proposed from Arthur Hughes’ book entitled testing for language teachers and J Charles Aldersons’ book entitled assessing reading.
33
Arthur Hughes, Testing for Language Teacher ..., 5. 34
33
a) Multiple choice
Multiple choice is a common techniques to test students’ comprehension toward the text. Students will answer multiple choice questions by eliminating the distractor by their logical analysis that they have gotten by reading the text35. For example:
Memorizing is easier when the material to be learned is
a) In a foreign language 2. To tell how to make something 3. To amuse the reader
35Charles Alderson, Assessing Reading (New York: Cambridge University Press, 2000), 211. 36
34
They will be intended to match the word on the left to the meaning or explanation in the right.
c) Ordering tasks
In ordering tasks students are given scrambled set of words, sentences, paragraph or texts and have to put them into their correct order.37 For example:
A it was called the last waltz 1 D
B the street was in total darkness 2 __
C because it was one he and Richard had learnt at school 3__
D Peter looked outside 4__
E he recognized the tune 5__
F and it seemed deserted 6__
G he thought he heard someone whistling 7__
d) Dichotomous items
This technique is called true and false techniques whereas the students are presented with a statement which is related with the text and decide whether the statement is true or false.38 This test is intended to test the ability of infer meaning. For example:
1. John is the youngest in his family T/F 2. John sister always comes home late T/F
37Charles Alderson,Assessing Reading..., 219. 38
35
Here the students will decide whether the statement above is true or false based on the text that they have read.
e) Short answer test
In this technique the students are asked a question which requires a brief response. The short answer technique will work well to test the ability to identify referents, predict the meaning of unknown words from context, and write items related to the structure of text.39 For example:
What does the word ‘it’ (line 26) refers to? ____________ Which town listed in table 4 has the largest population?_____ f) Gap filling
This technique is particularly useful in testing reading. It can be used to get the students response. Students read the text and in the same time they read the summary of the text whereas some of the keywords have been removed. Their task is to restore or fill the missing keyword; consequently they have to understand the main ideas of the original text to find the right keywords.40 For example:41
To support his claim that the mafia is taking over Russia, the author points out that the sale of __________ __________in
39
Arthur Hughes, Testing for Language Teacher..., 145. 40Charles Alderson,Assessing Reading..., 240.
41
36
Moscow has increased by ____________ per cent over the last two years.
c. English Assessment Instrument in Indonesian Curriculum
Dealing with English assessment instrument, there are two kinds of way to assess it. From Indonesian curriculum technique or from the original techniques of assessing reading based on Arthur and Charles books, both of the technique in Indonesian curriculum or English assessment is similar, it can be seen from the table below.
Table 2.2 Technique in Assessing Reading Assessment Technique in
Indonesian Curriculum
English Assessment Instrument Based on Arthur Hughes and J
Charles Alderson
Multiple choice Multiple choice
True-false Dichotomous Items(T/F)
Matching Matching Technique
Fill in the blank Gap filling
Short answer Short answer
Descriptive test Ordering tasks
37
4. Testing the Assessment Instrument
Test is not only about testing the students but also it is about how the test can measure what is going to measure. It is essential since testing is the only way to monitor and systematically rank the students. In measuring and evaluating the test, it has to consider on its validity and reliability. In this case, reliability will be omitted, since it can be measured if the test is already used to examine the student’s competence. While in this research, the researcher takes the assessment instrument which is designed by student-teachers during their practical teaching, whereas the test is used to examine other student-teachers. It means that the students are not the real students; therefore the reliability cannot be measured. As a result, validity will be the focus to measure and evaluate the test.
According to Scarvia B Anderson in his book "Encyclopedia of Educational Evaluation" stated that "a test is valid it measures what it purpose to measure". That is why the validity will be the concern before the test is conducted. It is used as the quality control of the test. In addition, there are two areas which need to be considered when discussing the validity of the test.42
a. Consider how closely the test performance resembles the performance we expect outside the test.
42
38
b. Consider to what extent evidence of knowledge about the language can be taken as evidence of proficiency.
In this research, validity is classified into two criteria. First is content validity, this validity is defined as any attempt to show that the content of the test is the representative sample from the domain which needs to be tested.43 It is related with the compatibility of the test use. For example the teacher design reading test for their students, here the content validity can be seen from the content of the test; if it is appropriate with the level of the student then, this test has its content validity. In addition, content validity shows how far the assessment instruments are able to represent the sample behavior which is used as the teaching purposes.44 It is related with the material of the test, if the material in the test has been taught by the teacher, it means that the material in charge with the content validity. Moreover, content validity is also known as curriculum validity whereas it will be related with the syllabus and lesson plan. In this case the other professional teacher is involved in assessing the content validity.
The next one is construct validity; this validity is the most possible one to be done by the teacher itself before conducting the test. It is concerned with the level of accuracy a construct within a test is believed to
43Glen Flucher - Fred Davidson, Language Testing and Assessment (USA: Routledhe, 2007). 4. 44
39
measure.45Construct validity can be fulfilled if the test are able to measure what’s going to measure, so it will be related with the correlation between the indicator and the purpose of the study. The requirement of proposing a good test can be the requirement to test the test, the instrument of the test is required to reflect the requirement of the standardization. It can be seen from the material, construction and language criteria. Then, between the content validity and construct validity is related to each other.
B. Review of Previous Study
Here, the researcher reviews some researchers which were related to this research, as follows:
It was a research which was done in 2009 conducted by IffahMursyidahMayangsari, entitled “an analysis of UAS English Test of Second Semester 2008/2009 by Teacher Made English Test at SMA 2 MuhammadiyahSidoarjo.”46 In this research, Iffah analyzed test by analyzing its validity, reliability, and its index of difficulty and discrimination. Descriptive qualitative is implemented as the methodology to answer the problem and she found that the teacher made test which is designed by SMA Muhammadiyah 2 Sidoarjo is good since it fulfills the characteristics of a good test.
45
Douglas Brown, Language Testing: A Critical Survey and Practical Guide (San Fransisco:Prentice Hall, 1994), 256.
46
40
The other similar study was conducted by AbidatulKhoiro in 2012, entitled “An Analysis of Teacher-Made English Try out Test form of National Examination 2010-2011 for the Third Graders of MAN Sidoarjo.”47 The object of the study is similar with the research which is done by Iffah. Here Abidatul did the research dealing with the teacher made test and the problem that she was going to find is also similar, it was about validity, index of difficulty and index of discrimination. By using descriptive method as her methodology, she found that there are several indicators which were not able to be covered by the test. In conclusion, although the result of the student test was good but the test has low quality as a test.
Still in 2012, another similar research was done by ItaFaradillah, entitled “An Analysis of Essay Test on English Final Test for Grade Eleven Students of SMAN 1 Lamongan.”48 Such as another previous research which was done by Abidatul and Iffah, this research also uses qualitative research to find the content validity, index difficulty, and index discrimination of the final test. As the result of the research, the researcher found that the content validity is in good level whereas 80% items represent all materials; Both the Index of difficulty and the Index discrimination is acceptable.
47
Abidatul Khoiro, Thesis: An Analysis of Teacher-Made English Try out Test form of National Examination 2010-2011 for the Third Graders of MAN Sidoarjo.(Surabaya: Perpustakaan IAIN Sunan Ampel Surabaya. 2012)
48 Ita Faradillah, Thesis: An Analysis of Essay Test on English Final Test for Grade Eleven Students of
41
In short, three of the previous study above were commonly focus on analyzing the assessment which had been used to assess the. It analyzed the content validity, index difficulty, and index discrimination of the test. While in this research the researcher focusing on the assessment instruments which were not used to test the students and assessed it based on test standardization. Therefore by assessing or analyzingthe test before it is used to test the students, the test is expected to measure what it is going to measure.
Another similar study related with reading test was done by SukmaliaYashintaNovansa in 2012, entitled “The Implementation of Reading Aloud Test in Reading Comprehension on the Grade X1 Class Students of SMA Negeri 3 Lumajang.”49 Quasi experimental is used as the methodology. In this research Sukmalia found that the student response of reading aloud test in reading text materials was better when using reading aloud; it is proven with the comparison of the mean score. Aloud” pretest 63, 0625 and posttest 79,375, usual 63, 75 post 67, 5. Here sukmalia focused on the implementation of testing reading using reading aloud test, while in this research the researcher focuses on types of reading test technique in written form; multiple choice, descriptive text and others.
49
42
The newest research was done by ArisBahariRizki in 2014 with the research entitled “A Study of Formative Assessment under the 2013 Curriculum at SMP WachidHasyim 7 Surabaya”.50 By using descriptive qualitative research, here, Aris tries to find the challenges of formative assessments under the 2013 curriculum and the implementation of it. He found that the implementation of formative assessment through self and peer assessment are well implemented. Here the subject of the research is the real teacher.
Seeing from the study that has been conducted before, the researcher concludes that all the previous studies have the similarity and different areas of study. Those previous studies could be the foundation of conducting this research. The previous study mostly focusing on the implementation of the assessment technique and the assessment which was designed by the real teacher, the content validity, index discrimination and index of difficulty, while in this research, the researcher focused on the quality of the test based on the standardized test and the causing factor.
In addition, some of the setting of the research above is school and the real teacher as the subject, the real teacher might be understood well of how to design the good test. Consequently, the significance of the research was not really impacted. While in this research, the subject of the research was the teacher candidate, whereas they need to know of how to design the good test as their
50Aris Bahari Rizki. Thesis: A Study of Formative Assessment Under the 2013 Curriculum at SMP
43
experience for being the real teacher someday, so the significance of the research could have more impact.
CHAPTER III
RESEARCH METHOD
This chapter deals with the procedures of conducting the research toward student teachers’ assessment instrument, it covers research approach and design, researcher presence, research location, data and source of data, research instruments, data analysis technique, checking validity of findings and research stages.
A. Research Approach and Design
This research was typically designed as qualitative case study. Qualitative approach is used to describe student teachers point of view and identify recurrent patterns or themes of their assessment design.1 In term of analyzing their assessment instrument based on standardization test and the causing factor of the weakness point, qualitative will lead the researcher in having deep understanding toward student teacher as the subject of the research. It would deal with interpreting the result of the assessment analysis and seeking the student teacher point of view toward the analysis result.
Furthermore, this research used case study as the research design. Case study has aim to investigate the certain phenomenon with specialized subjects.2 In line with it, in this research, the researcher had aim to investigate the student teacher as the subject by representing the phenomenon of their assessment
1
Donald Ary, Introduction to Research in Education (USA:Wadsworth, 2010), 452.
45
instrument design. Here, they designed the English assessment instrument which based on Indonesian curriculum. Instead of consider the English assessment technique; they have to consider the applied curriculum which regulate the assessment design. It only occurred within practice teaching class at English Teacher and Education Department, UIN SunanAmpel Surabaya. Here the researcher concerned to identify their assessment instrument whether it can fulfill the test standardization or not and find the causing factor of the weakness point in their assessment by having interview with them.
Moreover, in this research, the researcher deals with the research of student teachers’ assessment instrument and the causing factor of their weakness point in their assessment. It means that it would need depth description related to the assessment instrument itself. As a result, this case study is conducted through descriptive approach to get a depth understanding.3 Descriptive approach would recognize whether the assessment instrument fulfill the requirement of standardization test or not and describe the causing factor of the weakness point. Thus, the type of qualitative case study with descriptive approach was used in this research with aimed to investigate the student’s assessment instrument and its weakness factor.
3
46
B. Research Presence
In this study the researcher presents as a non-participant observer. Non-participant observer means the researcher does not reach the full involvement in participants’ activity.4 Therefore, the researcher came to the Practice Teaching class
for checking the students who teach reading skill, then asked their lesson plan and
their assessment instrument. The interview dealing with the causing factor of their
weakness point in their assessment instrument was done outside the class. Here the
researcher did not introduce herself, or bother the class, so, both of the student
teacher and also the lecturer could do teaching and learning activity as usual.
C. Research Location
This research was held in English Teacher Education Department, Faculty of Education and Teacher Training UIN SunanAmpel Surabaya. The researcher conducted the research for the students who enroll in Practice Teaching (PPL I). There are 7 classes in practice teaching 1; A, B, C, D, E, F and G which are taught by three different lecturers. Lecture 1 taught A, B and C class, lecture 2 taught D and E, and lecture 3 taught F and G class.
In term of getting research subject, the researcher designed the sample of the research purposively. Purposive sampling is common sampling technique which is used in qualitative research.5Different with quantitative research which need generalization of the subject, purposive sampling in qualitative research
4
J. Amos Hatch, Doing Qualitative Research in Education Settings(Albany: State University of New York, 2002), 72.
5
47
tend to determine sample based on certain characteristics or purpose.6 Since the researcher focused on reading skill, it means that the participant of this research was the student teachers who taught reading in their practice teaching. The schedule of practice teaching (PPL I) classes and the students who had been taken as respondents are specified in this table:7
Table 3.1The Schedule of Practice Teaching Class
Day Time Class Room Lecturer Number of
Sugiyono, MetodePenelitianPendidikan: PendekatanKuantitatif, Kualitatif, dan R&D (Bandung: Alfabeta, 2012), 300.
7Lecturing Schedule of Sixth Semester in English Teacher Education Department of SunanAmpel
48
their assessment instrument within practice teaching class. This primary data were obtained by collecting student teachers’ assessment instrument and analyzed it using Kunandar and Sri Wahyuni rubric. In addition, to find the causing factor of the weakness point, the researcher obtained the data by doing interview.
b. Secondary Data
The secondary data was the forms of supporting data obtained from some sources, such as the schedule of practice teaching class and also the attendance list of the students. Some theories were also taken by the researcher to support the data gained.
2. Source of Data
The sources of both primary and secondary data were obtained from teachers, students, and the research presence at practice teaching class in English Teacher Education Department, UIN SunanAmpel Surabaya. The primary data was obtained by analyzing the assessment instrument by using rubric which was adapted from Kunandar and Sri Wahyuni book and interview which was used to find the causing factor of the weakness point in their assessment instrument.
49
E. Data Collection Technique
In case of collecting the data of the research, the researcher needed the instrument. In this study, the researcher obtained the data by collecting documents and conducting interview. In short, the process of collecting the data is specified below.
Table 3.2 Techniques for Collecting Data Techniques
Research Questions
Documentation Interview
RQ 1: How do student teachers’ assessment instruments fulfill the requirement of the standardization of cognitive or knowledge competence assessment?
RQ 2: What are the causing factors that make student teacher unable to fulfill the indicator of the standardization assessment?
1. Documentation
Documentation is a wide range of written materials which can produce qualitative information. They can be particularly useful in trying to understand the philosophy of an organization as may be required in action research and case studies including policy documents, mission, annual, reports, minutes or meetings, codes of conduct, etc.8 It is used to answer the first research questionwhich need the assessment instrument document. The documents needed of this research are obtained from student teachers.
8
50
2. Interview
Since the researcher needs the information from the student teachers’ perception regarding with the causing factor of their weakness point, interview is chosen as the data collection technique. Interview is discussions, usually one-on-one between an interviewer and an individual; it is used to gather information on a specific set of topics.9 It is used to answer the second research question. In this case, unstructured interview is applied. After getting the analysis of the assessment instrument, there will be the strength and weakness, here the researcher is going to find the causing factor of the weakness in assessment instrument design. Every student teacher might have different weakness from their work, so the question of the interview will be formed from the weakness point of their assessment instrument.
F. Research Instrument
Research instrument is the device used by investigators for analyzing data.10 Here the researcher attempted to measure the variable and the items of the research through rubrics and interview guidelines.
1. Rubrics
Rubric which was used to analyze student teachers’ assessment instrument was adapted from PenilaianAutentik book by Kunandar and
9
GroupsMargaret, C. Harrell, Data Collection MethodsSemi-StructureInterviews and Focus (National Research Defense Institute, 2009), 6.
