i
DESIGNING A SET OF WEB-BASED MULTIPLE-CHOICE READING
TEST FOR THE GRADE XII STUDENTS OF SENIOR HIGH SCHOOL
IN KECAMATAN CIBINONG
A THESIS
Presented as Partial Fulfillment of the Requirements to Obtain the Sarjana Pendidikan Degree
in English Language Education
By
Antonia Fitri Lestari Student Number: 051214147
ENGLISH LANGUAGE EDUCATION STUDY PROGRAM DEPARTMENT OF LANGUAGE AND ARTS EDUCATION FACULTY OF TEACHERS TRAINING AND EDUCATION
SANATA DHARMA UNIVERSITY YOGYAKARTA
iv
vii
ACKNOWLEDGEMENTS
My endless gratitude goes to The Almighty God for His love, blessing, and opportunities given to me since I started writing this thesis. When I was happy and sad, He always accompanies me. As I believe, God designs my life and keeps me saved in His protection.
I would like to express my greatest appreciation to my major sponsor, Gregorius Punto Aji, S.Pd., M.Hum., for always guiding, encouraging, and supporting me patiently in finishing my thesis through his suggestion, guidance, feedback, and criticism. I am also faithfully thankful to all lecturers of English Language Education Study Program who have taught and guided me in passing my study time in PBI and all staffs in PBI who help me in administrative needs.
viii
My greatest gratitude goes to my family. I express my grateful to my parents, Lucia Mujiyanti, S.Pd. and Antonius Sartana, S.Pd, and my grandparents Mbah Minarjo, Mbah Paikem, and Mbah Fatimah (†) for their greatest patient, support, and prayers. I apologize for not being on schedule in finishing my study. Then, I also thank my sisters, Agustina Fitriana, Sesilia Windy Gloriana, and Clarisa Primaviolinita, for giving me a lot of affections, for making me laugh, and for giving me support in every ways.
From the bottom of my heart, I would like to express my gratitude to my best classmate friends RA. Triastuti Dian K., Yohanna Kurnita P., Dionisia Novianti, Galuh Dwi Ajeng, Eska Dwi Palupi, Steffani Ratna K., Nancy Paula H., Delfina Dae, Pascalia Yovita, Taufik H. and all of PBI ’05 students
for their cherished friendship. Special thanks are also addressed to my cyber tutor who helps me in accomplishing this website, Alexander Cahya Kurniawan.
For what is called comradely, I sincerely thank to all my boardinghouse friends, Trias, Astre, Riska, Ana, Puput, Nana, Eni, Ka Achied, Eli, Lucy, Vita and Regina. I thank them for every laugh and happiness in our lovely time.
My special gratitude goes to my dearest one, Bonaventura Trias Y.P. for his never-ending love, support, and motivation. He taught me to overcome my fear and protected me when I was down. God bless you, always!
Finally, I thank everyone in my life whom I cannot mention all their names for the lovely experience that I felt together with you all. God bless you all!
ix
ABSTRACT
Lestari, Antonia Fitri. 2011. Designing a Set of Web-based Multiple-choice Reading Test for the Grade XII Students of Senior High School in Kecamatan Cibinong. Yogyakarta: English Language Education Study Program, Sanata Dharma University.
Reading is one of the essential skills for success in all educational contexts because it has function to convey information, to entertain, and to codify. Reading skill may facilitate the learners to master the language. It is because reading integrates the text and the readers’ background knowledge to build meaning. Hence, reading is often implemented and refers to be dominant in a language test. In this study, the writer considered using web-based test in order to create a different way in conducting test. Besides, the web-based test can be accessed by many students without limitation of time and place.
This study is aimed to design a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong. The writer intended to answer a problem: What is the appropriate design of a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong?
Dealing with the design, the writer applied theories related to theory of Educational Research and Development (R&D), Instructional Design Model, Language Testing, Reading Skill, Designing Multiple-choice Items, Web-based Test, and Application Software. Moreover, the design was performed step by step according to the R&D cycle combined with Kemp’s model. They are (1) Learners’ Characteristics and Needs, (2) Goal, General Purposes, and Topics, (3) Learning Objectives, (4) Test Content, (5) Activities and Sources, (6) Support Services, (7) Feedback of the Preliminary Designed Test, (8) Revision of Preliminary Designed Test, (9) Feedback from the Implementation, and (10) Final Test Revision.
The appropriate design in the form of web-based multiple-choice reading test is based on data obtained from a research. The writer developed the website address at www.tryoutbahasainggris.co.nr. The writer utilized application software such as Adobe® Flash® and Wondershare QuizCreatorv 3.2 serial. The website includes seven main pages. They are index.html, panduan.html, tentang.html, tipea.html, tipeb.html, tipec.html and link.html. The designed test contains of four topics: narrative, explanation, discussion, and review texts. The test content is divided into two groups: reading test and grammatical test. For the activities, the writer implemented three tasks classified into reading comprehension passage, multiple-choice grammar task, and multiple-choice cloze vocabulary task. Each test consists of 35 questions, 5 options and 100 minutes time constraint. The result of the implementation of designed test showed that the average mean of the whole data was 3.77 fromthe scale 5.0.
x
ABSTRAK
Lestari, Antonia Fitri. 2011. Designing a Set of Web-based Multiple-choice Reading Test for the Grade XII Students of Senior High School in Kecamatan Cibinong. Yogyakarta: Program Studi Pendidikan Bahasa Inggris, Universitas Sanata Dharma.
Membaca merupakan salah satu keahlian yang bagi keberhasilan dalam semua konteks pendidikan karena memiliki fungsi untuk menyampaikan informasi, untuk menghibur, dan untuk menyusun kata-kata. Keterampilan membaca dapat memfasilitasi peserta didik untuk menguasai bahasa. Hal ini dikarenakan membaca menggabungkan teks dengan latar belakang pengetahuan pembaca untuk membangun makna. Oleh karena itu, membaca seringkali dilaksanakan dan menjadi acuan utama dalam sebuah test bahasa. Dalam penelitian ini, penulis mempertimbangkan untuk menggunakan tes berbasis web untuk menciptakan cara yang berbeda dalam melakukan tes. Selain itu, tes berbasis web dapat diakses oleh siswa tanpa batasan waktu dan tempat.
Penelitian ini bertujuan untuk merancang satu set multiple-choice reading tes berbasis web untuk siswa kelas XII SMA di Kecamatan Cibinong. Penulis bermaksud untuk menjawab suatu permasalahan, yaitu: seperti apa rancangan satu set multiple-choice reading tes berbasis web yang sesuai untuk siswa kelas XII SMA di Kecamatan Cibinong.
Berhubungan dengan rancangan tersebut, penulis menggunakan teori-teori yang berkaitan dengan metode Penelitian dan Pengembangan (R&D), Instructional Design Model, Language Testing, Reading Skill, Designing Multiple-choice Items, Web-based Test, dan Aplikasi Software. Selain itu, desain menggunakan langkah-langkah sesuai dengan siklus R&D dikombinasikan dengan model design Instruksi Kemp. Langkah-langkah yang digunakan yaitu (1) Karakteristik dan kebutuhan siswa, (2) Tujuan umum dan topik, (3) Tujuan pembelajaran, (4) Konten uji, (5) Kegiatan dan sumber, (6) Layanan dukungan, (7) Komentar dari rancangan awal tes, (8) Revisi rancangan awal tes, (9) Komentar dari implementasi tes, dan (10) Revisi dari final tes.
Pembuatan tes didasarkan pada data yang diperoleh dari penelitian. Penulis mengembangkan website dengan alamat www.tryoutbahasainggris.co.nr. menggunakan aplikasi Adobe® Flash ® dan Wondershare QuizCreatorv 3.2. Situs ini mencakup 7 laman, yaitu: index.html, panduan.html, tentang.html, tipea.html, tipeb.html, tipec.html dan link.html. Rancangan tes terdiri dari 4 topik yaitu narrative, explanation, discussion, dan review. Isi tes ini dibagi menjadi dua yaitu reading tes dan gramatikal tes. Untuk kegiatan dalam tes, penulis menerapkan tiga jenis tugas yaitu reading comprehension passage, multiple-choice grammar task, and multiple-choice cloze vocabulary task. Setiap tes terdiri dari 35 pertanyaan, 5 pilihan jawaban dan waktu pengerjaan 100 menit. Hasil pelaksanaan tes menunjukkan bahwa nilai rata-rata dari seluruh data adalah 3,77 dari skala 5,0.
xi
TABLE OF CONTENTS
Page
TITLE PAGE ... i
APPROVAL PAGES ... ii
STATEMENT OF WORK'S ORIGINALITY ... iv
LEMBAR PERNYATAAN PERSETUJUAN PUBLIKASI ... v
PAGE OF DEDICATION... vi
ACKNOWLEDGEMENTS ... vii
ABSTRACT ... ix
ABSTRAK ... x
TABLE OF CONTENTS ... xi
LIST OF TABLES ... xv
LIST OF FIGURES ... xvii
CHAPTER I : INTRODUCTION ... 1
A. Research Background... 1
B. Problem Formulation ... 5
C. Problem Limitation ... 6
D. Research Objectives ... 7
E. Research Benefits ... 7
F. Definition of Terms ... 8
CHAPTER II : REVIEW OF RELATED LITERATURE ... 12
A. Theoretical Description ... 12
1. Educational Research and Development ... 12
2. Instructional Design Model ... 16
3. Language Testing ... 20
a. Types of Language Test ... 21
b. Test Qualities ... 23
c. Designing Classroom Language Test ... 25
4. Reading Skill ... 27
xii
b. Reading Test Types ... 28
c. Reading Assessment ... 30
5. Designing Multiple-choice Items ... 34
6. Web-based Test ... 37
a. The Advantage of Web-based Test ... 37
b. The Role of Computer and Teacher in Web-based Test ... 38
c. Website Evaluation ... 39
7. Application Software ... 41
a. Adobe® Flash® ... 41
b. Wondershare QuizCreator 3.2 ... 41
B. Theoretical Framework ... 43
CHAPTER III : METHODOLOGY ... 47
A. Research Method ... 47
1. Research and Information Collecting ... 48
2. Preliminary Form of Product Development ... 48
3. Preliminary Product Verification ... 49
4. Preliminary Product Revision ... 49
5. Main Field Testing ... 50
6. Final Product Revision ... 50
B. Research Participants ... 51
1. The Participant in Research and Information Colleting Data .... 52
2. The Participant in Preliminary Product Verification ... 52
3. The Participant in Main Field Testing ... 52
C. Research Instruments ... 53
1. The Instrument for Research and Information Colleting Data .. 53
2. The Instrument for Preliminary Product Verification ... 54
3. The Instrument for Main Field Testing ... 54
D. Type of Data and Data Gathering Techniques ... 55
E. Data Analysis Techniques ... 59
xiii
2. Analysis Techniques in Preliminary Product Verification ... 60
3. Analysis Techniques in Main Field Testing ... 62
F. Research Procedure ... 64
CHAPTER IV : RESEARCH RESULT AND DISCUSSION ... 66
A. The Appropriate Design of a Set of Web-based Multiple-choice Reading Test for Grade XII Students of Senior High School in Kecamatan Cibinong ... 66
1. Learners’ Characteristics and Needs ... 67
2. Goal, General Purposes, and Topics ... 78
3. Learning Objectives ... 80
4. Test Content ... 82
5. Activities and Sources ... 84
6. Support Services ... 86
7. Feedback of Preliminary Designed Test ... 95
8. Revision of Preliminary Designed Test ... 99
9. Feedback from the Implementation ... 109
10.Final Test Revision ... 122
B. The Final Version of the Designed Test ... 125
CHAPTER V : CONCLUSIONS AND SUGGESTIONS ... 126
A. Conclusions ... 126
B. Suggestions ... 128
REFERENCES ... 130
APPENDICES Appendix A: Letters ... 133
1. Letter of Permission to the Headmaster of SMA Mardi Waluya ... 134
2. Letter of Permission to the Headmaster of SMA Eka Wijaya ... 135
3. Letter of Permission from SMA Eka Wijaya ... 136
Appendix B: Instruments ... 137
1. Questions List for Interviewing Teachers and Students ... 138
a. Transcripts of Interview with Teachers ... 140
xiv
2. Questionnaire for Feedback of Preliminary Designed Test
a. Sample of Filled-questionnaire ... 153
3. Questionnaire for Implementation a. Sample of Filled-questionnaire ... 163
4. Test for the Implementation a. Sample of Filled-test ... 169
b. Results of Open-form Questions from Students’ questionnaire 171 c. Results of Students’ Conducting Tipe B test ... 178
Appendix C: A Set of Web-based Multiple-choice Reading Test for the Grade XII Students of Senior High School in Kecamatan Cibinong ... 181
1. Printed Tipe A test and Explanation ... 182
2. Printed Tipe B test and Explanation ... 196
3. Printed Tipe C test and Explanation ... 210
4. Website Address ... 223
xv
LIST OF TABLES
Page
Table 3.1: Description Table of the Experts ... 52
Table 3.2: Description Table of the Students ... 53
Table 3.3: Type of Data ... 56
Table 3.4: The Interpretation of the Degree of Agreement ... 61
Table 3.5: The Descriptive Statistics and Transcripts of Respondents’ Opinions... 61
Table 3.6: The Data Collection from Students’ Answer Sheets ... 62
Table 3.7: The Data Collection on Item Facility (IF) ... 63
Table 3.8: The Data Collection on Item Discrimination (ID) ... 63
Table 3.9: The Data Collection on B-Index ... 64
Table 4.1: Summary of Learners’ Characteristics ... 77
Table 4.2: Summary of Learners’ Needs ... 77
Table 4.3: The Goal and General Purposes of the Study ... 79
Table 4.4: The List of the Topics ... 79
Table 4.5: Learning Objectives of the Designed Tests ... 80
Table 4.6: Test Content Description of the Designed Tests... 83
Table 4.7: Task Content Description of the Designed Tests ... 85
xvi
Table 4.9: Descriptive Statistics of the Participants’ Opinion ... 96
Table 4.10: Revision on the Goal and General Purposes ... 100
Table 4.11: Revision on the Indicators of Learning Objectives ... 101
Table 4.12: Revision on the Texts and Test Content Description... 104
Table 4.13: Revision on the Task Content Description ... 107
Table 4.14: The Description of Participants in the Implementations... 111
Table 4.15: The Description Statistics of the Close-Form Questions On the Implementation ... 112
Table 4.16: The Item Facility (IF) Results of Tipe B ... 117
Table 4.17: The Item Discrimination (ID) Results of Tipe B ... 119
Table 4.18: The B-Index Result of Tipe B ... 120
xvii
LIST OF FIGURES
Page
Figure 2.1: Kemp’s Instructional Design Model ... 20
Figure 3.1: The adapted R&D cycle combined with adapted Kemp’s Model ... 51
Figure 4.1: Flowchart of the website ... 88
Figure 4.2: Home page display ... 90
Figure 4.3: Tipe A preview using Wondershare QuizCreator 3.2 serial ... 92
Figure 4.4: The mechanism on the use of media ... 94
1 CHAPTER I
INTRODUCTION
This chapter is divided into six important parts. They are research background, problem formulation, problem limitation, research objectives, research benefits, and definition of terms.
A. Research Background
As an international language, English is important to communicate and to share information toward people from different countries. Since English becomes a bridge to succeed for any work field, Indonesian government decided to include English language as the compulsory subject in the curriculum within its Competence Standard and Basic Competence (based on the Decree of the Minister of National Education No. 22 Year 2006). In accordance with the School-based Curriculum or Kurikulum Tingkat Satuan Pendidikan (KTSP), teachers select appropriate materials based on Competence Standard (CS) and Basic Competence (BC) guided in Permen No. 22 Tahun 2006.
dominant in the English UN. The focus in reading test consists of text content, rhetoric, structure, and vocabulary.
Moreover, reading is argued as the most essential skill for success in all educational contexts (Brown, 2004: 185). Reading has its function to convey information, to amuse and entertain, to codify, etc. In line with Thompson (1987: 50) reading is an activity integrating the text and readers’ background knowledge to build meaning. It means that readers require other element to find the meaning of the test including their knowledge about grammar, vocabulary, spelling, eye movement, and intellectual comprehension. However, reading skill mastery may facilitate the learners to master the language.
To accomplish the study, the writer focuses on designing a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong. The reason for conducting the test in senior high school in Kecamatan Cibinong is because students in Kecamatan Cibinong are familiar with computer technology and the Internet, while some schools also provide those facilities to support the students’ need. In fact, the use of computer and the Internet have not been utilized effectively for educational purposes. Based on informal interview with SMA Eka Wijaya teachers, it was found that computer and Internet facilitation are used only for computer course. Besides, English teacher have not explored the use of computer and Internet facility to support English subject.
3
for Grade XII students. Moreover, Grade XII students require various tests to measure their readiness in facing final examination. The web-based test can facilitate students in providing online test which can be assessed anytime without limitation of time and place. The design may create different atmosphere, out of conventional ways, in conducting test. The online process requires connection to internet service provider with a certain address of the website, but students can download the test and print it.
According to Dudeney (2007: 7), the use of technology in the classroom is becoming increasingly important, and it will become a normal part of English language teaching practice in coming year. The writer applies web-based language testing to motivate learners in the use of technology in education. As accessed from Lau Sie Hoe in Web-based Computer-Adaptive Multiple-choice Assessment, computer-based assessment has become increasingly attractive way in educational field. It is proved by the increasing number of students who obtain on-line test via internet because computer-based assessment can reduce cost, and improve the quality and accountability of the assessment process.
computers. As the result, website is only a part of a huge data of the Internet. By using the Internet for conducting the test, students gain benefit in which they can practice everywhere and every time. As accessed on Kurnia Nusa’s in Web-based Distance Learning, students will not be limited by time and space to study; what they need is a computer or a notebook that has connected with the Internet. The test can be accessed easily via Internet that might content test work-sheet display, answer keys display, the score and feedback column. By conducting web-based test, the students do not need the presence of the teacher to provide the test and to assign feedback because one promising aspects of web-based assessment is instant feedback (Peat, 2003:19).
In contrary, computer-based assessment is infrequently conducted in Indonesia especially for senior high school level. Yet, most of Grade XII students of senior high school require some test references to prepare final exam. By conducting a web-based test, the students can do the test outside the class by using Internet facilities from their house or Internet cafés. Besides, the Internet offers chances for students to be familiar with ICT.
5
and feedback, it reduces content misconceptions, reasoning errors, and learning difficulties which often face by students after conducting test (Paul et al, 1994).
The writer focuses on reading test because it is commonly used to evaluate students understanding about language that includes comprehension text, grammar, and vocabulary understanding. Besides, reading takes place among other skills. Wallace (2003: 3) also stated that reading as one of the skills plays an important role in English language. It is because reading is the awareness of the way we used language. Besides, reading is considered difficult by many learners because of all the components and linguistic features, that is why people have difficulties in reading because they have to master all the language components and linguistics features as the same time before they can master the reading skills.
B. Problem Formulation
Dealing with designing a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong, the writer intended to answer a problem: What is the appropriate design of a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong?
C. Problem Limitation
The writer determined some limitations in order to make constraints of this study. First, the study focused on the developing application using software in the form of multiple-choice reading test. The writer decided to design web-based test by using Adobe® Flash® and Wondershare QuizCreatorv 3.2 program. Second, the study employed reading exercises to measure student’s comprehension on the text using web-based test. Third, the study was limited to particular reading materials addressed in School-based Curriculum (KTSP). The tests were based on reading materials which are appropriate with School-based Curriculum, Competence Standard, and Basic Competence.
7
grammar form. For the macroskills, the test includes detecting main idea, supporting idea, information, generalization, and exemplification; inferring links and connection between the information; distinguishing between literal and implied meanings; and developing reading strategies for interpretation of texts. Each test consists of 35 questions including 5 options which contain of one answer and four distractors.
Here, the subject of the study focused only for Grade XII students of senior high school in Kecamatan Cibinong. The writer decided this level because they had already understood the language components and linguistic features, they are able to read an English text, and thus they can understand direction in order to operate the computer. On the other hand, the design can also be accessed as an exercise for preparing the National Examination because it covers some materials for the national examination such as narrative, explanation, and discussion text. Moreover, the designed test provides some questions that are generally applied in national examination such as reading comprehension questions (main idea, supporting idea, word/ phrase meaning, and true/ false statement) and grammatical questions (connectors and conditional sentence).
D. Research Objectives
school in Kecamatan Cibinong. In this study, the writer would like to design and to present the design.
E. Research Benefits
This study was expected to bring about some contributions for students, the teachers and also the other test designers as follows:
1. For students
This study may help students to prepare the national examination especially in reading test. Then, the tests may enhance students’ knowledge of grammar, vocabulary, and reading comprehension. Besides, the students are able to practice the tests without the presence of teacher and students obtain the result immediately. While students conduct the web-based test, they learn a modern type of test and can be familiar with ICT.
2. For teachers
This study may facilitate teachers in conducting different ways of giving a test by using computerized test. Moreover, the teachers can apply the design as a media that carries ICT for teaching English.
3. For other test designers
9
F. Definition of Terms
In this part, the writer provided the definition of each term so that the readers have the same view and it may reduce misunderstanding.
1. Designing
Designing is defined as drawing plans or planning a system to decide how something will look or work. According to Briggs (et al, 1977), designing is a process of analysis or learning need and goal and the development of a delivery system to meet the needs; includes development of instructional materials and activities; and try out and revision of all instruction and learner assessment activities. The design must be planned and it is developed for specific purposes. In this study, designing is preparing a plan for assessing students’ reading ability in the form of multiple-choice test using computer. The design in this study is a set of web-based multiple-choice reading test. 2. Web-based Test
According to Roever (2001: 84), Web-based Test (WBT) is a Computer-based Language Test which is delivered via the World Wide Web (www). The term Web-based Testing is defined as a small-scale “home-grown” test available on website (Brown, 2004: 14). WBT can be offered any time unlike mass paper and pencil administrations. Moreover, it also provides immediate feedback for the test takers.
In this study web-based test means that the test is provided in the website and conducted using computer as testing medium which is connected via Internet. 3. Multiple-choice
Multiple-choice items are a form of assessment item in which respondents are asked to select one or more of the choices from a list. Multiple-choice item has options where one of those options is a key or correct response and the others are distractors (Brown, 2004:56). In this study, the test-takers select one of five options as a correct answer. The multiple-choice tests focus on formal aspects of language (lexical, grammatical, and few discourse features).
4. Reading Test
11
5. Grade XII Students of Senior High School
Grade XII students of senior high school are the third-year level students in senior high school. Generally, the students have already achieved all materials in senior high level because they are prepared for national examination. Grade XII students often review and tryout materials for the preparation of national examination to drill their knowledge and skill. Consequently, they are able to understand and accustomed to analyze questions in the form of multiple-choice test.
6. Kecamatan Cibinong
Kecamatan Cibinong is a sub-district which becomes the central of governmental activity in Bogor Regency (Kabupaten Bogor), West Java. There are six senior high schools in Kecamatan Cibinong, three state schools and three private schools. They are SMAN 1 Cibinong, SMAN 2 Cibinong, SMAN 3 Cibinong, SMA Mardi Waluya Cibinong, SMA Eka Wijaya Cibinong, and SMA PGRI 1 Cibinong. This study was conducted for students of SMA Eka Wijaya Cibinong, SMA Mardi Waluya Cibinong, and SMAN 1 Cibinong because those schools are favorite schools in Kecamatan Cibinong. Annually, those schools achieved higher range of score of national examination result in Kecamatan Cibinong. Besides, the locations of those schools are nearby the public centre in Kecamatan Cibinong.
12 CHAPTER II
REVIEW OF RELATED LITERATURE
This chapter deals with the discussion on the related literatures to support the study. The discussion includes two main subtopics. They are theoretical description and theoretical framework.
A. Theoretical Description
In this part, the writer concerns with the theories which are used to the accomplishment of this study. This part is divided into seven parts. They are the theory of Educational Research and Development (R&D), Instructional Design Model, Language Testing, Reading Skill, Designing Multiple-choice Items, Web-based Test, and Application Software.
1. Educational Research and Development (R&D)
13
and using the result to build test products. In Educational Research and Development, the ten steps used to develop mini-courses are described as follows: a. Research and Information Collecting
This step includes review of literature, classroom observations, and preparation of report of state of the art (Borg and Gall, 1983: 775). Literature review is conducted to collect information that sustain on the design planning. To support the information, classroom observation is conducted to gain information about classroom activity and situation.
b. Planning
It includes defining skills, starting objectives, determining course sequence, and small scale feasibility testing (Borg and Gall, 1983: 775). The focus on this step is determining the skill being improved by considering the objectives of the study that have to be achieved by the product.
c. Developing Preliminary Form of Product
This step includes preparation of instructional materials, handbooks, and evaluation devices (Borg and Gall, 1983: 775). Questionnaire and interview list, as evaluation devices, are designed to obtain information and feedback of the preliminary form of the product. According to Borg and Gall (1983: 781), the procedure of developing the product will differ depending on the nature of the product.
d. Preliminary Field Testing
questionnaire then analyzes the data collected (Borg and Gall, 1983: 775). This step is suggested to test the feasibility of the product before it is implemented. e. Main Product Revision
In this step, the designer revises the product as suggested by the Preliminary Field Test results (Borg and Gall, 1983: 775). The data obtained from interview and questionnaire, are analyzed to improve the product.
f. Main Field Testing
In this step, the revised product is conducted in 5 to 15 schools with 30 to 100 subjects. The data on subjects’ pre-course and post-course performance are collected and analyzed in the form of qualitative data. Then, the results are evaluated with respect to course objectives and are compared with control group data, when appropriate (Borg and Gall, 1983: 775).
g. Operational Product Revision
The designer revises the product as suggested by Main Field Test results (Borg and Gall, 1983: 775).
h. Operational Field Testing
15
i. Final Product Revision
This step is the final revision of the product as suggested by Operational Field Test result. The final version for the complete package of product is carried out after the data from the operational field test have been analyzed (Borg and Gall, 1983: 786).
j. Dissemination and Implementation
This last step is the technique to adjust the cost of the product developing process by operating effective dissemination and implementation the product to intended users (Borg and Gall, 1983: 786). The designer reports on product at professional meetings and in journals; works with publisher who assumes commercial distribution; and monitors distribution to provide quality control.
In general, the Education Research and Design (R&D) cycle consists of studying research findings pertinent to the product to be developed, developing the product based on those findings, field testing it in the setting where it will be eventually, and revising it to correct the deficiencies found in the field stage.
The criteria for product selection used include the following:
1) Does the proposed product meet an important educational need?
2) Is the state of the art sufficiently advanced so that there is a reasonable probability that a successful product can be built?
3) Are personnel available who have the skills, knowledge, and experience necessary to build this product?
4) Can the product be developed within a reasonable time?
This planning provides the foundation upon which later revisions are built.
2. Instructional Design Model
In this study, the writer adopts Kemp’s instructional design model to design a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong. The writer decided to adopt Kemp’s model because it has a flexible process and can be applied in any educational level. According to Kemp (1977:6) the method can be applied on any educational level: elementary, secondary, or college. In Kemp’s book, he states that the instructional design plan is designed to supply answers to three questions, which may be considered as the essential element of instructional technology (Kemp, 1977:8):
a) What must be learned? (Objectives)
17
c) How will we know when the required learning has taken place? (Evaluation)
In order to answer those questions, Kemp forms an instructional material design which consists of eight steps (1977: 13-100) that can be applied to individual topics and then to units and then to complete courses, involving one or few teachers. The eight steps are described as follows:
Step 1: Goal, Topics, and General Purposes
Kemp’s model starts with recognition of the broad goals of teaching learning process. Those goals may be derived from three sources: society, students, and subject areas. The next step is selecting the topics for teaching learning process. Those topics will become the scope of the course program, the basis for the instruction. Then, the next is listing general purposes of teaching learning process.
Step 2: Learner Characteristics
It deals with information about the learner’s capabilities, needs, and interests. According to Kemp (1977: 18-21), there are some factors affecting learner characteristics. They are academic factors, social factors, learning conditions and learning style.
Step 3: Learning Objectives
in this study is a multiple-choice reading test, the writer categorizes the objectives into cognitive group which includes objectives concerning knowledge, comprehension, application, analysis, synthesis, and evaluation (Kemp, 1977: 25). Step 4: List the Subject Content
According to Kemp (1977:43), student’s learning experiences must involve subject content. The content should close-relationship to the objectives and to the student’s needs. In this designed test, the writer listed test contents in line with the learning objectives. However, the test-materials adjust Competence Standard and Basic Competence.
Step 5: Pre-Assessment
The goal of this step is to gain information whether the student prepared to study the topic or unit and the student already competent in some of the stated objectives or not (Kemp, 1977: 50). Pre-assessment is divided into two: prerequisite testing and pretesting. Prerequisite test determines whether students have appropriate background preparation for the topic. While pretesting determines which of the objectives that students may already have achieved. In contrary, this study does not conduct pre-assessment to measure students’ preparation, but it conducted the product verification to measure the feasibility of the design from the experts.
Step 6: Teaching Learning Activities and Resources
19
the materials to provide learning experiences that will utilize the content associated with the objectives. The writer skips teaching learning activities because the focus is conducting test. Hence, the writer determined the activity as implementing the web-based test to the students. For the activities of the test, the writer applied three tasks classified into reading comprehension passage, multiple-choice grammar task, and multiple-multiple-choice cloze vocabulary task.
Step 7: Support Services
Teachers need to coordinate the service including funds, facilities, equipments, time, and schedule. In this study, the writer designed a web-based test which required support services such as web-based test program, computer laboratory and the Internet facility.
Step 8: Evaluation
Figure 2.1: Kemp’s Instructional Design Model (Source: Kemp, 1977:9)
3. Language Testing
In teaching, there are three components that are closely related. They are objectives, teaching activities, and testing. Their relation is shown by the effect of one component to the others. Tests are part of teaching activities where in the teaching process teachers need to observe students’ progresses in learning the materials by giving them some assessment (Brown, 2004: 4). Tests are a subset of an assessment. According to Brown (2004: 3), a test, in the simple form, is a method of measuring a person’s ability, knowledge, or performance in a given
Evaluation Learners’
charac- teristics
Learning objectives
Subject content
Pre-assessment Teaching
learning activities & sources Support
services
Revision
Goal, topics, &21
domain. First, the method must be explicit and structured to qualify the test. Second, a test should measure general ability, competencies, or objectives. Third, a test measures an individual’s ability, knowledge, or performance. It means that the test could measure student’s personal ability, and then the teacher will know who the students are. Finally, a test measures a domain which means the test shows students’ particular level in the class.
According to Bachman and Palmer (1997: 4), a model of tests on the large scale English Foreign Language tests usually include sections testing grammar, vocabulary, reading, and listening comprehension. These tests measure language ability as a set of components which are grammar, vocabulary, pronunciation, and spelling. Besides to evaluate student’s language ability, language tests can also be used as a tool for clarifying instructional objectives, for evaluating the relevance of the objectives, the materials, and the activities of students following program (Bachman & Palmer, 1997: 4).
a. Types of Language Tests
decision, such as diagnostic and achievement decisions. CRT measures specific objectives based on language points.
Test can be distinguished through some criterions. Based on the procedure, the test can be distinguished as formal test and informal test. According to Brown (2004: 5), informal test can take a number of forms, starting with incidental, unplanned comments and responses, along with coaching and other impromptu feedback to the students. On the contrary, formal tests are exercises or produces specifically designed to tap into a storehouse of skills and knowledge (Brown, 2004: 6). The tests are systematic, planned sampling techniques constructed to give teacher and student an appraisal of student achievement.
Then, test can be characterized as formative and summative test. Formative test evaluates students in the process of forming their competencies and skills with the goal of helping them to continue that growth process (Brown, 2004: 6). The opposite, summative assessment measures and summarizes what a student has grasped, and typically occurs at the end of a course or unit of instruction (Brown, 2004: 6).
23
1979). In accordance with the above explanation, this study can be classified as a criterion-referenced test, formal test, summative test, and discrete-point test. b. Test Qualities
Then, to know if the test is effective and useful there are some test qualities according to Brown (2004: 19) such as practicality, reliability, validity, authenticity, and washback. Further, Bachman and Palmer (1997:17) added some qualities such as interactiveness, and impact. Those qualities are explained as follows:
1) Practicality
It means that the test is not expensive, stays within appropriate time constraints, easy to administer, and easy to score. As a consideration, Bachman and Palmer (1997: 36) determined the practicality based on the consideration of the resources that would be required in the test had the balance qualities the teachers want, and the allocation and the management of the resources that were available.
2) Reliability
difference to a particular test taker whether the test taker takes the test on different occasion and setting.
3) Validity
Validity means that the tests are appropriate, meaningful, and useful in terms of the purpose of the test (Gronlund, 1998: 226). It means that the test really measures what teacher wants to measure, for example: a valid reading test will measure student’s reading ability by asking the students to read and by answering comprehension questions. Moreover, the test designer has to consider the content related evidence, criterion related evidence, construct related evidence, consequential validity, and face validity (Brown, 2004:26-27). The content related evidence means that the content of the test is suitable with the test, for example: if teacher tests speaking, she should give a test in which lets the students to speak. The criterion related evidence is usually related with the results that are supported by the other concurrent performance, and the prediction the test taker’s likelihood of future success. The construct related evidence related with any theory, hypothesis, or model that attempts to indicate the area of language ability teachers want to measure. The consequential validity refers to all the consequences of a test. The face validity refers to the test appearance.
4) Authenticity
25
contextualized items, relevant and interesting topics for the learners, thematic organization items, and real-world task.
5) Interactiveness
Bachman and Palmer (1997: 25) defined interactiveness as the degree and type of involvement of the test taker’s individual characteristics in accomplishing a test task. The interactiveness means as a quality of any tasks (target language use tasks and test tasks) can potentially vary in test taker interactiveness.
6) Washback or Impact
Washback or impact is the effects of the test for students. It means that a variety of individual will be affected by and thus have an interest, or hold a ‘stake’, in the use of a given test in particular situation Bachman and Palmer (et. al, 2004).
c. Designing Classroom Language Tests
1) Aptitude Test
A language aptitude test is designed to measure capacity or general ability to learn a foreign language and ultimate success in that undertaking (Brown, 2004: 43).
2) Proficiency Test
A proficiency test measures overall ability. It has traditionally consisted of standardized multiple-choice items, grammar, vocabulary, reading comprehension, and aural comprehension (Brown, 2004: 44). The tests are almost summative and norm-referenced. The example of this test is Test of English as a Foreign Language (TOEFL®).
3) Placement Test
The test has a purpose to place students into a particular level or section of language curriculum or school (Brown, 2004: 45). The test usually includes a sampling of the material to be covered in the various courses in a curriculum.
4) Diagnostic Test
A diagnostic test is designed to diagnose specified aspects of a language (Brown, 2004: 46). For example, pronunciation test might diagnose the phonological features.
5) Achievement Test
27
is categorized as achievement test, because the test is related directly to curriculum.
4. Reading Skill
This part provides explanation about reading. It includes the nature of reading, reading text types, and reading assessment.
a. Nature of Reading
In teaching language, reading is a skill that teachers expect learners to acquire. It is because reading includes all components of the language such as vocabulary, grammar, and even the pronunciation (Brown, 2004:185). Reading is an activity integrating the text and readers’ background knowledge to build meaning (Thompson, 1987: 50). Moreover, reading is a process of understanding the meaning of written or printed words or symbols. From the definition above, reading is an active process of getting meaning from written text in order to understand and comprehend written form of language.
b. Reading Text Types
There are some reading text types taught by students of senior high. They are analytical exposition, anecdote, description, narration, procedure, news item, discussion, explanation, report, recount, review, and hortatory text. Further, Competence Standard and Basic Competence in KTSP mention the goals of study which influence reading text types for English subject. The CS for reading skill for the Grade XII students includes functional texts, essay, and monolog of narrative, explanation, discussion, and review text in daily context for accessing science or knowledge. This study only presents four text types based on competence standard, as follows:
1) Narrative
It has a purpose to entertain, to amuse or to tell the readers about the fiction or non-fiction experience or story that has a problem and the resolution. The generic structure of this type is:
orientation: introduction about the characters, places, and times,
complication: development of conflicts or problems, and
resolution: the reconciliation of the problems.
The dominant language features are using past tense, using action verbs,
chronologically arranged, using conjunctions (then, before, after, soon, etc), and using direct and indirect sentences.
2)Explanation
29
The generic structure of this type includes:
general statement: general explanation,
explanation: a sequenced explanation of why or how something occurs,
and
closing.
Moreover, explanation text has some dominant language features such as using Simple Present Tense, using action verbs, using passive voice, using complex sentences, using noun phrases (e.g. the large cloud), using adverbial phrases, using technical terms, using general and abstract noun (e.g. the temperature), and using conjunction of time and cause-effect.
3)Discussion
Discussion text is aimed to present an issue overseen by two point of views (pros and cons) before it ends on certain conclusions or recommendations. The generic structure of discussion text is presented as follows:
issue: involves the statement of issue and preview,
arguments of pros: include several main ideas and elaborations of
supporting opinions from the issue,
arguments of cons: consist of statements of various viewpoints that
oppose the issue and its elaborations, and
conclusion or recommendation.
to present information about discussed issue such as smoking is harmful, homework is necessary, etc. Discussion text conveys thinking verbs to express personal views (e.g. believe, feel, hope); involves modalities (e.g. perhaps, must, should, could be); uses adverbials of manner (e.g. deliberately, hopefully); includes detail noun groups to obtain information (e.g. the dumping of unwanted kittens); and uses additives, contrastive, and causal connectives to join the arguments (e.g. however, on the hand, in contrary).
4)Review
This type is purposed to critique or evaluate an art work or event (e.g. movies, show performance, books, etc) for public. The generic structure of this type is:
orientation: introduction of the title, characters, the author, etc.,
evaluation: critique on several point,
interpretation,
evaluation, and
evaluative summation.
The text generally focuses on specific participants. The text also uses adjectives to point out attitudes (e.g. bad, good for teenagers, interesting), long and complex clauses, and metaphor.
c. Reading Assessment
31
also strategic to full understanding including fundamental bottom-up and top-down strategies. Bottom-up strategies begins with processing separate letters, words, phrases, clauses and then sentences of the text. Bottom-up strategies helps the readers to catch and understand the meaning of information of the text. Whereas top-down strategies, driven strategies for comprehension, consists of background information, cultural experience, life experience, and background knowledge that helps the reader to carry out interpretation of the text. Moreover, the test maker should also consider a number of different types or genres of written texts, the components of reading ability, and specific task. 1) Genres and Types of Reading Assessment
According to Brown (2004: 186), there are three specifications of common genres for assessments of reading ability. They are academic reading, job-related reading, and personal reading. In this study, the writer considers academic reading. The genre of text allows readers to apply certain representation that assist in extracting appropriate meaning.
2) Microskills, Macroskills, and Strategies for Reading
The skills and strategies for accomplishing reading appear as crucial consideration in the assessment of reading ability. Brown made the specific skill more detail into micro- and macroskills based on fundamental top-down and bottom-up strategies. Brown (2004: 187) represented micro- and macroskills for considering the objectives.
Microskills for reading comprehension are represented as follows:
a) Discriminate among the distinctive graphemes and orthographic patters of English.
b) Retain chunks of language of different lengths in short-term memory. c) Process writing at an efficient rate of speed to suit the purpose.
d) Recognize a core of words, and interpret word order patterns and their significance.
e) Recognize grammatical word classes (verb, noun, etc), system (tense, agreement, pluralization, etc), patterns, rules, and elliptical forms.
f) Recognize that a particular meaning may be expressed in different grammatical forms.
g) Recognize comprehensive devices in written discourse and their role in signaling the relationship between and among clauses.
Macroskill for reading comprehension are represented as follows:
33
b) Recognize the communicative functions of written texts, according to form and purpose.
c) Infer context that is not explicit by using background knowledge.
d) Infer links and connection between the information and detect such relations as main idea, supporting idea, new information, given information, generalization, and exemplification.
e) Distinguish between literal and implied meanings.
f) Detect culturally specific references and interpret them in a context of the appropriate cultural schemata.
g) Develop and use battery of reading strategies for interpretation of texts. Furthermore, according to Brown (2004: 188), there are some principal strategies for reading comprehension as follows:
a) Identify the purpose in reading a text.
b) Apply spelling rules and conventions for bottom-up decoding. c) Use lexical analysis to determine meaning.
d) Guess at meaning when you aren’t certain. e) Skim the text for the gist and for main ideas. f) Scan the text for specific information.
g) Use silent reading techniques for rapid processing.
h) Use marginal notes, outlines, charts, or semantic maps for understanding and retaining information.
In this study, the test covers some micro- and macroskills. For the microskills, the test includes recognizing grammatical world classes, recognizing system, and recognizing particular meaning in different grammar form. Those microskills are implemented in grammatical questions. For the macroskills, the test includes considering reading comprehension strategies; detecting main idea, supporting idea, information, generalization, and exemplification; inferring links and connection between the information; distinguishing between literal and implied meanings; and developing reading strategies for interpretation of texts. Those macroskills are implemented in reading comprehension questions.
5. Designing Multiple-choice Items
Multiple-choice appears as a simple kind of item to construct, but it is difficult to design correctly. Multiple-choice test are also commonly used in the elementary, junior and high school in order to test students’ achievement toward their study especially for summative test. However, multiple-choice formats are practical and reliable (Brown, 2004: 55). The practicality to administer and to score become the reason of why multiple-choice become the favorite test used in final examination.
35
have a considerable effect on the test scores. If the student does not know the answer, they consider guessing the options. The answer can be right depending on their luck. Then, it is very difficult to write successful items because the teacher should consider distractor efficiency. The washback may be harmful for the students because the test is restricted what can be tested. Then, cheating may be facilitated since the students only mention the letter A, B, C, D, or E.
According to Brown (2004: 56), a multiple-choice format has a primer terminology. The first, multiple-choice items are all receptive or selective. It means that the test-takers can choose from a set of responses or usually called supply. The second, multiple-choice has a stem that presents a stimulus and options or alternatives to choose from. The third, one of those options is key (correct response) and the others are distractors.
To make appropriate multiple-choice items, teachers should consider guidelines for designing multiple-choice items. In the following, there are four guidelines for designing multiple-choice items (Gronlund, 1998: 60).
a. Design each items to measure a specific objectives.
In this step, the designer should consider or revise the items so that can be appropriate with the comprehension question.
b. State both stem and options as a simply and directly as possible.
c. Make certain that the intended answer is clearly the only correct one.
The designer has to eliminate the unintended possible answer so the test takers will not be confused in answering the question.
d. Use items indices to accept, discard, or revise items.
To measure whether the items are appropriate and suitable, teachers can conduct three indicators: item facility, item discrimination, and distractor analysis (Brown, 2004: 58).
In contrary, CRT development and improvement projects is quite different because the purpose of CRT is to assess how much objective has been learned by each student, CRT assessment has to occur before and after instruction in order to determine whether there was any gain in sources (Brown, 2005: 79). CRT involves administering Item Facility (IF), Difference Index (DI), and B-Index (Brown, 2005: 80). However, the process of gaining information about DI is consuming time, so the writer replaced DI with Item Discrimination (ID) to analyze the MC items.
37
6. Web-based Test
In recent years, the advancement of technology, particularly the Internet, has affected in the way of people learn. Through the Internet, students are urged to research, explore and express themselves in ways which are not possible, practical or powerful with more conventional means (Castro et al, 1999). The term web-based testing is defined as a small-scale “home-grown” test available on website (Brown, 2004: 14). Besides, the large-scale test which is standardized, involves thousand or even tens thousands of test-takers such as TOEFL® and TOEIC®. Web-based test is also known as computer-assisted or computer-based test.
a. The Advantages of Web-based Test
The use of computer in giving web-based test offers several advantages. Below, they are the advantages according to Brown (2004: 14):
1) self-directed testing on various aspects of a language (vocabulary, grammar, discourse, one or all of the four skills, etc),
2) practice for upcoming high-stakes standardized tests,
3) large-scale standardized test that can be administered easily to thousand of test takers at many different stations, then scored electronically for rapid reporting of result, and
4) it provides opportunity to students for increasing their technological knowledge (Chafe et.al, 1999).
students, WBT can increase motivation levels for educational tasks because it involves learning computer. Muehleisen (et al, 1997) also states that computer utilizing provides a strong intrinsic motivation for students as they are still relatively new and exciting. CBT has flexibility to be offered to the students. It will suit the students who are ready to take the test. Besides the advantages, CBT also has disadvantages such as lack of security and the possibility of chatting, less of human interaction especially in oral production.
b. Role of Computer and Teacher in Web-based Test
39
Teacher’s position in conducting WBT is not directly needed because teacher provides the tests which can be implemented anytime and anywhere through the Internet provider. Although WBT does not need the presence of teacher, he or she has a responsibility to be a facilitator in school by preparing the school facilitation. Means and Olson (et al, 1997) state that there is a certain amount of overhead that goes with learning to use any new technology. It seems that teachers need to acquire skills of operating product of technology in order to provide better education.
c. Website Evaluation
To develop a good website, designer requires considering some criteria. Website development should be user-centered, hence evaluating the evolving design against user requirements based on the objectives of the website design. As assessed in (http: www.usabilitynet.org), the evaluation of the website can be measured from site structure and content, support navigation, page design, management and maintenance, as follows:
1) Structure and content
The substance and information on the site should be relevant to the site and should target the area of the public that the website is concerned with. 2) Support navigation or usability
3) Page design or appearance
The graphics and text should include a single style that flows throughout, to show consistency. The style should be professional, appealing and relevant. 4) The management and maintenance
The site must be easy to find via search engines or advertisement media. Then, continuance of the site must be preserved.
Roever (et al, 2001) added some qualification on website evaluation. He stated that qualitative and quantitative validation of web-based test does not differ in principle from validation of other types of test. However, there are specific validity issues introduced by testing medium.
5) Computer familiarity
It deals with test taker’s varying familiarity with computers. It can influence their scores and introduce construct-irrelevant variance (Kirsch et al., 1998). Tutorials to increase computer familiarity can eliminate this effect.
6) Typing speed
Differences in test takers’ typing speed are potentially more serious sources of error variance and are not amenable to quick training. However, typing speed is influencing written test only.
7) Delivery failures and speediness
41
8) Loading time and timer
This factor is important for timed tests during download and restarts the page.
7. Application Software
Application software is utilized to develop web-based design. In this study, web page authoring software facilitates users to create web pages which attach pictures, graphics, audios, animations, and particular display-interactive effects. Below, there are applications software applied in this study.
a. Adobe® Flash®
According to Gyncild (2007: 1), Adobe® Flash® provides a comprehensive authoring environment for creating interactive websites and digital animation. Adobe® Flash® is widely used to create engaging applications that are rich with video, graphics, and animation. The output from Adobe® Flash® has file extension in .swf and can be opened by using Flash Player. People can create content in Flash or import it from other Adobe applications and use Adobe ActionScript™ 3.0 to develop sophisticated interactive projects.
b. Wondershare Quiz Creator 3.2
assessments. The software enables trainers and educators to make quizzes for online testing. Besides creating Flash-based interactive quizzes and assessments, the software also can track the quiz results and analyzes the statistic report with Quiz Management System (QMS). Wondershare Quiz Creator 3.2 has nine different types of quiz questions. They are true-false, multiple-choice, multiple-response, fill-in the blank, matching, sequence, word bank, click map, and short essay. It can integrate each question with images, audio and narration, and Flash video. There are some benefits from the Wondershare Quiz Creator 3.2 presented in the next page.
1) Create a flash quiz within minutes
It is easily built with no complicated programming skill and drudgery needed any more. The program provides nine types of quiz questions.
2) Customize with comprehensive properties
43
3) Publish with flexible quiz export options
The quiz can be directly published to Flash and to website. Uploading a quiz is easy with the FTP (File Transfer Protocol) upload feature. Then, the quiz can be protected by hosting limit, password or build-in accounts
4) Track and report test result online
The test takers can get complete result report with a free Quiz Management System (QMS) Wondershare provides.
B. Theoretical Framework
This study intends to design a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong. To design the product, the writer considers some supporting theories such as: theory of language testing, theory of reading skill, theory of designing multiple-choice items, theory of web-based test, and theory of application software. In this study, the writer designs reading test not the materials. For that reason, the writer adopts some steps of Kemp’s instructional design model as the framework of the study. The framework of the designed test in this study consists of seven steps discussed below.
1. Identifying Learners’ Characteristics and Needs
literatures is certainly needed for collecting information on theories of designing web-based test. Study of related literatures was accomplished by reviewing literatures and other sources related to the theory of language testing, theory of designing educational product, and theory of web-based design.
2. Determining Goal, General Purposes, and Topics
Before designing the product, the writer should consider the goal, general purposes, and topics for the basis of the study. In accordance to Kemp (1977: 17), it aims to build the starting point that definitely becomes the root for the study. In this step, the writer determines the goal to design a web-based multiple-choice reading test for Grade XII students of senior high school. The next step is to determine general purposes. In this study, the general purposes are derived from Competence Standard and Basic Competence. Then, the next step is selecting the topics for designing test.
3. Specifying Learning Objectives
45
4. Specifying Test Content
In this study, the content should close-relationship to the objectives and to the student’s needs. In this designed test, the writer listed test content for the designed test. Based on the Brown’s theories of language testing (2004: 5-8), this study can be classified as a criterion-referenced test, formal test, summative test, discrete-point test, and achievement test. Thus, the designed test should be practical, reliable, valid, authentic, interactive, and giving feedback for students.
For the test content, this study focused on reading test which involved grammatical question and reading comprehension question. In accordance to KTSP, the test provided narrative, discussion, explanation, and review text. For designing the items, the writer considered theory of designing multiple-choice item which would be analyzed by IF, ID, and B-Index.
5. Specifying Activities and Sources
6. Determining Support Services
In determining support services, the writer refers to the theory of web-based test and theory of application software in order to develop the web-web-based test design. Besides, the writer needs to coordinate the other aspects including funds, facilities, equipments, time, and schedule. In this study, the writer requires Adobe® Flash® and Wondershare Quiz Creator 3.2 to design the website. The program of designed test may include home page, work-sheet page, score page, and feedback page. For the implementation, the study needs other facilities such as computer laboratory and the Internet facility.
7. Evaluation
In this step, the designed test is implemented in order to evaluate the test and students’ achievement. Meanwhile, the theory of designing multiple items is implemented here. After implementing the test, the writer analyzed the students’ data through questionnaire and collected data for the improvement of the design test. To determine the appropriate items, the writer analyzes the items through its IF, ID, and B-Index.
47 CHAPTER III
METHODOLOGY
This chapter presents the methodology or how the study is carried out.
This chapter consists of six sections. They are research method, research
participants, research instruments, data gathering techniques, data analysis
technique and research procedure.
A. Research Method
This study attempted to solve problem about what an appropriate design set
of web-based multiple-choice reading test for Grade XII students of senior high
school in Kecamatan Cibinong is. In order to achieve the goal, the writer conducted the strategy of Educational Research and Development (Borg and Gall,
1983). Borg and Gall (1983:772) stated that Educational Research and
Development (R&D) was a process used to develop and validate educational
products. R&D consisted of a cycle in which a version of the product was
developed, field-tested, and revised on the basis of field-test data. In addition, the
goal of R&D was to take the research knowledge and incorporate it into a product
that could be used in the school. In this study, the writer only provided six steps
because of the limitation of time and budget. They were (1) Research and
Information Collecting, (2) Preliminary Form of Product Development, (3)
Preliminary Product Verification, (4) Preliminary Product Revision (5) Main Field
1. Research and Information Collecting
It included review of literature, classroom observations, and preparation of
report state of the art (Borg and Gall, 1983: 775). This step became the basic
foundation in Educational Research and Development because it provided the
basic knowledge to develop educational product. Therefore, the writer conducted
study of related literature before designing a set of web-based multiple-choice
reading test. The purpose of the literature review was to determine the state of
knowledge in area of concern which would be developed (Borg and Gall, 1983:
777). Study of literature included reviewing theories about reading, language
testing, web-based test, designing multiple-choice items, and theories of
application software. Besides, the writer considered KTSP, Competent Standard,
Basic Competence, and materials that were appropriate for the Grade XII.
Furthermore, the writer conducted interview in order to gather information about
classroom media, students’ characteristics, materials, learning topics, and
students’ assessment.
2. Preliminary Form of Product Development
Based on the information gathered, the writer determined to conduct the
preliminary form of the product by considering some steps of Kemp’s
Instructional Design Model. Involving Kemp’s model, the writer determined goal,
general purposes, and topics for the test. The next steps were specifying learning
objectives, specifying test content, specifying activities and sources, and
