ANALYZING THE MULTIPLE-CHOICE ITEMS OF THE READING TEST IN SENIOR HIGH SCHOOL
NATIONAL EXAMINATION
A Thesis
Presented as Partial Fulfillment of the Requirements to Obtain theSarjana PendidikanDegree
in English Language Education
By
Hana Dyah Merina Student Number: 041214086
ENGLISH LANGUAGE EDUCATION STUDY PROGRAM DEPARTMENT OF LANGUAGE AND ARTS EDUCATION FACULTY OF TEACHERS TRAINING AND EDUCATION
SANATA DHARMA UNIVERSITY YOGYAKARTA
Hidup dan nasib bisa tampak berantakan, misterius, fantastis,
dan sporadis, namun setiap elemennya adalah subsistem dari
sebuah desain holistik yang sempurna.
Menerima kehidupan berarti menerima kenyataan
bahwa tak ada hal sekecil apa pun terjadi karena kebetulan.
Ini fakta penciptaan yang tak terbantahkan.
(Interpreted from a great idea of Harun Yahya)
Dedicated to My Heavenly Father, My late father Yohanes Baptista Sudi Hananto, My beloved mother Maria Margareta Sri Supadmi, and my lovely brother Fransiskus Xaverius Harris Dwi Prasetyo.
STATEMENT OF WORK’S ORIGINALITY
I honestly declare that this thesis, which I have written, does not contain the work or parts of the work of other people, except those cited in the quotations and the references, as a scientific paper should.
Yogyakarta, May 14, 2009
The Writer
Hana Dyah Merina
041214086
LEMBAR PERNYATAAN PERSETUJUAN
PUBLIKASI KARYA ILMIAH UNTUK KEPENTINGAN AKADEMIS
Yang bertandatangan di bawah ini, saya mahasiswa Universitas Sanata Dharma:
Nama :Hana Dyah Merina
Nomor Mahasiswa :041214086
Demi pengembangan ilmu pengetahuan, saya memberikan kepada Perpustakaan Universitas Sanata Dharma karya ilmiah saya yang berjudul:
Analyzing the Multiple-Choice Items of the Reading Test in Senior High School National Examination
beserta perangkat yang diperlukan (bila ada). Dengan demikian saya memberikan kepada Perpustakaan Universitas Sanata Dharma hak untuk menyimpan, mengalihkan dalam bentuk media lain, mengelolanya dalam bentuk pangkalan data, mendistribusikan secara terbatas, dan mempublikasikannya di Internet atau media lain untuk kepentingan akademis tanpa perlu meminta ijin dari saya maupun memberikan royalty kepada saya selama tetap mencantumkan nama saya sebagai penulis.
Demikian pernyataan in saya buat dengan sebenarnya.
Dibuat di Yogyakarta
Pada tanggal : 25 Juni 2009
Yang menyatakan
(Hana Dyah Merina)
ABSTRACT
Merina, Hana Dyah. (2009). Analyzing the Multiple-choice Items of the Reading Test in Senior High School National Examination. Yogyakarta: Faculty of Teachers Training and Education, Department of Language and Arts Education, English Education Study Program, Sanata Dharma University.
As an evaluation, National Examination had to be a tool in order to present an accurate result of students’ achievement. Due to that reason, the researcher conducted a study toward the National Examination. This study aimed to discover whether the examination measured the objectives stated in Kurikulum Tingkat Satuan Pendidikan (KTSP)covering the items, the written texts, and the question types. The focus of this research was limited on the reading test of the English subject since reading provides many contributions in public, in school, and in English learning.
The researcher applied some theories related to this study. The theories needed were those related to evaluation concepts (Mehrens and Lehman, 1973 and Gronlund, 1968), reading comprehension (Brown, 2001), language test theories, such as the validity of a test (Hughes, 1989), reading comprehension tests (Brown, 2004), and multiple-choice items (Marshall and Hales, 1972 and Harris, 1969), and taxonomy theory (Andersonet al,2001)
This study belonged to content analysis research which used qualitative approach since the primary data sources were from the National Examination and
KTSPdocument. Those primary data sources were supported by interviews as the secondary source. The researcher employed two instruments to collect the data needed, namely, the researcher and the interview guidelines. The data gathering techniques were documentary and interview method.
Firstly, the researcher conducted an analysis toward the National Examination items. The items were listed and classified based on the English reading objectives stated in KTSP. Then, the researcher analyzed the items based on taxonomy theory (Anderson et al, 2001) since the cognitive aspect developed inKTSPwas based on Bloom’s taxonomy (1979 as cited by Haryati, 2008). After analyzing the items, the researcher classified the written texts based on the types of the written texts mentioned in KTSP. The data obtained from the analysis above, was, then supported by the data obtained from the interview results.
After the analysis process, the findings revealed that the National Examination items measured the first Basic Competence related to students’ reading comprehension to the meaning of the short functional texts and the second Basic Competence related to students’ reading comprehension to the meaning of the simple essays. However, there were no items which measured the second Basic Competence related to students’ understanding toward the rhetorical steps of the simple essays. Furthermore, based on Anderson taxonomy theory (2001), the items measuring reading comprehension only measured two cognitive processes of the taxonomy, rememberandunderstand. It was not appropriate with the requirements mentioned in KTSP that all the cognitive processes should be
achieved. The findings also mentioned that the National Examination had employed the two types of the written texts mentioned in KTSP, short functional texts and simple essays. The question types employed in the National Examination could be categorized into four types: main idea, details, implied details, and vocabulary questions. The vocabulary mastery was measured explicitly and implicitly although the vocabulary mastery was not a requirement in
KTSP.
ABSTRAK
Merina, Hana Dyah. (2009). Analyzing the Multiple-choice Items of the Reading Test in Senior High School National Examination. Yogyakarta: Fakultas Keguruan dan Ilmu Pendidikan, Jurusan Bahasa dan Pendidikan Seni, Program Studi Pendidikan Bahasa Inggris, Universitas Sanata Dharma.
Sebagai sebuah bentuk evaluasi, Ujian Nasional harus bisa menjadi sebuah alat yang dapat menyajikan hasil akurat tentang hasil belajar siswa. Berdasarkan alasan tersebut, peneliti melakukan sebuah studi pada Ujian Nasional. Studi ini bertujuan untuk mencari tahu apakah Ujian Nasional mengukur tujuan pembelajaran seperti yang tertulis dalam Kurikulum Tingkat Satuan Pendidikan (KTSP) yang meliputi soal-soalnya, jenis-jenis teksnya, dan tipe-tipe pertanyaannya. Fokus utama dari studi ini adalah tes membaca dalam mata pelajaran Bahasa Inggris. Tes membaca dipilih karena membaca memberikan banyak sumbangan secara umum, baik di sekolah maupun dalam mata pelajaran Bahasa Inggris itu sendiri.
Dalam studi ini, peneliti memakai beberapa teori. Teori tersebut antara lain teori yang berhubungan dengan konsep evaluasi (Mehrens and Lehman, 1973 dan Gronlund, 1968), teori tentang tes bahasa, seperti kevalidan sebuah test (Hughes, 1989), tes pemahaman bacaan (Brown, 2004), dan soal-soal pilihan ganda (Marshall and Hales, 1972 dan Harris, 1969), serta teori taxonomi (Anderson et al,2001).
Studi ini merupakan analisa dokumen dengan pendekatan kualitatif karena sumber data primernya adalah soal-soal Ujian Nasional dan dokumen KTSP. Data utama tersebut didukung oleh wawancara sebagai sumber sekunder. Peneliti memakai dua instrument untuk mengumpulkan data yang diperlukan, yaitu peneliti sendiri dan panduan wawancara. Teknik pengumpulan data yang digunakan adalah metode dokumentasi dan wawancara.
Pertama-tama, peneliti melakukan analisa pada soal-soal Ujian Nasional. Soal-soal tersebut dibuat daftar dan diklasifikasikan berdasarkan tujuan pembelajaran seperti yang tertulis dalam KTSP. Kemudian, peneliti menganalisa soal-soal tersebut berdasarkan taxonomi teori (Anderson et al, 2001) karena penyusunan KTSP didasarkan pada taxonomy Bloom (1979 seperti dikutip oleh Haryati, 2008). Setelah menganalisa soal-soal ujian, peneliti menganalisa teks bacaan yang dipakai apakah sudah sesuai dengan jenis-jenis teks bacaan seperti yang disyaratkan dalam KTSP. Data-data dari analisa tersebut kemudian dikuatkan dengan data dari hasil wawancara.
Dari proses analisa, peneliti menemukan bahwa tidak semua Kompetensi Dasar yang menjadi tujuan pembelajaran diukur dalam soal-soal Ujian Nasional. Kompetensi Dasar yang diukur adalah Kompetensi Dasar pertama yaitu pemahaman siswa terhadap teks tulis fungsional pendek dan Kompetensi Dasar kedua yang berhubungan dengan pemahaman siswa terhadap esai sederhana. Sedangkan Kompetensi Dasar kedua yang berhubungan dengan pemahaman siswa terhadap langkah-langkah retorika esai sederhana tidak diukur. Berdasarkan teori taxonomy Anderson (2001), soal-soal Ujian Nasional yang mengukur pemahaman
siswa terhadap teks bacaan hanya mengukur dua tingkat proses kogntif, yaitu
remember dan understand. Padahal dalam KTSP, semua tingkatan kognitif harus bisa diterapkan dan diukur. Peneliti juga menemukan bahwa Ujian Nasional sudah menerapkan jenis-jenis teks seperti yang disebutkan dalam KTSP, yaitu teks fungsional pendek dan esei sederhana. Tipe-tipe pertanyaan yang digunakan dalam Ujian Nasional dapat dikelompokkan menjadi empat macam, yaitu pertanyaan tentang topik bacaan, penarikan kesimpulan, pencarian jawaban detail, dan kosakata. Selain itu, penguasaan kosakata juga diukur secara explisit dan implisit dalam Ujian Nasional meskipun penguasaan kosakata tidak menjadi salah satu tujuan pembelajaran dalam KTSP.
ACKNOWLEDGEMENTS
First of all, I would like to thank my Lord Jesus Christ for letting me to be
the architect in His great plan. This experience teaches me to be patient and
always trust Him that everything will be beautiful at its moment.
I thank Mrs. Dr. Retno Muljani, M.Pd. very much as my sponsor who
always helps me during the thesis making. I thank her for her guidance, patience,
and support. One year is not a short time to pass this exhausting but joyful
process. I thank Mr. Agustinus Hardi Prasetyo, S.Pd., M.A. who always gives
signatures in many letters of permission. It is very helpful to me. I also thank
MbakDani andMbakTari who help me in administration.
I would like to thank my lovely mother, Maria Margareta Sri Supadmi, for
her support and pray. I am sorry that it needs a longer time to make this thesis
complete. I also thank my grandfather and all of families for the supports.
I would also express my gratitude for Brother Yanu and Yeni who
provided me the very helpful informants. Many thanks I give to Bu Yanti in
Pangudi LuhurSenior High School andPakDana in 10 State Senior High School
for sharing helpful information in order to finish this thesis.
My gratitude also goes to Sariani Putranti, Yuni Astuti, and Theresia Rita. I
thank for the support and the time we have shared together. It is a wonderful
five-year time I will never forget. I also thank my big family of KKN Siluk I 2008,
Defifani, Beni, Ipung, Tere, Dika, Aster, Widya, Ita, and Cylas. I say sorry that I
can not make our promise comes true. Special thanks for PBI students Jody, Ani,
Novi, Riska, Nina, Alin, Nanda, Rina, Tutik, Evi, Bishop, Albert, Putri, Oon. I
thank Pius, Ahmed, Indri, and Tika for sharing and supporting me.
I also would like to give my never-ending thanks to Mitra Perpustakaan
and PUSD staff. Thanks for Budhe Asih, Amik, Cicik, Santi, Iin, Ruri, Yosep,
Prima and Mbak Sandra. You all are my new family. I thank Mas Risanto, Mas
Wid,MasBimo,MasDrajad, MbakIta,Mbak Ine, andMbakAmel for giving me
the best experience for my last days in the library.
I thank to Mas Rinto for being the father of our sweet daughters, Mbak
Santi for reflecting her spirit for me, and Finza for being my lovely friend. I also
thank Tika Deutch for sharing time together.
I also thank Sr. Benedicte who always supports me in writing this project. I
thank for the help in Stella Duce Senior High School. I thank very much to
Syantikara’s friends Boel2, Laura, Lani, Ike, Itjha, Elis, Veni, Sisca, Berta, Cindy,
Endah, Rany, Tata, Reena Ray, Mary Moru, Trisna for the supports. Special
thanks to Silvia Krisnawati. Finnaly, I won this war.
Finally, my gratitude goes to Sr. Margareth FCJ. Thanks for being the
reader of my thesis and praying for me. I also thank everyone who supports me
but I can not mention them one by one in this page.
The Writer
Hana Dyah Merina
041214086
TABLE OF CONTENTS
Page
TITLE PAGE ... i
APPROVAL PAGES ... ii
DEDICATION PAGE... iv
STATEMENT OF WORK’S ORIGINALITY ... v
LEMBAR PERNYATAAN PUBLIKASI ... vi
ABSTRACT ... vii
ABSTRAK ... ix
ACKNOWLEDGEMENTS ... xi
TABLE OF CONTENTS ... xiii
LIST OF TABLES ... xvi
LIST OF FIGURES ... xvii
LIST OF APPENDICES ... xviii
LIST OF ABBREVIATIONS ... xix
CHAPTER I. INTRODUCTION... 1
A. Research Background... 1
B. Problem Limitation ... 4
C. Problems Formulation... 5
D. Research Objectives ... 5
E. Research Benefits... 6
F. Definition of Terms... 7
CHAPTER II. REVIEW OF RELATED LITERATURE ... 9
A. Theoretical Description ... 9
1. Evaluation ... 9
a. Definition of Evaluation ……… 9
b. The Relationship between Evaluation and Objectives .. 10
2. Reading Comprehension ... 12
3. Language Test ... 13
a. Characteristic of a Good Test ... 13
b. Reading Comprehension Test ... 14
c. Test of Multiple-choice Format ... 15
4. Kurikulum Tingkat Satuan Pendidikan ... 17
5. Taxonomy Theory ... 24
B. Theoretical Framework ... 33
CHAPTER III. METHODOLOGY ... 35
A. Research Method ... 35
B. Sources of Data ... 36
C. Research Instruments ... 37
D. Data Gathering Techniques ... 38
E. Data Analyzing Techniques ... 39
F. Research Procedures ... 42
CHAPTER IV. RESEARCH FINDINGS AND DISCUSSION ... 44
A. Preliminary Analysis of the National Examination ... 44
B. Data Presentation ... 46
1. The Items ... 46
2. The Written Texts ... 50
3. The Interview Results ... 51
C. Discussion ... 53
D. Other Findings ... 62
CHAPTER V. CONCLUSIONS AND SUGGESTIONS ... 66
A. Conclusions ... 66
B. Suggestions ... 68
C. Implication ... 69
REFERENCES... 70
APPENDICES ... 74
LIST OF TABLES
Page
Table 2.1 The Comparison of the Cognitive Process
Based on Bloom (1956) and Anderson et al (2001) …… 26
Table 2.2 The Anderson Taxonomy Table (2001) ……… 33
Table 3.1 Points of Interview ……… 39
Table 3.2 The Classification of the National Examination Items
Based on the English Reading Objectives Stated inKTSP 40
Table 3.3 The Anderson Taxonomy Table (2001) ………. 40
Table 3.4 The List of the Written Texts ………. 41
Table 4.1 The Identification Result of the Question Types………… 47
Table 4.2 The Classification Result of the National Examination
Items Based on the English Reading Objectives …………48
Table 4.3 The Classification of the National Examination Items
Based on Anderson Taxonomy Theory (2001) ………… 49
Table 4.4 The Simplified Classification of the National Examination
Items Based on Anderson Taxonomy Theory (2001) …... 49
Table 4.5 The Anderson Taxonomy Table (2001) Applied for
the National Examination Items National Examination … 50
Table 4.6 The Result of Identifying the National Examination
Written Texts Based on the Written Text Types
Stated inKTSP……….. 51
Table 4.7 The Findings of the Interviews ………... 52
LIST OF FIGURES
Page
Figure 2.1 The Relationship Among Instruction, Objectives,
and Evaluation (Furst, 1958: 3) ………... 11
Figure 3.1 The Data Gathering Techniques ……… 41
Figure 3.2 The Research Procedures ………. 43
LIST OF APPENDICES
Page
Appendix 1 The Classification of the National Examination Items
Based on the English Reading Objectives Stated inKTSP... 75
Appendix 2 The Classification of the National Examination Items
Based on Anderson Taxonomy Theory... 79
Appendix 3 Interview Result with the English Teachers... 81
Appendix 4Ujian Nasional... 88
Appendix 5 The Competence Standard and Basic Competence
of the Senior High School English Reading... 155
LIST OF ABBREVIATIONS
KTSP Kurikulum Tingkat Satuan Pendidikan
BC Basic Competence (Basic Competence is equivalent to
Kompetensi Dasarwhich is the English reading objectives
stated inKTSP)
CHAPTER I INTRODUCTION
In this chapter, the researcher discusses six major parts, namely, the
background of the study, problem limitation, problem formulation, objectives of
the study, benefits of the study, and definition of terms.
A. Background of the Study
Reading plays an important role in the everyday life of a human being. A
fact which proves that reading is essential is the existence of visual media which
operates its function by using written words. The written media such as
newspapers, magazines, pamphlets, announcement or even a translated dialogue
in foreign movies is used to convey information, to entertain us, and to fulfil other
functions (Brown, 2004). Although there are translators, computer language, or
non verbal communication systems which are created by human beings, it is a fact
that much information is still locked up in the written words (Setyaka, 2002).
To respond to the texts in written media, people need to understand the
language of the text, vocabulary, sentence types, and the whole-text type. In other
words, reading involves responding to a text rather than producing the text. It is
necessary, then, that anyone has reading skills.
In school, reading is a skill that teachers basically expect students to
acquire. Brown (2004) supports it by arguing that reading is the most essential
skill for success in all educational contexts. Most subjects require students to read
many books in order to master the subjects. By reading, students acquire more
information and knowledge, in addition to what they learn from their teachers to
master the subjects.
In learning English, reading skills provide an important contribution.
Fiharsono (2005) states students may get inputs for the development of listening,
speaking, and writing skills through reading. They can gain knowledge of
speaking expressions, writing techniques, or deep comprehension in listening
through printed texts. Further, basic grammar and vocabulary may be introduced
through reading.
Concerning the contribution of reading skills which is elaborated above, it
is a fact that reading is an important skill at school. Students’ comprehension
toward what they read needs critical attention. The attention can be carried out by
conducting a test to measure reading skills. Further, Alderson (2004) supports it
by arguing that measuring of students’ reading skills is necessary in education.
Apart from that, teachers will not see to what extent students’ reading
mastery is by teaching only. By conducting a test, teachers can measure students’
reading skills and gain an input about students’ learning outcomes. The result of
the test can show to what extent the students have mastered the reading skills over
a certain period of time. Besides, the teacher can have reflection concerning their
teaching learning process in the classroom.
Harris (1969) discovers that there is always the same general type of
reading tests. The test consists of short passages of varying styles and contents.
questions’ technique is the oldest and the most common reading test. In many
reading tests, each passage is followed by a series of multiple choices of
comprehension items, short answer, matching words, or long answer, etc.
National Examination is an example of evaluation to measure students’
ability. The ability measured is represented in three main subjects: English,
Indonesian language, and mathematics. This evaluation implies a written test
which uses multiple-choice format items. National Examination is essential for
students to pass since this examination determines whether students will graduate
or not from senior high school.
In the last two years the existence of the National Examination brings
about controversy. The achievement standard that students must achieve is
considered higher than the students can attain. Apart from that, some people and
educational experts viewed the National Examination invalid (Kompas, 29 Mei
2006). The tests, seemingly, cannot be used to measure what students have
learned for three years.
One of the criteria of a good test which Hughes (1989) stresses is the
validity of the test. A test should have validity because the test should be an
accurate tool to measure what it is supposed to measure. Bloom et al (1981: 182
as cited by Setyaka, 2002) support it by saying that the starting point in test item
writing is the instructional objectives to achieve. Due to the importance of
objectives, the reading items in National Examinations should also measure
Tingkat Satuan Pendidikan (KTSP), the new curriculum which has been
implemented since 2006.
In this study, the researcher will analyze the multiple-choice items of the
reading test in the National Examination for senior high school. The researcher
has an interest in whether the reading test has reflected the English reading
objectives stated inKTSPor not. The result of the study can provide test designers
an input in designing the following tests for National Examination. The English
senior high school teachers can also obtain an input in recognizing the item
models of the National Examination.
B. Problem Limitation
To limit the discussion, this study is limited to three aspects. First, this
study focuses on the reading test of English in National Examination for senior
high school students using the multiple-choice format items. Second, this study
analyzes the 2007/2008 National Examination by usingKTSPas the parameter of
reading curriculum. KTSP is implemented due to the fact that teaching-learning
processes in schools are aimed at achieving the objectives stated in KTSP. Third,
the National Examination is also analyzed based on Bloom’s taxonomy revised
and proposed by Anderson et al (2001) since the KTSP construction is based on
C. Problems Formulation
The problems of the study can be formulated as follows:
1. Did the multiple-choice reading items of the National Examination test for
senior high school students measure the objectives of English reading
skills as stated inKTSP?
2. Did the texts used in the National Examination test for senior high school
students represent the written text types mentioned inKTSP?
3. Did the question types used in the multiple-choice reading items of the
National Examination test for senior high school students measure the
students’ comprehension through various question types?
D. Research Objectives
The objectives of the study can be formulated as follows:
1. To discover whether the multiple-choice reading items of the National
Examination test for senior high school students measured the objectives
of English reading skills as stated inKTSP.
2. To discover whether the texts used in the National Examination test for
senior high school students represented the written text types mentioned in
KTSP.
3. To discover whether the question types used in the multiple-choice reading
items of the National Examination test for senior high school students
E. Research Benefits
This study may provide benefits which are stated as follows:
1. For teachers
By using the results of this study, English senior high school teachers may
direct the teaching-learning product toward the objectives stated in KTSP,
not merely toward the National Examination items. The teachers can
develop the teaching materials not only related to reading comprehension
but also related to features of written texts and vocabulary.
2. For schools
This study can provide an input to carry out school programs such as
conducting mid semester or final tests. The tests can be designed to really
measure the objectives stated inKTSP.
3. For test designers
This study can be used as a reference to know whether the National
Examination items have fulfilled the requirements as stated in KTSP or
not. The test designers can also improve and develop their own test
materials in the future.
4. For curriculum designers
This study may be used as a feedback on whether the objectives stated in
KTSPhave been achieved or not. The curriculum designers may obtain an
input if they need to reformulate the objective statements in order to state
5. For future researchers
The study benefits for future researchers because it can be used as a basis
for conducting further research related to language assessments for high
school.
F. Definition of Terms
This section presents the definition of terms, namely, National
Examination, Reading test, Kurikulum Tingkat Satuan Pendidikan (KTSP), and
Senior High School students.
1. National Examination
National Examination is an evaluation that is conducted nationally at the
last grade of certain stage education. In this study, National Examination is
defined as an evaluation for English subject, reading skills particularly, by
using multiple-choice format items which is conducted nationally at the
end of school years for XII grade students.
2. Reading test
According to Samana (1982 as cited by Fiharsono, 2005), a reading test is
an evaluation instrument which measures to what extent students achieve
reading comprehension of texts. In this study, a reading test is defined as a
test which is composed of some written texts followed by two to four
multiple-choice questions and used to measure students’ reading skill
3. Kurikulum Tingkat Satuan Pendidikan (KTSP)
Muslich (2007) defines KTSP as an operational curriculum which is
constructed and conducted by each education level. In this study, KTSPis
defined as an operational curriculum which is constructed and conducted
by senior high schools.
4. Senior High School Students
Senior high school students mean students from grade X to XII. Those
who join National Examination are grade XII students. In senior high
school, the specification of the study field is given when the students are in
the XI grade. There are three specifications, namelyIPA, IPS, andBahasa.
For those three specifications students join the same National Examination
CHAPTER II
THEORETICAL REVIEW
There are two major points to be described in this chapter. The first deals
with the Theoretical Description, while the second discusses the Theoretical
Framework which is used in this research.
A. Theoretical Description
In this section, there are four topics to be discussed. They are (1)
Evaluation, (2) Reading Comprehension, (3) Language Test, (4) Kurikulum
Tingkat Satuan Pendidikan (KTSP), (5) Taxonomy Theory.
1. Evaluation
a. Definition of Evaluation
The terms test, measurement, and evaluation might be confused
because they may be involved in one single process. Mehrens and Lehmann
(1973) state that the definition of test is a presentation of a standard set of
questions to be answered, measurement is a result of a person’s answers to
such a series of questions, and evaluation is determination of the congruence
between performance and objectives.
Evaluation has a broader concept than measurement and measurement
has a broader definition than a test. If we ask students to answer a series of
questions concerning English, obtain their scores by counting the number of
correct answers, and conclude that those students are making good learning
progress, we are concerned with all those three concepts. The test is the set of
questions to be answered. The measurement is the students’ test result
according to specific rules. The evaluation deals with the value judgement
whether the students are achieving instructional objectives or not.
Gronlund and Linn (1990) conclude that evaluation is a systematic
process of collecting, analyzing, and interpreting information to determine the
extent to which instructional objectives are achieved by pupils. The process
may include either measurement procedures, such as a test or
non-measurement procedures, such as informal observation.
b. The Relationship between Evaluation and Objectives
Gronlund (1968) says that the first step of evaluation learning
outcomes is determining what is to be measured. He adds what are to be
measured in education are instructional objectives. Instructional objectives
provide focus and direction for guiding to educational planning and evaluation
process.
Gronlund (1968) states that objective is defined as a description of
outcomes, abilities, or qualities expected from each student. The purpose of
educational objectives is to give direction not only to the development of the
students but also the planning of the educational program. It means that the
materials, the assignments, the instructional methods, and all aspects of
Mehrens and Lehmann (1973: 10) say that there is a relationship
among instruction, objective, and evaluation. They cited a diagram from Furst
(1958: 3). The diagram is presented as follows.
Figure 2.1 The Relationship among Instruction, Objective,
and Evaluation (Furst, 1958: 3)
The diagram shows that objectives, instruction, and evaluation process
are involved in one process. Instruction refers to the means that the teachers
employ to help students realize the objectives of study. Objectives mean kinds
of changes in students that teachers wish to obtain. Evaluation means
determining the extent to which the instruction has been effective.
Objectives help to determine the instructional procedures to be used in
teaching learning and the method to be used to evaluate. The results of
evaluation provide feedback on the effectiveness of the teaching experienced
and the attainability of the objective. Both evaluation and educational
experiences help to clarify the objective.
Mehrens and Lehmann (1973) add that the evaluation helps the teacher
in (1) providing them with knowledge concerning the students’ entry
behaviour, (2) setting, refining, and clarifying realistic goals for each student, Objectives
Instruction Evaluation
(3) evaluating the degree to which the objectives have been achieved, (4)
determining, evaluating, and refining the instructional technique.
2. Reading Comprehension
In English language learning, there are some skills to be developed.
The term ‘skills’ in English language learning can be defined into two
levels-macro and micro skills (Dudley-Evans and John, 2006). Macro skills consist
of reading, listening, speaking, and writing. Each macro skill consists of a
number of micro skills.
According to Grabe (2002), reading comprehension skill is defined as
an ability to understand information in a text and interpret it appropriately.
According to Brown (2001), in reading skill, the micro skills may involved:
Recognizing grammatical word classes (nouns, verbs, etc.), patterns, and
rules.
Recognizing that a particular meaning may be expressed in different
grammatical form.
Recognizing the rhetorical forms of written discourse.
Recognizing the communicative functions of written texts according to
form and purpose.
Inferring context that is not explicit by using background knowledge.
Inferring connections between events or ideas, identifying cause and effect
models, detecting such relation as main ideas, supporting ideas, new
Distinguishing between literal and implied meanings.
Developing and using reading strategies such as scanning and skimming,
guessing the meaning of words from context.
3. Language Test
a. Characteristic of a Good Test
A test, according to Nunnally (1964), is a standardized situation that
provides an individual with score. Nunnally (1964) explains that there are two
points in the definition. Firstly, standardized means all procedures of testing
where all students are tested with same questions in the same way. Then
secondly, score means numerical indication of a student’s performance.
According to Marshal and Hales (1972), a test must have reliability,
validity, and practicality. They also state that validity is the single most
important aspect of a test. Hughes (1989) states that a test has validity if the
test measures accurately what is to be measured. According to Hughes (1989),
there are four kinds of validity that can be observed in a test.
a) Content Validity
A test is said to have validity which is viewed from its content. The
content composes a representative sample of the language skill or
structures. Heaton (1975) says that the relationship between the test
items and the course objectives always being apparent. In other words,
content validity also measures how the test item meets the course
information from a text is able to measure students’ reading
comprehension.
b) Criterion-related Validity
Criterion-related validity refers to instrumental validity. It is used to
demonstrate the accuracy of a measure or procedure by comparing it
with another measure or procedure which has been demonstrated to be
valid.
c) Construct Validity
A test is said to have validity which is viewed from test construction.
The word ‘construct’ refers to any underlying ability which is included
in a theory of language ability. For example, the ability to read
includes the ability in meaning guessing or scanning the general ideas.
d) Face Validity
A test is said to have face validity if it looks right as if it measures what
it is supposed to measure (Hughes, 1989). For example, a test measures
pronunciation ability by speaking or a test measures reading
comprehension by reading texts.
b. Reading Comprehension Test
According to Grabe (2002), reading comprehension is defined as an
ability to understand information in a text and interpret it appropriately. To
measure the ability, methods are offered in order to present the accurate result.
for testing reading. No single test method can fulfil the reading purposes for
which a teacher might test.
However, Alderson (2000) mentions that multiple-choice items are
common device for testing students’ text comprehension. There are some
advantages in implementing this kind of test. Test designers are allowed to
control the range of possible answers to comprehension questions. The test
designers can also control the students’ thought processes when they are
responding. In addition, multiple-choice questions can be marked by machine.
‘Reading a passage and answer some questions’ technique is the oldest
and the most common reading test used (Brown, 2004). A set of questions is
usually used for covering reading comprehension. The questions are related to
the main idea (topic), expressions/idioms/phrases in context, conclusion
(implied detail), detail (scanning for specifically detail), unwritten facts
supporting ideas, and vocabulary in context of texts.
c. Tests of Multiple-choice Format
A multiple-choice item is generally recognized as the most widely
applicable and an useful type of objective test items (Gronlund and Linn
1990). This type is the most popular and often used by classroom teachers.
Marshall and Hales (1972) describe the strengths of this item. Firstly,
multiple-choice item is flexible and versatile of all selection type
examinations. Gronlund and Linn (1990) explain versatility in multiple-choice
outcomes from simple to complex. Secondly, multiple-choice can be scored
rapidly, accurately, and objectively by all individuals who are unqualified to
teach the subject area examined. Thirdly, the scoring of multiple-choice is not
influenced by all outside performances. It means that the multiple-choice
items have range control of students’ way of thinking through the provided
alternative answers. Students have to know exactly what the items ask about
then they will choose the correct answer to answer the items.
However, multiple-choice item has some disadvantages. Firstly, Harris
(1969) mentions that constructing multiple-choice item requires considerable
skill and time to prepare. Secondly, Alderson (2000) adds that the tester does
not know how the students’ ways of thinking. The tester does not know
whether the students really know the answer or just guess the answer. The
students may guess at their choices or they may have totally different reasons
in mind from that which the test-maker intends when they are writing the
items. It is possible to get an item correct for ‘wrong’ reason or to get an item
wrong for the ‘right’ reason. Thirdly, Anderson et al (2001) states that
multiple-choice items cannot be used to measure all objectives at all levels of
cognitive domain. This type cannot be implemented to measure at the ‘create’
level. It means that multiple-choice item cannot be used to measure students’
ability in organizing materials or expressing their answer according to
4. Kurikulum Tingkat Satuan Pendidikan
Kurikulum Tingkat Satuan Pendidikan (KTSP) is a curriculum which is
being applied in Indonesian education. The curriculum started in 2006 has
been applied for two years. Muslich (2007) defines KTSP as an operational
curriculum which is constructed and conducted by each education level. It
means that each educational level has its autonomy in planning, conducting,
arranging, measuring, and evaluating learning based on schools’ condition and
characteristics. Yet, the implementation of KTSP still refers to the KTSP
construction guidelines which are constructed by Badan Standar National
Pendidikan(Muslich, 2007).
There are three aspects based on Bloom’s theory (1979 as cited by
Haryati, 2004) to be developed and assessed in KTSP.The aspects are (1) the
cognitive aspect which is closely related to thinking ability such as
knowledge, comprehension, and analysis, (2) the psychomotor aspect which is
related to physical activities such as writing, beating, jumping, (3) the
affective aspect involves attitudes, such as morality, self-concept, desire, or
behaviour. Further, Haryati (2008) states that each subject should consist of
those three aspects with different emphasis. Subjects which focus on thinking
ability emphasize cognitive process while subjects focus on physical activities
emphasize psychomotor aspect.
Haryati (2008) states that the three aspects should be realized in the
achievement of Competence Standard in each subject. Haryati (2008) explains
psychomotor, and affective aspects achieved by students to master a certain
subject. Since the Competence Standard statement formulation is still in a
wide area, the Competence Standard needs to be elaborated in Basic
Competence. Haryati (2008) defines the Basic Competence as the students’
minimal ability in demonstrating their mastery of cognitive, psychomotor, and
affective domains.
English, as one of senior high school subjects, realized its Competence
Standard into four English skills: listening, speaking, reading, and writing. For
English reading skills, the emphasis is on the cognitive aspect since English
reading skills focuses on students’ thinking ability of reading passages
understanding. Giscoe and Gee (1984 as cited by Setyaka, 2002) also argue
that reading comprehension is an active cognitive activity. It involves not only
readers’ comprehension skills but also their prior knowledge and experiences.
Haryati (2008) explains the cognitive aspect arranged in KTSP is adapted
from Bloom’s taxonomy (Sax, 1980). It consists of six processes which will
be explained in page 24, (4) Taxonomy Theory.
In English reading skills, there are six Standard Competence for senior
high school. These Standard Competence is specified and elaborated again
into 12 Basic Competence to be achieved in six semester from X to XII grade.
These Basic Competence needs to be achieved since these Basic Competence
becomes the objectives of English reading skills in senior high school. The
If we pay attention to word formulations of the Basic Competence from
X to XII grade, those Basic Competence can be simplified into two Basic
Competence. Each semester from X to XII grade requires students to achieve
those two Basic Competence. The first Basic Competence (1st BC) is students
are expected to be able to give responses to the meaning of short functional
texts such as announcement, advertisement, invitation, pamphlet, poster and
banner accurately and fluently. The second Basic Competence (2nd BC) is
students are expected to be able to give responses to the meaning and the
rhetorical steps of simple essays in form of recount, narrative, procedure,
descriptive, news item, report, analytical exposition, spoof, hortatory
exposition, explanation, discussion, and review accurately and fluently. What
makes the Basic Competence different for each semester in X to XII grade is
the types of the short functional texts and the simple essays.
Giving responses toward the meaning of the short functional texts and
giving responses toward the meaning and the rhetorical steps of the simple
essays in the two Basic Competence previously described are further specified
by Pujogiri (2009) as follows:
Identifying the main ideas of the short functional texts such as
advertisement or announcement (the 1stBC)
Identifying the topics of the short functional texts such as banner or
pamphlet (the 1stBC)
Identifying the implied details of the short functional texts such as
Identifying the unwritten supporting facts of the short functional texts
(the 1stBC)
Identifying the main idea of narrative texts (the 2ndBC)
Identifying the topics of recount texts (the 2ndBC)
Identifying the implied details in descriptive texts (the 2ndBC)
Identifying the rhetorical steps of the simple essays (the 2ndBC)
Identifying the process of an event (the 2ndBC)
Identifying the function of the simple essays (the 2ndBC)
Identifying the arguments mentioned in discussion texts (the 2ndBC)
Identifying the complication of the narrative texts (the 2ndBC)
Identifying the events in narrative, discussion, or explanation texts.
(the 2ndBC)
Based on the Competence Standard, the materials for senior high
school English reading skills are genre-base texts. It means that students need
to master types of written texts. The written texts consists of short functional
text and simple essay. The short functional texts are texts which exist in the
everyday life and provide information for public. The types of the texts are
(Jewell, 2005):
1. Announcement: a public and typically format statement about fact,
occurrence, or intention.
2. Advertisement: announcement in a public medium promoting a
3. Invitation: written request asking someone to go somewhere or to do
something.
4. Banner: a long strip of cloth bearing a slogan or design hung in a
public place or carried in a demonstration or procession.
5. Poster: a large printed picture, notice, or advertisement displayed in a
public place or used for decoration.
6. Pamphlet: a small booklet or leaflet containing information or
argument about a single subject.
The simple essays are texts in form of short paragraphs which have
characteristics in functions and structures. The types of the essay stated in
KTSP are recount, narrative, procedure, descriptive, news items, report,
analytical exposition, spoof, hortatory exposition, explanation, discussion, and
review. Some descriptions about the simple essays are provided in
http://www.testdigital.com/Description.htmas follows:
1. Recount
Recount text tells about experiences or events. The function of recount
text is to retell events or experiences for the purpose of informing or
entertaining. The events or experiences are reality. The structures of
text are orientation (introduction of who, where, and when are
involved), main event, comments or opinion, and closing. Generally,
recount text is more alive if the writer writes it himself because the
2. Narrative
Narrative is a text whose function is entertaining readers. The text may
be fiction or non-fiction which consists of conflict and resolution. The
structures of the text are orientation (introduction of characters, setting
of place and time), complication (conflict), resolution, and coda which
is a change occurring to character and moral message taken from the
text.
3. Procedure
Procedure is a text which provides steps of making or creating or doing
something. The structures of the text are goal of the activity, list of
materials, and steps.
4. Descriptive
Descriptive is a text which describes a thing, people, place specifically.
The structures of the text are introduction of subject and features of
subject such as physical appearances, quality, general behavior, or
characteristics.
5. News item
News item is a text which informs readers about events of the day
which are considered newsworthy or important. The structures of the
text are newsworthy events, background events (such as events
elaboration, involved people, or setting of place and time), and
6. Report
Report is a text which conveys information about something as a
research or an analysis result. The function of report is presenting
information of something based on fact. It can describe environment,
nature symptom, or human hand-made. The structures of the text are
general classifications which are statements describing the report
subject, explanation, and classification, then descriptions elaborating
the phenomenon under discussion in terms of parts, qualities, habit or
behaviors.
7. Analytical exposition
Analytical exposition is a text that elaborates the writer‘s idea about
the phenomenon surrounding. The function of the text is to persuade
the reader that something is in the case and needs attention. The
structures of the text are thesis (opinion statement), arguments, and
reiteration (statement reinforcement).
8. Spoof
Spoof is a text which retells an event with a humorous twist at the end.
The function is sharing a true story which is entertaining and usually
has an unpredictable ending. The structures of the text are introduction,
9. Hortatory exposition
Hortatory exposition is a text which is almost the same as analytical
exposition. The function of the text is to persuade the readers that
something should or should not be in the case.
10. Explanation
Explanation is a text which explains a process involved in the
formation or workings of natural or socio cultural phenomena. The
general structure is a general statement which is followed by
sequenced explanations of why or how something occurs.
11. Discussion
Discussion is a text which presents two points of view about an issue
and is ended by a conclusion or recommendations. The general
structures of this text are an issue, arguments for supporting the issue
followed by the elaboration, statements from various viewpoints also
followed by elaboration, and conclusion or recommendations.
12. Review
Review is a text which is used to critique or to comment on an artwork
or event for a public audience. The structures of the text are orientation
(introduction), evaluations, and evaluative summation.
5. Taxonomy Theory
Bloom (1956) defines taxonomy as a system of classifying objectives
from lowest to highest levels. He uses the taxonomy as an instrument of
classifying goals of educational objectives. Since this taxonomy is only
implied for cognitive process, it is called taxonomy of cognitive domain.
Bloom’s taxonomy of cognitive domain (1956) consists of 6 cognitive
processes, namely, knowledge, comprehension, application, analysis,
synthesis, and evaluation. All of these cognitive processes are in noun form
and considered as one dimension. Anderson et al (2001), then, revise the
taxonomy by changing the form of the cognitive dimension from noun to verb
and adding another dimension, namely, ‘knowledge’. The cognitive dimension
describes the cognitive processes how students think which consist of
remember, understand, apply, analyze, evaluate, and create. The knowledge
describes what students are expected to acquire consisting of factual,
conceptual, procedural, and metacognitive knowledge.
Anderson et al (2001) revise Bloom’s taxonomy (1956) based on a
consideration that a statement of objectives contains a verb and a noun. The
verb describes the intended cognitive process. The noun describes the
For clearer illustration, the comparison between the cognitive process
based on Bloom (1956) and Andersonet al(2001) are presented below.
Table 2.1 The Comparison of the Cognitive Process Based on Bloom (1956) and Andersonet al(2001)
Bloom (1956) Anderson et al(2001) Knowledge Remember
Comprehension Understand
Application Apply
Analysis Analyze
Synthesis Create
Evaluation Evaluate
Cognitive process emphasizes how students think. Anderson et al
(2001) define the cognitive processes within six processes. Those six
processes are remember,understand, apply, analyze, evaluate, andcreate.
a. Remember
The Remember by Anderson et al (2001) is similar to the knowledge by
Bloom (1956). Bloom (1956) defines knowledge as a cognitive process
which requires students to remember ideas, material, or phenomena while
Anderson et al (2001) defines remember where students needs to
remember things exactly as they heard, read, or saw them. Simply, this
cognitive process needs students to recall or recognize information. The
students are expected to store in their mind certain information. The form
knowledge was originally learned. Anderson et al (2001) divided
specifically remember into two subtypes, recognizing and recalling. In
recognizing, students needs to search long-term memory for a piece of
information which is identical to the presented information. In recalling,
students needs to search long-term memory for a piece of information to
complete a task.
b. Understand
Bloom (1956) and Andersonet al (2001) have the same argument that the
largest general class of intellectual abilities and skills emphasized in
schools is those which involves comprehension. When students deal with a
communication, they are expected to be able to understand what is being
communicated. Communication here refers to materials in concrete form
or the materials embodied in paper. Bloom (1956) adds that the term
‘comprehension’ has been frequently associated with reading
comprehension. Anderson et al (2001), which callcomprehension(Bloom,
1956) asunderstand, expect students to be able to construct meaning from
instructional messages, including oral, written, and graphic
communications by using their own words. In other words, students are
said to understand when they can explain ideas or concepts. Understand
are divided into 7 subtypes: (1) interpreting, (2) exemplifying, (3)
classifying, (4) summarizing, (5) inferring, (6) comparing, and (7)
The explanation is as follows.
1) Interpreting occurs when a student is able to transfer information
from one representational form to another. Interpreting may
involve transferring words to words (paraphrasing), pictures to
words, words to pictures, numbers to words, or words to numbers.
2) Exemplifying occurs when a student is able to give a specific
example of a general concept.
3) Classifying occurs when a student recognizes that something
belongs to a certain category. Classifying involves identifying
features or patterns that make something belongs to those certain
category.
4) Summarizing occurs when a student implies a single statement that
represents presented information. Summarizing involves
constructing a representation of the information.
5) Inferring occurs when a student is able to conclude the concept of
certain thing. Inferring also involves finding a pattern within a
series of examples.
6) Comparing involves detecting similarities and differences between
two or more objects, events, ideas, problems, or situations.
7) Explaining occurs when a student is able to construct and use a
c. Apply
Anderson et al (2001) mentions that apply involves using procedures to
perform exercises or solve problems. Students are said to apply when they
can use the information they have in a new way. In understand cognitive
process, the students are required to recognize information well enough so
that they can demonstrate correctly its use when asked to do so. In apply
cognitive aspect, the students are provided a new problem to apply the
appropriate information.
d. Analyze
Anderson et al (2001) explains that analyze involves breaking material
into parts and determining how the parts are related to one another and to
an overall structure. In other words, students can distinguish between the
different parts. Among the six cognitive processes,understand,apply, and
analyze are sometimes defined unclear. Bloom (1956) provides an
elaboration amongunderstand,apply, andanalyzebased on its definitions.
In understand, the emphasis is on the grasp of the material meaning. In
application, the emphasis is on remembering and bringing the appropriate
principle in given problems. Inanalyze, the emphasis is on the breakdown
of the material into constituent parts. Bloom (1956) defines analyze into
three levels: 1) students are expected to break down the material into its
elements/parts and identify it, 2) students are required to detect the
explicit relationships among the elements, 3) students are expected to
together the material as whole. The examples of apply activities in
language subjects are distinguishing fact from opinion, connecting
conclusions with supporting statements, finding evidence in support of the
author’s purpose.
e. Evaluate
Bloom (1956) definesevaluate,which is positioned in the sixth process of
his taxonomy, as making judgements of the entire process of a
performance related to its values, solutions, methods, or material based on
criteria and standards of quality, effectiveness, efficiency, and consistency.
The standards may be quantitative, such as ‘is this a sufficient amount?’ or
qualitative, such as ‘is this good enough?’. Simply, students are said to
evaluate when they can justify a stand or decision. However, not all
judgements are evaluative, such as students make judgements about
whether two objects are similar or different. Anderson et al (2001)
explains that the making judgement in evaluate here refers to the use of
standards of performance with clearly defined criteria.
f. Create
Create by Andersonet al (2001) is similar to synthesis by Bloom (1956).
Anderson et al (2001) revise the position of crate from the fifth to sixth
process since create is more than responding to a presented material but
producing a new product. Anderson et al (2001) definescreate as putting
together of elements and parts to form a functional whole. In this cognitive
these together into a structure or pattern unclear before. In other words,
students are required to make a new product by reorganizing some
elements into a pattern or structure not clearly present before. Although
the process of understand, apply, and analyze may involve detecting
relationships among the presented elements, create is different because it
also involves the construction of an original product. Understand, apply,
and analyze involves working with a given set of elements which are the
part of a given whole. They are parts of a larger structure where the
students are trying to understand. In create, the students must draw upon
elements from sources and put them together into a structure.
In the revised taxonomy, Anderson et al (2001) define knowledge as
what students know. Knowledge is divided into four types. Those four types
are factual knowledge, conceptual knowledge, procedural knowledge,
metacognitive knowledge.
a. Factual knowledge
Factual knowledge is basic elements that students must know. Those basic
elements will be used to understand certain discipline systematically.
There is little abstraction to factual knowledge. In English, the examples of
factual knowledge are vocabulary, kinds of tense, irregular and regular
verbs, parts of speech.
b. Conceptual knowledge
In conceptual knowledge, students apply relationships among basic
students can explain a concept in their own words and transfer information
to new situations, they have acquired conceptual knowledge. In English,
the examples of conceptual knowledge are concept of each tense,
understanding of reading passages, and concept of written texts types.
c. Procedural knowledge
Procedural knowledge is ‘knowledge of how’ to do something. The
knowledge often takes a series or sequence of steps to follow. In English,
the example of procedural knowledge is the sequence of narrative texts
construction.
d. Metacognitive knowledge
Metacognitive knowledge is more than knowledge. It tends to
self-regulation how students apply strategies to complete tasks. The highlight
of metacognitive knowledge is to make students more aware of and
responsible for their own knowledge and thought. Flavell (1979) concerns
that students may acquire some information from their study, but they may
not have the ability to monitor their learning conditions or make adaptation
within their learning process in monitoring and control their learning
condition. To help students develop metacognitive knowledge, reflection
is needed to make students aware of how much efforts they have done to
The table below presents the cognitive process and the knowledge in
Anderson taxonomy theory (2001).
Table 2.2 The Taxonomy Table (Andersonet al, 2001: 28)
B. Theoretical Framework
The aims of this study are to discover to how well the multiple-choice
reading items of National Examination test for senior high school students
measured the objectives of reading skills as stated in Kurikulum Tingkat Satuan
Pendidikan (KTSP),the written text types presented, and the question types used in
the National Examination. In order to conduct a systematic study, the researcher
provides a theoretical framework to explain the contribution of the theories
applied in solving the formulated problems.
Firstly, this study is related to National Examination. The researcher needs
to conduct a preliminary analysis to discover the general idea about the
examination. Since the examination is arranged based on some compilation
components, the researcher requires some theories in order to understand the main The Cognitive Process
ideas of the examination. The theories needed are those related to evaluation
concepts (Mehrens and Lehman, 1973 and Gronlund, 1968) and language test
theories, such as the validity of a test (Hughes, 1989), reading comprehension
tests (Brown, 2004), and multiple-choice items (Marshall and Hales, 1972 and
Harris, 1969). The preliminary analysis based on the theories will provide basic
comprehension of National Examination for the researcher.
Secondly, this study is analyzing the appropriateness of National
Examination compared to the English reading objectives stated in KTSP. KTSP
has an important role in this study since it is the parameter of the reading
curriculum. The researcher requires the KTSP document so the researcher is able
to obtain the general understanding of the curriculum and list the English reading
objectives in order to analyze the examination.
Thirdly, since the composing ofKTSP, particularly the cognitive aspect, is
based on Bloom’s theory (1979 as cited by Haryati, 2008) and the National
Examination items measure only the cognitive aspect, the researcher will classify
the items based on the taxonomy theory. The taxonomy theory which the
researcher employs to classify the items is Bloom’s taxonomy theory which has
been revised as proposed by Anderson et al (2001). One by one, the National
Examination items are classified based on the cognitive process and the
knowledge of the taxonomy theory (Anderson et al, 2001). The result of the
classification will reveal which cognitive process and knowledge the test items
CHAPTER III METHODOLOGY
This chapter illuminates detailed information about the research
methodology which the researcher used to answer formulated problems stated in
chapter I. Hence, the discussion consisted of six subdivisions, namely, research
method, sources of data, research instruments, data gathering techniques, data
analyzing techniques, and research procedures.
A. Research Method
Based on the formulated problems stated in chapter I, this research tried to
discover to what extent the National Examination had measured the English
reading objectives stated in KTSP. To answer the formulated problems, the
researcher applied National Examination items and KTSP document as the main
written materials to be analyzed. Since this research positioned the National
Examination items and the KTSP document as the main materials, this research
belonged to content analysis or document analysis (Aryet al, 2002).
Ary et al(2002) mentioned that document analysis was applied to written
or visual materials for the purpose of identifying specified characteristics of the
materials. By analyzing the National Examination items, the researcher tried to
identify the items of the National Examination for the purpose of discovering
whether the items had measured the objectives stated inKTSPor not. In this case,
the items were crosschecked on the English reading objectives stated inKTSP for
its appropriateness.
Based on the type of the data which was gathered, the researcher applied
qualitative approach to carry out this study. A qualitative approach was described
as an approach which was used to gather descriptive data and therefore not
numerical (Ary et al, 2002). This study belonged to the qualitative approach since
it gathered purely descriptive data of the National Examination items. If there was
numeric data, it was used to show the percentage number of the items.
B. Source of Data
Since this research belonged to content analysis, National Examination
items and KTSP document were the main sources of data (Carney, 1972). The
National Examination items were checked to the objectives stated inKTSP.
Besides the documents as the primary sources of data, the researcher also
used informants, namely, English senior high school teachers. The teachers
participated as informants to help the researcher understand more deeply the
object of the study (Lincoln and Guba, 1985 as cited by Moleong, 2006).
There were two teachers involved in this research. One teacher was from a
private senior high school and the other was from a state senior high school. The
research was conducted in each school where those teachers taught. Those schools
were chosen by a consideration that the National Examination items and KTSP
C. Research Instruments
Kountur (2003) explained that instruments of a research were all tools
which were used to collect data. In this study, there were two instruments applied
in order to collect the data needed. They were the researcher herself and interview
guidelines.
Researcher was the main instrument in this study since the researcher could
understand the facts related to the National Examination being studied in order to
obtain qualified findings (Moleong, 2006). The researcher collected the National
Examination, listed its items, and analyzed it. Meriam (2005) adds that the
researcher was the primary instrument not only for data collection but also for
data analysis. The researcher was the person who collected and analyzed the
National Examination.
The second instrument which the researcher employed was interview
guidelines. This instrument was used to interview some senior high school
English teachers. The type of the interview was open-ended questions since it
permitted free responses from the interviewees to answer the questions and to gain
as much information as possible. The data gathered from the interview was about
teachers’ opinion related to National Examination items, such as the capability of
the items to measure the objectives stated in KTSP, the types of the written texts,
etc. The data from the interview was used to support the primary data on the
D. Data Gathering Techniques
In conducting this research, the researcher employed two methods of data
gathering technique offered by Bungin (2007). They were the documentary
method and the interview method.
In this study, the researcher applied documentary method since this
research belonged to content analysis research which employed documents as the
main sources of data. The documentary method was a data gathering technique
where the researcher researched documents such as daily notes, letters, disc
(Bungin, 2007). The documents were National Examination and KTSP document
The researcher researched the facts existing in those documents. To process the
data systematically, the researcher used some tables to list the multiple-choice
items of the National Examination and several of written texts appeared in the
test.
The second method that the researcher employed was interview method.
The interview method was a data gathering technique where the researcher asked
some questions to informants (Bungin, 2007). The interviews were applied to
senior high school English teachers and conducted in informal situation. The first
interview was conducted on November, 20th 2008 forPangudi LuhurSenior High
School English teacher. The teacher was a representative for private senior high
school. The second interview was conducted on December, 3rd 2008 for SMA 10
English Teacher. The teacher represented state senior high school. The
information gained was the general description of the National Examination.
