THE MEANING OF THE 2015 ENGLISH NATIONAL EXAMINATION TO SENIOR HIGH SCHOOL STUDENTS
A THESIS
Presented as Partial Fulfillment of the Requirements for the Degree of Magister Humaniora (M. Hum.)
in English Language Studies
Sabina Thipani
Student Number: 146332021
THE GRADUATE PROGRAM IN ENGLISH LANGUAGE STUDIES SANATA DHARMA UNIVERSITY
i
THE MEANING OF THE 2015 ENGLISH NATIONAL EXAMINATION TO SENIOR HIGH SCHOOL STUDENTS
A THESIS
Presented as Partial Fulfillment of the Requirements for the Degree of Magister Humaniora (M. Hum.)
in English Language Studies
Sabina Thipani
Student Number: 146332021
THE GRADUATE PROGRAM IN ENGLISH LANGUAGE STUDIES SANATA DHARMA UNIVERSITY
vi ABSTRACT
Thipani, Sabina. 2015. The Meaning of the 2015 English National Examination to Senior High School Students. Yogyakarta: The Graduate Program in English Language Studies, Sanata Dharma University.
The National Examination was always defined by the authorities (the World Bank, the government, critics, teachers, and researchers). Especially by the World Bank and the government, learners are not involved in the decision making and defining process of the National Examination. This situation encouraged me to conduct research to dig learners‟ lived experience regarding the National Examination. It asks the meaning of the 2015 English National Examination to senior high school students.
To answer the research question, phenomenology approach is implemented. The texts of this researched were gained through interviews. The setting is Western Indonesia and the participants were Senior High School students which experienced the 2015 English National Examination as students. The nature of this research is hermeneutic phenomenology. In order to find the meaning, the text was analyzed by using the following steps: 1) horizonalization, 2) clustering and thematizing, 3) individual textural descriptions, 4) individual structural description, 5) individual textural-structural description, 6) composite textural description, 7) composite structural description, and 8) synthesis of structural and textural description.
Five themes are found from the text analysis process. They include: 1) others’ concern, 2) independency, 3) graduating as focus, 4) anxiety, 5) self-understanding. In the National Examination process, the participants are affected by parents, teachers, and friends. Most participants in this research considered independency important. Their independency was reflected through their initiative, stable motivation, and autonomy. Not only considering independency important, the participants also considered graduating important. During the preparation for and on the D-day of the examination, anxiety was the feeling which dominated the participants and affected their actions. The National Examination also helped the participants to understand themselves better. Because of the National Examination, they could identify their weaknesses, ways to improve their weaknesses, and aspects they improved because of the National Examination.
The research results showed that the participants‟ way of comprehending the National Examination is different from the authorities. However, the National Examination still needs to be conducted because the research results show it helped the students sharpen their independency and understand themselves better. Nevertheless, the policy makers, schools, and parents need to make sure the students improve themselves well in the process. Future researchers could involve participants from other levels of education or students who fail the examination.
vii ABSTRAK
Thipani, Sabina. 2015. The Meaning of the 2015 English National Examination to Senior High School Students. Yogyakarta: The Graduate Program in English Language Studies, Sanata Dharma University.
Ujian Nasional (UN) hampir selalu didefinisikan oleh otoritas (Bank Dunia, pemerintah, kritikus, guru, dan peneliti). Terutama oleh Bank Dunia dan pemerintah, siswa tidak dilibatkan dalam proses pengambilan keputusan dan pendefinisian UN. Situasi ini mendorong saya untuk melakukan penelitian yang bertujuan menggali lived experience siswa ketika menjalani UN. Rumusan masalah penelitian ini adalah: Apa makna UN Bahasa Inggris Tahun 2015 bagi siswa SMA?
Untuk menjawab rumusan masalah tersebut, metode fenomenologi digunakan dalam penelitian ini. Teks dalam penelitian ini diperoleh melalui wawancara. Latar penelitian ini adalah Indonesia Barat dan partisipan penelitian ini adalah siswa SMA yang mengalami UN Bahasa Inggris Tahun 2015. Penelitian ini adalah penelian fenomenologi hermeneutik. Untuk menggali makna, teks yang diperoleh dianalisa melalui langkah-langkah berikut: 1) horizonalisasi, 2) pengelompokan dan pentemaan, 3) pendeskripsian struktural masing-masing partisipan, 4) pendeskripsian tekstural masing-masing partisipan, 5) pendeskripsian tekstural-struktural masing-masing partisipan, 6) pendeskripsian struktural seluruh partisipan, 7) pendeskripsian tekstural seluruh partisipan, dan 8) sintesis deskripsi struktural dan tekstural.
Ditemukan lima tema dari proses analisa teks. Kelimanya mencakup: 1) pengaruh orang lain, 2) independensi, 3) lulus sebagai tujuan, 4) kecemasan, 5) pemahaman diri. Partisipan juga dipengaruhi oleh orang tua, guru, dan teman mereka. Sebagian besar partisipan dalam penelitian ini menganggap independensi penting. Independensi mereka terlihat dari inisiatif mereka, independensi mereka, dan kemandirian mereka. Selain independensi, partisipan juga menganggap kelulusan sebagai hal penting. Selama persiapan dan pengerjaan UN, kecemasan adalah perasaan yang mendominasi partisipan dan mempengaruhi tindakan-tindakan mereka. UN juga membantu partisipan memahami diri mereka dengan lebih baik. UN membantu mereka mengetahui kekurangan mereka membantu mereka meningkatkan kelemahan mereka, membantu mereka menemukan strategi untuk menghadapi UN, dan membantu mereka mengidentifikasi aspek-aspek dalam dirinya yang mengalami peningkatan karena UN.
Hasil penelitian menunjukkan bahwa partisipan memaknai UN secara berbeda dengan otoritas. Meskipun demikian, UN tetap perlu dilakukan karena hasil penelitian ini menunjukkan bahwa UN membantu siswa mengasah independensi mereka dan memahami diri mereka dengan lebih baik. Namun, pembuat kebijakan, sekolah, dan orang tua perlu memastikan bahwa siswa dapat mengembangkan diri mereka dengan baik dalam proses UN. Peneliti di masa mendatang dapat mengeksplorasi partisipan dari level pendidikan yang berbeda atau partisipan yang gagal dalam UN.
viii
ACKNOWLEDGMENTS
I would like to first of all thank my thesis advisor, Dr. J. Bismoko, for his guidance, patience, inspiration, and critical insights, not only during my research but also during my study in KBI. The energy he has always put when teaching and guiding us has always motivated me to be a better educator and researcher. I am also thankful for the chance he gave us to get acquainted with phenomenology, a study which helped me answer a lot of questions I had as a student.
I would also like to express my deepest gratitude to Dr. B. B. Dwijatmoko, M.A., F.X. Mukarto, Ph. D, and Dr. E. Sunarto, M. Hum for the constructive criticism and insights during the process of thesis review and thesis defence. I am also thankful that I have the chance to be taught by all lecturers in KBI. Their guidance had helped me to understand English language education study more comprehensively and to develop discipline in learning and researching.
My warmest thanks also go to my family: my mother, my father, and my brother. I thank them for the support, love, and inspiration they always give in every step of my life. I would also like to thank Wahmuji for our companionship, our conversations and insightful criticism he gave during my research and for his patience and support during the hard times.
I would also like to express my sincere gratitude to friends in KBI for the time we spent and the process we had during our study. I hope what we have learnt from each other will help us to grow in the next phase of our life.
Last but not least, I would like to express my deepest gratitude to all my participants for their willingness to share experiences and their cooperation during the research. Their experiences have helped me learn more and understand not only other humans but also myself. I owe them a lot and wish them all the best.
ix
STATEMENT OF LICENSE AGREEMENT... v
ABSTRACT... vi
CHAPTER II LITERATURE REVIEW... 14
A. THEORETICAL REVIEW... 14
a. Research on the National Examination... 21
b. Criticism of the National Examination... 25
1) The National Examination to Increase Intelligence and Improve Education Quality... 25
2) The National Examination Widen the Academic Achievement Gap... 26
3) An Incomprehensive Test... 28
4) The National Examination Had the Potential to Increase Dropping Out Risk... 29
5) Pressure on Students and Teachers... 30
6) Cheating... 31
7) On Mapping Education Quality... 33
c. Recommendation for a Better Assessment System... 34
1) The Importance of Evaluating the National Examination... 34
2) The Importance of Evaluating the Government... 34
3) A Better Assessment System... 36
4) How to Determine Students‟ Graduation... 36
d. The 2015 National Examination in Indonesia... 37
4. Senior High School... 39
x
a. Development of Phenomenology... 39
b. The Nature of Phenomenology... 40
c. Narrative and Hermeneutic Phenomenology... 42
d. Important Concepts in Phenomenology... 43
1) Consciousness... 43
e. Research on Lived Experience... 51
f. Phenomenological Research and Research on Perception... 56
B. THEORETICAL FRAMEWORK (PRE-ASSUMPTION) 59 CHAPTER III METHODOLOGY... 66
D. DESCRIPTION AND INTERPRETATION... 69
1. Horizonalization... 69
2. Invariant Constituent Clustering and Thematizing... 69
3. Individual Textural Descriptions (Stories) ... 70
4. Individual Structural Description (Interpretations)... 70
5. Individual Textural-Structural Descriptions (Stories and Interpretations) ... 71
6. Composite Textural Description (Stories) and Composite Structural Description (Interpretations)... 71 7. Synthesis of Textural and Structural Description (Meaning)... 72
E. TRUSTWORTHINESS... 72
CHAPTER IV DESCRIPTION AND INTERPRETATION... 75
A. INDIVIDUAL TEXTURAL DESCRIPTIONS (STORIES) 75 1. Dylan‟s Textural Description (Story)... 75
2. Baez‟s Textural Description (Story) ... 79
3. Sting‟s Textural Description (Story) ... 84
4. Dolores‟ Textural Description (Story) ... 88
B. INDIVIDUAL STRUCTURAL DESCRIPTIONS (INTERPRETATIONS) ... 94
1. Dylan‟s Structural Description (Interpretation)... 94
2. Baez Structural Description (Interpretation)... 99
xi
4. Dolores‟ Structural Description (Interpretation)... 107
C. INDIVIDUAL TEXTURAL-STRUCTURAL DESCRIPTIONS (STORIES AND INTERPRETATIONS)... 109
1. Dylan‟s Textural-Structural Description (Story and Interpretation) ... 109
2. Baez‟s Textural-Structural Description (Story and Interpretation) ... 112
3. Sting‟s Textural-Structural Description (Story and Interpretation) ... 114
4. Dolores‟ Textural-Structural Description (Story and Interpretation) ... 117
D. COMPOSITE TEXTURAL DESCRIPTION... 120
E. COMPOSITE STRUCTURAL DESCRIPTION... 124
F. TEXTURAL-STRUCTURAL SYNTHESIS (MEANING)... 128
1. Themes... 129
a. Others‟ Concern... 130
b. Independency... 131
c. Graduating as Focus... 132
d. Anxiety... 132
e. Self-understanding... 133
2. Summary... 133
CHAPTER V CONCLUSIONS... 136
A. RESEARCH RESULTS... 136
B. IMPLICATIONS... 138
C. RECOMMENDATIONS... 139
BIBLIOGRAPHY... 141
xii
LIST OF TABLES
Table 2.1.: The Focus of the Recent Research Compared with the Previous
Research Reports... 55 Table 2.2.: The Government and Critics/Teachers on the National
xiii
LIST OF APPENDICES Appendix 1
Interview Questions... 147
A. General Main Questions... 147
B. Follow Up and Probing Questions for All Participants... 148
C. Follow Up and Probing Questions for Dylan... 148
D. Follow Up and Probing Questions for Baez... 152
E. Follow Up and Probing Questions for Sting... 153
F. Follow Up and Probing Questions for Dolores... 155
Appendix 2 Invariant Constituent Clustering and Thematizing... 160
A. Dylan‟s Invariant Constituent Clustering and Thematizing... 160
B. Baez‟s Invariant Constituent Clustering and Thematizing... 164
C. Sting‟s Invariant Constituent Clustering and Thematizing... 172
D. Dolores‟ Invariant Constituent Clustering and Thematizing... 177
1
CHAPTER I INTRODUCTION
This chapter is to describe the justification and feasibility of the research
project. From discussion on the relevance and benefit in all sub chapters, the
justification of this research is reflected. The feasibility of this research can
especially be seen under the discussion of Background, Problem Identification,
Problem Limitation, and Problem Formulation.
A. BACKGROUND
In the popular context, the National Examination was always defined by the authorities, especially the World Bank and the government. Learners or students‟
opinions were almost never considered. For the World Bank, the National
Examination, or what was called national assessment in the more universal
context, was merely one of the tools they needed to have in order to achieve
inclusive and sustainable globalization. For the government of Indonesia, it was a
great assessment which they thought would help students to improve motivation
and achievement, and also the schools and education system to improve their
quality. Contradictory to the World Bank and the government, Indonesian critics
and teachers saw the National Examination as an assessment which was not
comprehensive, which could increase dropping out risk, which gave pressure to
teachers and students, and thus needed to be evaluated and be reformed into a
better assessment system. The critics were also more sympathetic to students than
2
the government seemed to listen to teachers and critics more, learners‟ thought
were still not seen as one of the sources used in policy or decision making.
When I was a student, I found this situation hard. I remember feeling nervous
and scared in Junior High School since I knew that the National Examination would be used to determine my and other students‟ graduation. I thought the
burden felt heavier especially because I was always among the best three in class.
Because I was always among the best three I thought it would be so embarrassing
if I did not pass the National Examination.
Not so long after the National Examination was over, the headmaster of the
school called me. She told me that I passed the examination, but not with my own
real scores. None of the students in my school passed if the school used their real
scores. My problem was not in my average score. It was one of my scores which
was less than what was standardized. She told me that she lobbied the authority in Education Office to fake my scores and other students‟ so that we could pass.
When I knew the fact, I felt shocked and sad. I was not glad even though I passed
the examination. I felt incapable, not deserving the graduation. I felt traumatic
ever since, felt that I could never be able to be good again academically. I did not
thank nor protest the decision taken by the school to all of us because I thought I
was powerless, I was just a student.
When I was in Senior High School, I have never got rid of the National
Examination. Its function to determine graduation always haunted me during my
study. Even so, the burden I had in Senior High School was not as heavy as the
burden I felt in Junior High School. Since my Senior High School was one of the
3
be even the best ten in class. However, I knew that not passing the National
Examination was a huge embarrassment for us. I still felt scared, panic, and most
of all mad with the fact that the National Examination was used to determine my
graduation.
The combination of these feelings, especially the anger, made me keep
complaining during my study. I often shared my anxieties to friends and my
mother. I complained about how unfair the examination was for me and other
students. The examination was unfair because it determined my three-year process
of learning in only three days. The examination only asked 50 questions for each
subject. Meanwhile, I learnt so much more. I did not think the examination could
be used to tell the world if I was a good student or not. I also complained about
how the government misunderstood me as a student. On television and
newspaper, the government kept saying about how great the examination was and
how important it was for the students. Meanwhile, at the same time I did not think
they knew the anxieties I had as a student. I felt my life was defined and
controlled by the people who could not sympathize with me or other students at
all. I complained almost all the time but I did not know how the complaints could
be heard. I thought I was just a student. There was nothing I could do.
The combination of the feelings I mentioned above, especially the panic, made
me decide to join a study course. I have never joined any courses before because I
could study better if I could understand things by myself. I could study better if I
could manage everything by myself. After I finally joined the course, I indeed did
not feel comfortable studying. In the course I was always told what to do. We
4
how to pragmatically solve the questions. It bored me because I felt studying was
fun if I could fix things myself and know the basic and concept map of the
subject. I had to adapt more because studying in the course was very different
from studying by myself. Eventually, after joining the course my scores decreased
drastically.
Another impact of the feelings I had during my study in Senior High School
was me ended up cheating. Other than all of those feelings, at that time I had some
other justification. Some of them were related to the arguments I had mentioned
above. I thought the government was unfair. Thus, it was also fine for me to do
the examination unfairly. I also thought, because I did not have any access to
express my opinion and anger, cheating was the only room left for me to express
it. What I meant by cheating, though, was not depending the whole examination to
others. When preparing for the examination I heard there would be leaks of
examination answers. I eventually knew that some of my friends bought them and
speeded them to the whole school. Almost all students used them. I used the
leaked answers only as reference. There were a couple of versions of answers. I
did all examination questions by myself and matched them with the leaked
answers. When they were different I rechecked my work again.
As a student I thought graduating was important. However, I thought it would
be fair if graduating was determined by my three-year studying process in Junior
or Senior High School. Since the graduation was determined by the National
Examination, an examination which I thought not considering process, I felt
burdened with it. During my study I, most of the time, felt scared and panic;
5
though I prefer studying alone. I even decided to cheat even though I have never
cheated before.
As a student I also felt powerless. When my Junior High School headmaster
told me that my scores were faked, I did not react. I also had intense anxiety
especially when I was in Senior High School. I felt the examination was unfair
and I felt misunderstood by the authority. However, I did not do anything than just
sharing the opinion to friends and my mother. I did not do anything than just
cheating to express my anger. I did not do anything because as a student I felt
powerless.
My lived experience encouraged me to conduct research on others‟ lived
experience on the National Examination. By conducting research on the topic, I
hope to give the chance to learners or ex-learners to deliver their lived experience
to important parties in education world. As a learner, I felt I never had the
empathy I needed when dealing with the National Examination. Thus, I also hope the research will evoke teachers‟, critics‟, government‟s and policy makers‟
empathy on learners‟ experience regarding the National Examination. Other than
that, by conducting this research I hope I will be able to identify myself with the participants‟ experience and to be able to have more scientific understanding on
my and others‟ lived experience regarding the National Examination. I also hope I
will be able to enhance my ability in digging and delivering my and others‟ lived
experience on the National Examination scientifically.
The lived experience on the National Examination that I had was experienced
when I was in junior and senior high school. However, in this research I chose to
6
first, I considered senior high school students have already had richer experience
on the National Examination since they had experienced the examination for three
times (in elementary school, in junior high school, and in senior high school). The
second reason is because I consider senior high school students have better
capability to articulate their lived experience compared to students in lower levels.
It is because hey are more mature compared to students from lower levels.
As stated previously, the authorities had not involved learners in their
arguments regarding the National Examination. Different attention to learners was
given in academic world. The attempt which academicians made to have learners‟
involvement in their research can be seen in three of six research reports regarding the National Examination I found. The first research discovered one student‟s
lived experience on the English National Examination (Fiharsono, 2011). It discovered four themes regarding the topic: (1) student‟s understandings, (2)
student‟s beliefs, (3) student‟s intentions, and (4) student‟s action. The second
research dug six qualities of useful language tests in the 2012 English National
Examination (Thipani, 2013). The findings involved the 2012 English National Examination‟s reliability, validity, authenticity, interactiveness, impact, and
practicality. The third research questioned the washback experienced by the
twelfth graders of a private religion-based vocational high school in Yogyakarta
regarding the National Examination preparation (Subagyo, 2014). The research
showed that the National Examination encouraged the students to be more prepared with “potential challenges such as pressure and time constraints” and
those teachers could “comprehend the phenomenon faced by their students”
7
Other than the three research reports, I found three other research reports
discussing the National Examination. The first research discussed the relevance
between the Senior High School National Examination multiple choice reading
items with objectives of English reading skills in KTSP (Merina, 2009). The
research showed that the National Examination only covered two cognitive
processes of Anderson taxonomy (remember and understand) and did not cover
all cognitive processes as required in KTSP. However, the National Examination
had covered two types of written text in KTSP (short functional text and simple
essays). The research also showed that vocabulary mastery was measured even
though it was not required in KTSP. The second research discovered EFL teachers‟ voices regarding the National Examination (Siswanti, 2010). The
researched showed the participants‟ different opinion how the National
Examination should be conducted. One of the participants thought the National
Examination should be conducted the way it had been conducted meanwhile
another one thought the National Examination should be conducted by adjusting
to vocational high school students‟ competencies. However, both of them thought
the National Examination should be based on English for Specific Purposes. The
last research discovered content validity and authenticity of the 2012 Senior High
School English National Examination (Widyaningrum, 2014). The research
showed that the 2012 Senior High School English National Examination was
98.8% valid in content and 79.5% authentic.
I found ten research reports discussing English acquisition with
phenomenology. Only one research discusses English education design. Another
8
them try to discover meaning from the perspective of the learners. The rest two
focused on meaning from the perspective of the teachers.
The first research on design discovered the meaning of textbook selection to
SMP English teachers (Kiswara, 2009). The research showed that all participants “tried to meet the students‟ learning needs, interest and cultural background”
(Kiswara, xiv) in the process of textbook selection, even though the focus they
have regarding learning needs was different. The participants also had different
level of autonomy which affected their perceptions and the end their process of
selecting textbooks.
The second research was elaborated on the discussion on the National
Examination above. It was written by Fiharsono (2011).
The third research, which focused on implementation, discussed the meaning
of the children lived experience in Turkey on learning English through computer
games (Turgut & Irgin, n.d.). The finding shows that the students found benefits
(including transfer, motivation, and awareness) by learning English through
computer games (Turgut & Irgin).
The fourth research, also focusing on implementation, discussed Colombian
adult English language learner selection and using of language learning strategies
(Paredes, 2010). The research showed that the learners used learning strategies which are not included in Oxfords‟ taxonomy (Paredes).
The fifth research discovered the meaning of learning English to Theology students (Trisnowati, 2011). The research finding showed five themes: “respect
for language use, reflection on language learning experience, self-knowledge,
9
themes she found two essences. The first one is “meaningful language learning in
the sense of respect for language use” and the second one is “a desire in seeking
for understanding of the meanings of life through language learning in theology studies” (Trisnowati, xv).
The sixth research discussed the meaning of blog-assisted learning in the ESL
writing classroom to Taiwanese students (Lin, Groom, & Lin, 2013). The research
showed that blog-assisted ESL writing learning made students with low
proficiency levels feel anxious and embarrassed of the reactions from other
students regarding their works (Lin, Groom, & Lin).
The seventh research discovered the meaning of vocabulary learning using
IPALL to students (Sastrikirana, 2015). From the research she found empirical themes (“better knowledge of word meaning and forms, better learning
atmosphere, chances to repeat, focus and concentration” (Sastrikirana, xvii)) and transcendent themes (“perseverance, honesty, acceptance, connection with a
bigger world, God talks to me” (Sastrikirana, xvii)).
The eighth research discovered the meaning of leaning English to International
Class students of University of Muhammadiyah Yogyakarta (UMY) (Stania,
2015). She found two interpreted themes which are related to body and mind and
to God.
The ninth research discovered the meaning of lived experience of “becoming
and being a woman teacher” and women experience on teaching boys and girls
(Kirk, 2004). The research result is description of the meaning of experience of
becoming and being a woman teacher and women experience on teaching boys
10
The last research discovered the meaning of teaching English large classes to a novice teacher (Astuti, 2010). Specifically she discovered what “the teacher
thinks about teaching English large classes, how she gives meaning to it, how she
conducts and approaches her teaching, how she experiences success and failure,
what she learns from it to become more empowered and self-fulfilling” (Astuti,
xiv).
The research I conducted aim to discover the meaning of the lived experience
on the 2015 English Senior High School National Examination to the Senior High
School students. Thus it is expected to contribute to lived-experience studies on
English Examination in particular and English evaluation in general from the
perspective of the learners. In the broader scope, this research is expected to
enrich discussion in English acquisition study.
B. PROBLEM IDENTIFICATION
This research aims to discover the meaning of the 2015 English National
Examination to Senior High School students. The English National Examination
is a smaller part of English assessment. Meanwhile, English assessment is a type
of English evaluation. In English education, English evaluation is one of the
divisions other than concept, design, implementation, and revision. English
education itself is part of English acquisition. This research is thus, as stated
previously, a research on a smaller part of English acquisition.
Other than that, since the research aim to discover meaning, this research can
be identified as a phenomenology research. More explanation on this can be seen
11
C. PROBLEM LIMITATION
One of the aspects of the research context is the National Examination. This
research will thus be mostly relevant to audiences in Indonesia even though there
is also possibility that the research may have relevance to broader audiences who
experience or are involved in or exposed to national assessment.
Another aspect is 2015. This research will thus be actual mostly to those
involved in the National Examination near the year 2015 in the future. However,
there is possibility that the research can be relevant to the need of the next
national examinations in the further future.
Senior High School is also one of other aspects of the problem. This research
will thus be mostly relevant to those involved in the Senior High School National
Examination. However, there is possibility that the research can be useful to other
levels of the National Examination.
Not only that, the research is a phenomenological research. This research thus
has strength in its depth. However, it is weak in its width.
D. PROBLEM FORMULATION
This research aims to discover the meaning of the lived experience on the 2015
English Senior High School National Examination to the Senior High School
students. In order to reach the aim, I proposed the following research question:
What is the meaning of the 2015 English National Examination to senior high
school students?
E. RESEARCH GOALS
The immediate goal of the research is to discover the meaning of the lived
12
Senior High School students. The intermediate goal is to contribute to the
discussion on the National Examination in particular and evaluation in general.
This research also aims to enrich discussion on phenomenological research.
The ultimate goals of the research are more reflective, illuminating, empathic,
self-actualizing, and self-fulfilling individuals. From the research, the participants
and the researcher are demanded to reflect on their and others‟ lived experience.
By reflecting, the participants and researcher are illuminated. From the research,
the participants and the researcher can also actualize themselves. The participants
actualize themselves by expressing their textural reflection while the researcher
actualizes herself by, as stated previously, digging phenomenological sensitivity, exercising hermeneutic ability, and exploring the participants‟ textual reflection.
From the research, the audience can understand others‟ lived experience and are
expected to have empathic understanding on it. Empathic understanding was the
key to better equity. With better equity, individuals can be more autonomous.
Autonomy brings us to empowerment and empowerment makes use be more
self-actualized, and in the end, more self-fulfilling.
F. RESEARCH BENEFITS
The research can bring benefits to the research participants, researcher, and audience. The research participants‟ benefits from having the chance and a
channel to deliver textual reflection on the lived experience they have. By doing
this, they have the chance to be illuminated and to actualize themselves.
Meanwhile, the researcher can actualize self by experiencing pedagogical
13
ability, and having the chance to explore others‟ textual reflection. This process
also brings the researcher to illumination.
The audience, on the other side, which includes students, teachers, critics, or
policy makers and the government, benefits from enhancing their empathic
understanding to learners and from having materials they can use as
considerations to their actions. Students can identify themselves with the research
results and reflect their actions with them, and use the reflection to act with more
consideration. Teachers can use the research results as reflection materials and to
use the reflection to teach more empathetically. Critics can use the research results
as reflection and insights to their criticisms. Similarly, policy makers and the
14
CHAPTER II LITERATURE REVIEW
This chapter is to discover the logical truth of the research. In the Theoretical
Review, relevant concepts of the study, which includes phenomenology and the
national examination pre-understanding, are presented. In the Theoretical
Framework, pre-assumption from the concepts are presented.
A.THEORETICAL REVIEW
In this part, universal truth of the relevant concepts and concepts relation are
exposed. The concepts meant are related to phenomenology and the National
Examination. This part is divided into five more parts: Assessment and Test,
National Assessment, National Examination, Senior High School, and
Phenomenology.
1. Assessment and Test
The following description contains discussion on assessment and test. It also
contains discussion on the relation between the two terms with National
Assessment and National Examination.
Further discussion on the National Assessment can be seen after the discussion
on assessment and test. Meanwhile, further discussion on National Examination
can be seen after the discussion on National Assessment.
a. Assessment
Assessments is a broad term defined as a process for obtaining information that
is used for making decisions about students; curricula, programs, and schools; and
15
example, we mean we are collecting information to help us decide the degree to
which the students has achieved the learning targets (Nitko and Brookhart, 2011,
3). Scheerens, Glas, & Thomas (2003) define assessment as the information
gathering on the achievement of individual student. In the context of language
teaching, assessment can be defined as a process of collecting information about aspects of test taker‟s language ability systematically, on the basis of substantive
ground (Bachman & Palmer, 2012).
Assessment can thus be defined as an information gathering process about
individual student language achievement/ability/competence which is done
systematically, based on certain substantial basis. The term „assessment‟ in the discussion under subtitle „National Assessment‟ refers to this definition. The
system which is supported by National Assessment is the system created by the
World Bank in its member countries in order to reach its vision.
From the discussion under subtitle „National Assessment‟ it can be concluded
that National Assessment belongs to analytical assessment. Analytical assessment
provides specific feedback along several dimensions (Lane, 2008). It breaks
down the performance into different levels of behavior expected, assigning each a
point value (which can be weighed if desired), and which are tolerated for a
quantitative measure (Blaz, 2001). In other words, it is an assessment which
measures several dimensions, each of which with single score, which are broken
down from a general performance quantitatively.
National Assessment also belongs to general assessment. General assessment
uses criteria and descriptions of performance that generalize across (can be used
16
National Assessment also belongs to educational assessment. It is because its
characteristic is similar to educational assessment, which is “integral to the educational process” (construct) (Chatterji, 2003, 2).
National Assessment is also a summative assessment because it has the characteristics of summative assessment. Summative assessment “aims to
measure, or summarize, what a student has grasped, and typically occurs at the
end of a course or unit of instruction” (Heritage, M., 2010, 6).
National Assessment is also related closer to formal assessment. Formal assessment is “a formal, systematic, summative information-gathering procedures
that are used to make decision about pupil grades, promotion, placement, or
honors. (Airasian, 1991, 191).
b. Test
A test is “a method of measuring a person‟s ability, knowledge, or performance
in a given domain” (Brown, 2004, 3). Method is “instrument (a set of techniques,
procedures, or items)” (Brown, 3). It “must be explicit and structured” (Brown, 3).
Measure refers to result offering to test takers (Brown). Individual refers to who
the test taker is, her/his experience and background, and her/his abilities (Brown).
Performance implies test taker‟s ability/competence (Brown, 3) while given domain refers to measurement of “the desired criterion and not include other
factors inadvertently” (Brown, 4).
A test can also be defined as a type of assessment in which formal tasks are
presented to pupils to obtain systematic evidence about their performance
17
representative indication of pupil learning based on the instruction provided.
(Airasian).
Test can thus be defined as a method to measure student‟s competence in
certain domain formally. The term „examination‟ in „National Examination‟ refers
to this definition.
National Examination can be categorized as paper-and-pencil language test.
Paper-and-pencil language test “take the form of the familiar examination question paper” (McNamara, 2000, 5). It is “typically used for the assessment
either of separate components of language knowledge (grammar, vocabulary, etc.) or of receptive understanding (listening and reading comprehension” (McNamara,
5). The test items in this type of test has “fixed response format” (McNamara, 5).
Multiple choice is one of the most important format (McNamara).
National Examination can also be categorized as achievement test. Achievement test is a test which is “associated with the process of instruction”
(McNamara, 2000, 6). It is usually conducted at the “end of course tests, portfolio
assessments, or observational procedures for recording progress on the basis of
classroom work and participation” (McNamara, 6).
2. National Assessment
One of the themes which World Bank sets to promote an inclusive and
sustainable globalization is World Bank role as a unique and special institution of knowledge and learning. This theme requires World Bank to “focus continually
and rigorously on results and on the assessment of effectiveness” (Greaney &
Kellaghan, 2008, ix). This assessment includes assessment in education world.
18
World Bank in economic development since economic development requires
human knowledge and skills (Greaney & Kellaghan). National assessment
provides the information regarding individual student achievements in knowledge
and skills which could be useful for World Bank (Greaney & Kellaghan).
Not only for World Bank, information on knowledge and skills is also
important for governments and students. Higher levels of knowledge and skills are
needed if students would like to participate in the working world in the future
(Greaney & Kellaghan, 2008). Skillful and knowledgeable workers are what the
government needs in order to compete successfully in globalization (Greaney & Kellaghan). Other than that, governments also “need to monitor achievement
levels to determine how changes in enrollment and budgetary conditions affect the
quality of learning” (Greaney & Kellaghan, 18). National assessment has helped
ministries of education to “describe national level of learning achievement,
especially in key subject areas, and to compare achievement levels of key sub
groups” (Greaney & Kellaghan, 2). It has also been useful to help the ministries “to support or refute claims that standards of student achievement are rising or
falling over time” (Greaney & Kellaghan, 2).
National assessment is also claimed to be significant for policy makers,
politicians, and educational community. It is considered effective in providing
feedback regarding significant measures to these parties (Greaney & Kellaghan,
2008). It provides information on aspects of the system and basic inputs needed
by the system for policy making, information which the government often lacks
19
classroom” (Horn, Wolff, and Velez 1992 as paraphrased in Greaney &
Kellaghan, 19).
However, World Bank considers assessment of effectiveness in education field
as the biggest challenge (Greaney & Kellaghan, 2008). This includes national
assessment. Even though is considered plays important roles in giving information
regarding individual student achievements in national level, not many countries systematically monitors the learning outcomes “either through conducting their
own assessments of student achievement or through participating in regional or
international assessments” (Greaney & Kellaghan, ix-x). The information
gathered from the national assessments which have been conducted in some
developed countries has not been maximally used (Greaney & Kellaghan). The
problems mentioned above might be caused by: the policy makers who are not
dedicatedly and committedly involved in the assessment and
unwell-communicated assessment analysis results (Greaney & Kellaghan). Another
reason is national assessment which is only used symbolically as a scientific and
modern tool to legitimate state action thus it was not seriously used in “management of the education system or in policy making“ (Greaney &
Kellaghan, 21).
Indonesian government has been implementing national assessment since
1950. The first term used to call national assessment was Ujian Penghabisan
(1950-1960) (Cessnasari, 2005). It was then called Ujian Negara (1965-1971)
(Cessnasari). The term was then changed again into EBTANAS (Evaluasi Belajar
Tahap Akhir Nasional) (1980-2001) (Cessnasari, 2005). In 2002 it was called
20
was centralized (Rachman, 2015, April 13). There was a national body preparing
questions (Rachman, 2015). This year the government set a minimum passing
grade, which would be set higher periodically (Rachman, 2015). The grades were
used to determine whether the students could graduate from certain education
level or not. Since 2005 the term has been known as UN (Ujian Nasional/National
Examination). The term UN has been used until now (2017). In 2011, not only the
National Examination was held. Local School Examination (Ujian Sekolah/US)
was also held at the end of each level. To decide whether a student could graduate
or not, the government combined the National Examination score (worth 60% to
the total score) with School Examination score (worth 40% to the total score).
“According to Peraturan Kementerian Pendidikan Nasional Nomor 34 Tahun
2007 (The Law of the Ministry of Education Number 34 Year 2007)”, the
National Examination functions as 1) the tool to map the quality of students and
schools and/or educational program, 2) select university student candidates, 3)
determine whether students pass or fail from school and/or educational program,
and 4) support and help schools to improve the quality of education (Sarasvati,
2012, 230). The National Examination was “held by Badan Standar Nasional
Pendidikan (BSNP)” (Sarasvati, 2012, 228). It was “the only body that is allowed
to hold the national examination in Indonesia” (Sarasvati, 2012, 228). The examination was compulsory (Sarasvati). It was “held at the end of each level of
study“ (for students in the 6th grade of elementary school, 9th grade of secondary
school, and 12th grade of high school) (Sarasvati, 229). The format of the examination was multiple choices (Sarasvati). The students used “a special paper
21
computational device” (Sarasvati, 229). As stated previously, since 2003, the
students must pass certain grade in order to graduate from certain level of study
(Sarasvati). If students failed the examination, they could “join a remedial examination” (Sarasvati, 230). It was “held a week after the notice” (Sarasvati,
230). If students still failed, up to 2012, Sarasvati stated that there was no clear
procedure that the students could follow in order to pass the examination.
Sometimes the students were suggested to keep continuing the study in the higher
level as long as in the first year they could get certificate showing they had passed
the examination (Sarasvati). Sometimes they are suggested to join PKBM and
finish their study there (Sarasvati).
This part elaborates concepts related to National Examination. It is divided into
three parts: national assessment, national assessment in Indonesia (before 2015),
and the 2015 national examination in Indonesia.
3. National Examination
a. Research on the National Examination
Previous research on the National Examination and the English National
Examination discussed washback/impact, validity, authenticity, practicality, reliability, interactiveness, administrability, student‟s action and teachers‟ voices.
All topics were discussed by six researchers. Three of them used (Merina, 2009;
Thipani, 2013; Widyaningrum, 2014) content analysis method. One of the three
(Thipani, 2013) used both content analysis and sample survey. One (Siswanti,
2010) used progressive qualitative approach. One research (Fiharsono, 2011) used
phenomenology and another one used (Subagyo, 2014) case study. I will describe
22
Research on the National Examination washback was conducted by Subagyo (2014). In his research he had two questions: (1) “What washback do the twelfth
graders of a private religion-based vocational high school in Yogyakarta
experience in response to the National Examination preparation?” and (2) “How
does the washback happen to the students, teachers, and the school?” (Subagyo,
vii). In his research he found that secondary students were more aware of the
impacts of the National Examination. He also found that the National
Examination encouraged the students to be more prepared with “potential
challenges such as pressure and time constraints” (Subagyo, vii). His research also showed that teachers could “comprehend the phenomenon faced by their students”
(Subagyo, vii). In my research on 2013 I also discussed it even though I used the term „impact‟. My research showed that “the test developer goals were in accord
with the society/education goals” but “did not bring significant impact to the
students and teachers” (Thipani, vii). Fiharsono‟s research also showed the
National Examination washback to his participant. The washback was seen from the participant‟s action to study previous tests and predictions (2011).
Research reports on the National Examination validity were conducted by
Merina (2009), Fiharsono (2011), Thipani (2013), and Widyaningrum (2014). In
her research Widyaningrum showed that 98.8% of the 2012 English National
Examination content was valid. My research on 2013 showed that the 2012
English National Examination was unconstructively valid. Not only discussed
validity in general like the two previous research reports, Merina (2009)
23
Examination test for senior high school students measure the objectives of English reading skills as stated in KTSP?”, (2) “Did the texts used in the National
Examination test for senior high school students represent the written text types
mentioned in KTSP?”, and (3) “Did the question types used in the multiple-choice
reading items of the National Examination test for senior high school students measure the students‟ comprehension through various question types?” (Merina,
5). The research showed that the National Examination only covered two
cognitive processes of Anderson taxonomy (remember and understand) and did
not cover all cognitive processes as required in KTSP. However, the National
Examination had covered two types of written text in KTSP (short functional text
and simple essays). The research also showed that vocabulary mastery was
measured even though it was not required in KTSP. Quite different from Merina, through the description of student‟s understandings and student‟s beliefs
Fiharsono (2011) explained not only the English National Examination‟s content validity, but also the English National Examination‟s criterion-related validity in
his research. According to Fiharsono, the English National Examination did not
represent all English competencies listed in the curriculum. Its content validity
was low. He also showed that the result of the English National Examination was
significantly different from other measurements conducted either at the same time
or in the future. Thus, the criterion-related validity was low.
Three research reports (Fiharsono, 2011; Thipani, 2013; Widyaningrum, 2014) discussed authenticity. Fiharsono‟s research showed that the competencies tested
in the English National Examination are not relevant to English competencies
24
showed that the 2012 English National Examination was “not utterly authentic”
(Thipani, vii). Meanwhile, Widyaningrum‟s research showed that the 2012
English National Examination was 79.5% authentic.
My research in 2013 discussed the 2012 English National Examination
practicality, reliability, and interactiveness. The research showed that the examination was “practical in the context of human resources and time allocation
availability but impractical in the context of material resources availability”
(Thipani, vii). It also showed that the exam was “reliable in the context of inter-rater and not reliable in the context of student, test administration, and test”. The
research also indicated that the examination could be considered interactive because the students‟ level and type of general education, their types and amount
of preparation, and their language ability helped them to be involved in the test.
However, it could also be considered not interactive because the students‟ family
background, topical knowledge, affective schemata, and strategic competence did
not help them to be involved in the test.
Fiharsono‟s research discussed the National Examination administrability and
student‟s action (2011). There were dishonest practices during test administration.
One of them was student buying test leak. Discussion on student‟s action include
Fiharsono‟s participant which did all intentions he had during the National
Examination.
Siswanti discussed teachers‟ voices regarding the National Examination in her
research (2010). Two of her participants had different opinion regarding how the
National Examination should be conducted. One of them thought the National
25
another one thought the National Examination should be conducted by adjusting to vocational high school students‟ competencies. However, both of them thought
the National Examination should be based on English for Specific Purposes.
b. Criticism of the National Examination
The National Examination had been a controversial issue in education in Indonesia. Harti writes “discussion or polemic on the National Examination
always happens every year, even all years in Indonesia until they become latent national education discussion” (n.d.). The controversy is constructed by two big
sides: the supporting side (the government) and the opposing side (the critics).
Retno Listyarti, general secretary of the Federation of Indonesian Teachers
Associations (Federasi Serikat Guru Indonesia/FSGI), said educators had been “protesting the system of the examination implementation for years” (Jakarta
Globe, n.d.). In 2006, “a group of teachers filed a lawsuit at the Central Jakarta
District Court demanding the abolition of the national exams. The court granted the request and the government appealed the verdict” (The Jakarta Post, 2014,
December 30). “The Supreme Court rejected the government‟s appeal” in 2009
(The Jakarta Post). They said “the examinations could only be held if their implementation was improved” (The Jakarta Post).
1) The National Examination to Increase Intelligence and Improve Education Quality
Jusuf Kalla, when serving as vice president of Indonesia in 2004-2009, stated
that the National Examination is important to help students increase their
intelligence. He believed that students can increase their intelligence if they study
26
examination would make Indonesian students, as he mentioned it, stupid
(Dharma, 2012). Because this examination (which he believed would make the
students smarter) was based on one standard, Kalla was sure that the examination
would make all students in Indonesia equally smart (despite different
infrastructure each school had) (Dharma). Believing in the magic this National
Examination had, Kalla also believed that in order to make our education level
was equal to Singapore or Malaysia, in five years, all the government must have
done was improving the examination difficult level every year (Dharma).
Different from Kalla who believed that the National Examination would be
able to increase students‟ intelligence, Dharma (2012) stated that, other than poverty, hunger, student mobility, health, safety, parent education, etc., students‟
intelligence would be affected by the input and process of education. Realistically,
the input and process were still far from standard. In rural areas, many school
buildings were broken; there were not enough teachers in schools, not enough
books (Dharma). The National Examination was not a tool to make Indonesian
students smarter. It was a tool to measure the output which was resulted from the
input and process of education.
2) The National Examination Widen the Academic Achievement Gap
The government believed that the National Examination was an appropriate way to improve students‟ motivation and achievement, even though, according to
critics, this believe the government had was just based on assumption (teKUN &
eF, 2012). According to critics, this belief showed that the government neglected other factors which might affect students‟ motivation, such as social and
27
socioeconomic status and school condition with academic achievement” (Willms,
2006; Fuchs & Womann, 2007, as paraphrased in teKUN & eF, 364).
As a part of universal interventions, the National Examination, which could be
included in graduating test, could indeed improve academic achievement in one
side (Willms, 2006 as quoted by teKUN & eF, 2012). However, on the other side, it could “establish or even widen academic achievement difference based on
socioeconomic status” (Willms, as quoted by teKUN & eF).
As a high stake test, the National Examination also had many consequences.
The positive consequence was that the examination would motivate the students
to be the best (teKUN & eF, 2012). Other than that, it could also help authority to
select graduates easily (teKUN & eF).
It also had negative consequences. Similar as its status as a part of universal
interventions, the National Examination as a high stake test could also “widen
academic achievement gap based on socioeconomic status” (Dee & Jacob, 2006;
Willms, 2006 as paraphrased by teKUN & eF, 2012, 364). It only improved
academic achievement of students with good social economical background or
students who studied in schools with good condition (Dee & Jacob, 2006, as
paraphrased by (teKUN & eF). If this type of examination is conducted since the
students were in their young age, the gap between the students will be more
drastic (teKUN & eF). This gap would form education elites (teKUN & eF).
High stake test, such as the National Examination, which was scientifically
proven made for those with good socioeconomic background can also be seen from Sarasvati‟s story in helping her students passing the National Examination
28
National Examination. According to Saravasti‟s explanation, her students did not
pass the National Examination because the school was not in standard condition
and her students were from lower class families. Her students also could not pass
the remedial examination and in order to join another remedial examination they
had to pay 250.000 rupiahs or had access to lobby the authority. Some of her
students did not pass the second remedial examination, even though one of them
had already got in to a vocational school and did well in the school. In order to
help this student passing the National Examination, Sarasvati had to lobby the ministry of education. Sarasvati‟s conclusion on her experience was, “without
money, power, and networks, it would be almost impossible for a student who
fails the examination to continue their schooling” (Sarasvati, 244).
3) An Incomprehensive Test
Even though as a high stake test the National Examination was considered able to improve students‟ academic achievement, the improvement meant was only
limited to the tested subjects (Bishop, Mane, Bishop, & Moriarty, 2001; Phelp,
2001 as summarized in teKUN & eF, 2012). The subjects which were not tested
tended to be neglected thus the consequence was that the curriculum became
narrowed (Gayler, Chudowsky, Hamilton, Kober, & Yeager, 2004; Jones, Jones,
& Hargrove, 2003; Watanabe, 2006 as paraphrased in teKUN & eF).
Other than that, student-centered learning process and learning process which
focuses on creativity were also neglected because this type of examination
required lots of drilling (Abrams, Pedulla, & Madaus, 2003; Jones, Jones, &
Hargrove, 2003, Vogler, 2005; Zao, 2006 as paraphrased in teKUN & eF).
29
aspects of objectives which include cognitive, affective, and psychomotor aspects
(Harti, n.d). It only covers cognitive aspect (Harti).
4) The National Examination Had the Potential to Increase Dropping Out Risk
High stake test such as the National Examination has the potential to increase
dropping out risk, especially for students from lower class (Cunningham & Sanzo,
2002; Dee & Jacob, 2006; Marchant & Paulson, 2005; National Research Council,
1997; Reardon, 1996; Warren, Jenkins & Kulick, 2006 as paraphrased in teKUN
& eF, 2012), students from minority group (Dee & Jacob, 2006; National
Research Council, 1997; Reardon, 1996; Warren, Jenkins, & Kulick, 2006 as
paraphrased in teKUN & eF), and students with low academic achievement
(Archer & Dresden, 1987; Bishop & Mane, 2001; Jacob, 2001 as paraphrased in
teKUN & eF).
Sarasvati‟s research also showed how the National Examination as a high stake
test made two of her students dropped out of school and one of her students almost dropped out of school (2012). Sarasvati‟s students were from lower class
family. Two students who dropped out did not join the second remedial
examination. One of them was asked to work by his parents and another one was too depressed to go on. One of Sarasvati‟s students who almost dropped out had
actually had entered a vocational school. Even though she finally joined the
second remedial examination, she failed. The vocational school told Sarasvati
they could not let her student continue her study if she did not pass. After
lobbying members of ministry of education, her student was given the chance to
30
finally dropped out of school because the National Examination demotivated the
students by judging them with results showing if they pass or fail. The students
also dropped out because the National Examination procedure, especially for
those who failed the examination, was not clear. Another reason was because the
input and process in her school was far from proper.
5) Pressure on Students and Teachers
High stake test can also give pressure on students (Gregory & Clarke, 2003 as
paraphrased in teKUN & eF, 2012). The National Examination itself had
reportedly causes depression and pressure on students (The Jakarta Post, 2014,
December 30; Rachman, 2015, April 13; Jakarta Globe, n.d.). Students were also
reported falling ill because they were too depressed in preparing for the
examination Jakarta Globe.
Sarasvati (2012) showed how the National Examination made her students
demotivated. Some of her students had to take two remedial examinations after
failing the National Examination. Two of her students decided to drop out of
school instead of joining the second remedial test. One of them decided not to join
the second remedial test because she was depressed and pessimistic When
studying for the second remedial test, the students joining the examination were
also exhausted thus had very low motivation.
Other than making students demotivated, Sarasvati (2012) also stated that the
National Examination put students, especially those who did not pass the
examination, in uncertain position. Those who failed the National Examination
were not told what was supposed to be done when they did not pass the
31
Should they have redone the examination in certain subjects, in all subjects?
Should they have just stayed home and wait for the next examination or join kejar
paket program (non-formal education)? (Sarasvati).
The National Examination also made it difficult for those who did not pass to
continue their education (Sarasvati, 2012). As stated previously, the unclear
procedure which the students should have followed when failing made only those
having money, power, and network could get through the remedial examinations
easily. It made those with no money, power, and network dropped out easily when
faced with the fact that they failed the examination. If students could pass, they
would not get the certificate. If they did not get the certificate, they could not
continue their study.
The government was negligent to the students who were or might have been depressed for the National Examination. “It is better to make a couple of students
stressful than a million of students stupid,” Kalla once said (Dharma, 2012, 86).
Not only to students, the National Examination was also reportedly brought
pressure to teachers. In her writing Sarasvati told how stressful the teachers were
in helping students be prepared for the remedial examination (2012). Indeed,
research reports had shown that high stake test (including the National
Examination) could give teachers pressure (Abrams, Pedulla, & Madaus, 2003, as
paraphrased in teKUN & eF, 2012).
6) Cheating
Cheating practices had gone on for years during the National Examination. “In
