STUDENT PERCEPTION OF THE SCHOOL SPEAKING TEST

(1)

i

STUDENT PERCEPTION

OF THE SCHOOL SPEAKING TEST

A THESIS

Presented in Partial Fulfillment of the Requirements

for the Degree of

Magister Humaniora (M Hum)

in English Studies

By

ZUMRI SUATMI

056332036

THE GRADUATE PROGRAM IN ENGLISH LANGUAGE

STUDIES

SANATA DHARMA UNIVERSITY

YOGYAKARTA

(2)

(3)

(4)

(5)

(6)

vi

ACKNOWLEDGEMENTS

First of all I would like to thank God, Allah, who has granted me spiritual force, health, safety and an opportunity to continue my education in the English graduate program. Everything I get in my life is not separated from His gift and generosity. In this occasion I also want to thank to my parents who have never been tired to encourage me to be a patient and wise mother, and a meaningful person for others.

Second, I would like to express my sincere thanks to my adviser, Pak Mukarto (Drs. FX. Mukarto, M.S., Ph.D.) for his guidance and constructive feedback throughout all the various stages of this research. I am also grateful to English Language Studies lecturers for the chance to participate in their classes. Participating in their classes has indeed encouraged me to regard the time, cooperate with others and increase insight and knowledge.

(7)

vii Fourth, my appreciation goes to both students from SMK N 1 Pengasih, Marsiyam and Lina as the research participants in this study. This study would not have been possible without their participation -any insight arising from the study is due to their willingness to share their experiences in doing the school speaking test.

Finally, I would like to express my gratitude to the headmaster (Drs. H. Rumawal) for facilitating my access to the students, and my colleagues who cannot be mentioned one by one in SMK N 1 Pengasih, for their support, assistance, encouragement and tolerance. My gratitude also goes to LPMP Yogyakarta, Pemda Kabupaten Kulon Progo and Dinas Pendidikan Kabupaten Kulon Progo for the assistance, encouragement and support given to me throughout the duration of this study. Without their support this thesis could not have been completed.

Yogyakarta, 24 June 2010

(8)

viii

Testing oral proficiency has become one of the important issues since the role of speaking became important in the direct/oral approach, so long before the communicative approach. Compared to other skill tests, the speaking test is far too complex a skill to permit any reliable analysis to be made for the purpose of objective testing. That is why many teachers feel more comfortable doing the writing tests. However, vocational high school students are required to have many skills to face the world of works. One of them is the ability to communicate in English whether spoken or written. The need of a speaking test is one of the responses to prepare the students for an increasingly global society because they are going to look for a job. Hence, their ability to speak must be evaluated.

This study tried to find out the student perception of the school speaking test. In this study, the aspects of the school speaking test which was assessed were the student view about the effectiveness of the school speaking test, the student action in doing the school speaking test and her expectation for the betterment of its implementation.

It was a qualitative progressive research and the main data were from interviews with the participants. The participants of this research were two vocational high school students from SMK N 1 Pengasih at the third grade. The reason was that they had a lot of experiences in doing the school speaking test. The interview data were transcribed and thematized according to the components and categories of the school speaking test. The data were analyzed in the process of coding. The final step was making the interpretation of the result of analysis.

The result of this study showed that from the student view, the school speaking test had face validity and content validity. It was said that the test was also practical in the case of cost, ease of administration and the use of visual aids. Unfortunately, it was not practical in the number of the test takers and time constraints. Related to reliability of the test, the intra-rater reliability was high but the inter-rater reliability was still low. The test tasks in the school speaking test had situational and interactional authenticity. The examiners’ feedback on the student performance had positive backwash effect on motivation, learning habits, and teaching materials. On the other side, the teaching method was still dominated by teachers’ explanation.

(16)

(17)

xvii INTISARI

Yogyakarta: Kajian Bahasa Inggris, Program Pasca Sarjana, Universitas Sanata Dharma.

Tes kecakapan lisan telah menjadi salah satu isu yang penting sejak peran berbahasa lisan menjadi penting di dalam pendekatan langsung/lisan, jauh sebelum adanya pendekatan komunikatif. Dibandingkan dengan tes ketrampilan yang lain, tes lisan jauh lebih komplek, suatu ketrampilan yang memungkinkan analisanya dapat dipercaya untuk dibuat dengan tujuan seperti tes obyektif. Itulah mengapa banyak guru merasa lebih nyaman melaksanakan tes tertulis. Namun, siswa sekolah kejuruan diharapkan memiliki banyak ketrampilan untuk menghadapi dunia kerja. Salah satunya adalah kemampuan berbahasa Inggris, baik secara lisan maupun tertulis. Kebutuhan akan tes lisan merupakan salah satu respon untuk menyiapkan siswa di masyarakat global yang semakin meningkat karena mereka akan mencari pekerjaan. Oleh karena itu, kemampuan berbicara mereka harus dievaluasi.

Penelitian ini mencoba untuk menemukan persepsi siswa terhadap tes lisan sekolah. Dalam penelitian ini, aspek tes lisan sekolah yang diuji adalah pandangan siswa tentang efektifitas dari tes lisan sekolah, aksi siswa dalam mengerjakan tes lisan sekolah dan harapan mereka terhadap perbaikan pelaksanaannya.

Ini adalah penelitian kualitatif progresif dan data utamanya adalah wawancara dengan partisipan. Partisipan dari penelitian ini adalah dua siswa sekolah kejuruan dari SMK N 1 Pengasih di tingkat tiga. Alasannya bahwa mereka mempunyai banyak pengalaman dalam mengerjakan tes lisan sekolah. Data wawancara ditranskrip and ditematisasi menurut komponen dan kategori dalam tes lisan sekolah. Data dianalisa dalam proses pengkodean. Langkah terakhir adalah membuat interpretasi dari hasil analisa.

Hasil penelitian ini menunjukkan bahwa dari pandangan siswa, tes lisan sekolah memiliki paras validitas dan isi validitas. Dapat dikatakan juga bahwa tes ini praktis dalam hal biaya, kemudahan administrasi dan penggunaan alat peraga. Sayangnya, tes ini tidak praktis dalam hal jumlah peserta tes dan lamanya waktu. Berkaitan dengan reliabilitas, reliabilitas diri penguji cukup tinggi tetapi reliabilitas antar penguji masih rendah. Tugas tugas tes dalam tes lisan sekolah memiliki kebenaran situasional dan interaksional. Ulasan balik penguji terhadap penampilan siswa mempunyai pengaruh balik yang positif pada motivasi belajar mereka, kebiasaan belajar dan bahan ajar. Di sisi lain, metode mengajar masih didominasi oleh penjelasan guru.

(18)

(19)

iv

STUDENT PERCEPTION

OF THE SCHOOL SPEAKING TEST

A THESIS

Presented in Partial Fulfillment of the Requirements

for the Degree of

Magister Humaniora (M Hum)

in English Studies

By

ZUMRI SUATMI

056332036

THE GRADUATE PROGRAM IN ENGLISH LANGUAGE STUDIES SANATA DHARMA UNIVERSITY

(20)

v

A THESIS

STUDENT PERCEPTION

OF THE SCHOOL SPEAKING TEST

By

Zumri Suatmi 056332036

Approved by

(21)

vi

STUDENT PERCEPTION

OF THE SCHOOL SPEAKING TEST

A THESIS

Presented by Zumri Suatmi

Student Number : 056332036

Defended before the Theses Committee And declared acceptable

THESIS COMMITTEE

Chairperson : F.X. Mukarto, Ph.D. _________________

Secretary : Dra. Novita Dewi, M.S., M.A (Hons)., Ph.D._________________

Members : 1. Dr. Fr. B. Alip, M.Pd., M.A. _________________

2. Dr. J. Bismoko _________________

Yogyakarta, 24 June 2010 The Graduate Program Director

Sanata Dharma University

(22)

vii

STATEMENT OF ORIGINALITY

This is to certify the originality of the written study. To the best my knowledge and belief, all ideas, statements, and sentences, unless otherwise stated, are ideas, statements, and sentences of the writer. The thesis contains no material previously published or written by another person except where due reference is made in the thesis itself.

(23)

viii

ACKNOWLEDGEMENTS

First of all I would like to thank God, Allah, who has granted me spiritual force, health, safety and an opportunity to continue my education in the English graduate program. Everything I get in my life is not separated from His gift and generosity. In this occasion I also want to thank to my parents who have never been tired to encourage me to be a patient and wise mother, and a meaningful person for others.

Second, I would like to express my sincere thanks to my adviser, Pak Mukarto (Drs. FX. Mukarto, M.S., Ph.D.) for his guidance and constructive feedback throughout all the various stages of this research. I am also grateful to English Language Studies lecturers for the chance to participate in their classes. Participating in their classes has indeed encouraged me to regard the time, cooperate with others and increase insight and knowledge.

(24)

ix Fourth, my appreciation goes to both students from SMK N 1 Pengasih, Marsiyam and Lina as the research participants in this study. This study would not have been possible without their participation -any insight arising from the study is due to their willingness to share their experiences in doing the school speaking test.

Finally, I would like to express my gratitude to the headmaster (Drs. H. Rumawal) for facilitating my access to the students, and my colleagues who cannot be mentioned one by one in SMK N 1 Pengasih, for their support, assistance, encouragement and tolerance. My gratitude also goes to LPMP Yogyakarta, Pemda Kabupaten Kulon Progo and Dinas Pendidikan Kabupaten Kulon Progo for the assistance, encouragement and support given to me throughout the duration of this study. Without their support this thesis could not have been completed.

(25)

x

Testing oral proficiency has become one of the important issues since the role of speaking became important in the direct/oral approach, so long before the communicative approach. Compared to other skill tests, the speaking test is far too complex a skill to permit any reliable analysis to be made for the purpose of objective testing. That is why many teachers feel more comfortable doing the writing tests. However, vocational high school students are required to have many skills to face the world of works. One of them is the ability to communicate in English whether spoken or written. The need of a speaking test is one of the responses to prepare the students for an increasingly global society because they are going to look for a job. Hence, their ability to speak must be evaluated.

This study tried to find out the student perception of the school speaking test. In this study, the aspects of the school speaking test which was assessed were the student view about the effectiveness of the school speaking test, the student action in doing the school speaking test and her expectation for the betterment of its implementation.

It was a qualitative progressive research and the main data were from interviews with the participants. The participants of this research were two vocational high school students from SMK N 1 Pengasih at the third grade. The reason was that they had a lot of experiences in doing the school speaking test. The interview data were transcribed and thematized according to the components and categories of the school speaking test. The data were analyzed in the process of coding. The final step was making the interpretation of the result of analysis.

The result of this study showed that from the student view, the school speaking test had face validity and content validity. It was said that the test was also practical in the case of cost, ease of administration and the use of visual aids. Unfortunately, it was not practical in the number of the test takers and time constraints. Related to reliability of the test, the intra-rater reliability was high but the inter-rater reliability was still low. The test tasks in the school speaking test had situational and interactional authenticity. The examiners‟ feedback on the student performance had positive backwash effect on motivation, learning habits, and teaching materials. On the other side, the teaching method was still dominated by

teachers‟ explanation.

(33)

(34)

xix ABSTRAK

Yogyakarta: Kajian Bahasa Inggris, Program Pasca Sarjana, Universitas Sanata Dharma.

Tes kecakapan lisan telah menjadi salah satu isu yang penting sejak peran berbahasa lisan menjadi penting di dalam pendekatan langsung/lisan, jauh sebelum adanya pendekatan komunikatif. Dibandingkan dengan tes ketrampilan yang lain, tes lisan jauh lebih komplek, suatu ketrampilan yang memungkinkan analisanya dapat dipercaya untuk dibuat dengan tujuan seperti tes obyektif. Itulah mengapa banyak guru merasa lebih nyaman melaksanakan tes tertulis. Namun, siswa sekolah kejuruan diharapkan memiliki banyak ketrampilan untuk menghadapi dunia kerja. Salah satunya adalah kemampuan berbahasa Inggris, baik secara lisan maupun tertulis. Kebutuhan akan tes lisan merupakan salah satu respon untuk menyiapkan siswa di masyarakat global yang semakin meningkat karena mereka akan mencari pekerjaan. Oleh karena itu, kemampuan berbicara mereka harus dievaluasi.

Penelitian ini mencoba untuk menemukan persepsi siswa terhadap tes lisan sekolah. Dalam penelitian ini, aspek tes lisan sekolah yang diuji adalah pandangan siswa tentang efektifitas dari tes lisan sekolah, aksi siswa dalam mengerjakan tes lisan sekolah dan harapan mereka terhadap perbaikan pelaksanaannya.

Ini adalah penelitian kualitatif progresif dan data utamanya adalah wawancara dengan partisipan. Partisipan dari penelitian ini adalah dua siswa sekolah kejuruan dari SMK N 1 Pengasih di tingkat tiga. Alasannya bahwa mereka mempunyai banyak pengalaman dalam mengerjakan tes lisan sekolah. Data wawancara ditranskrip and ditematisasi menurut komponen dan kategori dalam tes lisan sekolah. Data dianalisa dalam proses pengkodean. Langkah terakhir adalah membuat interpretasi dari hasil analisa.

(35)

(36)

(37)

1

CHAPTER I

INTRODUCTION

This chapter presents the background information of the importance of this research. The discussion is started with the background of the study and then followed by the problem identification, problem limitation, problem formulation, research goals, and research benefits.

A. Background

Testing oral proficiency has become one of the important issues since the role of speaking ability has become more central in language teaching with the advent of communicative language teaching (Sook, 2003; Nakamura, 1997). The shift toward more communicative approach of foreign and second language teaching around the world has led to a change in instructional styles – giving more time for students to actively communicate with one another. They must be conscious of the structural features of the target language, be able to associate those features to the functional usage, and have the ability to use both forms and functions properly for establishing meaningful communication. In other words, as global communication continues to

advance, one of the major focuses of language instruction is to enhance learners‟

(38)

2 language test should emphasize the competent use of language in spoken communication.

Compared to other skill tests, the speaking test is far too complex a skill to permit any reliable analysis to be made for the purpose of objective testing (Sook, 2003). That is why many teachers feel more comfortable doing the writing test. Years of experience marking work have made them familiar with the level of written competence students and they do this in order to succeed in a specific standard, for example UNAS. They may also feel much less secure when dealing with speaking test because it involves overall linguistic proficiency at the same time such as phonology, grammatical feature, vocabulary, fluency, accuracy, etc. Fulcher (2003) adds:

Speaking tests have always been credited with importance, but that the lack of consensus regarding criteria and the difficulty of fitting speaking into the framework of qualitative psychometric testing fashionable, speaking tests were generally not planned on the same level as pencil-paper tests. (Fulcher 2003:2)

(39)

3 The school speaking test is designed to measure the students‟ competence in English communication. The education institution in Kulon Progo has published the English module for SMK students in two series namely Modul Novice and Modul Elementary. At the end of each sub-competency (topic) of the module and before going to the next sub-competency, the students are required to perform their speaking ability in the form of interviewing with their English teachers, practicing dialogs or conversations with the partners, or playing a role in a certain situation reflected the topic of each unit, making monolog in describing something, etc. The teacher listens and sometimes gives questions orally to get deeply understanding. The feedback and comments are given as the students finished performing the test. He/she then gives score directly in the terms of pronunciation, fluency, appropriate words and accuracy, and attitudes.

The testing tool and procedures of this speaking test is characterized by a deliberate move from a traditional formal assessment to less formal one. Heaton (1974) states that

Successful communication in situations which stimulate real life is the best of mastery of a language. It can thus be argued that fluency in English (a person‟ ability to express facts, ideas, feelings and attitudes clearly), and his ability to understand what he hears can be measured by tests which evaluate performance in the language skills. (Heaton 1974:6)

(40)

4 Perception is the ability to recognize familiar persons, objects or events with meaning and expectation (Elliot, 1996). In this case, students will recognize the familiar things around them and realize what they know. They interpret their understanding in doing the school speaking test as a part of their experiences in study. Student perception of the school speaking test is influenced by internal and external conditions, the environment. It involves the feeling of subjectivity. It is a cognitive process and can change depend on the stimulus which is experienced by him / her. It is influenced by many factors such as motivation, attitude, interests, personality, etc.

Miller (1974) argues that an observer perceives something because he is interested in such things, knowledge about them and also that he is eager to perceive and learn more about them. Hence, the student perception of the school speaking test will be accurate because he / she are familiar and have experience with it.

There seems no prominent research on this school speaking test from the point of view of the student perception, especially in Kulon Progo. In particular, the perception of vocational high school students in doing the school speaking test remains largely unexplored. In order to gain a more holistic understanding of the phenomenon and experiences of this test, it needs to be explored. Fulcher (2003:2) holds that “Since examinee perception is fundamental to the functioning of the basic

(41)

5 B. Problem Identification

Louma (2004) recognizes that speaking, particularly in a second or foreign language, is the most difficult language skill to assess. The various directions and foci in the testing of speaking abilities of learners frequently lack solid grounding theory and pedagogy and reliable test designs. This is due, for the most part, to the difficult matching of the testing goals and the appropriate instruments and the tasks for assessment. She also suggests that because testing situations and participants vary considerably in language classrooms, it is necessary that a review of the construct validity of the testing process is conducted.

Chuang (2009) argues that in English proficiency testing, oral performance appears to be one of the most difficult skills to assess since there are many external

and internal factors that influence a rater‟s impression toward how well someone can

speak a foreign language. In other words, the reliability of scoring has always been doubted as the oral proficiency test inevitably involves raters‟ personal / subjective views instead of their objective points of view.

(42)

6 It is not easy to establish criteria to evaluate a speaking test. The reasons that teachers avoided doing oral tests include the amount of time it took, the large size of student population, and students‟ negative reactions toward speaking test.

From the learner perspective, speaking test is the most complex and difficult task among the language skills since their preparations should include knowledge about the language and the skills involved in using it. This means that many students think their oral skill is deficient. Chuang (2009) has observed from some studies which indicate that Asian students indeed have comparatively high anxiety in English learning since most of them lacked speaking practice in the target language both inside and outside of the classroom. This limited real-life practice and experience appears to have eroded their confidence and weakened their willingness to speak. Moreover, they experience a sense of panic when pressured into doing an English oral test.

(43)

7 alternative assessment methods are compared, students perceive alternative assessment as being more 'fair' than the traditional 'normal' assessment methods.

C. Problem Limitation

Speaking test, in fact, brings some problems for both teachers and students. The teacher as the examiner should be aware of the purpose of the speaking test and the difficulty in constructing its criteria. The students, on the other hand, face many factors in their speaking ability, such as their limited knowledge of language system, language anxiety, language classroom settings and procedures, etc. From the numerous problem identified, I would like to narrow down the scope and methodology. I limit the scope on the student perception of the school speaking test.

D. Problem Formulation

The research, therefore, aims to answer theoretically and empirically the following question: what is the student perception of the school speaking test?

E. Research Goals and Objectives

(44)

8 and control the student perspectives. Instead, it is to reveal, describe and understand what the student thinks, acts and expects about the school speaking test.

F. Research Benefits

Scientific and technological benefits are obtained in this research. At least, there are two scientific and five technological benefits.

1. Scientific benefits

This study may have two scientific benefits to empower students and promote their autonomy to express their understanding on the school speaking test.

a. This study may provide narrative description of the student perception of the school speaking test as an attempt to understand the student‟s understanding in doing that test and to support continuous learning opportunities.

(45)

9 2. Technological benefits

This study has several technological benefits to contribute in the improvement of the school speaking test. Moreover, it also facilitates the students to communicate their opinion on the school speaking test they have done.

a. The result of the study may give substantial information for language testing field, especially the effectiveness of the school speaking test based on the student perception.

b. The revelation of the student perception of the school speaking test may expectantly be valuable for the betterment of the school speaking test itself. c. The betterment of the school speaking test will devotedly enhance the student

motivation in learning language

d. The clear understanding of the school speaking test based on the student perception may encourage English teachers to be smarter, more communicative and careful in handling the speaking test and more creative in designing test tasks.

(46)

10

CHAPTER II

REVIEW OF RELATED LITERATURE

As stated in the previous chapter, the objective of this study is to describe and interpret the student perception of the school speaking test. It is important to base this study on some related theories to see how the study should be conducted so that the expected result can be obtained. The supporting theories are under two sub topics in this chapter. The sub topics are theoretical review and theoretical framework.

A. Theoretical Review

The main purpose of this chapter is to ensure the status of the research topic, answer the research question theoretically and how I will use it to account for the research problem. The principal areas to be discussed are perception, the school speaking test and some test principles in the speaking test.

1. Perception

(47)

11

student‟s view of reality. In this case, therefore, perception refers to the way of process by which the students organize and interpret the school speaking test.

Perception is important because student‟s behavior is based on their perception of

what reality is, not on the reality itself.

People learn about something before perception. Each person will have different perception about particular things because it involves the feeling of subjectivity. It happens in our real life that two persons will percept something based on their likes or dislikes. Hence, we can conclude that two persons looking at the same phenomenon may see very different things.

In the organizations theory, perception can be as the cognitive process where a person tries to give meaning about his/her environment. It is accordance to what Gibson, as quoted by Leathers (1992) says that perception is the cognitive process that individuals use to interpret and understand the world around them. Perception can be made through the cognitive process in our mind. Hence, the students learn about something before perception on it.

Elliot (1996) defines perception as the ability to recognize familiar persons, objects, or events with the meaning and expectation. If someone perceives something, means that he can recall past experiences with objects or events; he experiences meaning and has certain expectations about the object or

event. The student‟s experience and expectation about the school speaking test,

(48)

12 Vernon (1987) points out the three dimensions of perception. They are the understanding about the object, the view and action toward the object. Kalish (1973) adds one more dimension of perception, namely set or expectation.

“Because of previous experiences and learning, we often anticipate that certain things will occur before they actually happen. That is we have a set or

expectation, that they will occur” (Kalish 1973:83)

Based on some theories above, in this research, therefore, perception is

someone‟s belief and understanding about something because he / she has knowledge and experience with it. His / her understanding about a certain object will influence the way he / she views and acts on it. His / her previous experience with a certain object will lead him / her with expectation.

Related to the student perception of the school speaking test, three dimensions of perception will be involved: view, action and expectation. The concern of this study is an attempt to capture the student views, opinions, experiences and expectation based on the implementation of the school speaking test. The student perception of the school speaking test will be strong because he / she has knowledge and experience about it.

2. Speaking Skills

(49)

13 which includes both verbal and nonverbal components. It focuses on basic competencies needed for everyday life – for example giving directions, asking information, or providing basic information in an emergency situation, etc.

Nunan (1989) defines speaking as the ability to understand the spoken language on everyday topics at a native – like conversational pace, and formulate appropriate and intelligible spoken responses in that language. He summarizes successful oral communication involves the developing: the ability to articulate phonological features of the language comprehensibly; mastery of stress, rhythm, intonation patterns; an acceptable degree of fluency; transactional and interpersonal skills; skills in taking short and long speaking turns; skills in negotiating meaning, etc.

Harmer (2007:343) says that if students want to be able to speak fluently in English, they need to be able to pronounce phonemes correctly, use appropriate stress and intonation patterns, speak in a range of different genres and situations, and also use a range of conversational and conversational repair strategies.

From the discussion above I conclude that speaking is the person‟s ability

(50)

14 3. Speaking Test

Leo (2006) suggests that the most suitable way to know learner‟s ability

to speak is through a speaking test. We may think that a speaking test is time consuming, especially with big classes. Hence, the size of the class should be considered when planning a speaking test.

Hymes (1974) as cited by Chuang (2009) argues that the main purpose of

communicative language tests is to assess the test taker‟s ability to use the

language in real -life situations. In testing speaking skills, the focus should center on producing the appropriate and meaningful messages rather than grammatical accuracy. She also argues that communicative language tests are those which make an effort to test language in a way that reflects the way that language is used in real communication; they focus on language meaning and function rather than language form. If students are encouraged to learn the target language through more communicative ways, it would make a positive effect on their language learning.

(51)

15 4. The Area of Speaking Test Situation

Daisy (2004) divides speaking test situation into four interactive areas:

speakers, test tasks, raters and criteria.

a. Speakers

The speaker is the person whose responses are to be assessed. The speaker is assumed to possess speaking ability, which is the construct oral assessment intends to measure. In this discussion, the speaker refers to the student. When he/she takes the speaking test, he/she comes to the testing situation having developed his/her ability in various ways, from formal classroom study to communicative interaction. Daisy further explains that the speaker and his/her speaking ability are influenced by more than cognitive or psycholinguistic factors, for example, background (age, sex, nationality, L1, etc), general behavior and attitude (e.g. shy, confident, etc), personality, and the sample of language elicited.

b. Test Tasks

(52)

16 characteristic which consist of the following aspects: characteristics of the setting, the test rubrics, input, the expected response, and the relationship between input and response.

Related to the speaking tasks, Beale (2000) categorizes them into two general approaches that meet the criteria for interactive test. Those are interview and role play. The interview tasks are a direct language use because they measure oral skills by having the examinee actually speak. He divides the types of interview tasks into structured interview and unstructured interview. For role play tasks he has two categories namely structured role play tasks (information gap) and unstructured role play tasks.

A structured interview composed of set questions which has many advantages. It can be reliably used to determine someone's general level in terms of grammatical knowledge, vocabulary, pronunciation and fluency. It can also be used to find out how well the candidate can structure a short narrative, and to what degree they can express more complex points of view. It is relatively cost and time efficient to administer, and if the interview is recorded properly then marking can be a fairly reliable standardized procedure.

(53)

17 requires experienced interviewers who can facilitate the conversation in an unforced manner.

Beale (2000) further explains about the role play tasks. A role play is language use in a simulated real life situation. Unlike the interview format, role play can focus on a variety of different language functions. The assessor can be involved as a participant in the role play, or simply as an observer of two or more testees. The structured or controlled role play gives the candidates a detailed set of instructions to follow, usually with some kind of form to complete as they go. These are usually called information gap activities since they involve the transfer of information with others to complete a set task.

(54)

18 speaking test becomes a place for the students to express their ideas and thoughts.

c. Raters

Namara (2000) provides a thorough description of the rating process, including problems with raters. The largest concerns from the perspective of

raters seem to be the reliability. If raters‟ scores do not agree consistently with

each other, the problem is one of inter-rater reliability. Likewise, if an individual rater applies criteria inconsistently or shift standards haphazardly, the problem is one of intra rater reliability.

Daisy (2004) argues that individual rater characteristics, such as experience, training, occupation, native language, and rater bias may influence perception of speaker performance. The raters, therefore, should be aware that current approaches assuming theoretical framework of language as communication and speaking / listening as interactive activities with blurred

boundaries raise several questions about the impact of pragmatics on rater‟s

perception and reliability.

d. Criteria

(55)

19 guidelines for judgment. These should describe the various levels of performance in a way that can be tested both logically and consistently.

Douglas as quoted by Daisy (2004) says that criteria are established on the basis of some theoretical framework outlining the components of speaking ability. Thus, the definition of criteria is dependent on the definition of speaking ability.

Mc Carty (1998) holds that

Many criteria for evaluating English speaking test are possible, and there are as many ways to prioritize them, especially when the parties involved are from different cultures, these criteria can not be taken from granted as mutually understood and agreed. (Mc Carty 1998:6)

Beale (2000) holds that applying descriptive assessment criteria to a candidate's oral performance requires making subjective (or impressionistic) judgments. He mentions two kinds of performance criteria namely global rating scale and analytic rating scale. The former criterion, a global or wholistic scale provides a general description of ability, in which the various components of language use are grouped together in a single 'band' descriptor. He gives a sample taken from the Australian Second Language Proficiency Ratings (ASLPR) scale, developed by Ingram and Wylie in 1982:

Band 6: Competent Speaker.

(56)

20 will not impede exchange of views. Shows some independence in discussion with ability to initiate. (Carroll cited in Weir 1993: 44)

The second criterion, analytic rating scale refers to the breaking down of an object into its constituent parts or aspects. He gives some samples of assessment categories from different analytic rating scales. Speaking Proficiency English Assessment Kit (SPEAK), Educational Testing Service, USA uses categories “Fluency, Pronunciation, Grammar, and

Comprehensibility”. “Fluency, Accuracy, comprehension, communicative

ability” is used by the Placement rating scale, Nova conversation school, Japan. Negotiated performance profile, Tokyo Denki University, Japan uses categories

“Attitude and Confidence, Expressiveness (pronunciation, intonation and

volume), Body Language, Understandability (for the listener, is the message delivered clearly?), and Communicative Ability (can the speaker say what he/she want to say?)

(57)

21 overall purpose of assessment and founded on a clear theory of language use, it is possible to choose relevant assessment tasks.

5. Some Principles in Speaking Tests

A test needs to be a theory driven. Bachman and Palmer (1996) say that the most important consideration in designing a language test is its usefulness. There are some consideration in evaluating the qualities of the test: reliability, validity, authenticity, interactiveness, impact (backwash effect), and practicality. Beale (2000) adds that an effective test of interactive oral skills is not a haphazard selection of tasks chosen at random. Instead each assessment situation presents a set of practical demands that need to be specifically addressed. The principles of validity, reliability, practicality and bias for best provide basic guidelines for evaluating the effectiveness of a test instrument. In this section, therefore, the various concepts of test effectiveness will be discussed.

a. Validity

(58)

22 relates to study of the perceived content of the test and its perceived effect, and

external validity relates to studies comparing students‟ test scores with

measures of their ability gleaned from outside the test.

Internal validity can be examined through face validity and content validity, while external validity can be seen through concurrent validity and predictive validity. In this research, however, I emphasize the internal validity of the school speaking test in the term of face validity and content validity. Sook (2003) holds that a test of speaking ability in a classroom setting is usually an achievement test. The achievement test should have content and face validity.

1) Face Validity

Ingram as quoted by Alderson at al (1995:172) defines face validity as

the test‟s surface credibility or public acceptability. The attention focuses

upon particular poor items, unclear instructions or unrealistic time limit. The data on face validity may be gathered by interviewing students about attitudes and reaction to, their feeling about, or a test they have just taken or looked at. Tests that do not appear to be valid to users may not be taken seriously for their given purpose.

Sook (2003) states that face validity is a must in a classroom speaking

(59)

23 good face validity. While Brown as quoted by Harmer (2007) further says that face validity means that in the student perception, the test is valid. The researcher can help them to foster that perception with items that are clear and uncomplicated, directions that are crystal clear, tasks that are familiar and relate to their course book, a difficulty level that is appropriate to them, a carefully constructed and well through out format, etc. In this research, therefore, the face validity of the school speaking test will concern with those items.

2) Content Validity

Content validity is the representativeness or sampling adequacy of the content – the substance, the matter, and the topics – of a measuring instrument (Alderson at al, 1995:173). A test has a good content validity if it is actually samples the subject matter about which conclusions is to be drawn. It requires the test takers to perform the behavior that is being

measured. For example when we are assessing a person‟ ability to speak, in

(60)

24 In my understanding, therefore, the content validity of the test can be seen from suitability between test content and teaching objectives, enough samples of spoken language and level of its difficulty. Although there were still many other things that could be explored to know about the validity of the test, but in this research I could only cover those two things because of the limitation of knowledge and time.

b. Reliability

Reliability, as Brown (1997) defined, refers to the extent to which assessment are consistent; consistent instruments to measure student achievement. The test will be reliable if it does not make any difference whether a student takes the assessment in the morning or in the afternoon; one day or the next. There are three types of reliability: stability or test-retest (e. g. Give the assessment twice, separated by days or weeks or months), alternate form (create two forms of the same test) and internal consistency (compare one half of the test to the other half). He also argues that reliability is affected by many factors. Three most important factors are the length (or total number of questions), the quality of questions, and the fit to the group being measured.

(61)

25 professional judgment impact decision-making in scoring, but their reliability also influences the meaning and quality of the scores. In this case, I limit the research on inter-rater reliability (the consistency of judgment between different markers of the same test) and intra-rater reliability (the consistency of judgment by the same person on different occasions and under different circumstances. What raters need to do for this purpose is to achieve high inter- rater reliability. The degree of inter- rater reliability is established by correlating the scores obtained by candidates from rater A with those from rater B.

In short, the reliability of an examiner to consistently grade a test using the same assessment standard is of utmost importance. There are essentially two issues: first, a student will receive a different grade if he/she takes the test with a different examiner using the same marking scale (inter-rater reliability).

Second, an examiner‟s grade will be the same if the test is taken at a different time or date with the same assessor (intra-rater reliability).

c. Practicality

(62)

26 resources, time constraints, financial limitation and the ease with which a test can be administrated and scored.

Beale (2000) holds that the practicality of a test refers to the degree to which it is cost effective and easy to administer. The number of testees, the time constraints for testing and marking, and the available human and physical resources all need to consider carefully before an assessment scheme is chosen. In short, this is not only an issue of money, but also of the perceptions of those who will be taking and using the test. Also, if a test can be administered efficiently by assessors and markers, this increases the validity and reliability of the results as a whole.

(63)

27 d. Authenticity

Authenticity is a quality of the relationship between features of the test and of those of the non-target use context (Bachman, 1990: 301). There are two approaches on authenticity; the real life approach and the interaction ability

approach. Real life (RL) approach tries to develop tests mirror the „reality‟ of

non test language use. This approach has been considered as naïve because the setting itself does not exactly resemble its real life setting.

In the second approach, the authenticity of language tests arises from

their „situational‟ and their „interactional‟ authenticity. Bachman (1990) holds

Situational authenticity refers to the relationship of features of the test method to particular features of the target-use situation. Interactional authenticity mentions the extent to which an examinee‟s language ability is engaged in the test task. Thus the emphasis in this model shifts from attempting to sample actual instances of non-test language use to that of determining what combination of test method facets is likely to promote an appropriate interaction of a particular group of test takers with the testing context. (Bachman 1990:317)

(64)

28 situation or they have an interview which needs interaction with the teacher as the examiner. Hence, I am going to see the authenticity of the school speaking

test from point of views: the test takers‟ performance mirrors the target use situation, the existence of interaction between the student and her partner, and the existence of interaction between the student and the examiner.

e. Backwash Effect

Hughes as quoted by Bachman and Palmer (1996) define backwash as the effect of testing on teaching and learning, and assert that testing can have either a beneficial or harmful effect on teaching and learning. Positive backwash happens when students study and learn those things which teachers intend them to study and learn. It will result when the testing procedures reflect the skills and abilities that are taught in the course. By assessing communicatively, teachers would expect the backwash to be beneficial. If teachers wish students to learn communicate effectively in a variety of practical situation, teacher

should test them on these skills. Heaton (1974) adds that “…..oral test can have

an excellent backwash effect on teaching that takes place prior to the test”

(Heaton 1974:84)

(65)

29 pressure to practice such items rather than speaking itself. In this case, the backwash would be negative.

Backwash involves the various ways in which test use affects society, an education system and individuals with in those (students and teachers). Therefore, the research will be focused on backwash to individuals. Bachman and Palmer (1996) argue that there are some aspects of impact on individuals; the experience of taking the test, the feedback test takers received about their performance on the test, the influence on teaching and learning process.

Backwash is not entirely determined by test writers; it also depends on the ways in which teachers and learners interpret test requirements, and work with them in the classroom (Prodomou, 1995, as quoted by Wharton, 2005:4).

Backwash effects may be determined by teachers‟ and students‟ experience and

attitude toward the test. Thus they have their own ideal to testing, which is based on all aspects, which may be different from other parties. It is important,

therefore, to listen the student‟s voice about the backwash effect of the school speaking test.

(66)

30 strategies, topical knowledge, etc. Besides that, finding the evidence of scoring is more related to the teacher perception.

6. Current Issues in the School Speaking Test in Kulon Progo

Educational Institution in Kulon Progo has distributed English modules for all vocational high schools in its area. The modules were written by a team consists of some English teachers of vocational high schools in Kulon Progo. The title of the module is Buku Modul Bahasa Inggris untuk SMK, and up to now there are two series, Modul Novice and Modul Elementary. These English modules are prepared and launched to support the curriculum reforms of the vocational education and training. It presents language theories and practices which will build up a perfect communication skill. It will enable students to study on their own and develop all their potentialities. In short, almost vocational high schools in Kulon Progo Regency use these modules as one of English source books.

(67)

31 There are some scoring criteria in this speaking test. Some components of the language evaluated are accuracy in pronunciation, fluency, choice of words and expressions, and attitude. The scoring criteria of each component starting from the lowest to the highest are “poor-fair-good-excellent”. These scoring criteria are written in the evaluation sheet (see appendix 2). There is

also a space for teacher‟s comments. Each student must give this evaluation

sheet to the teacher (also as the examiner) before he / she performs.

The existence of the school speaking test is one form of manifestation toward the application of Competence- Based Curriculum (CBC) in evaluation.

It is aimed to measure the students‟ speaking ability. In 2004 when the new

curriculum 2004 was being tried out at some schools in Indonesia, and as a new way of coming into line with this new movement, SMK I Pengasih developed a pioneering school speaking test.

(68)

32 B. Theoretical Framework

From the theoretical review above I get some understandings about perception

and speaking test. Student perception is the student‟s beliefs about something which

is influenced by some factors such as knowledge / understanding, feeling, motivation, personality and previous experience. Those factors influence him/her in viewing a certain object. What he/she does and expects with that thing can not also be separated from them.

Student perception of the school speaking is what the student believes, acts and expects about the school speaking test. He/she knows, understands and believes something in the school speaking test because he/she has experiences with it. What he/she does with the the school speaking test is the form of the action. From the

student‟s knowledge, feeling, motivation and previous experiences with the school speaking test will lead him/her mind to some changes or expectations on it.

Speaking test involves interactive areas: speakers, test tasks, examiners and criteria. In this matter, the speaker refers to the student. When he/she takes the speaking test, he/she comes to the testing situation having developed his/her ability in various ways, from formal classroom study to communicative interaction. His/her speaking ability is influenced by cognitive or psycholinguistic factors.

(69)

33 are structured interview and unstructured interview. Role play tasks are designed to test language use in various setting and focus on general language functions or purposes. Both types are a direct language use because they measure oral skills by having the examinee actually speak.

The problems in the reliability of the speaking test sometimes come from the examiners because its scoring is influenced by the subjective factors. A good speaking test should have high intra-rater and inter-rater reliability. So the examiner‟s consistency and fairness in judgment are not questioned, the appropriate criteria or categories in the test are much needed. To get more detail information about the student speaking ability, the analytic rating scale criterion can be applied.

The school speaking test is one form of manifestation towards the implementation of Competence- Based Curriculum (CBC). It is an achievement test

used to measure the students‟ ability to speak. At the end of each sub-competency or topics, the students are required to perform their progress and understanding about what they have already learnt from the module. It gives many opportunities for them to express what they know and understand after learning a certain topic.

The school speaking test is a work sample of output which treats the performance itself as the target. Of course its implementation has influences towards

the teaching learning process, the students‟ learning habits and also for the teachers‟

(70)

34 react to the task and what they expect from the test. In short, the school speaking test must fulfill the criteria of a good test if it is said can measure the real students‟ ability in English speaking. Therefore, to get a deep understanding of the implementation of the school speaking test, the research will focus on the student view, action and expectation.

Because the students are the subject of the test, hence their perception is fundamental to the functioning of the basic test construct. It is important to know their perception about what the test looks like. In this case, their experiences and familiarity with the test is a must so that their perception is accurate. If we want to know about the effectiveness of the test, we should know about its validity, reliability, practicality, authenticity, backwash effect, etc.

(71)

35

Figure 2.1 Construct mapping of the student perception of the school speaking test

(72)

36

CHAPTER III

METHODOLOGY

The purpose of this study is to answer the question on the student perception of the school speaking test. Therefore, this chapter will present the sequential procedure of this research. There are several interconnected parts that build the research method. Those parts are research method, nature of data, data setting and sources, data gathering instruments and data collection, data collection techniques, data processing, data analysis, and trustworthiness in research findings.

A. Research Method

The progressive qualitative approach was applied in this study. Qualitative research was concerned with the opinions, experiences and feelings of individuals producing subjective data (Hancock, 1998:2). He also added that the data in qualitative research are used to develop concept and theories that helps us to understand the social world. Since I was interested in understanding the student has constructed about her world and experiences in doing the school speaking test, thus, the aim of this study would try to achieve „depth‟ rather than „breadth‟.

(73)

37 areas. It aims to understand human experience, to reveal both the processes by which people construct meaning about their worlds and to report what those meanings are and help us to understand the world in which we live and why things are the way they are.

This approach allowed me to capture the student own voices and to examine his/her words and beliefs. It was an attempt to understand the phenomenon and to answer the main research question. This was due to the fact that English education deals with the most complicated human behavior.

B. Nature of Data

Qualitative data, as Holliday (2002) states, is what happens in a particular social setting or a particular place or amongst a particular group of people. There are many types of data, including the researcher‟s description of what she sees and hears, what participants say and write, and what participants use and produce. Hence, in order to answer the research question, the nature of data in this research was narrative. It was gained from a sequence of in-depth interview with the participants.

C. Data Setting and Sources

1. Setting

(74)

38 has been noted as having an important role in the reform of the national vocational education and training, since the implementation of the national curriculum try-out in 2003. One of its implementation was the use of the school speaking test as the tool to evaluate the student‟s mastery of English. Therefore, this school would provide rich and manageable data.

The second one was that SMK N 1 Pengasih is the only school in Kulon Progo which implements the school speaking test. Although there were many vocational high schools in Kulon Progo which used Modul Novice and Modul Elementary published by Dinas Pendidikan Kabupaten Kulon Progo, they did not

assess the student‟s speaking ability. At the end of each topic in Modul Novice and Modul Elementary there was a formative test called “Competency Test” and it was what I mean as “The School Speaking Test” in this research.

The third reason was that I was one of the teachers and also the examiners. So at least I knew the student and the implementation of the school

spaking test well. Therefore, it was like what Holliday (2002) said, “This setting

can itself motivate the researcher….the setting is connected closely to the

(75)

39 2. Participants

SMK N 1 Pengasih had about 90 teachers and 864 students distributed in 6 mainstreams: Accounting, Secretarial Affairs, Marketing, Multimedia, Hotelier, and Clothing Design. It made this school highly feasible for the study. The number of the students in one class was varied between 32 to 36 students.

This research needed only a few participants to share their experiences. The reason was that I could go deeper in searching the meaning of their experiences in doing the school speaking test. The choice of participants here was regarded to the accessibility and adequacy of information which was relevant to the topic research.

The participants of the research were two students. One of them was from Accounting Department. Her name is Marsiyam Wijiastuti. And the other was from Hotelier Department, Lina Kusuma Dewi. Both of them were at the third grade, therefore, their experience in doing the speaking test was almost three years. I offered them to use pseudonyms for the identity but they thought that it was not necessary. Therefore I made them a letter or a statement of acquiescence and they agreed to sign it. It was put in appendix 3.

(76)

40

Elementary and regarded to be passed for Modul Novice. On the other hand, the student was in low level or in novice level if her score was under 400 in the placement test. Those criteria were determined by the agreement among English teachers in SMK N 1 Pengasih. Although those participants were from different departments, but basically, the English materials for all departments were the same. Additional materials for each department could be different to support the

student‟s skills.

The first participant was regarded as the high level student because her mastery of English was good. Her English score in national examination when she graduated from junior high school was 9.60. The TOEIC score she got in the placement test when she just accepted as the vocational high school student in SMK N 1 Pengasih was 405. She got score 600 in PTE (Proficiency Test of English) in October 2009. It was the highest score in SMK N 1 Pengasih. Her success in English subject was also shown in the result of the National examination at vocational high school levels. Although her mark was not the best, she could reach 9.00.

STUDENT PERCEPTION OF THE SCHOOL SPEAKING TEST

STUDENT PERCEPTION

OF THE SCHOOL SPEAKING TEST

Presented in Partial Fulfillment of the Requirements

for the Degree of

Magister Humaniora (M Hum)

in English Studies

ZUMRI SUATMI

THE GRADUATE PROGRAM IN ENGLISH LANGUAGE

STUDIES

SANATA DHARMA UNIVERSITY

YOGYAKARTA

ACKNOWLEDGEMENTS

TABLE OF CONTENTS

STUDENT PERCEPTION

OF THE SCHOOL SPEAKING TEST

Presented in Partial Fulfillment of the Requirements

for the Degree of

Magister Humaniora (M Hum)

in English Studies

ZUMRI SUATMI

A THESIS

STUDENT PERCEPTION

OF THE SCHOOL SPEAKING TEST

STUDENT PERCEPTION

OF THE SCHOOL SPEAKING TEST

STATEMENT OF ORIGINALITY

ACKNOWLEDGEMENTS

TABLE OF CONTENTS

CHAPTER I

INTRODUCTION

CHAPTER II

REVIEW OF RELATED LITERATURE

CHAPTER III

METHODOLOGY