T1 112005156 Full text

(1)

The Perceptions of English Department Students on the Use of

Multiple-choice Questions

THESIS

Submitted in Partial Fulfillment of the Requirements for the Degree of

Sarjana Pendidikan

Merry Tan 112005156

ENGLISH DEPARTMENT

FACULTY OF LANGUAGE AND LITERATURE

SATYA WACANA CHRISTIAN UNIVERSITY

SALATIGA

(2)

i

The Perceptions of English Department Students on the Use of

Multiple-choice Questions

THESIS

Submitted in Partial Fulfillment of the Requirements for the Degree of

Sarjana Pendidikan

Merry Tan 112005156

ENGLISH DEPARTMENT

FACULTY OF LANGUAGE AND LITERATURE

SATYA WACANA CHRISTIAN UNIVERSITY

SALATIGA

(3)

(4)

(5)

iii

COPYRIGHT STATEMENT

This thesis contains no such material as has been submitted for examination in any course or accepted for the fulfillment of any degree or diploma in any university. To the best of my knowledge and my belief, this contains no material previously published or written by any other person except where due reference is made in the text.

Copyright@ 2013 Merry Tan and Dra. Martha Nandari S. Handoko, MA.

All rights reserved. No part of this thesis may be reproduced by any means without the permission of at least one of the copyright owners or the English Department, Faculty of Language and Literature, Satya Wacana University, Salatiga.

(6)

(7)

1 Introduction

There are pros and cons perceptions in the society about multiple-choice test questions (MCQs). Many people think that MCQs are not quite effective for the educational world because it is hard to measure students‟ ability level. However,

MCQs are the most popular question type used in teaching and learning process (Oosterhof, 2001, Tozoglu et al., 2004). According to Lee (2011) as cited in Ihsan Ibadurrahman, MCQs are the most commonly used testing type in many academic achievement tests but MCQs are also the most criticized type. Some students might be more able to perform on MCQs better than other question types. Generally, students' perceptions or attitudes toward the types of question could differ according to their outlooks about their performance on the test. Moreover, although most of the students report that multiple-choice questions are easier than the other question types; they could consider multiple-choice questions are more difficult than the other question types in some situations (Holtzman, 2008). The explanation of those results might connect with students‟ belief about the process of preparing for this type of

questions. So, they perhaps think that with objective questions, it is easier to study the material. Zeidner (1987) stated that students' perceptions' toward test types can differ in terms of different works, such as how they perceived interestingness, usefulness, trickiness, fairness and other elements.

Multiple-choice questions also have advantages and disadvantages in teaching and learning process. It is supported by the statement from Ben-Simon, Budescu and Nevo, (1997) that stated “Multiple-choice tests also have advantages

(8)

2

exams.”. Overall, there are many problems that appeared from this type of questions, such as the guessing issue, the difficulty to measure the real students‟ ability, and the

process of preparing questions which is not easy because it needs time and efforts (Oosterhof, 2001). Some researchers still argued that this type of questions can measure the complexity of academic processes effectively (Mehrens and lehman, 1984). The literature indicates that students prefer this type of exam comparing with other types exam (Zeidner, 1987; Choi, 1998; Birenbaum and Feldman, 1998). The details of these results might linked with students‟ perceptions about the process of

preparing for this type of exam, as the result, they perhaps believe that with objective exams it is easier to study the material, and they might think that multiple-choice type question is more fair than the other types question.

Zeidner (1987) tried to study secondary school students' perceptions of essay and multiple choice type questions, the participants of his study consisted of 174 male and female students. Two instruments were applied in this study. First, he represented students' perceptions in terms of ten scopes: difficulty, complexity, clarity, interest, trickiness, fairness, value, success expectancy, degree of anxiety evoked, and feeling at ease. The second instrument was used as standard instrument, to observe the validity of the major instrument, this instrument included seven scopes. In the second tool the students were asked to answer whether they prefer essay or multiple choice type questions. The results of this study revealed that secondary school students preferred multiple-choice question more than essay question.

(9)

3

developing students' skills in dealing with the mentioned questions. In general, Nassar observed that the biggest problem that students dealt with was obtained in students' achievement on essay questions. He explained that this result due to students' perceptions of essay questions before and after the training. As the result, students might have negative perceives toward this type of questions because they do not have sufficient qualifications of dealing with such questions.

The study of English Department (ED) students‟ perceptions about the use

of multiple-choice type questions on the learning process is being focused to determine the beneficial and harmful effects on doing the MCQs. This study surely has a major effect to establish and to recognize the important of multiple-choice questions. The project‟s goal is designed to help students improve their achievement level, to immobilize the misunderstanding toward multiple-choice questions, and to decide the validity of multiple-choice test questions for sure. This goal can be achieved through the teacher who can teach the students the tricks on doing the multiple-choice test questions and English Department/ED students‟ help to talk about their perceptions toward the use of multiple-choice test questions on the learning process. The output of this study is various perceptions about the use of multiple-choice test questions on the learning process that can make the students think about the benefits and harmful of multiple-choice test questions and disseminate by renewal the negative perceptions to positive perceptions.

The research questions to be answered are:

(10)

4

2. According to the English Department (ED) students, what are the beneficial effects and the harmful effects of multiple-choice questions to their learning process?

Multiple-choice questions are basically recognition tasks because students must identify the correct response. The multiple-choice questions are supposed to be a very flexible part of assessment format that are used to measure knowledge, skills, abilities, values, thinking skills, etc (Haladyna, 1999; McDougall, 1997). Besides that, a test usually consists of a number of questions and the students must choose the right answer among the number of choices. Therefore, students are trained to use their brain to think logically. Then, multiple Choice Questions (MCQs), also called as an objective test, is one of the test types in language assessment where participants are typically asked to respond by giving one correct answer from the available options (Hughes, 2003). Hughes (2003, pp.76-78) stated that multiple-choice questions are all receptive or selective, response items chosen by the test-taker from a set of responses or usually called as a supply type of response rather than creating a response. Multiple-choice questions have a stem, which includes a stimulus, and options usually between three and five to choose. One of the options is the correct response, while the others are the distractors.

As we know, there are many beneficial effects of the use of multiple-choice questions on teaching and learning process. According to Haladyna (1999), multiple-choice questions are considered to be beneficially an efficient and effective way to assess a wide range of knowledge, skills, attitudes and abilities. Multiple-choice tests can also be a useful tool to develop students‟ thinking skill (Haladyna, 1999).

(11)

5

scoring procedures, it is easier for the teacher on grading and scoring. It is also easier for the students to answer the MCQs because in each question, there is a correct answer in between the options for sure.

Generally people have the opinion that MCQs only deal with lower-order analytical skills that require knowledge, application and comprehension only, but not the high-order skills including critical thinking and problem solving skills. According to William (1996) & Mehrens (1998), the latest test appliances spending multiple-choice questions to test higher order analytical skills or which include analysis, synthesis, and evaluation to display students‟ point. Higher order analytical

skills make the limited errors in doing multiple-choice questions. To develop these skills, students can debate, write and master the structured argument. Smothering education is needed but of course with high responsibility. The example of this analytical skill is students have to collect the data by using the internet. McGuinness (1999) points out that higher order analytical skill includes categorizing and analyzing information, drawing conclusions from the information, „brainstorming‟

new ideas, and solving the problem. Besides that, he also added that higher order analytical skill includes determining cause and effect, evaluating options, planning and setting goals, monitoring progress, making decision, and reflecting on one‟s own

progress.

(12)

6

electronic textbook. According to McGuinness (1999) collecting information is the lower order analytical skills.

Although some people think that multiple-choice questions are effective for the teaching and learning process, actually there are also many harmful effects that students and teacher faced from the use of multiple-choice tests. Some people claim that multiple-choice questions are too easy for the students. It makes the students lazy to study. Multiple-choice questions are easiest to write when there is a definitively right or wrong answer (Bloom, Benjamin B, 1956).

The bad effect of multiple-choice questions that clearly appear on the learning process is the guessing issue. The score is unpredictable because this is just depending on the luck. However, the students would usually prefer multiple-choice questions than the essay or other question types because of that reason. Students think that MCQs are helpful when they forgot the material because they can guess the answer. According to Espinosa & Gardeazabal (2009), students are motivated to have a high possibility to guess in doing multiple-choice questions. Besides that, candidates may be able to pick the correct answer without having to read the passage or understanding the meaning in it (Lee, 2011: 31). Similarly, a correct answer can also be the result of eliminating incorrect answers, without the candidates knowing the right answer in the first place (Woodford & Bancroft, 2005).

People think that multiple-choice questions are always objective and highly reliable for most students to improve the skills, but this makes the students idle in the learning process. They just have to choose one of the options that they think is the true one whereas we don‟t know if they understand the lesson or not. Related to the learning skills‟ disability, people argued that students can improve their marks on

(13)

7

strategies will not give learners with the most effective ways of improving their language ability (Hughes, 2003).

Multiple-choice questions are time consuming. Just like Burton, Sudweeks, Merrill, & Wood (1991) argued that good multiple-choice questions are more difficult and more wasting the time to make than other types of test questions that require skills. This skill, however, may be increased through study, practice, and experience.

Multiple-choice questions are known as a simplest kind of question types to create, but it is extremely difficult to design appropriately. Hughes (2003, pp. 76-78) mentioned that the weaknesses of multiple-choice questions are only technique tests that recognize knowledge and severely limit what can be tested. It mentioned that guessing and cheating may have a significant effect on test scores.

Adapted from Gronlund (1998, pp. 60-75) and J.D. Brown (1996, pp. 54-57), there are four guidelines for designing multiple-choice questions for both classroom-based and large-scale situations. The first one is designing each item to measure a specific objective (the comprehension of wh-questions. Next is stating both stem and options as simply and directly as possible. To make it as a good multiple-choice question, it‟s better to make it not too wordy and get directly to the point. The third one is being sure that the intended answer is clearly the only correct one. Then, the last guideline for designing multiple-choice items is using item indices to accept, discard, or revise items.

(14)

8

a test. First is item facility (IF). Item facility is the scope to which an item is easy or difficult for the planned group of test-takers. The teacher should classify the questions carefully to make it not too difficult and too easy because it‟s invalid to

separate the high- and low-ability test-takers. Second is item discrimination (ID). Item discrimination is the scope to which an item differentiates between high- and low-ability test takers. Poor ID would appear when the high- and low-ability students score equally well because it did not discriminate between the two groups. The last index is distractor efficiency which is related to item discrimination. The example is in a test at least two distractors attracting some attention from lower ability test takers.

The Study

In order to answer the questions of this study, the quantitative method, namely survey research, was used. Then, to convince and analyze my own perspective of my study, I used qualitative descriptive method with an open-ended style. Open-ended (or “open question”) is a question where the researcher doesn‟t provide the respondent with a set answer from which to choose. Rather, the respondent is asked to answer "in their own words". In this case, I used an essay question to provide students‟ perspective. The participants of this study were

seventy-four students from basic grammar classes in the English Department of Satya Wacana Christian University in the second semester of the academic year 2012-2013.

Data Collection Instrument

(15)

9

questions which the students had to choose for the best answer, and the second section consisted of an essay question which asked about students‟ perceptions of the use of MCQs in their tests. The questions of perceptions‟ were prepared by the researcher and the ideas of these questions were taken from some researchers‟

perceptions toward the use of MCQs. The first three questions of the first section were made to find out the students‟ perceptions on the difficulty and possibility of

guessing, cheating, and ambiguity of MCQs. The same procedure was developed with the questions which were constructed to measure the students' skills by using MCQs on their tests. Next parts of the questions were developed to find out students‟

perceptions toward the benefits of the use of MCQs. The instrument was constructed as a Likert-type rating scale on a four-point range that was scored and categorized as following: Strongly Agree=4, Fairly Agree=3, Fairly Disagree=2, Strongly Disagree=1. Higher scores on the questions indicate more favorable dispositions toward the use of multiple-choice questions type under consideration. Finally, the writer counted the results one by one each questions.

Data Collection Procedures

(16)

10

choose whether they agreed or disagreed with the list of beneficial and harmful effects of MCQs. Students respond to the instrument with the name and student ID written on, and they need in average 5-10 minutes to accomplish the task. The process of collecting the data most of the time was done by the researcher or with the help of friends or some teachers who were teaching in the selected classes.

Quality of Data

For the purpose of examining the validity of the instrument (face validity evidences), I used SPSS to make sure for the validity. After inputted the data to SPSS, I used the descriptive reliable to measure it. After got the reliability statistics and item-total statistics, the questions could be determined the validity and reliability. If the value of Cronbach‟s Alpha was > 0.70, it means that the test was

reliable. And if the value of corrected item-total correlation was > r table, it means that the test was valid.

Data Analysis Procedure

Besides that, an essay question was made to find out if there were other opinions about the use of MCQs to support the answers in the first section (list of questions). Sample questions from the instrument are as follow: "It is easier to get high score in MCQs", "MCQs are practical and reliable", "MCQs decrease the anxiety", "and MCQs differentiate good, average, and weak students". In order to answer the research questions, I explained questions with the dominant answers one by one with the supporting statements from the essay question.

Discussion

In order to describe the English Department students‟ perceptions toward

(17)

11

Those questions were about the advantages and disadvantages of multiple-choice questions.

Table 1. MCQs increase the chance of guessing.

SA FA FD SD

13 32 19 10

SA = Strongly Agree FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree

As shown in the first question, there were thirty-two participants who fairly agreed that multiple-choice questions increased the chance of guessing. Some of them thought that the chance of guessing was the good idea to raise their mark and to help when they forgot the material. Aprilia L, one of the students in the class of basic grammar, said that “If we don’t have any idea about the answer, we can choose one of the choices”. Not much different with Aprilia L, Laurensia Pamela assumed that “If we really don’t know the answer, we can guess the answer and if we are lucky we

can get the great score”. That means those students perceived the positive points of guessing. Multiple-choice assessments with a high number of stems and options helped the students to get at least 25% if the test-takers picked one of the options continuously by purely guessing. Intan Ariani argued that “Multiple-choice

questions give a chance for the students to guess if they really don’t know the

answer. It is really helpful, but on the other hand, students become lazy to study because they still have at least 25% chance to guess the right answer.”

Table 2. MCQs reduce the chance of cheating.

SA FA FD SD

5 18 28 23

(18)

12

few who disagreed with the question. Some students perceived that “Multiple-choice questions will increase the chance of cheating.” It had been spreading over the world that multiple-choice questions could not be separated with the issue of cheating. Yayas K.S. also said that “multiple-choice will increase the chance of cheating with

other students.”

Table 3. MCQs have more options that help the students in comparison and increase the ambiguity.

SA FA FD SD

29 32 9 4

Towards question number three “multiple-choice questions have more

options that help students in comparison and increase the ambiguity” students strongly agreed. Many of them also argued that “multiple-choice questions are really

confusing and so ambiguous because the answer is quite similar and it’s trapping.”

Actually, the confusing part was located in the question construction. Focusing on important content was not enough.

Table 4. MCQs improve students‟ language skills.

SA FA FD SD

6 37 25 6

As predicted, the students fairly agreed with the fourth question which asked if multiple-choice can improve students‟ language skills. One of the students argued that “we can increase our ability with new vocabulary.” Although it was difficult to increase the students‟ language ability with multiple-choice questions,

actually it was simpler when the students measured their language ability.

(19)

13

SA FA FD SD

7 47 18 2

Then, as shown in the fifth question, the participants showed their interest and agreement in this question (multiple-choice questions develop students‟ thinking skills). Kartika Dianing R wrote that “Multiple-choice questions can develop

students’ thinking skills.” Similar to Kartika, Raissa Junita also said that “

Multiple-choice questions can help her thinking skills.” There were many positive opinions

about developing students‟ thinking skills.

Table 6. MCQs make students save time in doing the test.

SA FA FD SD

27 34 12 1

Concerning question number six about time-saving in doing the test, the students strongly agreed with that idea. Nadia Angela told that “We can save a lot of

time to check our answer again.” Not only Nadia who argued that multiple-choice

questions make students save time in doing the test, but also Santa Maya Pramusita said that “The benefit of multiple-choice questions is we don’t have to write too

much, so it can save the time.”

Table 7. MCQs encourage the students to study.

SA FA FD SD

3 37 31 3

(20)

14

answer, so it motivates me to study harder. I just need to take notes and study the

important points for my strategies.” However, there were some statements from the

participants that said “Multiple-choice makes students lazy to study.”

Table 8. MCQs are easier to get the high score.

SA FA FD SD

15 39 19 1

About the question number eight, the participants fairly agreed that multiple-choice questions enabled them to get the high score easier. Rizza Faosiah said that “Multiple-choice questions help students to easier get the high point.” Same with Rizza, Laurensia Pamela also said that “If we are lucky we can get the

great score.” In fact, multiple-choice questions were easier to get score because

there were more options that made students easier to do the test.

Table 9. MCQs measure the real students‟ ability.

SA FA FD SD

2 24 39 9

Dealing with question number nine about measuring the real students‟

ability, the participants also fairly agreed. Naomi Tade Djo agreed that “Mul tiple-choice questions can measure her real ability because she can learn about

problem-solving.” The ability to find out and to solve the stressed problem was more

important.

Table 10. MCQs are practical and reliable.

SA FA FD SD

10 47 13 4

(21)

15

As shown in the next question that stated that multiple-choice questions are practical and reliable, the participants agreed. Anissa Putri argued that “Multiple -choice is a simpler way to answer the questions.” Another similar opinion came from Jap Victoria, who said that “MCQs are quite helpful for students in the test

because it is easier compared to essay.”

Table 11. MCQs are valid.

SA FA FD SD

12 44 14 4

The next statement was about multiple-choice questions that were valid. The participants fairly agreed with. Most of the students answered that “Multiple

-choice questions are valid.”

Table 12. MCQs are objective.

SA FA FD SD

17 46 11 0

As expected, in question number twelve, “Multiple-choice questions called as an objective”, it was revealed that the participants in this research totally agreed.

The students agreed that “Multiple-choice questions are helpful for the students. It’s

also easier to do than the other type of questions.” It could not be denied that

multiple-choice questions were so helpful especially for the students and the teacher too.

Table 13. MCQs are easy to do.

(22)

16

18 38 14 4

There were a lot of the participants who agreed that multiple-choice were easy to do. It was just like what Jessica said, “In my opinion multiple-choice makes it easier for the students to answer.” Almost similar with what Jessica said, Dara also argued that “It’s easier to do the exercise with multiple-choice because we just can

learn the important point, but for essay questions we have to think harder.” It means

that multiple-choice questions type is easier than the other question types.

Table 14. MCQs decrease the anxiety.

SA FA FD SD

9 50 12 3

The result of question number fourteen which asked if multiple-choice questions decrease the anxiety was agreed. La Valeroe Novianne agreed that “MCQs

decrease the anxiety.” Similar with the students, Snow (1993) said that “MCQs

reduce students’ anxiety.”

Table 15. MCQs differentiate good, average, and weak students.

SA FA FD SD

6 26 33 9

Last question was the very crucial question. The participants were asked about MCQs that could differentiate good, average, and week students, and they fairly agreed. Luky Rias Pambudi argued that “The discrimination of good, average, and week students happened sometimes, because sometimes also good, average, and week students get the similar score that make it harder for the teacher to

(23)

17

As expected, almost all of the students preferred multiple-choice questions, even for the higher-achiever students. It seems that students have built their thought that multiple-choice is easier to do because it has more options which are directed to the real answer. Personally, I agree with my participants‟ perceptions and I think that my participants weren‟t only copy paste from my statements only, because they made their own statement and there was a statement that made me realize with the fact that “always pick answer C and you‟ll be right about 25% of the time”. As we know in general, it is impossible because the teacher should make the questions more complex and unpredictable in scoring. However, the researches convince that it is possible and even it is a must that at least 25% of the score will be given if the students choose “C” all of the time. It is very interesting because it will happen if the teachers are competent in the process of questions‟ making.

Conclusion and Suggestions

(24)

18

guessing and cheating. Some stated that although there were options on the question, if they chose the wrong answer, the point was obviously zero. As a conclusion, it doesn‟t mean that if the students choose to guess and cheat in the test, they will get at

least 25% of the score no matter of what. The previous result might indicate that the students perceive multiple-choice as a positive and the beneficial one. Although there were pros and cons about multiple-choice around the participants, most of them agreed that multiple-choice is very helpful. It is very obvious either through the results of this study or the results of the other studies in literature that students in general have positive thoughts toward multiple-choice questions. Surprisingly, students fairly agreed with the statement “MCQs measure the real students‟ ability” but heed that MCQs increase the chance of guessing. It doesn‟t mean that students indicate guessing as a part of real students‟ ability, but sometimes not only real students‟ ability that needed in MCQs but also the sense of proper guessing should

be needed. It proves that even the high-achievers also try to guess sometimes. The more interesting result was that although students preferred multiple-choice questions more than the other question types. They thought that multiple-choice questions were the most confusing question type.

Implication

To make sure that students succeed in their multiple-choice questions exam, the teachers should motivate the students to study harder when they have exam with multiple-choice questions. It can improve students‟ thinking skills and language skills with deep learning.

Suggestions

(25)

19

1. The students cannot underestimate the multiple-choice questions because the more options means the more confusing they can be. 2. The teachers should make a report about students‟ perceptions of

multiple-choice questions. Since students' scores on this question type is used to take decision about their real ability.

3. The teachers must be careful in making the good multiple-choice questions. So that they are valid and reduce the ambiguity.

Limitation

The participants of this study were Basic Grammar students' of the first grade in English Department of Satya Wacana Christian University of the academic year 2012-2013. Also the results of this study should be interpreted in the context of its definitions of the variables, its instrument, and the research method used to answer the research questions.

Further Research

Other researchers could describe the beneficial and harmful effects of other question types in the next research. They also could compare multiple-choice question type with the other question types.

Acknowledgement

(26)

20

Besides my advisor, I would like to show my biggest gratitude to best of the best advisor of my life, God, You help me pass all of the difficulties I faced. Please bless me for the next steps of my life too. Next, I would like to thank my family: my lovely parents for the praying and the words that motivate me to pass this study. Thanks to my cousins who help me a lot: Didi, Aweng, Chang Sese, Jessica, Silvia, Agnes, Jenni, Weni, Hia Rein, Mengkhoi hia, Sok Pheng hia, LingCe, Cehu, EngCe, Aso, etc. For my aunts and uncles who support me. I love you all.

(27)

21 References:

Aiken, L. R. (2000). Psychological Testing and Assessment (10thEdition). Boston, MA: Allyn and Bacon.

Bachman, L. & Palmer, A. (1996). Designing and developing useful language tests. Oxford: Oxford University Press.

Ben-Simon, A., Budesco, D.V. and Nevo, B. 1997. A comparative study of measures of partial knowledge in multiple-choice tests. Applied Psychological Measurement, 21 (1): 65-88.

Biggs, J. B. (1973). Study behavior performance in objective and essay formats. Australian Journal of Education, 17, 157-167.

Brown, J.D. (1996). Testing in language programs. Upper Saddle River, NJ: Prentice Hall Regents.

Bush, M. (2001). A multiple-choice test that rewards partial knowledge. Journal of Further and Higher Education, 25 (2), 157-163.

Cannon, R.A. and Newble, D. (1983) A Handbook for Clinical Teachers, Lancaster, MTP Boston: p 97-105.

Carey, J. (1997). Everyone knows that E = MC2, who can explain it? Business Week, 3547, 66-68.

Chatterji, M. (2003). Designing and Using Tools for Educational Assessment. Boston, MA: Allyn and Bacon.

Dufresene, R. J., Leonard, W.J., & Gerace, W.J. (2002). Making sense of students‟ answers to multiple-choice questions. The Physics Teacher, 40, 174-180. Gronlund, N. E. (1998). Assessment of Student Achievement. Sixth Edition. Boston,

MA: Allyn and Bacon.

Holder, W. W., & Mills, C. N. (2001). Pencils down, computer up: The new CPA exam. Journal of Accountancy, 191 (3), 57-60.

Hughes, A. (2003). Testing for language teachers (2nd edition). Cambridge: Cambridge University Press.

Johnson, D. W. & Johnson, R. T. (2002). Meaningful Assessment: A Manageable and Cooperative Process. Boston, MA: Allyn and Bacon.

Kreig, R.G., and Uyar, B. (2001). Student performance in business and economics statistics: Does exam structure matter? Journal of Economics and Finance, 25 (2), 229-241.

Lee, J. (2011). Second Language Reading Topic Familiarity and Test Score: Test-Taking Strategies for Multiple-Choice Comprehension Questions (PhD. Thesis). Retrieved from ProQuest Dissertations and Theses. (Accession Order No. 3494064).

Marshall, J. C., & Hales, L. W. (1971). Classroom test construction. Reading MA: Addison-Wesley

(28)

22

McKeachie, W. J. (1999). Teaching Tips: Strategies, Research, and Theory for College and University Teachers (10thEdition). Boston, MA: Houghton Mifflin Company.

McTighe, J. & Wiggins, G. (2005). Understanding by design (2nd ed). Alexandria, VA: Association for Supervision and Curriculum Development.

Popham, W. J. (2000). Modern Educational Measurement: Practical Guidelines for Educational Leaders (3rdEdition). Boston, MA: Allyn and Bacon.

Stoynoff, S. & Chapelle, C. (2005). ESOL tests and testing. Alexandria, VA: TESOL.

Snow, R. E. (1993). Construct validity and constructed-response tests. In E. B. Randy, & C. W. William (Eds.), Construction versus choice in cognitive measurement. Hillsdale, NJ: Lawrence Erlbaum, 45-60.

Trice, A. D. (2000). A Handbook of Classroom Assessment. New York: Addison Wesley Longman, Inc.

(29)

Appendix A

Questionnaire:

I am doing a research about the students’ perceptions on the use of multiple-choice questions on the learning process. I would like to

know your opinions about the beneficial and harmful effects of multiple-choice questions.

INSTRUCTIONS: Please rate how strongly you agree or disagree with each of the following statements by placing a circle in the appropriate box. Choose “strongly agree”, “fairly agree”, “fairly disagree”, or “strongly disagree” based on your opinion.

1. Multiple-choice questions reduce the chance of guessing. Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree 2. Multiple-choice questions reduce the chance of cheating. Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree 3. Multiple-choice questions have more options that help

the students in comparison and reduce the ambiguity. Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree 4. Multiple-choice questions improve students‟ language

skills. Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree

5. Multiple-choice questions develop students‟ thinking

skills. Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree

6. Multiple-choice questions make students save time in

doing the test. Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree

7. Multiple-choice questions encourage the students to

study. Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree

8. Multiple-choice questions are easier to get the high score. Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree 9. Multiple-choice questions measure the real students‟

ability. Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree

10.Multiple-choice questions are practical and reliable.

(30)

12.Multiple-choice questions are objective.

Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree 13.Multiple-choice questions are easy to do.

Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree 14.Multiple-choice questions decrease the anxiety.

Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree 15.Multiple-choice questions differentiate good, average,

and weak students. Strongly Agree Fairly Agree Fairly Disagree Strongly Disagree

What do you think about the beneficial and harmful effects of multiple-choice questions on your learning process? Please give your opinion about it and specify if you have the other ideas about characteristic of multiple-choice questions.

(31)

(32)

(33)

(34)

(35)

(36)

Appendix C

Opinions about the benefial and harmful effects of multiple-choice questions 1. Beneficial: I don't need to study hard because I just need to read the material. 2. I think MC is easy to do, but it doesn't good way to measure students' understanding. 3. Beneficial: Practice, save time. Harmful: Reduce to give our own answer.

4. Make me more easier to answer questions. 5. -

6. Multiple-choice questions decrease the anxiety. If I can't answer the question, I can pick one of the answer randomly. 7. We can save a lot of time to check our answer again and we can increase our ability with new vocabulary.

8. That can help me to remember the answer and elimination wrong answers. It has bad effect because easy to cheat.

9. Multiple-choice is needed to save the time in doing the test because sometimes we don't have enough time to do many questions. Multiple-choice can also decrease our ambiguity. We can compare the answer from A until E and decide which one will be the best. But, sometimes multiple-choice can confuse us too if the option almost same.

10. The multiple-choice most have some key word of the answered from the question. And make it more easier to understand. 11. If we do multiple-choice test sometimes we confused about the answer because there are many choices that may be we think that it was the best answer.

12. For the MC question, it's hard to choose if there is misstyping on its answer. 13. -

14. If don't have any idea about the answer, we can choose one of the choices. 15. Multiple-choice questions make student easier to cheat.

16. -

17. Sometimes, in some subject, multiple-choice is the best way to answer the question. Multiple-choice shouldn't more than 4, because it would be confusing for the students.

18. Multiple-choice questions are easy to do but sometimes can make confuse.

19. Multiple-choice questions are easy to do, easier to get high score but sometimes make confused.

20. The beneficial effect from MCQs in my opinion is that it makes it easier for students to answer but the harmful effect

would be that some students believe that if they answer MCQs in whichever way they like, they'd still get some of the answers right. Therefore, it can make the students a little bit lazy in thinking which answer is right.

21. We can do the MCQs easier but sometimes there are some of them confusing choice.

22. MCQs give a chance for the students to guess if they really don't know the answer, it really help, but on the other hand student become lazy to study because they still have at least 25% to guess the right answer.

23. -

24. It helps student to answer question by giving choices, in order make the effective time during the test.

25. MCQs often confuse me to answer the question. For some subjects MC may be better tha essay or short answer questions, but for other subjects it will be harder to do.

26. The beneficial of MC is it can reduce time to answer the question, but the effect is sometimes it's confusing optional because there are some optional to be chosen.

27. MC is more confusing than essay, because we usually confuse the answer (A is right but B is also right) so, what is more correct. If we do essay we can explain about the question.

28. When I did my test, sometimes I blank with the answer so that I think multiple-choice can remain me. What the answer of the question, it also save in time in doing the test, we can guess the answer too.

29. Beneficial: we can guess and we don't have to write too much, it makes it easier sometimes, objective. Harmful: if we choose the wrong one, the point is obviously zero, no tolerance and sometimes confusing.

30. I think MC are reduce the chance of cheating, guessing and develop students' thinking skills.

31. MC is easier to do and save a lot of time. We don't have to guess and in fact when I had a test, we have a limit time.

32. From where I stand, MC isn't the best way for learning. MC can be really tricky sometimes. It also full of trap. Once you give the wrong answer, even though it's because of marely a light mistake of us and does not have any connection with the question, the result is still a false answer.

33. Easier to do the exercise with a MC because we just can learn the important point, but essay question we have to think harder. 34. -

35. Benefits: we don't have to write too much, so it can save the time. It help us if we forget the answer because it has a keyword. Harmful: make us confuse. Easy to be cheated.

(37)

38. The beneficial is help student to guess the best answer. The harmful is reduce students' creativity. 39. Help the student to easier get the high point.

40. MC is simple way to answer the question, but I think it is not help me to reach the high score. 41. MCQs can help my thinking skills. Besides, it's more practice and save time.

42. Student can choose the correct answer. They can predict the correct answer if they not sure about that.

43. I think MC is good to increase my score but sometimes it makes me confuse because sometimes the questions are confusing. I'd rather do the essay than MC.

44. I think it's quite easy to do the MCQs because the answer is already in the options. But sometimes MCQs is really confusing and so ambiguous because the answer quite similar and it's just like a trap. MC sounds like a riddle that makes me curious with the answe, so it motivates me to study harder. I just need to take notes and study the important points for my strategies. 45. I think MC is more simple way to answer questions, and to get a good score maybe :)

46. MC is easy to do.

47. I think MC is good because when we didn't know the answer, we can guess what is the answer.

48. If we really don't know the answer we can guess the answer and if we are lucky we can get the great score.

49. I think it's a good idea to have a MCQ because it will help us to think and choose the right answer. MC is not very easy, you have to know which is a tricky answer and which is not. So, it'll be more interesting if we use MCQs compare to write an essay! 50. Benefit: helps me if I don't study. Harmful effect: makes me lazier. Wastes my time while doing the test.

51. The beneficial is we can compare the other choice then it help me to guess the answer. 52. MC are very helpful.

53. The fifteen questions above have been represent all of my opinion. Maybe one suggestion is MC could be extended from A-C maybe.

54. -

55. MC easier to do, if i can't dp the questions I can guessing the answer. 56. -

57. -

58. MCQs quite helpful for students in the test because it is easier compared to essay. 59. When I can't remember the material, MCQs help me to remember it again. 60. The benefit is we can compare the answer with the other.

61. - 62. -

63. Beneficial: save time, brief. 64. MC is good.

65. MC can help us to differentiate the true or the false sentence. So we can learn from the sentences which one the wrong or right and teach us to be more carefully to answer the question.

66. Beneficial: give the clue to us to get the best answer. Harmful: increase the ambiguity. 67. I think MC will increase the chance of cheating with other students.

68. It saves time to remembering the material. 69. -

70. I think the MCQ will increase the chance of cheating and sometimes the choices are ambiguous. 71. -

72. The discrimination of good, average, and week students happened sometimes, because sometimes also good, average, and week students get the similar score that make it harder for the teacher to differentiate the ability.