Chapter 4 Research Results 39
4.2 Qualitative Data Analysis 49
paper-based test since they accessed and completed the electronic-based English proficiency test anytime and anywhere. Below were the interviewees’ responses regarding this theme.
“It was convenient, and I could do the test everywhere I had enough time to recheck the right answers and make sure if I could do well.” (Interviewee 1)
“It was convenient and easy to do this test. I could check my score and also review the video clips as many times as I liked. This way, I could easily recheck for the correct answer.” (Interviewee 9)
“It was convenient to do this test. We did not have to write anything on the paper but just clicked at the correct answer. I liked it since I didn’t have to turn the paper pages here and there while doing the test. I could do it anywhere; not just only in the exam room.”
(Interviewee 14)
▪ Pressure and distraction free –
Several interviewees remarked that they had less pressure to do the electronic-based English proficiency test than to do the traditional style of test. The atmosphere in the exam room made them tense and there was a lot of distraction there. Conversely, they tended to have better concentration and attention while doing this electronic-based test. This analysis can be supported by the interviewees’ responses as follows.
“I had less pressure to do the electronic-based test. To do the traditional style of test made me tense due to the atmosphere in the exam room that was quiet and chilling. To do electronic-based test was more relaxing.” (Interviewee 15)
“There was no pressure in doing this test. I could do the test anywhere with a relaxing mood.” (Interviewee 18)
“I had better concentration with this new kind of the test. I had better attention than doing the test in the exam room and I was not distracted while doing the electronic-based test.” (Interviewee 20)
▪ Efficient time management –
Some interviewees reported that the electronic-based English proficiency test helped them to manage the time and complete the test efficiently.
“It was convenient, and I could do the test everywhere I had enough time to recheck the right answers and make sure if I could do well.” (Interviewee 1)
“It was more convenient than a paper-based test. I didn’t have to erase anything when I made mistakes, so I had more time to do the test. I could correct the answer right away, so it saved a lot of time.” (Interviewee 5)
“There was less pressure to do electronic-based test. I usually had pressure when I did the test in the exam room. I felt nervous when I saw my classmates finished the test early. Also, we could manage the time more efficiently with this new kind of test.” (Interviewee 17)
▪ Revision of test –
A few interviewees stated that they were able to recheck and correct mistakes for the right answer with this electronic-based test since they could review the test items as many times as they liked easily.
“It was convenient, and I could do the test everywhere I had enough time to recheck the right answers and make sure if I could do well.” (Interviewee 1)
“It was more convenient than a paper-based test. I didn’t have to erase anything when I made mistakes, so I had more time to do the test. I could correct the answer right away, so it saved a lot of time.” (Interviewee 5)
“It was convenient and easy to do this test. I could check my score and also review the video clips as many times as I liked. This way, I could easily recheck for the correct answer.” (Interviewee 9)
▪ Safety –
One interviewee mentioned that it was impossible to cheat or copy other students on the electronic-based English proficiency test since all students were required to have a password to access the system. Below was this response.
“I could not check the correct answers on this electronic test, so I didn’t know which choice was the correct answer. It was impossible to cheat or copy others because we needed a password to access the system.” (Interviewee 2)
▪ Insufficient time –
Only a few interviewees provided a negative remark on the time given for completing the test. They mentioned that there was not enough time for them to complete all questions in time. However, one of the interviewees added that this issue would be solved by
a good time management skill by in the examinees themselves. The following were the responses in this regard.
“Generally, the electronic-based test was all right. I thought that we should manage the time well by ourselves while doing any test though.” (Interviewee 23)
“The system of the electronic-based test was all right because it was convenient. By the way, I could not finish the last part in time because there was not enough time for me.” (Interviewee 24)
“The system of the electronic-based test was good. It was convenient because we could do the test anywhere; just having the Internet. About the time given, we must manage it by ourselves. Anyway, it was not enough for me to complete all questions in time.”
(Interviewee 25)
2. What did you like most in the electronic-based English proficiency test? Why?
▪ Reading part –
One third of the interviewees were fond of the reading part ofthe electronic- based English proficiency test most. They stated that they could do well on this part since they had time to translate the words and understood the content as well as the meaning of the sentences. They were able to learn, practice and improve their English vocabulary skill and critical thinking skill through reading the texts. In addition, some of them liked the physical features of the reading part on the electronic-based test such as the font sizes and the font colors of the reading texts which were easy for them to read. The following were some of the interviewees’ responses on this question.
“I liked the reading part of this test. I was not good at English vocabulary.
To do this part, I was able to translate each word until I understood the meanings of each sentence and word. I learned a lot.” (Interviewee 10)
“I liked the reading part the most because I had an opportunity to improve my English vocabulary skill.”(Interviewee 12)
“I liked the reading part the most because the listening part was more difficult. It was easy to read the texts on the reading part with suitable colors and font sizes.”
(Interviewee 22)
▪ Listening part –
With the same number of the interviewees who liked the reading part, nine interviewees noted that they liked the listening part on the electronic-based English proficiency test most. This part appeared to be easier for them since there were pictures, audio and visual clips as well as multimedia features provided on the test and these facilitated their understanding. They were able to replay the audio and visual clips to find out and recheck for the correct answer. One of them liked this part the most because of the opportunity to listen to the English accent. This analysis was supported by the interviewees’ responses as follows.
“I liked the listening part because I was able to replay the audio clips as many times as I liked. When I forgot the content, I was able to listen again to find out the correct answer. It was different from the paper-based test where the examinees were controlled by the examiner. We could not tell the examiner to replay the audio clips as we liked.”
(Interviewee 14)
“I liked the listening part and the format of the test that I was able to do any part first or later. I was also able to recheck and correct the choice I had selected anytime. I did not waste my time to erase any mistakes.” (Interviewee 20)
“I liked the listening part the most because I was able to listen to the English accent.”(Interviewee 26)
▪ System of the electronic-based English proficiency test –
Several interviewees favored the system of the electronic-based English proficiency test concerning the suitable time duration for doing the test, the visibility of testing time, the test format, safety, the paper-less style of the test, and more opportunity for cross- checking. Some of them added that the system of this electronic test was fast, convenient and stable. These remarks were obtained from the interviewees’ responses and some of them were as follows.
“I liked the time duration for doing the test that was suitable. I didn’t have to rush. The test questions were not too easy or too hard. Also, with the limited time, it was not possible for any students to cheat in the test. I thought that we could manage the time better with this electronic-based test.” (Interviewee 1)
“I liked the system and paper-less style. This was so convenient, fast, and stable.” (Interviewee 4)
“I liked the system of this electronic-based test since it was convenient. I did not have to use a pencil but just clicked on the choices. Then, I was able to submit the paper right after I finished the test. I did not have to turn the pages of the test while doing the test.”
(Interviewee 25)
3. What did you dislike most in the electronic-based English proficiency test?
Why?
▪ System of the electronic-based English proficiency test –
Several interviewees mentioned about the limitations of the system of the electronic-based English proficiency test. They wished they had had freedom to randomly complete the test items; in other words, the examinees did not have to finish the early part before doing the later one. Many of them also remarked about the function buttons of the video clips on the listening part such as the time cursors that they were too small to handle on the gadgets. Some of them wished to have the subtitles provided on the video clips. Some of the remarks were given as below.
“I wish I could have freedom to do any part first or later. It was not necessary to do the parts of the test in order or to finish the early part before doing the later part.”
(Interviewee 9)
“I wished it was more convenient on the video clips buttons. It was difficult to control the time cursor especially when I wanted to restart the clip. I hoped that there would be the better system for the users to select any part of the clip more easily.” (Interviewee 17)
“I did not like the video clips although I was able to re-listen to them anytime. I could not catch up the scripts because there was not subtitle there.” (Interviewee 18)
▪ Listening part –
As the same number of the interviewees who disliked the system of the electronic-based English proficiency test, six of them showed their unfavorable responses on the listening part of this test since they could not catch up the audio clips due to the difficulty in understanding the main idea and content, unfamiliar accents, and very fast speed of the scripts on the clips. Below were their comments.
“I did not like the listening part because I could not catch up the audio clips because they were too fast.” (Interviewee 7)
“I did not like the listening part because I could not understand the main idea and they were unclear for me.” (Interviewee 10)
“I did not like the listening part because I could not understand the scripts.
The speakers on the clips spoke too fast and the accents were hard to understand.” (Interviewee 22)
▪ Reading part –
Five interviewees stated that they disliked the reading part on the electronic- based English proficiency test since they did not like reading the long paragraphs or texts. They had to spend more time to read, translate and comprehend the texts. Simply put, they had to put high effort to finish the test in time. This regard was supported by the interviewees’
responses below.
“I did not like the paragraphs. I could not do well in the reading comprehension part so I made many mistakes.” (Interviewee 14)
“I did not like the reading part because I could not understand the texts. I had to read them several times and put more effort to manage the time. I had to be in a rush before the time was up so that I could finish all parts in time. I wished I had had more time to do the test.”(Interviewee 20)
“I did not like the reading part because I could not finish it in time. I could not translate some sentences or lines and spent too much time to translate. I felt that there was too much to read on this part.”(Interviewee 25)
▪ Behavior with the electronic-based English proficiency test –
A few interviewees said that they literally had pressure on the time duration given for the test. In other words, the pressure was from the interviewees themselves, not the test. They were nervous and worried in trying to complete the test in time although they emphasized that they disliked nothing on this test. In addition, one interviewee added the comment on the safety of the test and stated that it might be difficult to control any examinees not to cheat during the test. The responses on the examinees’ behavior with the electronic test were shown below.
“I did not like when I had pressure before the time was up. This was from myself not the system. The system and the test questions were good.” (Interviewee 1)
“There was nothing that I did not like on the test. I did not like when I could not finish the test in time. I had some pressure because I focused on the time too much so the pressure was from myself not from the test.”(Interviewee 2)
“It might be difficult to control the students not to cheat especially when the students did not turn on the camera while doing the test.”(Interviewee 5)
▪ Test time duration –
Three interviewees remarked that they did not like the time that was given for doing the test. They felt that the time was not enough for them to complete all items on this electronic-based English proficiency test, especially on the reading part since they had to read the texts and answer all questions. Below were their remarks on this regard.
“I did not like the time that was given for the test. I could not finish in time.
I thought that the time given was not enough.” (Interviewee 3)
“I did not like the time given for doing the test. I spent too much time on some items and this frustrated me. I was afraid that I might not finish all parts in time. The time given for the reading part was not enough.”(Interviewee 4)
“I did not like the time duration given for doing the test which was not enough especially in the reading part. I could not read the texts and complete all questions in time. I wish I had more time to complete the test.”(Interviewee 8)
▪ Nil –
Interestingly, there were 3 interviewees expressed that they disliked nothing on the test and everything on this electronic-based English proficiency test was all right. Their expressions were as follows.
“I disliked nothing on the test. Everything was all right.”(Interviewee 16)
“Nothing I did not like on the test. Everything was ok.” (Interviewee 26)
“I like everything of this electronic-based test.” (Interviewee 27)
4. Did you have any difficulties in taking the electronic-based English proficiency test? Explain.
▪ Questions on the electronic-based English proficiency test –
More than fifty percent of the interviewees found that they had some difficulties in the questions of this electronic-based English proficiency test. Half of them indicated thatthe listening part was difficult for them and they took time to figure out the correct answer. Also, many of them revealed that the reading part of the test such as the comprehension questions was difficult since there were a lot to read; they could not understand the questions and literally made mistakes. What mentioned above was supported by some of interviewees’ responses below.
“Most of the test questions were not difficult. The system was fast but I had to take time to read some questions. So, I felt that those questions were quite difficult and I could not do well in some comprehension questions. The rest of the test questions were fine. I could do them quite easily.” (Interviewee 1)
“No problem on the system and it was not difficult to use. However, there were a lot to read especially in the reading part. Sometimes I could not understand the questions and made the mistake. With the time limit, I could not finish the test in time.” (Interviewee 2)
“The system was easy and convenient to access but only the questions in the reading part were difficult.”(Interviewee 25)
▪ System of electronic-based English proficiency test –
Four interviewees mentioned that the difficulties in taking the electronic- based English proficiency test was caused by the Internet system or a WIFI signal. They had to reconnect the Internet system when the connection was weak or unstable and it wasted their time. However, a few of them added that this difficulty was out of their control. In addition, only one interviewee mentioned that the difficulty in taking this test was from the cursor on the video clips due to its small size. Below were some of the responses of this issue.
“The difficult part of the test was about the Internet. When the connection was weak, I had to reconnect it and it wasted my time.” (Interviewee 3)
“The difficult part of the test was about the Internet system because it was out of our control. When the connection was weak, it wasted my time to reconnect.”
(Interviewee 5)
“Nothing was difficult in doing the electronic-based test. If there was a difficulty, it might be only from the Internet connection. Also, the design of the test was so easy to read and follow. Nothing was difficult.”(Interviewee 17)
“It was hard to control the cursor on the video clips to select the specific time on the video.” (Interviewee 15)
▪ Nil –
Rather surprisingly, almost 30 percent of the interviewees remarked that they had no problem in doing theelectronic-based English proficiency test and nothing was too difficult for them. They stated that the system was easy and convenient to access and no part on this electronic-based test was difficult for them. Below were some their responses in this regard.
“Nothing was difficult in doing this new kind of test. The system was easy and convenient to access.” (Interviewee 20)
“Nothing was difficult. I could do the test questions quite well. Also, it was not difficult to access the system.” (Interviewee 22)
“No part of the electronic-based test was difficult for me and the system was easy to access.” (Interviewee 24)
5. Could you give the advantages or benefits of the electronic-based English proficiency test? Explain.
▪ Convenience of the electronic-based English proficiency test –
Interestingly, all interviewees agreed that the advantages of the electronic- based English proficiency test were all about its convenience. They pointed out that the electronic-based test was remarkably more convenient than the paper-based test since they were able to do the test anywhere without traveling to the exam room, anytime, and with any devices that they were able to afford. In addition, they occasionally searched for the meaning of words that they did not know and the information online so easily while doing the electronic- based test. Also, some of them stated that they liked it when they checked the scores instantly as well as reviewed the test items for the correct answers. The followings were their responses on the convenience of this electronic-based test.
“The electronic-based test was more convenient than the paper-based test.
We could search for the meaning of words that we did not know so easily. When doing the paper-based test, it was impossible to do like this and, we would forget all knowledge right after we finished the test. To be able to search for the meanings by ourselves helped us remember the words better.” (Interviewee 2)