Students’ Perceptions toward Analytic Scoring Rubric in Academic Writing
Novia Surya M 112009121
Abstract
This study aims to find out ED students’ perceptions toward analytic scoring in academic writing. The study was conducted as writing is challenging, especially in academic writing course which is the requirement to take proposal & report writing and thesis writing. In Academic Writing, the students have to produce their writing product in accordance with the aspects that is assessed to get a good grade. The aspects are provided in the term of rubric. The rubric that is used in academic writing course is analytic scoring rubric. This rubric has aspects such as thesis/argument, writing the quote, quality of quotation sandwich, hedging, coherence, references, argument/purpose, supporting details, word choice, grammar, and organization. To obtain the data, this study used interview questions. There were twenty participants for this study. Each of them was asked seven interview questions with additional follow up questions. The interview sessions were conducted for approximately ten to fifteen minutes to provide detail information of ETEP students’ perceptions toward analytic scoring in academic writing. The results show that the students’ perceptions about the use of analytic scoring rubric in Academic Writing are mostly perceived positively, even though there are some students who came up with negatives thought. The reasons why the students perceived positively toward analytic scoring rubric are because analytic scoring rubric provides clear description of each criterion, provides information about students’ strengths and weaknesses, can guide the students in revising their draft, can be a guideline for lecturers to grade students’ writing as well as for students to understand how to get good grades, and helps the students improve their writing.
Keywords: students’ perceptions, analytic scoring rubric, Academic Writing
INTRODUCTION
In my writing journey, I realize that writing is not an easy task to do. The reason is in
addressing so what and who cares, and so on. Those things have to be done to produce a good writing product, if writer misses one or more of those, the writing product will not be as good as what the writer and also the examiner/grader are expected. Those things are similar with what Husada (2007) mentioned in his study that to produce a piece of writing the students need to follow several stages of the writing process, such as choosing a topic, brainstorming, drafting, feedback, revising, and editing.
In the beginning of the academic writing course, teacher gave the students a rubric. In the rubric, there are many aspects on writing that would be graded; topic sentence, writing the quote, quality of quotation sandwich, hedging, coherence, references, argument/purpose, supporting details, word choice, and grammar. The requirements of each aspect are supposed to be fulfilled by the students to produce good writing products. Rubric that was used in academic writing course is analytic scoring rubric. Some research mentioned that analytic scoring is useful for writing. According to Zane (2011), research has shown that using analytic scoring rubric has positive effect on students’ learning and the quality of their written
products. It is possible to say that analytic scoring is good for students in writing. The students would know the requirements of each aspect that should be fulfilled, and they could recognize if there is an aspect that the students have not fulfilled yet. One possibility is using analytic scoring as their guidance; the students can understand their weaknesses and strengths in writing. As Lee (2008) mentioned in his study that analytic scoring rubrics can be useful in capturing their weaknesses and strengths. Zane (2011) also mentioned in his study that when analytic scoring rubric are given to the students before writing, they are able to evaluate their own writing. In other words, with an analytic scoring rubric, the students can know more clearly what areas need work and what areas are mastered.
way of getting information about their writing whether or not theirs is improving. Thus, I want to do this study about students’ perceptions toward analytic scoring rubric in Academic
Writing. Besides, because there are still few who do research about analytic scoring, I am interested to do research about analytic scoring. Some research, such as, Bacha (2001) and Zane (2011) mentioned that analytic scoring can be useful for writing. I am interested to find out whether my study has similarities with previous researches. This study was guided by following research question: “What are students’ perceptions toward analytic scoring in their
Academic Writing?” It was an interesting subject to further investigate. This study is important because writing skill is the most essential component of language and also needed in learning language. In addition, analytic scoring also takes important rule in writing to improve students’ writing ability.
This study is not the first study that conducted on the field of students’ perceptions
toward writing assessment. There are a number of previous studies on the perceptions toward assessment, such as which were recently conducted by Rashid (2007) and Wood (2009). In his study, Rashid (2007) revealed that the assessment tool that was used in ESL writing classroom in a Malaysian university has some advantages. The subjects of his study used portfolio assessment which was implemented in writing classroom. He found that the students felt there was improvement in their writing abilities. Additionally, he found that the assessment reflect students’ real performance. Nonetheless, in his study he found that not
only the assessment which contributed to improve students’ writing abilities but also lecture’s feedback, self-evaluation, and peer review took a role in improving their writing abilities. Another recent study was conducted by Wood (2009). She conducted a study on basic writers’ perceptions of writing assessment practices. The data analysis revealed and described
attention and didn’t feel empowered to challenge the ways in which their writing is assessed.
In other word, her study found that the students viewed the assessment tool, or rubric which is used to teach and assess their writing is helpful.
Looking at the two studies previously mentioned, this study is more or less similar to both studies. Nevertheless, this study only focuses on one type of assessment which is analytic scoring and it was used in academic writing classroom. The aim of this study is to investigate and to describe what students’ perceptions are toward analytic scoring in
academic writing course. The findings of this study are expected to contribute valuable insight in the field of teaching and learning writing in ESL classroom. The finding of this paper can help student teachers consider maintaining the application of analytic scoring in evaluating students’ writing product. Besides, it can help teachers to give input to their
students to use analytic scoring as their guidance.
LITERATURE REVIEW Analytic Scoring
There are two commonly grading types in writing which I have mentioned before – analytic and holistic scoring. This study is focused on analytic scoring which deals with “the
separation of the various features of a composition into components for scoring goals”
emphasize. The effect of analytic scoring for the students is they can improve their awareness to know what their weaknesses and strengths as mentioned by Lee (2008) in his study.
Students are graded by analytic scoring in level of proficiency is higher or significant than students are usually graded by other methods (Weigle, 2002). In other words, students that are graded using analytic scoring rubric tend to have better writing abilities than students who are graded by other methods. The analytic scoring involves deducting students’ points based on the number of errors in grammar or other linguistic features. The students can see from the analytic scoring rubric to find out what being measure of their writing by the teacher. This rubric should emphasize on grammar, punctuation, mechanics, range of vocabularies, content and ideas, and other features that they think are important (Sokolik, 2003). The one of analytic scales in ESL which is the best and most widely used was created by Jacobs et al. (1983). In Jacobs et al. scale, scripts are rated on five aspect of writing: content, organization, vocabulary, language use, and mechanic. The five aspects are differentially weighed. The highest score is given first to content (30 points), and then language use (25 points), with organization and vocabulary weighted equally (20 points) and mechanics receiving very little emphasis (5 points) (Weigle, 2002). Kroll (1990:87) supported this scoring, especially Jacobs et al. She stated that:
“The best-known scoring procedure for ESL writing is the ESL composition Profile (Jacobs et al. 1983), which uses scale with four steps to judge five different traits, each trait being differentially weighed with scores reported both separated and in combination.”
The analytic scoring has some advantages (Hughes, 2003). It “disposes of the problem
added that analytic scoring has beneficial effects to guide students to produce their writing as it provides more information about their strengths and weaknesses and they became aware of what they should write following analytic scoring instruction. However, she also mentioned that analytic scoring also has disadvantage. The main disadvantage of analytic scoring is related with time allocation. It usually takes longer time to correct. In other words, the students cannot know their score directly after they submit their writing product, because teachers should grade their writing product following the analytic scoring criterion separately. Elbow (1993) states that grading has caused so much discomfort to so many teachers and students, because there is confusion toward the process itself. Zane (2011) has similar opinion about that, nevertheless they feel anxious about grading, but they still need it, especially the students – because grading gives many advantages to them.
Analytic Scoring Rubric
Richardson (2005) in his study mentioned that a rubric is “a means of scoring a performance assessment where in multiple criteria are being assessed and quality of performance or product is important.” There are two kinds of rubrics that are commonly used
to assess writing; holistic scoring rubric and analytic scoring rubric. The use of scoring rubric itself is to measure proficiency. Becker (2011) stated that in L2 writing assessment, an individual’s writing ability is often graded based on the criteria included in the scoring
rubrics. Besides, scoring rubric is also to measure achievement. According to Hughes (2002), students can be more aware of their progress or readiness that was provided in the assessment rubric for subsequent levels of instruction.
According to Zane (2011), analytic scoring rubric is a table that outlines the criteria for scoring a student’s work. Each row lists a specific criterion. He added that columns to the
Moreover, the scores can be either summed to find a total score or averaged across the criteria. Richardson (2005) in his study mentioned that the scoring provides more information to the student and teacher about the specific strengths and weaknesses of the students’ skills.
When using analytic scoring rubric to measure proficiency and achievement, it brings advantages such as categories are not collapsed into one inflated score; can train raters easily (Cohen, 1994), generalization to different writing tasks is possible (Weigle, 2002), reliability is improved (Huot, 1996; Knoch, 2009), and it can help to identify writers’ strengths and weaknesses; provides diagnostic information (Bacha, 2001; Carr, 2000) as cited in Becker (2011). To sum up, that statement is that students can realize their strengths and weaknesses when they looking at their analytic scoring rubric and by realizing those, the students can improve their writing abilities to write well.
Students’ Perceptions
This paper focused on students’ perceptions toward analytic scoring in academic
writing. Additionally, students usually bring their assumptions and beliefs about what a wiring course should offer them and in what way (Ismail,2011). Ismail (2011) added that when taking an academic writing course, students’ main purpose is to learn to write
effectively within their particular discipline and to succeed in their academic studies. Hence, students’ needs, perceptions, and present writing proficiency levels should be sought first in
order to offer them the required assistance. Research on students’ perceptions and beliefs has
students’ perceptions and preferences of the kinds of feedback, demonstrated that students
possess very strong views about the types and amount of feedback offered by their teachers (Ferris, 1995; Montgomery and Baker, 2007).
THE STUDY
Context of the Study
[image:8.595.72.524.221.615.2]This study was a qualitative descriptive research. It described the ETEP (English Teaching Education Program) students’ perceptions toward analytic scoring in Academic Writing. The setting of the study was a private university located in Salatiga, Central Java, Indonesia. This study focuses on the one course, Academic Writing course. The type of scoring rubric that was used to grade students’ writing product by Academic Writing’s teacher in 2013/2014 semester is analytic scoring rubric. Here is one example of the rubric,
figure 1. analytic scoring rubric type 1
It has the importance for the students’ future writing, thesis writing. Participant of the Study
2011 year of study. They have been selected to be the representative sample of English Department students.
Data Collection Instrument
To gather the information from them, this study used structured interview. The reasons why structured interview is used are because the order of the questions is strictly specified in advance, and all the participants are asked the same questions in the same order. Moreover, the structured interview used open-ended questions; seven questions. My intention of using those questions is to make greater flexibility on changing the order of the questions to provide opportunity for follow up questions. As in Zacharias (2011), Allison (2002) explains that open-ended items can serve as a “safety valve” (p. 137), because participants have more opportunities to fill in what they want to say rather than fit their responses into predetermined confines of closed ended items. Sound recorder was used to record the conversation for this purpose.
Data Collection Procedures
To gather the participants, I sent messages to each participant and I met them one by one personally after making appointment to each of them based on their time availability and willingness to be interviewed. There are seven questions which were asked to each participant and with follow up questions in the same order of each. The participant answered the questions using Bahasa. The interview sessions of each participant had been taken 10 to 15 minutes on average and had been recorded and transcribed.
Data Analysis Procedure
data into those categories using the transcriptions, I came with the conclusion using my interpretation according to the data which had been collected.
FINDINGS AND DISCUSSION
This section presents the finding of the study and the discussion of the finding including the writer’s interpretation of students’ perceptions toward analytic scoring rubric in their writing. The students’ perceptions about the use of analytic scoring rubric are according
to their experience in the academic writing class that they had been taken. The finding were classified the students’ perceptions about the use of analytic scoring rubric into two specific sides; positive and negative. The positive perceptions have five separable reasons and the negative perceptions have four separable reasons that will be mentioned and explained. Therefore, the use of analytic scoring rubric in Academic Writing for students’ writing should bring impact to the students in their writing either positive or negative.
The students’ perceptions about the use of Analytic Scoring rubric in Academic Writing
This study found that there are five reasons why the students perceived positively toward Analytic Scoring rubric used in their Academic Writing, which are: Analytic Scoring rubric provides clear description of each criterion, it provides information about students’ strengths and weaknesses, it can guide the students in revising their draft, it can be a guideline for lecturers to grade students’ writing as well as for students to understand how to
Analytic Scoring Rubric provides clear description of each criterion
Analytic Scoring Rubric has several aspects that are graded separately such as thesis/argument, grammar, word choice, coherence, organization, so what & who cares, and also critical incident (CI) as you can see in Appendix B. Besides, each criteria has own each description in order to make the students understand why they got good or bad score for their writing. From the interview, study found out that 18 out of 20 students mentioned Analytic Scoring rubric is appropriate to be used in academic writing course as mentioned by students C in excerpt 1
Excerpt 1
“Saya suka dengan penggunaan analytic scoring rubric di kelas academic writing, karena analytic scoring rubric jelas, semua aspek dari apa yang saya tulis dan yang akan dinilai dipisah-pisahkan secara jelas. Aspek-aspek tersebut di deskripsikan dengan jelas, baik bagaimana mendapat nilai sempurna atau kurang dari nilai sempurna. Saya suka penggunaan scoring rubric ini di academic writing karena rubric ini menyediakan informasi yang jelas tentang nilai kami dan menyediakan detail yang spesifik.”
“I like analytic scoring rubric that used in academic writing class, because it is clear – all of the parts that will be graded in our writing were separated clearly. Those were having clear description whether how to get excellent score or less than excellent score, I like the use of analytic scoring rubric in academic writing because it provided clear information of our grade and specific detail given.”
(Writer own translation)
This may suggest that analytic scoring rubric has good point of giving clear description of each criterion. Moreover, students’ thought it is clear because they know each aspects of
analytic scoring rubric should be followed by the students to complete their writing product and the aspects are the ones graded by the lecturer. In other words, analytic scoring rubric is considered useful and helpful for the students because analytic scoring provides enough or clear information about what should the students write and also what are being assessed by the teacher. Other, Student L also says similar thing, student’s L response in excerpt 2
Excerpt 2
grammar, word choice, coherence,dan lainnya. Selain itu, penjelasan di setiap poin itu jelas, jadi rubric ini sangat bagus digunakan di academic writing.”
“I think analytic scoring rubric that I got in academic writing is appropriate, because there are points there; thesis argument, grammar, word choice, coherence, and so on. Besides, the explanation in each point is detail - so this rubric is really good in academic writing.”
(Writer own translation)
This is in line with Bacha (2001) in his study mentioned analytic scoring scales, in being more criterion-referenced, have also been found to be better suited in evaluating the different aspects of the writing skills. It is similar with Weigle (2002) who mentioned that writing products which are grade on analytic rubric that includes several aspects representing the construct of writing. It means that analytic scoring rubric contains several aspects of writing product that can be used to evaluate their writing skills, since each aspect has each description. Hence, the clear description of each aspect in the analytic scoring rubric brings positive impact to the students in writing.
Analytic Scoring Rubric provides information about students’ strengths and weaknesses
Since analytic scoring rubric provide detail information about their work, the students also believe that analytic scoring rubric that they got provide information about their strengths and weaknesses. This study found 19 out of 20 believed that analytic scoring rubric gave them clear explanation about their strengths and weaknesses in writing, as student L response in excerpt 3
Excerpt 3
“Tentu saja memberikan informasi tentang kelebihan dan kekuranganku. Hal ini karena seperti yang tertulis dalam rubrik, ada banyak poin seperti thesis, critical incident, word choices, grammar, dan coherence. Thesisku mendapat poin 5 berarti kelebihanku adalah aku bisa membuat thesis statement yang sangat baik, untuk word choices aku hanya mendapat 3 seperti yang kubilang sebelumnya, berarti aku lemah dalam memilih vocabulariesuntuk tulisan saya.”
can make the thesis statement very well, for the word choices I only got 3 as I said before, means that I am weak at choosing vocabularies for my writing. “
(Writer own translation)
This may signify that analytic scoring rubric in Academic Writing provides the students’ writing ability. It is because there are many aspects that are graded in the analytic scoring. Besides, those aspects were graded separately, thus, the students can realize either where they are good at or where they still needs improvement. This is in line with what Lee (2008) mentioned in his study that analytic scoring rubrics can be useful in capturing their weaknesses and strengths. Similarly, Bacha (2001) also mentioned that analytic scales can be very informative about the students’ proficiency levels in specific writing areas. The other
researcher, Wiseman (2012) stated that analytic scoring rubrics provide more information about a test taker’s performance of the areas of language ability that is rated.
By knowing that analytic scoring rubric provides information about their strengths and weaknesses in writing, the students do believe that their writing progress was improving from the beginning. The improvement can be seen from their writing rubric that they got from lecturers. This study found out that 19 out of 20 students mentioned that they experience progress in their writing -- at the beginning the students got not a good score, yet, their score was increasing to be better and better. It means that analytic scoring rubric does not only provide information about aspects of writing that students are good or bad at but also their improvement in producing writing product. Besides, one of the students mentioned that she got improvement of her writing ability, not only be seen from the rubric but also from the lecturer’s role in the use of the rubric that occur, as revealed in excerpt 4
Excerpt 4
coherence yang bagus seperti apa yang dituliskan dirubric dan berhasil. Namun, saya tidak mengatakan bahwa patokan bagaimana saya bisa meningkatkan penulisan saya hanya karena rubrik, tetapi masukan dari dosen dan juga rubrik yang membuat saya bisa meningkatkan kemampuan menulis saya.”
“At first, I still got bad grade, 5; low grade. Then, I was already familiar with the rubric, so I knew which part that I am good at and still need improvement. I was improving by the time. For example, making coherence , the coherence of my writing is not good still need improvement, so I tried to revise it and ask the teacher how to making good coherence as what mention in the rubric, and It worked. However, I didn’t say that the measurement of how I can improve my writing because of rubric, but the teacher feedback and also the rubric that made me improve my writing ability.”
(Writer own translation)
This indicate that students can improve their writing skills because the existence of the analytic scoring rubric. However, in his/her opinion the lecturers’ role is needed to guide the students in developing their writing ability. To put it another way, what is needed by the students to do well for their writing product is not only the rubric; the lecturers are also needed by the students. Those two elements; rubric and lecturer can make the students go well with their writing products.
Analytic Scoring Rubric can guide the students in revising their writing drafts
In academic writing, the students did not only receive one of the analytic scoring rubrics. They also did not only make one draft of their writing products. There were several drafts that should be made in Academic Writing course. Most of them mentioned that they revised their draft after received analytic scoring rubric from the teacher as student J response in excerpt 5
Excerpt 5
“I read it, and I reflect on my paper that I have worked on, and then I revise according to the reflection that I have done and using my knowledge in understanding the rubric. For example, in the rubric point of coherence - flow smoothly, if I could read my paper two times and I don’t feel confuse in reading my paper, I think that it almost flows smoothly. However, defining a good paper is quite hard; if I think that it flows smoothly but the other readers or the teacher didn’t think so.”
(Writer own translation)
The other student who says so, student C mentioned, Excerpt 6
“Yang pasti merevisi, karena dari rubrik aku bisa melihat bagian mana yang harus kutingkatkan atau kuperbaiki, sehingga aku bisa mendapatkan nilai yang lebih baik untuk tugas. Menurutku hanya itu, karena yang penting dalam melakukan academic writing saat itu adalah proses revisi.”
“Of course revising, because from the rubric I can see which part I have to make improvement or changes, so I could get better grade for the assignment. I think just it, because the important thing in doing the academic writing back then, the revising process.”
(Writer own translation)
Analytic Scoring Rubric can be a guideline for lecturers to grade students’ writing as well as
for students to understand how to get good grades
The rubric tends to be nothing if there is no score in it. The students need the score and also the detail why they got that score in the rubric. The students believe that analytic scoring is appropriate to be used in academic writing course. They mentioned the appropriateness of analytic scoring in different ways with different reasons, as student C response in excerpt 7
Excerpt 7
“Analytic scoring itu sangat detail dan ditulis dengan rapi, kriteria penilaian dijelaskan dengan lumayan jelas untuk mahasiswa dan dosen yang menggunakan rubrik ini di academic writing. Menurut saya, rubrik ini bisa menjadi pedoman yang baik untuk dosen dalam menilai tulisan mahasiswa di academic writing, dan untuk mahasiswa, dapat mengerti bagaimana bisa mendapat nilai bagus, jadi ini lumayan jelas. Menurut saya, penilaian ini adil untuk mahasiswa karena daripada hanya memberi masukan general di essai mahasiswa, mahasiswa bisa memahami dengan mudah bagian mana yang kita masih lemah atau yang kita sudah bagus.”
“The analytic scoring was very detail and neatly written, the criteria of the assessment were explained quite clear for the students and also the lecturer that use it in academic writing. I think, it provided good guidance for the lecturer to grade the students’ writing product in academic writing, and for the students could understand how I could get good score, so it is quite clear. I think it is fair for the students because rather than only giving general comment on the writing product of the students, the students we could easily understand which part we were still lack of or we were doing well.”
(Writer own translation)
The reason why it can be a guideline for the lecturers to grade students’ writing as well as for
students to understand how to get a good grade is the lecturers grade the students based on the aspects that exist in the rubric and the students produce their writing products in accordance with the aspects in the rubric, example as mentioned by Student S in excerpt 8,
Excerpt 8
“I think the scoring is fair enough, because it has certain criteria, if I didn’t have the thesis statement, if I didn’t have critical incident, and I made many mistakes in grammar, my score would be decided in the rubric.”
(Writer own translation)
This may indicate that analytic scoring may help lecturers to score the students’ writing as the score itself was explained clearly in the rubric. Thus, the students can understand the reason why they got high score, good score, or low score. Each student tends to get different score of each aspect in accordance with their writing product. The lecturer did not grade the students’ writing based on his/her overall impression but gave the score based on the quality of the aspects in the students’ writing product. This is in line with Wood (2009) mentioned in her
study that students perceive that the way in which their writing is assessed is fair because of the result of transparency and an environment of trust – both of which are created around the program-wide common rubric. In other words, the scoring is fair because of the score and the details are displayed in the rubric. Besides, the students put their belief in their lecturer as a writing expert to assess their writing product.
Mostly, the students mentioned that the lecturer’s role is needed in guiding the
students to write. Besides, the lecturer should provide detail information related with the score that the students got in the analytic scoring rubric. Similarly, the others also mentioned that the lecturer’s role is needed in the use of analytic scoring rubric in the students’ writing,
as student R in excerpt 9
Excerpt 9
“Peran dosen dan rubrik dibutuhkan oleh mahasiswa, karena dosen menggunakan rubrik untuk menilai essai mahasiswa dan mahasiswa menggunakan rubrik untuk mengevaluasi penulisan mereka dan sebagai pedoman.”
Here is the other one, student C in excerpt 10 who mentioned lecturer’s role is needed in the use of rubric,
Excerpt 10
“...terkadang penjelasan detail bagus untuk mahasiswa tetapi kurasa ini akan lebih jelas dan menarik untuk mahasiswa mendapat masukan detail dari dosen tentang essai yang sudah mereka buat – jadi essaimu itu ...., jadi kita sebagai mahasiswa tidak harus hanya fokus ke penjelasan dirubrik. Menurutku, peran dosen di rubrik dibutuhkan oleh mahasiswa untuk memberikan masukan.”
“…sometimes the detail information are good for the students but I think it will be more clear and exciting for the students to get detail comment from the lecturer about the writing product that students had made –so overall your writing is .... , so we as the students do not have to focus only on the detail of the rubric. I think that the teacher’s role in the rubric is needed by the students to provide comments.” (Writer own translation)
Those may confirm that lecturers’ role is providing feedback or comments for each student’s
writing product and also provide detail information about the score and general comment that written in the rubric. It means that not only the rubric that is needed by the students but also lecturers in guiding them for producing writing product. This is in line with Wood (2009) who found out that assessment and feedback on their writing helps them become better writers. Wood added that when the students use the rubric, it was mostly to help them decode their lecturer’s comments on their writing. In short, the rubric used by the students is to comprehend the lecturer’s interpretation of their writing. Hence, the students can understand
Analytic Scoring Rubric helps the students to improve their writing
Mostly, students have positive thought toward analytic scoring rubric related to their writing. Since students believe that analytic scoring rubric provides clear description of each criterion, shows their strengths and weaknesses in writing, and also can be guideline for them to write well, those drove them to consider that analytic scoring rubric helps them to improve their writing as mentioned by student R in excerpt 11
Excerpt 11
“… karena beberapa poin yang di centang/di tandai oleh dosen bisa dipahami dan dimengerti olehku untuk mengevaluasi penulisanku...”
“… because some of the points that are thick by the lecturer were understandable for me to evaluate my writing...”
(Writer own translation)
By looking at that excerpt, this study found that the appropriateness of the scoring is because the detail explanation why they got those kind of scores. Thus, the students can revise and write better for the next writing assignments. Furthermore, analytic scoring rubric helps them to improve their writing because they can understand each aspect which is explained in the column. They also understand what they are good at or bad at, what they need to revise in accordance with the rubric. The lecturer put the thick in the box/column and the explanation where the lecturer put the thick is understandable for the students and it is clear for the students.
lecturers about students’ writing, and it does not provide enough information about students’
writing.
Analytic Scoring Rubric creates negatives feelings
Some students thought that analytic scoring rubric can create negatives feelings on them because of the aspects that are graded in the rubric. The students have to fulfill all the aspects with good quality to get a good grade. Two out of 20 thought that analytic scoring rubric is not good for them, as mentioned by student A in excerpt 12
Excerpt 12
“Menurut saya rubricnya menakutkan, karena tulisan saya dinilai berdasarkan thesis/argument, so what and who cares, critical incident dan sebagainya, itu semua dinilai menggunakan poin – needs improvement, average, dan also absent. Ada banyak hal yang akan dinilai dalam tulisan saya.”
“I think it’s intimidating, because in my writing, it was scored based on thesis/argument, so what and who cares, critical incident, and so on, they were graded using point – needs improvement, average, and also absent. There are many things that being graded in my writing.”
(Writer own translation)
This may show that because of the several judgments of the students’ writing that provided in
the analytic scoring rubric, those make students feel stressful, they should focus not only on one aspect but many aspects if they want to get a good grade for their writing product. It can be hard for the students to write because of many aspects that should be fulfilled. This is in line with Zane (2011), it is nearly impossible to create mutually exclusive categories of quality – regardless of the type of categories used. In other words, it can be hard to bring into line those aspects of writing that being graded that exist in analytic scoring rubric.
Analytic Scoring Rubric has no further explanation about students’ strengths and weaknesses
rubric provide information about their strengths and weaknesses as student N response in excerpt 13
Excerpt 13
“Kurasa nggak, karena guru hanya menggunakan rubrik ini tanpa memberikan komentar atau masukan, aku tidak yakin bahwa itu menunjukkan kekuatan dan kelemahanku.”
“I think no, because the teacher just used this rubric without giving comments or feedbacks, I don’t think that it showed my strengths and weaknesses.”
(Writer own translation)
Students thought that their lecturer plays a very important role to explain further information about the description of the aspects in the rubric in relation to their writing draft in the term of feedback or comment. Otherwise, in this student opinion, he/she might be suggested that the students can improve their writing skills if they get further information about their strengths and weaknesses. Teachers’ comments and feedback will help students to be more conscious
about what they need to improve for their writing and what they have to preserve for their writing. Hence, they can write better for their next writing assignments.
Analytic Scoring Rubric has no enough feedback from the lecturer about students’ writing
This reason still has correlation with the previous which mentioned that lecturer plays important role for their writing. Some students felt that his/her lecturer did not provide enough feedback in the rubric about their writing. They thought that the description in the rubric is not enough for them to do the next writing assignment. They need the lecturers’
feedback and comments about their writing, student N response in excerpt 14
Excerpt 14
“I think the analytic scoring rubric in academic writing is not so good, because the teacher only thick on the box and they rarely gave the comments, so that I didn’t really know what is my strengths and weaknesses in writing.”
(Writer own translation)
Students thought that analytic scoring rubric is not good for them because of the lecturer did not really provide enough feedback for them. According to student N, analytic scoring rubric cannot help her if the lecturer did not guide her to do her writing product. She wanted the role of both; the lecturer and also the rubric to make it good and she can know her writing ability if both play the role.
Although particular students think that analytic scoring rubric can help them knowing their strengths and weaknesses, there were some points that needed to be improved, student G response in excerpt 16
Excerpt 16
“…Kelemahannya, kolom kosong tanpa penjelasan untuk kriteria tertentu, aku merasa sulit untuk memahami apa yang seharusnya aku perbaiki. Terkadang, penjelasan dirubrik tidak terlalu jelas, seperti di thesis/argument –presenting in interesting way, apa yang di maksud interesting way? Karena menurutku thesis/argument fokus pada apa yang kamu bicarakan. Apa yang dimaksud dengan interesting, aku membutuhkan masukan lebih. Jadi, peran dosen dibutuhkan untuk menjelaskan rubrik.”
“…The weaknesses is the blank space with no explanation of the criteria, I found it difficult, what I should do to improve it. Sometimes, the explanation in the rubric is not really clear like in the thesis/argument – presented in interesting way, what kind of interesting way? Because I think the focus of thesis argument should be what you are going to talk about. What the teacher means by interesting, I need more feedback. So, the teacher’s role is needed to explain the rubric.”
(Writer own translation)
though it means she had good thesis/argument, she still needs the further detail feedback from the lecturer about it. So that, she can write better for the next writing assignments.
Analytic Scoring Rubric doesn’t provide enough information about students’ writing
Analytic scoring rubric is used by the students to be guideline for their writing. They could revise their writing based on the rubric that they got. They could understand what they are good at or bad at by looking at the rubric that they have. Analytic scoring rubric cannot be guideline for the students to write well and recognize where they’re at if there is no enough
information about their writing. One of the students – student N thought that there is no significant improvement of her writing progress because the analytic scoring rubric doesn’t provide enough information about her writing, as in excerpt 15
Excerpt 15
“Peningkatan yang aku alami tidak stabil, nilaiku terkadang meningkat dan menurun, tidak stabil. Saya tidak tahu apa yang harus saya lakukan untuk penulisanku, karena rubriknya tidak menjelaskan informasi yang lengkap tentang penulisanku.”
“I did not make stable progress, my score sometimes; it is increase and decrease, not stable. I don’t know what to do for my writing, because the rubric did not provide further information about my writing.”
(Writer own translation)
This may suggest that analytic scoring rubric perhaps did not help the students in improving their writing skills. The reason in his/her opinion is analytic scoring rubric did not provide detail information about students’ writing and therefore, the students did not know what he/she should write and what he/she should revise for the next assignments.
CONCLUSION AND PEDAGOGICAL IMPLICATION
who have already taken Academic Writing course. The results show that analytic scoring brings positive and also negative impact to students’ learning in Academic Writing. However, mostly, their perceptions are more on positive sides. It can be seen from students’ perceptions that were specified into positive and negative about the use of analytic scoring in Academic Writing. There are several reason why the students perceived positively toward analytic scoring used in Academic Writing, such as (1) Analytic Scoring Rubric provides clear description of each criterion (2) It provides information about students’ strengths and weaknesses in writing (3) It can guide the students to revise their writing draft (4) It can be guideline for the lecturers to grade students’ writing as well as for students to understand how
to get a good grades and (5) It helps the students to improve their writing. This is in line with what Zane (2011) mentioned in his study that research has shown that using analytic scoring rubric has positive effect on students’ learning and the quality of their written submissions.
Besides, analytic scoring rubric also brings positive effect for them and also their future writing. They believe that analytic scoring is fair for them because the rubric shows transparency of their score which is in line with Wood (2009).
However, some student may perceive negatively toward analytic scoring rubric used in Academic Writing. The reason are (1) Analytic Scoring Rubric creates negative feelings (2) It has no further explanation about their strengths and weaknesses (3) It has no enough feedback from the lecturers for the students and (4) It doesn’t provide enough information about students’ writing. Therefore, Analytic scoring’s criteria cannot be applied in their
writing if there is no teacher’s guidance. This study found that the students believe in their teacher’s guidance. They mentioned that lecturer plays important role in the use of analytic
Nevertheless, the finding of this study has limitations that should be known. First, the result of the study cannot be generalized to all cases because there will be many factors which could make the differences in findings. Additionally, if the study was conducted in the different context, for example students’ perceptions of analytic scoring in literature course, it
would have produced different results in the three categorize. Second, the study might generate different results if the study used the bigger number of participants. The other reason, since it is about students’ perceptions, the characteristic of the participants can also
affect the findings.
The results of this study show that the existence of analytic scoring rubric and the scoring itself in academic writing is important to help the students’ learning in writing,
improve their writing skills, and also guide them for their better future writing. Regarding the findings, it might be better if the lecturers apply analytic scoring in writing classes. While others who have analytic scoring in writing classes can maintain and more concerned in giving feedback and should provide consultation for its application. Furthermore, it is important for the students to be more aware of the use of analytic scoring because it can help them to write well. Lastly, it becomes my recommendation for future researchers to conduct a further study in this field, especially the students’ perceptions that differ from other points
related to analytic scoring in other writing classes that were not discussed in this study and different context of this study.
REFERENCES
Bacha, N. (2001). Writing Evaluation: What Can Analytic Versus Holistic Essay Scoring Tell Us? System, 371-183. retrieved April 8, 2013, from:
http://www.sanjeshserv.ir/Hamgam/Files/file/Writing%20Evaluation.pdf. Becker, A. (2011). Examining Rubrics Used to Measure Writing Performance in U.S
Intensive English Programs. The Catesol Journal . retrieved April 6, 2014 from: http://www.catesol.org/Becker%20113-130.pdf
Cohen, A. (1987). Student Processing of Feedback on their Compositions. (A. W. Rubin, Ed.) Learner Strategies in Language Learning, 5.
Elbow, P. (1993). Sorting Out Three Forms Of Judgement. Ranking, Evaluating and Liking : , 55 (2), 187-206.
Gaudiani, C. (1981). Teaching Composition in the Foreign Language Curriculum. Language in Education: Theory and Practice Series .
Hughes, A. (2003). Testing for Language Teachers (2nd. edition). Cambridge: Cambridge University Press.
Husada, H. (2007). Writing Process, Assessment and Students’ Error Correction. Second Language Writing .
Ismail, S. (2011). Exploring Students' Perceptions of ESL Writing. English Language
Teaching , 4 (2). retrieved August 3, 2013 from:
http://www.ccsenet.org/journal/index.php/elt/article
Jacobs, H. Z. (1983). Testing ESL Compositions: A Practical Approach. Rowely, MA: Newbury House.
Kroll, B. (2001). “Considerations for Teaching an ESL/EFL Writing Course" In Teaching English as Second or Foreign Language (3rd Edition ed.). (M. H. M. Celce-Murcia. Boston, Ed.)
Kroll, B. (1990). Second Language Writing: Research insights for the classroom. Cambridge University Press.
Lee, Y-W et al. (2008). Analytic Scoring of TOEFL CBT Essays: Score for Huma ns and E-raters. ETS. retrieved January 19, 2014 from: https://www.ets.org/Media/Research/ McKay, S. L. (2006) Researching second language classrooms. Mahwah, NJ: Lawrence
Erlbaum Associaties, Inc.
Nunan, D. (2000). Second Language Teaching and Learning. Heinle & Heinle Publisher. Rashid, M. (2007). Malaysian University Students' Perceptions on The Use of Portfolio as an
Assessment Tool in an ESL Writing Classroom. Masalah Pnedidikan , 30 (2). retrieved, February 18, 2014 from: http://www.pdfio.com/k-2022151.html#
Richardson, J. (2005). Students' perceptions of academic quality and approaches to studying in distance education. British Educational Research Journal , 31 (1), 7-27. retrieved
April 8, 2014 from:
http://paws.wcu.edu/wgordon/documents/studentperceptionofqualityindistanceeducati on.pdf
Sampath, S. (2005). Sampling Theory and Methods (2nd Edition ed.). Alpha Science International, Ltd.
Sokolik, M. (2003). “Writing”. In Practical English Language Teaching. (D. Nunan, Ed.) New York: McGraw-Hill.
Zacharias, N.T (2011). Qualitative Resarch Methods for Second Language Education :A Coursebook. 12 Back Chapman Street, Newcastle upon Tyne, NE6 2XX, UK: Cambridge Scholar Publishing
Wiseman, C. (2012). A Comparison of the Performance of Analytic vs. Holistic Scoring Rubrics to Assess L2 Writing. Iranian Journal of Language Testing , 2 (1). retrieved April, 6, 2014 from:http://ijlt.ir/portal/images/PDF/408-2012-2-1.pdf
Wood, S. A. (2009). Basic Writers' Perceptions of Writing Assessment Practices. retrieved November 26, 2013 from: http://web.nmsu.edu/~sandrusw/dissertation.pdf
Zane, T. (2011). How to Create Your Written Communication Rubrics. Salt Lake Community
College. retrieved January 19, 2014 from:
APPENDIX A - Questions for Interview
1. What do you think about analytic scoring rubric used in academic writing? 2. What do you think about the analytic scoring in academic writing?
3. What do you think about your score in academic writing?
4. After receiving the analytic scoring rubric from the teacher, what do you usually do with it?
5. Do you believe analytic scoring and the rubric itself provide information about your strengths and weaknesses? Why or why not?
6. Based on your experiences using analytic scoring rubric, how was your writing progress from the beginning?
7. Based on your experiences, what is benefit/weakness of analytic scoring rubric?
APPENDIX B
Analytic Scoring Rubric – also used in academic writing semester 1 – 2013/2014
Points Absent Needs
Improvement
Average Very Good Excellent
Thesis/Argument Arguments are inadequately presented Arguments are clearly presented but may lack of relevance Arguments are presented with relevant superting material & presented in an interesting way
So what? & who cares?
Readers need to struggle to understand why the thesis is important & who it is for?
Provide a sufficient reason of why the argument is important & presented in a way that engages the readers/
Critical Incident (CI)
Present; hard to see the relationship between the CI & the argument
A clear relation between the CI & the thesis
A clear & direct relationship between the CI & the thesis
Grammar Glaring errors, making it difficult to understand the text
A few errors Few errors No detecable errors
Word Choice Many
inappropriate use of vocabulary Some inappropriate use of vocabulary
Appropriate & varied use of diction
Coherence Lack of
connection between ideas Showing attempts to connect ideas although not all successful Flow smoothly
ACKNOWLEDGEMENT
I would like to express my greatest attitude to Allah SWT. for His amazing kindness, guidance, and strength so that I can finish my thesis. Also, I would like to express my sincere gratitude to my thesis supervisor, Maria Christina Eko Setyarini, M.Hum. for giving me her valuable time, support, suggestions, and guiding me patiently during consultations that help me finishing my thesis. I would also show my appreciation to my second reader, Victoria Usadya Palupi, MA-ELT, for her willingness to read and examine my thesis.
