The results of the follow-up survey were overwhelmingly positive
98 CLASSROOM ASSESSMENT TECHNIQUES
divided the class into groups of four or five and asked them to read and critique each other's drafts.
Despite her best attempts to organize and direct the group-critiquing sessions, her students were not giving or getting much help. Even though she had given the students a format to follow and questions to answer about each other's papers, their comments were vague and their critiques pro forma. She could see that the students were beginning to dread these small-group sessions. In reading their subsequent drafts, she found very little improve-ment that could be linked to the group work. She was determined to find out why the small-group critique sessions were not working, so that she could either improve them or give up group work entirely.
Assessment Technique. The instructor decided to assess her students' awareness of how their own and their peers' behavior affected their learning in small-group work. Specifically, she wanted to find out whether students thought their writing was benefiting from the group critiques - and, if so, in what way. She came up with two questions and created the simple assessment form illustrated below (Exhibit 5.12). At the end of the subsequent half-hour critique session, the instructor handed out copies of the form to all students and gave them five minutes to answer. Students were asked not to put their names on the forms but to indicate their group number.
Results. As she read through the students' responses after class, she was struck by two things. First, although she had made a point of asking students to be specific in their responses, and had included in that request directions for the Group-Work Evaluation Form, many of the responses were general and vague. For example, students wrote, "I told X that I didn't understand his essay" or "The group didn't like my essay topic." In addition, those responses that were specific focused almost entirely on what she viewed as trivial spelling, punctuation, and grammar mistakes - even though she had repeatedly told the students not to worry about those kinds of "surface"
mistakes in critiquing and revising first drafts. Over and over again, she had directed the students to read first drafts for meaning and to worry about the nonessentials only in final rewrites. Since most of the text in a first draft will be changed or thrown out during the revision process, she reasoned, why invest time and energy polishing prose before you are sure that it says what you want it to? Clearly, that message had not gotten through, despite many repetitions.
Exhibit 5.12. Group-Work Evaluation Form.
Directions: As soon as you have finished your group-work assignment, please take five minutes to answer the two questions below. Give as many specific examples as you can and answer in full sentences.
1. What specific comments, criticisms, and/or suggestions did other members of your group offer to you that are likely to help you improve your draft essay?
2. What specific comments, criticisms, and/or suggestions did you offer to other mem-bers of your group that are likely to help them improve their draft essays?
The second thing about the responses that caught her attention was their unequal distribute ion. Most, though not all, of the students could remember some example of a comment, criticism, or suggestion they had received that might be
helpful. Almost none of them, however, could think of anything useful they might have said. The writing instructor had actually expected the reverse: that her students would recall what they had contrib-uted to the group, but not what the group had contribcontrib-uted to them.
Since the evaluation responses were numbered by group, she could also see that the better answers were not randomly distributed but, instead, tended to cluster in three of the seven groups. When she checked her group assignment sheet, she realized that those three groups contained most of the women in her class. What were the women doing differently in their groups,
she wondered, that was leading to more effective learning?
Response. At the midpoint of the next class meeting, the instructor
summa-rized the results outlined above. She then asked the class to help her figure out what those results meant and what to do next. Specifically, she asked the students why so many of them were focusing on spelling, punctuation, and grammar instead of on meaning. The students gave a variety of informative answers. As the students spoke, the instructor listened carefully and took notes. Several said that they couldn't even tell what a piece of writing meant if it was full of "mistakes " so it needed to be cleaned up before they could really understand it. Other students admitted that they were simply follow-ing the example set by previous writfollow-ing teachers, who had mainly pointed out their mechanical errors. A number of students agreed with the classmate who said that he didn't feel capable of criticizing his classmates' draft essays and was uncomfortable -when he tried to do so. A young woman added that she didn't want to criticize her friends and they didn't want to criticize her.
The students made it very clear that they expected the teacher to do the criticizing and correcting
She reminded the students that, in their responses on the evaluation form, many of them mentioned helpful comments they had received, but few of them remembered comments they had offered. She asked the stu-dents why they thought :heir responses were so lopsided. One after another, they gave variations on the same two responses; and, seen in a certain light, the responses were reasonable. At that particular moment, however, they did not sound reasonable to the instructor. Resisting her strong impulse to take issue with the students, she forced herself to listen carefully and take notes, speaking only to move the discussion along or clarify their statements.
First, the students reminded her that she had instructed them to get comments from their groups so that they could improve their own drafts; she had never emphasized their responsibility to give helpful comments. If helping their classmates was so important, why hadn't she told them to do it?
Second, the students pointed out that, although they knew what had or had not been helpful to them as individuals, they had no idea what any other person in their group considered helpful; so how could they tell whether their comments were helpful to others? By this time, to the instructor's relief, the class period was ending. She promised the students that she would
100 CLASSROOM ASSESSMENT TECHNIQUES
respond to their comments at the next class meeting, reminded them of their homework assignment, and sent them off.
Follow-Up.
The writing teacher was more than surprised; she was stunned by her students' comments during the class. Her initial reaction was disbelief.
How could the students suggest with straight faces that it was her responsibil-ity to require them to be helpful and to teach them how to be helpful?
Wasn't it obvious that, in group work, students were supposed to try to be helpful and were supposed to find out from their peers what kind of help they needed and wanted? After recounting the experience, she confided to a colleague that she thought the students were simply making excuses, trying to escape responsibility.
But her colleague was not persuaded. He asked her where the students would have learned how to function effectively in groups. After discussing their own experiences, they agreed that productive cooperation was not usually taught in school or on the job. They also agreed, laughing, that-if the productivity of committee meetings were any indicator-there wasn't much evidence that most of their faculty colleagues had developed effective group-work skills. Since she did not assume that students came to college with adequately developed writing skills, her colleague pointed out, why should she assume that their group-work skills were any better?
The more the instructor thought about it, the more she realized that her students probably had been honest with her. She recognized that if she was going to use group work in the classroom, she would have to be willing to train her students to do it well. And if she was going to ask them to critique each other's papers, she clearly had to teach them how to do so. It would mean taking time away from other elements of the course, but she felt that learning to cooperate effectively was such a valuable skill that the tradeoffs would be worth it.
At the next class meeting, she thanked the students for their feedback and told them that she had decided to be more directive and explicit in her group-work assignments. She explained why she considered cooperation a necessary skill for college and work success and gave some examples to illustrate its value. Since several students had indicated their reluctance to be
"critical," she pointed out the difference between "critiquing" a first draft and "criticizing" someone's writing. Using the students' earlier responses as a starting point, she worked with the class to identify the qualities that made critiques either helpful or harmful.
The class agreed that the most helpful critiques were specific, pointed out strengths and weaknesses, provided explanations for positive and nega-tive judgments, and offered options for improvement whenever possible. In contrast, harmful or useless critiques were very general, focused exclusively on strengths or exclusively on weaknesses, provided no reasons for the judgments made, and offered no suggestions for improvement or insisted that there was only one way to improve.
The writing in structor used the criteria for helpful critiques in planning
the next group-wo rk session. She also reviewed the original group-work
evaluations, using the results to help her reconstitute the groups to get a
better mix of strong and weak contributors. Instead of asking students to read each other's draft: in class, she gave each student copies of two drafts from his or her new group to read as homework. Along with the drafts, students got two copies of a critiquing form. The form asked them to point out some positive element in the draft, a strength, and explain why they thought it was good;
tc d(othe same for a weakness; and to offer two or three suggestions for improvI.rtg the weakness. Students were told that their group work would be based on these forms and that the instructor would collect
them after the critiquing session.
As she had hoped, the next critiquing session was much more lively and seemed to be more productive. At the end of the group work, after she had collected the critiquing forms, the instructor handed out copies of the same Group-Work Evaluation Form she had used several sessions earlier. The overall level of feedback on the session was much higher. There was more writing, responses were more specific, the tone was more positive, and most students were able to answer both questions-commenting on what they had contributed as well as on what they received. Moreover, the quality of the revised essays that they handed in subsequently was markedly higher.
The writing instructor was, understandably, very pleased with the improvement, and she told the class so. Most agreed with her that the group work had been more useful and enjoyable. When she asked them what accounted for the improvement in the critiquing session, however, she once again got an answer that she would not have predicted. The teacher expected students to talk about Jhe direct instruction she had given them, the list of characteristics of helpfu-l and harmful critiques, or the critiquing form.
Students did mention these instructional changes-but not immediately.
The first student to respond said he had made sure to give helpful comments because he knew he was going to be asked about them as soon as the group work ended. Several other students agreed. They said they knew in advance that she was going to hand out the Group-Work Evaluation Forms again, and they wanted to be able
togive better answers the second time around.
While the students probably could not have improved their group performance without more guidance and instruction and additional prac-tice, it was the Classroom Assessment and the teacher's response to it that focused their attention on the importance of improving their performance.
And some students were motivated to improve simply because they antici-pated the assessment, even though they knew it would be ungraded.
102 CLASSROOM ASSESSMENT TECHNIQUES
T W 0
Classroom Assessment
Techniques
C H A P T E R
Choosing the Right Technique
HOW TO FIND THE CLASSROOM ASSESSMENT TECHNIQUE YOU NEED
Our aim in presenting the fifty different Classroom Assessment Techniques described in Chapters Seven, Eight, and Nine is to provide college teach-ers-from various disciplinary specialties and backgrounds-with a com-pendium of good ideas developed by their colleagues for assessing and improving student learning. These chapters are, in many ways, the heart of this handbook. On one level, they resemble a tool chest containing fifty different "feedback devices," from which teachers can select the right assess-ment tool to fit the particular assessassess-ment job at hand; on another level, these chapters can be regarded as a collection of favorite recipes, or a "how-to"
book, a vehicle for sharing tried-and-true approaches among practitioners.
We urge readers to view and use these CATs as starting points, ideas to be adapted and improved upon. And we hope these fifty will serve as models and inspiration for many more new CATs yet to be invented.
Part Two is the largest part of the handbook, and the one least likely to be read straight through. Instead, we expect that many readers will make use of the three indexes in this chapter to find specific Classroom Assessment Techniques. By using the alphabetical index of CAT names (Table 6.1), faculty can quickly locate techniques that they may have read about earlier in the handbook, or elsewhere, or heard colleagues mention. By referring to the index of examples by disciplines (Table 6.2), teachers can look at CATs that other faculty in their fields have already used. Finally, instructors can find assessment techniques related to their Teaching Goals Inventory re-sponses in the index of CATs by related TGI clusters (Table 6.3).
For ease of use, the indexes are placed at the end of this chapter, immediately preceding the Classroom Assessment Techniques in the follow-ing three chapters. These three indexes, along with the topical index that the Table of Contents offers, provide readers with several ways to locate appropri-ate and useful Classroom Assessment Techniques. In addition, we hope that the organization of the following three chapters will encourage faculty to
Choosing the Right Technique 105
browse through the chapters for promising techniques. Chapter Seven includes twenty-seven Classroom Assessment Techniques for assessing course-related knowledge and skills. Chapter Eight contains thirteen CATs for assessing students' course-related attitudes and values, self-awareness, and learning strategies. Finally, the ten techniques in Chapter Nine are designed to assess student reactions to instruction.
The first and largest set of techniques are those for assessing academic skills and knowledge. Helping students learn the subject matter of their courses is the most common goal of college teachers, and virtually all teachers try to measure what students are learning about the content being taught. But most college teachers aspire to more than simply teaching students information about subject matter. They hope to use subject matter to teach students to think - that is, to develop higher-level cognitive skills: to solve problems, analyze arguments, synthesize information from different sources, and apply what they are learning to new and unfamiliar contexts.
From the 1950s through the 1970s, major advances were made in the description and measurement of "higher" cognitive outcomes, as Benjamin Bloom and other measurement specialists developed taxonomies of educa-tional objectives (Bloor and others, 1956) and guidelines for formative and summative evaluation (Bloom, Hastings, and Madaus, 1971). During the 1980s and the early 19')0s, educators and cognitive scientists joined in attempts to promote the development of "creative" and "critical" thinking skills through direct instruction. In choosing and organizing the techniques presented in Chapter Seven, we have drawn on these streams of research and pedagogical practice. We have given particular attention to the various ways in which researchers and theorists conceptualize intellectual development.
Chapter Eight presents techniques for assessing learners' course-related interests, opinions, attitudes, and values, and their views of themselves as learners. These "affective" dimensions of learning critically affect student motivation and success in the "cognitive" dimensions presented in Chapter Seven. Chapter Eight also focuses on skills that cognitive psychologists call
"metacognitive," to convey the importance of students' attention to and awareness of their own learning processes. Research shows that successful students engage in metacognition to plan their learning, monitor it "in process," and continually assess their own skills as learners. Less successful students, in contrast, are more likely to view learning as memorizing a set of facts and answers that are "out there," apart from themselves, and are less likely to assume that they can control their own learning.
Chapter Nine contains assessment techniques for collecting informa-tion on teaching performance and instrucinforma-tion. In this chapter, we ask not so much what students are learning, or how they are learning it, as how they perceive and react to t~he classroom experience. Recent research on the evaluation of teaching and learning suggests that students are valid and reliable sources of information about the effects of teaching or its impact on their learning (Cross, 1988). Many instructors, especially beginning teach-ers, will find the direct, timely, and safe access to student responses provided by these CATs the most helpful feedback of all.
Chapter Nine is also based on the extensive and still-developing liter-ature on students' reactions to their classroom experiences. Among the most
informative studies are those by Centra (1977a, 1977b), Feldman (1977), Gleason (1986), Kulik and McKeachie (1975), and Seldin (1984). To date, student evaluations have been used primarily to help administrators make decisions about promotions and tenure. However, the assessment techniques presented in Chapter Nine are not for that purpose. Rather, they are designed to help teachers and students make decisions about which methods and materials are most helpful in promoting learning in the classroom.
Chapters Seven, Eight, and Nine represent broad-stroke topic-centered classifications of the CATs. The groupings of CATs in each of these chapters represent functional clusterings of the assessment techniques.
Within each of these ten CAT groups, we have presented quicker and easier techniques first, more difficult and demanding ones last.
FORMAT USED TO
Each Classroom Assessment Technique format contains at least thirteen of
PRESENT CLASSROOMthe following elements; many of them contain all fourteen.
ASSESSMENT
TECHNIQUES 1.