Phase II: Implementing the Assessment
Step 4: Teaching the Target Lesson Related to the Goal or Question Being Assessed. Plan to integrate the Classroom Assessment activity into your
during a single class session. As professionals and adult learners, faculty should set and follow their own agendas. Nevertheless, we have become convinced that it is advisable to begin by going through the whole cycle with a single, simple assessment first, then branching out to plan a more compre-hensive project. It is important to experience both the positive outcomes and the possible difficulties of each phase.
In summary, when you create the plan for your Classroom Assessment Project, you should describe the assessment strategy you plan to use, making sure that your description answers the following questions:
1. What "assessable" question are you trying to answer?
2. What specific Classroom Assessment Technique or instrument(s) will you use to collect data?
3. How will you introduce the assessment activity to students?
4. How will you integrate it into ongoing classroom activities?
5. What technique will you use to collect feedback?
6. Realistically, how much time can you devote to this project?
7. Will that be enough time to accomplish what you are planning?
8. What will a successful outcome look like?
9. What is the minimum outcome that you would consider worthwhile?
10. What steps can you take to "build in" success?
anonymity is important but not ensured, can lead to student frustration and confusion. If an assessment interrupts the flow of a lesson, it will yield less useful feedback and may actually hamper learning.
Step 5: Assessing Student Learning by Collecting Feedback. Choose a simple Classroom Assessment Technique. Introduce the technique posi-tively and explain it carefully. Make sure to ensure anonymity if responding to the assessment poses any risks-real or perceived-to the student.
The assessment tool can vary from a simple list or one-sentence sum-mary to an elaborate survey with multiple-choice, scaled-response, and/or open-ended questions. To avoid being overwhelmed by more data than you have time to analyze, start with a simple technique and develop a sense of how much information you can glean from a single question or two. Later, if necessary, you can expand a short and simple technique into a longer, more complex one. You will also find it easier to respond and adjust your teaching if the messages that come back from students address only one or two issues at a time.
w- CASE IN POINT
An English instructor, on his first go-round with Classroom Assess-ment, decided to solicit feedback from students regarding his comments on their essays in a college composition class. He asked one simple question:
"How could my comments on your essays be more helpful?" He got back two very important messages. First, many of the students could not read his handwritten comments; second, many wished for more guidance on how to correct the deficiencies he pointed out in their writing.
The design of the assessment tool is crucial. Multiple-choice or scaled-response items are easier to tally, but they allow students less freedom of expression than written responses. Open-ended questions, while more diffi-cult to summarize, encourage students to bring up issues or mention prob-lems that teachers might never suspect exist. For that reason, you may often be better off using one or two open-ended questions rather than a set of multiple-choice items. The wording of questions on Classroom Assessments, as on tests, must be extremely clear. Confusing or ambiguous questions will
elicit confusing and ambiguous responses, clouding your analysis and inter-pretation. To ensure that your questions are clear, you might want to do a
trial run of the technique on a fellow instructor or a student assistant.
Introducing the assessment to students is an important step in the process. Students usually are pleased that an instructor might want to evaluate their learning for some other reason than to assign a grade. By announcing to your class that you want to assess how much and how well they are learning so that you can help them learn better, you can lower barriers to effective learning set up by the grading system and power structure of the classroom. By including students in Classroom Assessment Projects, you suggest that everyone is really pursuing the same ultimate goal-better learning.
Before using Classroom Assessment Techniques, you will want to consider the possible need for anonymity. Where student mastery of content or a skill is being assessed, you may want to know who gave which responses, and students may not feel threatened or hesitant about providing names.
However, when students are being asked to give evaluative comments about the class, the materials, or the instructor, they should be directed to give anonymous responses. Students' anonymity also must be safeguarded when-ever they are being asked questions that concern their attitudes, opinions, values, or personal histories. Even when the questions concern facts and principles, however, asking for anonymous responses can mean the differ-ence between finding out what students think is the "right" answer and finding out what students really think.
- CASE IN POINT
When an astronomy instructor asked students to classify a list of statements according to whether they were "descriptive and factual" or
"interpretive and theoretical," she found it helpful to have names on the responses. She followed up puzzling responses by asking a few students why they had made certain selections. In so doing, she gained significant new insights into why her students had difficulty in making the distinction between facts and theories in astronomy. These insights allowed her to develop more explicit and effective ways to illustrate and teach the contrast.
Later, however, when she asked students to rate the value of the course to their lives, she insisted on anonymity in order to encourage honesty and protect students.
Step 6: Analyzing Student Feedback.
Prepare yourself for surprising
feed-back. Look carefully at both positive and negative results. Carry out only as much analysis as is useful and reasonable, given the time and energy available.This step in the Classroom Assessment Project Cycle often has the greatest impact on instructors, because here the faculty member's assump-tions about what students are actually learning, thinking, or feeling in a particular class come into direct contact with information from the students.
Many teachers are, of course, already in touch with and aware of student reactions, but few go through a Classroom Assessment Project without uncovering at least one or two surprises.
For many teachers, analysis is the first step in refocusing their efforts on student learning, in moving from a teacher-centered toward a learner-centered approach to teaching. The awareness of student needs and percep-tions suddenly thrust upon them can stimulate or fortify a resolve to improve the quality of learning. The meaning of "doing a better job" in the classroom begins to change from "performing better as a teacher" to "promoting better learning in students." Although most teachers are dedicated in principle to this idea, Classroom Assessment can help them put it into practice.
When you begin to analyze the data from a Classroom Assessment, remember to look at the whole range of student responses. Some responses,
Planning and Implementing Classroom Assessment Projects 51
especially negative ones, may strike sensitive chords and have a selectively greater impact than others. However, a balanced picture of the feedback will be more truthful and, in the end, more helpful to you and students alike.
General Comments on Analyzing the Data You Collect. Nearly all the tech-niques presented in this handbook generate data that can be analyzed and assessed quantitatively, qualitatively, or by a combination of the two ap-proaches. Given the formative nature of Classroom Assessment, an approach that combines numbers and narrative is often the most enlightening and useful.
Before deciding how to analyze the data that you have collected through the use of assessment techniques, you will need to clarify why you are analyzing these data Your original purpose for assessing learning in your classroom-to assess your students' knowledge of a particular concept or their reactions to a lesson, for example-should determine how you analyze
the data.
All quantitative data analysis is based on measuring, counting, cate-gorizing, and comparing amounts, numbers, and proportions of items. At the simplest level, we use quantitative analyses to answer the questions "how much" or "how many." A quantitative analysis of student responses to Classroom Assessment Techniques can be as simple as counting the total number of relevant items listed or mentioned. One easy way to analyze such
lists is "vote counting," tallying the number of responses on the students' lists that match those on a master list. This is basically what happens when teachers score multiple -choice tests. Qualitative analysis, on the other hand, provides answers to questions such as "what kind," "how well," and "in what ways." A simple and familiar example of qualitative analysis occurs when teachers read and evaluate essay exams or term papers. In Classroom Assess-ment, to report the results of qualitative data analysis, we depend on narrating, explaining, and giving examples.
The skills required to analyze data collected through Classroom Assess-ment Techniques are familiar to most teachers and, as noted above, can be as simple as those used to evaluate and score tests and term papers. However, the task of analyzing data for Classroom Assessment differs in three impor-tant ways from the normal evaluation of student learning that takes place in classrooms. These differences concern purpose, unit of analysis, and the criteria on which the analysis is based.
When teachers analyze and assess tests and major assignments, they often have multiple and sometimes even conflicting purposes. One purpose is usually to give students feedback on how much and how well they are learning. At the same time, teachers almost always have to grade the students' work for administrative purposes. In the end, the extrinsic, institu-tional demand for grades is often the most powerful motivator for the evaluation of student learning. The data collected through use of Classroom Assessment Techniques, in contrast, should not be analyzed for grading purposes. The motivation for analyzing data collected through Classroom Assessment is intrinsic and personal. The analysis is driven by the teacher's desire to learn more about student learning in order to improve it.
The second major difference concerns the unit of analysis. In tradi-tional classroom evaluations of student work, the primary unit of analysis is
the individual student. It is the individual student's work that is evaluated, rated, ranked, and graded. Even when class averages are computed and students are graded on the curve, the real focus is on the individual student's performance. Classroom Assessment, on the other hand, focuses on the whole class, or groups within the class, as the primary unit of analysis. While results of this aggregate analysis may subsequently lead the teacher to focus on individual students, that is not its principal function. Classroom Assess-ment Techniques are aimed at collecting data on the class as a "learning community. "
The third difference concerns the criteria used and the explicitness with which teachers must define these criteria in their analyses. In traditional classroom evaluation, the criteria for assessing student work are intimately linked to grading. Such criteria are sometimes binary: right or wrong, acceptable or not, pass or fail. More often, they are scalar, as represented by the letter grades A through F or the numbers 1 through 100. In either case, the criteria may be linked to an external standard, such as a national norm, or an internal standard, such as the class average. All such criteria are useful insofar as they make it possible to indicate, with one letter or a single number, the level of performance that the student has demonstrated. As such, they are primarily criteria for rating the products or outcomes of learning. In teacher-directed Classroom Assessment, the criteria used in analyzing the students' responses depend a great deal on the individual teacher's purpose for assessing and very little on institutional needs. The teacher is responsible for choosing or developing criteria to fit that purpose.
We suspect that many teachers find it easier to judge an answer right or wrong than to analyze and assess a written record of the process that led to the answer. Nonetheless, the most useful criteria in Classroom Assessment are often those that can be used to analyze processes and intermediate outcomes of learning
-ratherthan the final products.
Perhaps the simplest way to sum up the difference between classroom evaluation used to grade individual students and Classroom Assessment is to note that the primary purpose of evaluation is to classify students, whereas the primary purpose of assessment is to understand and improve learning.
Suggestions for Analyzing the Data You Collect.
What specific kinds of analysis do you need to transform raw data into information that will help you make instructional decisions? One way to begin to answer that question is to consider several possible general questions about student learning that you might want to answer through Classroom Assessment.
Questions About Your Students
How many students are learning well and how many are not?
Which students are learning well and which are not?
What do successful learners do that other learners don't do, or don't do as well?
What do less successful students do that might account for their failures?
Planning and Implementing Classroom Assessment Projects 53
Questions About Course Content How much of the course content are students learning?
Which elements of the course content are students learning?
How well are students learning the various elements of the course content?
How well are students integrating the various elements of the course content?
Questions About Teaching
How does my teaching affect student learning, positively and negatively?
What, specifically, could I change about my teaching to improve learning inside the classroom?
What, specifically, could I change about my teaching to improve learning outside the classroom?
Phase III: Responding to the Results