Projects
66 CLASSROOM ASSESSMENT TECHNIQUES
This finding disconfirmed the instructor's prediction that the statements on gender and ethnicity would be most often miscategorized.
Follow-up interviews with students revealed that negative or highly skeptical attitudes toward science were responsible for incorrect responses from some of the brightest students. For example, several A and B students told her that they simply did not believe that astronomers could judge the temperature of a star from its brightness, even though they knew that the statement "Brighter stars are generally hotter ones" is considered in astron-omy to be a fact.
Such responses were quite revealing to the instructor. She saw clearly that students could do well in her class on most objective measures, and could succeed in distinguishing facts from theories, without necessarily accepting the general premises or fundamental values of the discipline. In other words, they could play the game without believing in it.
Response. The results of this Classroom Assessment convinced the astron-omy professor to devote more class time to making explicit the similarities
-and differences-between distinguishing facts from opinions in "everyday"
settings and the more explicit and rigorous rules used by scientists. She also used the outcomes of this experiment to convince other general education instructors to work on explicitly teaching students how to distinguish facts from opinions in questions about race, ethnicity, gender, and the like.
PSYCHOLOGY:
ASSESSING SKILL IN APPLICATION AND PERFORMANCE
Example 3: Assessing Students' Skill in Applying What They Have Learned Discipline: Psychology
Course: Introduction to Psychology
Teaching goal: To help students develop an ability to apply principles and generalizations already learned to new problems and situations (TGI Goal 1)
Teacher's question: Have/how have students applied knowledge and skills learned in this class to situations in their own lives?
CAT used: Applications Cards (CAT 24, Chap. 7)
Background. Regardless of the discipline or level of the courses they teach, most faculty aim to help students use what they have learned to further their academic, professional, or personal goals. In assessing his introductory survey course, a veteran psychology instructor focused on the goal of applica-tion of skills and knowledge. He wondered to what extent his students were applying what they were learning in psychology to their lives. In designing this assessment, this professor gained insight into a question he claimed he had wanted to ask throughout twenty years of college teaching.
Assessment Technique. The psychology professor decided to ask students directly whether they had applied lessons learned in the survey course to their
Exhibit 5.3. An Applications Card.
Have you tried to apply anything you learned in this unit fn human learning to your own life?
Yes No
If "yes. " please give as many specific, detailed examples of vour applications as possible.
If "no," please explain briefly why you have not tried to apply what you learned in this unit.
lives. In addition, to encourage honest self-reporting, he decided to ask for specific examples of their applications. Realizing that some of the course content was more likely to be applied than others, he chose to assess a unit on human learning, a part of the course that he felt had clear relevance to students' everyday live:3 and experience.
As soon as the class had completed a three-week unit on "human learning," the instructor wrote his assessment question on the chalkboard and asked students to copy it (see Exhibit 5.3). He told the class that this assessment would be their homework assignment for that evening. He also announced that they would receive credit for completing the assessment but that it would be ungraded. To ensure anonymity, he asked students not to write their names on the cards and encouraged them to type their responses.
He explained why he wanted their responses and promised to discuss the results in class after he had carefully read them.
Results. At the beginning of the following class, the instructor assigned one student at random to collect the cards and check off the names of students who had handed them in, so that they would receive credit. That night, the psychology professor took the applications cards home and read through them, tallying positive and negative responses and categorizing the types of
applications reported.
To his astonishment, twenty-two of the thirty-five students in the survey course not only said "yes," that they had applied something they learned about learning to their own lives, but also were able to give convincing examples. Several students gave examples of ways in which they had altered their study or test-taking behaviors as a result of what they had learned about their own learning. Not surprisingly, many of the applications were attempts to improve learning by using mnemonics, applying specific learning strat-egies, or lowering stress. A handful of students who were parents mentioned ideas or techniques from class that they had applied to help their young children learn.
On the other hand, several of the thirteen students who said "no"
responded that they had never before been asked to apply things learned in school to their lives, and so they had not tried to do so. Some pointed out, quite correctly, that the instructor had never assigned them to find applica-tions as they were studying the unit. A few students wrote that the question was not relevant, since they were not psychology majors. The instructor reported that the responses of those who had not tried to apply course learning were at least as valuable to him as the positive responses.
68 CLASSROOM ASSESSMENT TECHNIQUES
Response. During the following class meeting, the psychology professor summarized the results and asked students to share their applications with the class. He made a list of their examples on the board and then challenged each student to think of one other possible application that no one had mentioned. To his surprise, the class had little difficulty coming up with another, longer list of possible applications. The class discussion was one of the liveliest he had witnessed in years, and students indicated that they found it very useful to consider applications.
As a result of his assessment, this instructor made it a point to include discussions of possible applications in each subsequent unit. He found that class participation increased noticeably and that students improved their ability to propose reasonable applications based on psychological theories and research findings. From the first week of class, his students now know that they will be expected to propose applications, even if they choose not to experiment with them in their lives.
MATHEMATICS:
ASSESSING SKILL IN PROBLEM SOLVING
Example 4: Assessing Students' Problem-Solving Skills
Discipline: Mathematics
Course: Second-Semester Calculus
Teaching goal: To help students develop problem-solving skills (TGI Goal 3)
Teacher's questions: To what extent are students aware of the steps they go through in solving calculus problems and how well can they explain their problem-solving steps?
CAT used: Documented Problem Solutions (CAT 21, Chap. 7)
Background. In mathematics and the sciences, it is common to find students who can solve neat textbook problems but who cannot solve similar but messy real-world problems. One reason for this failure to transfer problem-solving skills may be that few students develop an explicit awareness of the ways they solve problems and, therefore, cannot easily adjust their ap-proaches to new situations. To improve their problem-solving skills, students need to develop metacognitive skills-ways of thinking about how they think.
This calculus instructor was frustrated by his students' seeming in-ability to analyze and discuss their own problem-solving processes. Al-though these students were successful math, science, and engineering majors, they found it nearly impossible to explain how, why, or where they were stuck when they could not solve a particular homework or quiz prob-lem. As a result, class discussions almost always degenerated into lecture-demonstrations in which the teacher would show students how to solve the problems they had failed to solve. The instructor suspected that most of his students solved problems by memorizing formulas and plugging familiar-looking problems into them, a strategy that he recognized as necessary but far from sufficient for success in mathematics.
Exhibit 5. 4. A Documented Problem Solution.
5. Look over the four probinis above. Choose one and write out your solution to it step by step. Draw a line down the middle of a piece of paper. On the left side of the paper, show each step in the math; on the right side, explain in words what you were doing in each step. Write as though you were explaining the steps to a first-semester calculus student who had never solved this type of problem. Be prepared to talk through your solution in the next class meeting.
Since the instructor was convinced that to become competent mathe-maticians students needed to become self-aware and self-critical problem solvers, he decided to assess his students' skills in order to find ways to improve them.
Assessment Technique. This calculus teacher regularly assigned five home-work problems each night, to be discussed the next day in class. To assess students' problem-solving skills without increasing the overall workload, he dropped the number of homework problems to four, eliminating the easiest problem in each set. The new, fifth homework "problem" was the assessment technique itself (see Exhibit 5.4).
After solving the four calculus problems-or going as far as they could go in trying to solve them-the students were directed to select one of these problems and explain, step by step in complete sentences, how they had solved it. In other words, they were to document their solutions.
Results. The instructor met with a lot of resistance at first. Several students, including a few of the best, handed in perfunctory and inadequately documented solutions. Students complained about the fifth "problem" and argued that they should not be required to explain how they solved prob-lems, only to learn to solve them. The instructor was initially taken aback by the vehemence of his students' protests. And he was particularly surprised that the most able students seemed to be leading the charge.
After two weeks of struggling to implement this technique, with minimal success, he asked students point-blank why they objected to the assessment. They gave two different sorts of answers. First, nearly all argued that the process of writing out their problem-solving steps was very time-consuming and that they considered the time wasted. The instructor had initially required the fifth problem but had not graded it or given any credit for its completion. "Why bother doing it," several students asked, "if it isn't going to count?"
The second type of objection related to students' conceptions of mathe-matics. As mentioned in the previous paragraph, most of the students viewed getting the tighi. answer as the only means of success. They felt strongly that it did not matter how they got the right answer, or whether they understood how they got it, as long as they got it right and received full credit. The most successful students, by definition, already got the right answers most of the tirne. All the students agreed that no previous math teacher had ever asked them to "document" their problem-solving steps in writing. Why was he inflicting this hardship on them? What could they possibly gain from this exercise?
70 CLASSROOM ASSESSMENT TECHNIQUES
In discussing the students' responses with his colleagues in the campus Classroom Research group, the calculus teacher admitted his frustration and disappointment. Although he had held his tongue and temper in class, he complained to the faculty group that his students were acting like "grade grubbers," unwilling to learn anything unless they were rewarded or punished with grades. He was particularly puzzled by their insistence that since their previous mathematics teachers had not asked them to focus on problem solving, he had no right to do so.
Most of his colleagues had also come up against student resistance to innovation at one time or another; so his reaction was one that the faculty Classroom Research group could easily empathize with. Several faculty pointed out that the students' reactions were understandable and even rational, however, given their prior "conditioning" in mathematics. Many of his students had no doubt learned to equate success very simplistically with getting the right answers. A colleague suggested that the calculus teacher consider offering the students some incentives for changing their behavior.
She argued that his students might be more willing to cooperate if they received some credit for what was clearly hard work and if they clearly saw the value of the task.
Response. The calculus instructor saw the merit in his colleague's suggestion and decided to give the procedure one last try. During the following week, he announced that he had listened carefully to the students' concerns and that, as a result, he would begin giving credit for the fifth problem, but not a grade. If students made a serious, good-faith effort at documenting the steps taken in solving one of the first four problems, they would receive full credit;
if not, no credit. In effect, then, the fifth problem was worth 20 percent of the homework grade. If they failed to document a problem solution, the best they could hope for was 80 percent.
It became apparent during the following class discussion that offering credit for the assessment had provided the missing extrinsic motivation. The calculus teacher then took on the second and greater challenge: convincing students of the intrinsic value of becoming explicitly self-aware problem solvers in calculus. In class and through comments on their homework papers, he praised their attempts at explanation. He also began to ask individual students to go to the board and demonstrate their solutions, and he then encouraged others in the class to show how they had solved the same problems in different ways. The instructor brought in parallel but "messier"
problems and asked the students to outline in words the steps they would take to solve them. The students then worked in groups for a few minutes to compare their planned approaches.
After only a few more applications of this technique, the class recog-nized the improvement in their understanding and in the level of class discussion. All the students seemed to benefit from focusing on problem solving, but the weaker students improved dramatically. Many students had begun with virtually no capacity to explain their work, but they had learned from their peers - through group work and board work - to articulate their steps. Their instructor remained convinced throughout that students could gain more control over problem solving when they understood the process.
At the end of the semester, the calculus teacher reported that the percentage of students receiving A's and B's was much higher than usual, reflecting much higher average midterm and final examination scores. For the first time in nearly thirty years of teaching calculus, he did not fail a single student in his calculus course. His midterm and final exams were drawn from the same item bank he had used in the past; so it was unlikely that the exams were easier than in previous years. His cautious explanation was that students had simply learned the calculus better and more deeply by focusing on problem solving and that the information provided by their documented problem sers had allowed him to teach more effectively. He also speculated that his renewed enthusiasm for discussing the "how" of calculus might have boosted student interest.
POLITICAL SCIENCE:
ASSESSING SKILL IN SYNTHESIS AND CREATIVE THINKING
Example 5: Assessing Students' Skill in Synthesizing and Summarizing Information
Discipline: Political science
Course: Introduction to United States Government
Teaching goal: To help students develop an ability to synthesize and integrate information and ideas (TGI Goal 5)
Teacher's questions: How well can students distinguish peripheral material from key points in lectures and how well can they integrate those key points into a coherent summary?
CAT used: The One-Sentence Summary (CAT 13, Chap. 7)
Background. To learn from a lecture, class presentation, or reading assign-ment, students must differentiate between relevant and irrelevant informa-tion. They must distinguish the few key points from the many supporting details. And to retain what they learn, students must actively integrate and summarize new information.
This political science instructor wondered how well his students were coping with his information-rich lectures on United States government. He was concerned that they might be paying too much attention to taking notes on details and too little on synthesizing those details into a bigger picture.
Therefore, he decided to assess how well he was achieving his goal of helping students synthesize and integrate new information.
Assessment Technique. To get feedback on how he could help students achieve the goal, the instructor selected the One-Sentence Summary, a technique in which the learner tries to summarize a specific amount of material by answering the questions "Who does/ did what to whom, when, where, how and why?" Hie chose to focus the assessment on a class session devoted to presidential elections. As he prepared for class, he identified the key concepts that he would discuss in the lecture; and then, working from his