However, as this expert noted, the meta-cognitive nature of self-regulation makes it extremely difficult to assess use. the si-.andard kinds of paper and pencil instruments. FINDING One implication the experts draw from their analysis of CT skills is the following: "while RT skills themselves transcend specific subjects.
CONSENSUS LIST OF CRITICAL THINKING COGNITIVE SKILLS AND SUB-SKILLS
- Self-Regulation
- CATEGORIZATION
- DECODING SIGNIFICANCE
- CLARIFYING MEANING
- ANALYSIS: To identify the intended and actual inferential
- EXAMINING IDEAS
- DETECTING ARGUMENTS
- ANALYZING ARGUMENTS
- EVALUATION: To assess the credibility of statements or other representations which are accounts or descriptions of a person's
- ASSESSING CLAIMS
- ASSESSING ARGUMENTS
- INFERENCE: To identify and secure elements needed to draw reasonable conclusions; to form conjectures and hypotheses; to
- QUERYING EVIDENCE
- CONJECTURING ALTERNATIVES
- DRAWING CONCLUSIONS
- EXPLANATION: To state the results of one's reasoning; to justify that reasoning in terms of the evidential, conceptual,
- STATING RESULTS;
- JUSTIFYING PROCEDURES
- PRESENTING ARGUMENTS
- SELF-REGULATION: Self-consciously to monitor one's cognitive activities, the elements used in those activities, and the results
- SELF-EXAMINATION
- SELF-CORRECTION
To assess the credibility of statements or other representations that are accounts or descriptions of a person's representations that are accounts or descriptions of a person's perception, experience, situation, judgment, belief or opinion;. and to assess the logical strength of the actual or intended inferential relationships between statements, descriptions, questions or other forms of representation. to recognize the factors relevant to the assessment of the. degree of credibility to attribute to a source of information or opinion. to assess the contextual relevance of questions, information, principles, rules or procedural directions. to determine the acceptability, the level o, confidence to place in the probability or truth of any given representation of an experience, situation, judgment, belief or opinion. For example: to recognize the factors that make a person a. credible witness regarding a given event or credible authority on a given subject; to determine whether a given behavioral principle applies to deciding what to do in a given situation; on. determine whether a given statement is likely to be true or false based on what one knows or can reasonably find out. to judge whether the assumed acceptability of the premises of a given argument justifies one's acceptance as true. deductively certain), or very probably true (inductively justified), the expressed conclusion of that argument.
AFFECTIVE DISPOSITIONS OF CRITICAL THINKING APPROACHES TO LIFE AND LIVING IN GENERAL
In addition to possessing CT skills, the good critical thinker can therefore be characterized by certain affective tendencies or habits of mind. There is consensus that good critical thinkers can exhibit these tendencies.
APPROACHES TO SPECIFIC ISSUES, QUESTIONS OR PROBLEMS
The majority of experts (617.) consider the dispositions listed in Table 5 as part of the conceptualization of. The experts are not saying that someone whose metaphysical, epistemological, political, cultural, or religious view of the world differs from his own is ipso facto not a good critical thinker.
CONSENSUS STATEMENT ON TEACHING AND ASSESSING CT SKILLS A CT skill, like any skill, is the ability to engage in an activity, process or
Although the experienced CT instructor is able to continually assess students, CT assessment must be made explicit to reinforce its value in the eyes of the students, their families, and the public. Regardless of the field, students should be encouraged to be curious, raise objections, ask questions, and point out difficulties in the teacher's position.
PARTICIPATING CRITICAL THINKING EXPERTS Jonathan Adler
Massachusetts, Amherst Towsen State University
APPENDIX A
Described and reviewed by two people in NMMY many citations of other research regarding this instrument. For junior grades, this test purports to evaluate a variety of research, inquiry, and critical thinking skills in the sciences. For grades 2-121, this test includes subscores on sequencing, analogies, numerical, and verbal reasoning.
Designed to measure the effectiveness of remedial reading programs, this test includes subscores on the.
APPENDIX B
Kean, Michael H., "Testing Future Challenges, Future Responses," Presented at the Annual Meeting of the National Committee on Measurement in Education, San Francisco, CA, April 1986. Moss, Pamela A., and Petrosky, Anthony R., "A Proposal for measuring critical thinking,” revision of a paper presented at the annual meeting of the American Educational Research Assoc., Montreal, Canada, April 1983. Duellmalz, Edys S., “Designing an Integrated Instructional and Assessment System for CT SKills,” Presented at the American Educational Research Assoc. , Montreal, Canada, 1983.
Jr., “CT and Pre-College Philosophy,” presented to the American Educational Research Assoc., San Francisco, CA, 1986.
ROUND
There are, no doubt, a great number of other people who are interested in the question of testing critical thinking and who have
QUESTIONS
- Do you agree with the two assumptions? How would you amend/clarify them? Why?
- To which educational level (from Kindergarten through post- Baccalaureate) should the committee give priority? Why so?
- After looking the attached preliminary list, whom else would you recommend be added to our committee?
- Mar. 14, 1988) the focus was on establishing group membership and agreeing on preliminary working assumptions
If we're going to get anywhere, we're going to have to get clear in our own minds about how CT is to be distinguished from other kinds of thinking...” Notes that we are a subcommittee of the APA pre-college committee a person argued, “We must prioritize K-12; That's our mission.” Taking APA's interests into account, one person argued, “Since our ad hoc committee is convened within the framework of the APA, perhaps our focus should be primarily on improving postsecondary education. .".
34;My comments will no doubt mirror those you will receive from my colleagues who are also concerned about the problems facing our education system, particularly in relation to the teaching of CT skills and concepts.'
ROUND THREE ****
QUESTION: WHAT OPERATIONS ARE CENTRAL TO CT?
Dear Delphi Colleagues,
- Interpreting -- Observing, Decoding, Clarifying
- Inferring CLu Conjecturing, Drawing Conclusions
- Analyzing -- Locating A.-guments, Parsing Arguments
- Evaluating -- Verifying Claims, Assessing Logical Strength
- Monitoring -- Regulating, Reviewing, Correcting
Applying this lesson to the problem at hand, there are many ways to organize CT skills and subskills. True, skills and subskills could be called by other labels or combined in other ways. Instructions: BeloN you will find a.sheet that outlines and describes six basic CT skills: and subskills.
I urge you to read the entire list of skills and sub-skills before beginning to respond to a specific item.
DESCRIPTIONS OF SKILLS AND SUBSKILLS
In this model, c;Elru.sin.g, receives results in the form of information from the knowledge base, the results of other CTs. Cxpreinq then formats it: output for use by another CT function, for storage in the knowledge base, or for transmission outside the CV loop. Thus, he can send the results of one's inferences for review by directing them to eya/qAt.thg before allowing yxpressing.
And then they send the results to the expression function to be formatted for delivery elsewhere.
AN INPUT/OUTPUT FLOW MODEL OF CRITICAL THINKING SKILLS
EXPRESSING
Fart What CT is Not
Just because something is on this list doesn't mean the person isn't using Cr in it, nor that the person might.
CRITICAL THINKING IS NOT YES/NO
- Speaking or writing,
- Motivating, persuading, selling,
- Trouble-shouting, problem-solving, puzzle solving,
- Communicating using lanyuaqe
We also agreed to begin by identifying the core elements of CT expected on. We agreed to use this theoretical construct of CT at the college level to guide what is said about CT at the K-12 levels. Once we reach consensus on the core list of CT skills expected at the lower level, we will have completed.
My first two readings of the data in round 4 are that we have a great deal on the list of CT sk.11s, although there will be many adjustments and changes to be made.
DELPHI ROUND 5-A
PART 1 -- CT SKILLS
As a goal statement of what a generally educated critical thinker in college should be able to do, we hope that our consensus description of CT skills will inform CT assessment and CT curriculum development at both the college and K-12 levels. Since becoming proficient in CT involves learning to effectively use CT skills in many different contexts, we cannot overemphasize the value of a solid liberal arts education that complements honing one's CT skills and cultivating a CT aptitude. To assess the credibility of statements, descriptions, questions or other representations that express experiences, situations, . beliefs, judgments or opinions; and evaluate the strength of the expressed inferential relationships between such claims, descriptions, questions or
To assess the degree of credibility to be attributed to a source of information or opinion; to a:.) sr relevance of questions,.
A, PART 2 -- FNDORSEMENTS,
- Do you endorse the above statement as useful for purposes of asst sing tht; skills dimension of CT at the lower division college
- Would you be willing to lave your name listed in association with the above description of CT skills as a contributing member
It's the best I know. 34;..to make sense of CT we must pay attention to the critical thinker. Let's not reduce rich and complex realities like CT to what can be directly tested in multiple-choice format."
We don't have that here. a "[Conjecture] is creative thinking not CT. * "[Having included searching and gathering evidence] is ambiguous.
DELPHI ROUND 5B
The Dispositional Dimension of CT
- Should our final report include a statement on the dispositional
- What implications or recommendations for K-12 and college lower division level assessment, curr.culum development and pedagogy follow
INSTRUCTIONS Consider the following statement.. correct as you see Lit. Some basic information is presented in .. the second half of this letter. In addition to the dimensioning of cognitive skills, KT also includes certain cognitive dispositions, personal qualities or intellectual virtues, which are key to its wide and successful use. Cultivating KT cognitive dispositions is especially important in ensuring the use of KT abilities outside of the narrow KT.
The critical thinker is someone who possesses and cultivates the cognitive disposition, personal qualities or intellectual virtues.
The Normative Dimension of CT
- Should our final report include a statement on the normative
- What implications or recommendations for K-12 and college lower division level assessment, curriculum development and pedagogy follow
4) "CT can be characterized as skeptical, radical, disillusioned, problem-seeking, holistic, judgment-oriented, non-algorithmic, constructive, comprehensive, empathetic, meta-cognitive, higher-order, reflective reason-controlled, logical, self-correcting , context-sensitive, criterion-referenced, crisis-oriented, normative, inferential, pragmatic, goal-oriented, relational and open-minded." Matthew Lipman, 1988. 5) "A given student, S, is a critical thinker in some area requiring mental effort, X, if S has the disposition and skill to solve problems in X using of some subset of the available evidence E relevant to activities in area X. . . 34;CT" has an identifiable meaning, but the criteria for its correct application vary from field to field." Richard Paul has us most effectively taught (that] there is, or should be, a causal relationship between skill and disposition such that the disposition can/must elicit, motivate and act as co-constraints for the skills.
34;Of course, some descriptions of dispositions are ways of describing what people who are good at CT are like, but I fail to see how it follows that they are not also ways of describing r7 itself."
DELPHI ROUND 5-C
Proposed Outline of Our Report
- Delphi Conceptualization of CT B. List of Delphi Recommendations
- Endorsement of Members of Delphi Research Panel
- Introduction
- The Concern for CT Assessment
- National interest and large scale assessment 2. CT assessment in the inciividual classroom
- Charge to the project director
- Description of Delphi Research Methodology 3. Building the Delphi Panel of Experts
- Conceptualization of CT, [Rounds 3, 4, and 5]
- CT -- the skills dimension
- General Comment on Assessing a Skill. [Rounds 4 and 5 -C]
- Recommendations
- General Considerations Regarding Educational Assessment
- Validity, Reliability
- Difficulty and Discriminability 3. Purposes of a CT assessment
- Characteristics of persons being assessed
- Strategies for Classroom CT assessment 1. Some questions to ask one's self
- Assessment, pedagogy and curriculum development
- Suggestions on putting together a classroom CT test
- Comments on Large Scale CT assessment IV. Appendices
- A Quick List of Purported CT Assessment Tools B. A CT Bibliography with emphasis on Assessment
- Response rates to each Delphi and other tabular data E. List of Delphi Panel of Experts
INSTRUCTIONS: Consider the following summary of our Delphi research report to the American Philosophical Association.
General Comment on Assessing A Skill
Recommendations
34;..Do we not observe performances that we interpret as skill or as evidence of skill at a certain level or level.". 34;because it can be less time-consuming and because it would be easier to design instruments for..". If we have the product and a description of the process by which it was obtained, we would be in a better position to judge the level of skill."
34; because the third method could reveal that a person has good CT skills, while the second method indicated poor skills."
FINAL ROUND ***
DELPHI ROUND 6
