Principles in Test
Construction
Dr. Sumardi, M.Hum
OUTLINE OF THIS COURSE:
Part I
What is a test for?
Principles in Test Construction
Steps in Preparing Test Questions
Preparing Multiple Choice Questions
Preparing True or False Questions
Part II
Review of Part I
Preparing Matching Type Questions
Preparing Sentence Completion Questions
Preparing Essay Questions
Other types of Test Questions
Wrap-up/Things to Remember
TYPES OF TEST
Achievemen t test Achievemen
t test
Standardized test
Standardized test Teacher-made test Teacher-made test
Written test
Written test Oral test Oral test Performance /Practical
test
Performance /Practical
test
Essay-type test
Essay-type
test Short-answer type test Short-answer
type test Objective- type test Objective-
type test
THE PURPOSE OF TESTING:
To provide a record for assigning grades.
To provide a learning experience for students.
To motivate students to learn.
To serve as a guide for further study.
To assess how well students are achieving the stated goals of the lesson.
To provide the instructor with an opportunity to
reinforce the stated objectives and highlight what is important for students to remember.
GENERAL STEPS IN TEST CONSTRUCTION
OUTLINE
DRAFT
ORDER
TEST
ANALYZESUBMISSION
PRODUCE A
T.O.S.
STEP 1: OUTLINE
The unit learning objectives, or
The unit content or major
concepts to be covered by the
test
STEP 2: TABLE OF
SPECIFICATIONS (TOS)
A two way chart that relates the learning outcomes to the course content
It enables the teacher to prepare a test containing a
representative sample of
student behavior in each of the
areas tested.
Don’t make it overly detailed.
It's best to identify major ideas and skills rather than specific details.
Use a cognitive taxonomy that is most appropriate to your discipline, including non-specific skills like communication skills or graphic skills or computational skills if such are important to your
evaluation of the answer.
TIPS IN PREPARING THE TABLE
OF SPECIFICATIONS (TOS)
Weigh the appropriateness of the distribution of checks against the
students' level, the importance of the test, the amount of time available.
MATCH the question level appropriate to the level of thinking skills
TIPS IN PREPARING THE TABLE
OF SPECIFICATIONS (TOS)
EXAMPLES OF STUDENT ACTIVITIES AND VERBS FOR BLOOM’S COGNITIVE LEVELS
Bloom’s Cognitive Level
Student Activity Words to Use in Item Stem
Knowledge Remembering facts, terms, concepts,
definitions, principles
Define, list, state, identify, label, name, who?, when?, where?, what?
Comprehension Explaining/interpreting the meaning of material
Explain, predict, interpret, infer, summarize, convert, translate, account for, give example, paraphrase Application Using a concept or
principle to solve a problem
Apply, solve, show, make use of, modify,
demonstrate, compute
Examples of Student Activities and Verbs for Bloom’s Cognitive Levels
Bloom’s Cognitive Level
Student Activity Words to Use in Item Stem
Analysis Breaking material down into its component parts to see interrelationships/
hierarchy of ideas
Differentiate,
compare/contrast, distinguish ____from ____, how does
____relate to ___, why does ____work
Evaluation Making a judgment based on a pre-established set of criteria
Appraise, evaluate,
justify, judge, which would be better?
Creation Producing something new or original from component parts
Design, construct, develop, formulate,
imagine, create, change, write a poem or short story
TIPS IN PREPARING THE
TABLE OF SPECIFICATIONS (TOS)
The following array shows the most common questions types used at various cognitive levels.
Factual Knowledge Application Analysis and Evaluation Multiple Choice
True/False
Matching Type Sent. Completion Short Answer/RRT
Multiple Choice Short Answer Problems
Essay
Multiple Choice Essay
STEP 3: DRAFT
THE QUESTIONS COVERING THE CONTENT IN THE OUTLINE:
1. Are the directions to each section absolutely clear?
2. Is there an example item for each section?
3. Does each item measure a specified item?
4. Is each item stated in clear, simple language?
5. Does each multiple choice item have appropriate
distracters; that is, are the wrong items clearly wrong and yet sufficiently “alluring” that they aren’t ridiculously
easy?
6. Is the difficulty of each item appropriate for your students?
7. Is the language of each item sufficiently authentic?
8. Do the sum of the items and the test as a whole adequately reflect the learning objectives?
STEP 4: ORDER
The selected questions logically:
Place simpler items at the beginning to ease students into the exam.
Group item types together under common instructions.
If desirable, order the questions logically from a content standpoint (e.g. chronologically or by conceptual groups, etc.)
STEP 5: TEST
PUT the questions away for one or two days before rereading them or have someone
else review them for clarity.
TEST the questions by actually taking the test.
STEP 6: ANALYZE
The items to give you an idea
whether the questions were well-
written or poorly written as well as if there were problems in
understanding instruction.
GENERAL RULES IN WRITING TEST QUESTIONS / STEMS
Number test questions continuously.
Keep your test question in each test group uniform.
Make your layout presentable.
Do not put too many test questions in one test group.
T or F: 10 – 15 questions
Multiple Choice: max. of 30 questions
Matching type: 5 questions per test group
Others: 5 – 10 questions
GENERAL RULES IN WRITING TEST QUESTIONS … (CONTINUATION)
o Avoid humorous items. Classroom testing is very important and humorous items may
cause students to either not take the exam seriously or become confused or anxious.
o Items should measure only the construct of interest, not one’s knowledge of the item context.
o Write items to measure what students know, not what they do not know. (Cohen & Wallack)
1. Multiple Choice
2. True or False
3. Matching Type
4. Fill-in the blanks (Sentence Completion)
5. Essay
5 Most Commonly used Test Format
Source: Turn-out of Test Questions in SSI (2003-2007)
1. MULTIPLE CHOICE TEST
When checking the stems (questions) for correctness:
Ensure that the stem asks a clear question.
Reading level is appropriate to the students
The stem is grammatically correct.
Negatively stated stems are discouraged.
What to Look for on
Multiple Choice Tests
MULTIPLE CHOICE QUESTIONS
1. Use negatively stated stems sparingly and when using negatives such as NOT,
underline or bold the print.
2. Use none of the above and all of the above sparingly, and when you do use them, don't always make them the right answer.
3. Only one option should be correct or clearly best.
MULTIPLE CHOICE QUESTIONS:
4. All options should be homogenous and nearly equal in length.
5. The stem (question) should contain only one main idea.
6. Keep all options either singular or plural.
7. Have four or five responses per stem (question).
MULTIPLE CHOICE QUESTIONS:
7. When using incomplete statements,
place the blank space at the end of the stem versus the beginning.
8. When possible, organize the responses.
9. Reduce wordiness.
10. When writing distracters, think of
incorrect responses that students might make.
EXAMPLE OF POOR STEM
1. Sheldon developed a highly controversial
theory of personality based on body type and temperament of the individual. Which of the following is a criticism of Sheldon's work?
a. He was influenced too much by the Freudian psychoanalysis.
b. His rating of physique and temperament were not independent.
c. He failed to use empirical approach.
d. His research sample was improperly selected.
EXAMPLES
Better: (Eliminate excessive wording and irrelevant information)
1. Which of the following is a criticism of Sheldon's theory of personality?
EXAMPLE OF POOR STEM &
OPTIONS
1. The first paragraph is …
a. about the materials of weathering.
b. about the process of weathering.
c. about the impact of weathering.
d. about the result of weathering.
EXAMPLES
Better: (Include in the stem any word(s) that might otherwise be repeated in each option.)
1. The first paragraph is about _______.
a. the materials of weathering.
b. the process of weathering.
c. the impact of weathering.
d. the result of weathering.
EXAMPLE OF POOR STEM
1. Which is not a major technique for studying brain function?
a. Accident and injury b. Cutting and removing c. Electrical stimulation d. Direct phrenology
EXAMPLES
Better: (Use negatively stated stems sparingly.
When used, underline and/or capitalize the negative word.)
1. Which is NOT a major technique for studying brain function?
The related examples:
The following statements are TRUE based on the text …
These are any activities done by the farmers, EXCEPT …
EXAMPLE OF POOR STEM
4. ________________ is the least form of behavior disorder.
a. Psychosis
b. Panic disorder c. Neurasthenia d. Neurosis
EXAMPLES
Better: (When using incomplete statements avoid beginning with the blank space.)
1. The least severe form of behavior disorder is __________________.
EXAMPLE OF POOR OPTIONS
1. The number of photoreceptors in the retina of each human is about . . . .
a. 115 million b. 5 million c. 65 million d. 35 billion
EXAMPLES
Better: (When possible, present alternatives in some logical order.)
1. The number of photo receptors in the retina of each human is about
a. 5 million b. 35 million c. 65 million d. 115 million
EXAMPLE OF POOR OPTIONS
6. Latane and Darley's smoke-filled room
experiment suggested that people are less likely to help in groups than alone, because people ...
a. in groups talk to one another.
b. who are alone are more attentive.
c. in groups do not display pluralistic ignorance.
d. in groups allow others to define the situation as a non-emergency
EXAMPLES
Better: (All alternatives should be approximately equal in length.)
6. Latane and Darley's smoke-filled room
experiment suggested that people are less likely to help in groups than alone, because people in groups ...
a. talk to one another
b. are less attentive than people who are alone
c. do not display pluralistic ignorance
d. allow other to define non-emergencies
Strengths of Multiple-choice Test
More representative in term of material/content.
Easy to administer and analyze
More objective in scoring
Weaknesses of Multiple-choice Test
The technique tests only recognition knowledge
Guessing may have a considerable effect on test scores
The technique severely restricts what can be tested
Washback may be harmful
Cheating may be facilitated
2. TRUE OR FALSE
Each statement is clearly true or clearly false.
Trivial details should not make a statement false.
Statements are written concisely without more elaboration than
necessary.
Statements are NOT quoted exactly from text.
What to Look for on
True/False Tests
Give emphasis on the use of
quantitative terms than qualitative terms.
Avoid using of specific determiners which usually gives a clue to the
answer.
False = all, always, never, every, none, only
True = generally, sometimes, usually, maybe, often
Discourage the use of negative statements.
Whenever a controversial statement is used, the authority should be
quoted.
Discourage the use of pattern for answers.
Tips in Making True/False Tests
EXAMPLES:
____ 1. Repetition always strengthens the tendency for a response to occur.
(Using "always" usually means the answer is false.)
Find the errors, and/or problems with the following true-false tests.
EXAMPLES:
_____ 2. The process of extinction is seldom
immediate but extends over a number of trials.
(Words like "seldom" usually indicate a true statement.)
EXAMPLES:
_____ 3. The mean, median, and mode are measures of central tendency,
whereas the standard deviation and range are measures of
variability.
(Express a single idea in each statement.) e.g.“The mean, median, and
mode are measures of central tendency.”
3. MATCHING TYPE TEST
PARTS OF THE MATCHING TYPE TEST (HORIZONTAL TYPE)
Column A (Premise)
Column B (Response)
PARTS OF THE MATCHING TYPE TEST (VERTICAL TYPE)
(Response)
(Premise)
The list of responses should be relatively short.
Response options should be arranged alphabetically or numerically.
Directions clearly indicate the basis for matching.
Can responses be used more than once?
Where will you place your answer?
Can students infer relationships or are they based on real world logic?
What to Look for on
Matching Type Tests
Position of matches should be varied. Avoid using patterns.
The choices of each matching set should be on one page
There are more responses than premises in a single set if responses cannot be used
more than once.
What to Look for on
Matching Type Tests
The premises are homogeneous as well as the responses and are grouped as one item.
If responses can be used more than once, it should be proportional to the number of
premises (3:5 or 4:10)
What to Look for on
Matching Type Tests
EXAMPLES OF POOR MATCHING:
Directions: Match the following.
1. Food A. Primary reinforcer
2. Psychoanalysis B. Sigmund Freud 3. B.F. Skinner C. Operant conditioning
4. Standard deviation D. Measure of variability 5. Schizophrenia E. Hallucinations
EXAMPLES:
Better: (Use homogenous material in matching
items, and if responses are not to be used more than once, include more responses than stimuli.)
Match the theories in Column A with their proponents in Column B. Write the letter of the correct answer.
Column A Column B
___ 1. Psychodynamic Theory A. Albert Bandura ___ 2. Trait Theory B. B.F. Skinner
___ 3. Behaviorism C. Carl Rogers ___ 4. Humanism D. Gordon Allport
___ 5. Social Learning Theory E. Karn Horney F. Raymond Cattell
G. Sigmund Freud
THE OTHER EXAMPLES:
In the blank space in column A, put the letter of that in column B that has the same or similar meaning.
Column A Column B
___ 1. Board A. Catch up with ___ 2. Call B. Get in
___ 3. Postpone C. Get on ___ 4. Remove D. Keep off ___ 5. Continue E. Keep on
___ 6. Overtake F. Look out for ___ 7. Arrive G. Look up to
___ 8. Be careful of H. Point out I. Put across
J. Put off
K. Run across L. Take off
4. SENTENCE COMPLETION / FILL-
IN THE BLANKS
Only significant words are omitted.
When omitting words, enough clues are left so that the student who knows the correct answer can
supply the correct response.
Ensure that grammatical clues are avoided.
Blanks are at the end of the statement.
The length of the responses are limited to single words or short phrases.
Questions are not lifted as verbatim quotes from text.
What to Look for on
Sentence Completion Tests
EXAMPLES:
1. An animal with six legs is called _________.
The item is so indefinite. It can be
completed with answers such as bee, mosquito or any other insect
Better:
1. Animals with six legs are called ___________.
EXAMPLES:
1.The __________ is the answer in _____.
Too many key words are omitted. Lines are not in equal length.
Better:
1. The product is the answers in _________.
EXAMPLES:
1. If a mango weighs 250 grams, 10 mangoes would weigh ______.
There are two possible answers – 250 grams and 0.25 kilos.
Better:
1. If a mango weighs 250 grams, 10 mangoes would weigh ____ grams.
5. ESSAY / SHORT ANSWER TEST
TYPES OF ESSAY ITEMS:
Extended response type
The test may be answered by the examinee in whatever manner he wants
Example: Do you think teachers should be allowed to work abroad as domestic helpers? Explain your answer!.
Restricted response type
The test limits the examinees response may be answered by the examinee’s responses in
terms of length, content, style or organization.
Example: Give and explain three reasons why the government should or should not allow
teachers to work abroad as domestic helpers.
The task is clearly defined. The students are given an idea on the scope and direction you intended for the answer to take. The question starts with a
description of the required behavior to put them in the correct mind frame.
E.g. “Compare” or “Analyze”
The questions are written in the linguistic level appropriate to the students.
Questions require a student to demonstrate
command of background information, not simply repeating information.
What to Look for on
Essay Tests
Questions regarding a student’s opinion on a certain issue should focus not on the opinion but on the way it is presented and argued.
A larger number of shorter, more specific questions are better, than, one or two longer questions.
What to Look for on
Essay Tests
PROPOSED CRITERIA IN GRADING ESSAY TEST
Ideas (20%)
Weight of Evidence Presented (40%)
Correct Usage (20%)
Logical Conclusions drawn from the
evidence (20%)
EXAMPLE:
What is wrong with this question?
Describe asthma?
Better: (Clearly explain what is expected of the student.)
Describe asthma. Include in your answer : a. the pathophysiologic features of asthma b. the clinical manifestations associated
with an asthma episode
c. the management of an asthma episode.
(10 points)
EXAMPLE:
What is wrong with this question?
Who is better, Rizal or Bonifacio?
Better: ( The students are given an idea on the scope and direction you intended for the answer to take.)
Compare and contrast the method used by Rizal and Bonifacio in promoting
nationalism. (5 points)
6. OTHER TYPES OF TEST QUESTIONS
RESTRICTED RESPONSE TEST (RRT)
Test takers are not given choices as possible answers. Items ask for a specific answer to each questions.
Example:
Who discovered the America continent?
Enumerate the four elements of the state?
PRINCIPLES IN CONSTRUCTING RRT
Do not ask for trivial facts or details. It is not only useless but also frustrates the students.
How many balls are used in a 9-ball match?
Questions should elicit facts not opinions?
What do you think President SBY should do for the country to recover from its’ economic deficit?
Minimize questions that call for sheer memory work unless if the answer has important analytical significance.
When will the next president be sworn to office?
Test takers are asked to arrange items in a systematic or logical order.
Arrange the presidents according to their term of office.
_____ Soeharto
_____ Megawati Soekarno Putri _____ Ir. Soekarno
_____ B.J. Habibie
_____ Abdurrahman Wahid
CHRONOLOGICAL SEQUENCING TEST
(CST)
PRINCIPLES IN CONSTRUCTING CST
Items should be homogenous and are related to each other.
There should not be more than 5 items in each set.
Do not number the items. This confuses the students.
All items to be arranged should be in the same page.
Directions should be clearly stated and that each set should be labeled about their
relevance.
WHAT IS WRONG IN THIS TEST QUESTION?
Arrange the following events in their chronological order.
1. The first thing I saw was the Oceanorium.
2. It’s the Australia’s largest marine park.
3. Where you can watch all sorts of sea fish and animals underwater.
4. Last Sunday, I visited marine park called Sea World at Surfer’s Paradise near Brisbane.
5. After the show, I had my lunch at a shape-like-ship restaurant.
6. The show was in a big outdoor of swimming pool 7. Then I watched the performance of sea animals.
BETTER:
War in the Pacific
Arrange the following events in chronological order.
Write the numbers 1-5 on the blanks provided.
___ USAFEE forces in Bataan surrender to the Japanese.
___ Japanese forces attacks the US fleet in Pearl Harbor, Hawaii.
___ Japan breaks diplomatic ties with the US.
___ The US declares war with Japan.
___ Gen. MacArthur escapes to Australia from Corregidor.
PROPOSED ARRANGEMENT OF TEST ITEMS
True or False
Multiple Choice
Matching Type
Sentence Completion
Others (RRT/Analogy/CST)
Essay
THINGS TO REMEMBER:
Making a good test takes time
Teachers have the obligation to
provide their students with the best evaluation
Tests play an essential role in the life of the students, parents,
teachers and other educators
A good lesson makes a good question A good question makes a good content A good content makes a good test
A good test makes a good grade
A good grade makes a good student
A good student makes a good COMMUNITY
POINTS TO PONDER…
Thank you …
