PEMBUATAN
DAN
MENELAAH
SOAL MCQ
(
ITEM DEVELOPMENT AND REVIEW
)
Komite Bersama
Uji Kompetensi Dokter Indonesia
A simple model of competence
Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.
Knows
Shows how
Knows how
Does
P
ro
fe
ss
io
n
a
l
a
u
th
e
n
ticit
y
Cognition
Behaviour
Climbing the pyramid...
Knows
Shows how
Knows how
Does
Knows
Factual tests:
MCQ, essay type, oral…..
Knows how
(Clinical) Context based tests:
MCQ, essay type, oral…..
Shows how
Performance assessment in vitro:
OSCE, SP-based test…..
Sulitkah membuat soal MCQ?
Tijauan1: SKDI Tinjauan 2 Tinjauan 3 Tinjauan 4 Tinjauan 5 Tinjauan 6 Tinjauan 7 Ketrampilan dasar klinis Kognitif Recall Reproduksi Pertumbuhan,
perkembangan, dan degenerasi Promosi kesehatan dan pencegahan penyakit Individu Aplikasi biomedis,
behavior, clinical, & epidemiologi pada kedokteran
Procedural skill
Reasoning Kepala dan leher Kelainan genetik dan kongenital
Penapisan/Dia gnosis
Keluarga Komunikasi efektif Afektif Saraf & Perilaku Penyakit Infeksi
dan Imunologi Manajemen/T erapi Masyarkat Manajemen masalah kesehatan primer
Kulit, otot, tulang, jaringan lunak
Penyakit neoplasma
Rehabilitasi Penelusuran, kritisi, dan
manajemen informasi
Darah & Sistem Kekebalan Penyakit akibat trauma atau kecelakaan Aspek hukum dan etika Profesionalisme, moral,
dan etika praktik kedokteran
Jantung, Pembuluh darah & Sistem limfatik Kesadaran, pemeliharaan, dan pengembangan personal Saluran cerna, hepatobilier, dan pankreas Saluran Pernapasan Ginjal dan saluran
Blue Print
UKDI
K
ESALAHAN PADA SOAL PILIHAN GANDA
(MCQ)
Hanya menguji kemampuan mengingat – sering
hanya hal yang trivia
Tidak menguji aplikasi atau integrasi pengetahuan
Tidak menguji kemampuan daya nalar dan
pemecahan masalah
Tidak konsisten dengan prinsip-prinsip pendidikan
masa kini
Lebih meningkatkan kemampuan memorisasi dari
pada pemahaman
H
AL
-
HAL YANG HARUS DIPERHATIKAN
DALAM MEMBUAT SOAL
“MCQ”
Hindari pertanyaan “recall” simpel : jawaban
yang dapat diperoleh dari suatu halaman buku
ajar
Hindari pertanyaan yang mendorong
memorisasi sederhana
Selalu menguji kemampuan aplikasi, integrasi,
atau pemahaman
P
RINSIP
-
PRINSIP DASAR
Nilai aplikasi dan integrasi pengetahuan
Gunakan skenario klinik untuk menyajikan masalah
Uji kemampuan daya nalar dan pemecahan masalah
Fokuskan pada materi yang penting – hindari “trivia”
Sediakan satu jawaban “paling benar “
Buatlah distraktor homogen
Hindari memorisasi fakta sederhana
P
ANDUAN UMUM KONSTRUKSI
MCQ (1)
1. Yakinkan bahwa setiap soal dapat dijawab tanpa
harus melihat pilihan jawaban atau yakinkan bahwa
pilihan jawaban adalah 100 % benar atau salah.
2. Usahakan agar data atau informasi penting
seoptimal mungkin dicantumkan pada soal (stem),
sedangkan pilihan jawaban usahakan sesingkat
mungkin.
M
ULTIPLE
-C
HOICE
Q
UESTION
F
ORMATS
Multiple-choice Questions Standard True-false-type R-type Extended matching A-type True-false Questions One-best-answer Questions K-type Complex True FalseC
OMPONENTS OF AN
A-
TYPE
Q
UESTION
A 35-year old man has had a stomach
ache all afternoon. He ate the
following lunch: two big McDonalds
hamburgers, an ice cream shake,
large fries.
Which is the most likely diagnosis?
A. Appendicitis
B. Bowel obstruction
C. Colon cancer
D. Pancreatitis
E. Too much lunch
Lead-in
Vignette,
Scenario
,
Or Stem
Options A, B, C, and D
are Distractors
Option E is the Key
S
AMPLE
G
OOD
Q
UESTION AND THE
„C
OVER
T
HE
O
PTIONS
‟ R
ULE
A 32-yo man has a 4-day history of progressive weakness
in his extremities. He has been healthy except for an
upper respiratory tract infection 10 days ago. His
temperature is 100 F, BP 130/80, pulse 94, respirations
42 and shallow. He has symmetric weakness of both sides
of the face and the proximal and distal muscles of the
extremities. Sensation is intact. No deep tendon reflexes
can be elicited; the plantar responses are flexor. Which of
the following is the most likely diagnosis?
A. Acute disseminated encephalomyelitis
B. Guillain-Barré syndrome
C. Myasthenia gravis
D. Poliomyelitis
E. Polymyositis
Which of the following is true about
pseudogout?
A. It occurs frequently in women.
B. It is seldom associated with acute pain
in a joint.
C. It may be associated with a finding of
chondrocalcinosis.
D. It is clearly hereditary in most cases.
E. It responds well to treatment with
allopurinol.
S
AMPLE
F
LAWED
Q
UESTION AND THE
P
ANDUAN UMUM KONSTRUKSI
MCQ (2)
Hindari informasi berlebihan pada soal.
Hindari soal-soal yang bersifat menjebak dan
terlalu kompleks serta tidak relevan.
Gunakan pilihan jawaban yang secara
gramatikal konsisten dan secara logis
kompatibel dengan soal.
P
ANDUAN UMUM KONSTRUKSI
MCQ (3)
Hindari kata-kata absolut seperti, selalu, tidak
pernah, dan semua pada pilihan jawaban, dan
hindari pula istilah yang tidak jelas (meragukan)
seperti biasanya atau sering.
Hindari kalimat atau frase negatif seperti
KECUALI, TIDAK, atau BUKAN.
Fokuskan soal pada konsep-konsep penting, dan
hindari hanya menguji kemampuan mengingat
fakta.
P
ANDUAN UMUM KONSTRUKSI
MCQ (4)
Fokuskan soal-soal pada konsep penting dan
kasus-kasus yang sering serta memiliki potensi masalah
serius.
Setiap soal diusahakan agar menilai aplikasi
pengetahuan, tidak hanya menguji daya ingat
terhadap fakta terisolasi.
Seluruh distraktor (pengecoh= jawaban salah) dibuat
homogen (dari sisi gramatikal, taksonomi).
P
ANDUAN UMUM KONSTRUKSI
MCQ (5)
Hindari kesalahan teknis membuat soal, seperti :
Pilihan jawaban terlau panjang, kompleks, atau
ganda.
Istilah yang rancu pada pilihan jawaban.
Bahasa pada jawaban tidak parallel.
Pilihan jawaban tidak menurut urutan logik.
Bukan salah satu di atas atau semua benar
P
ANDUAN UMUM KONSTRUKSI
MCQ (6)
Pilihan pada jawaban memberikan petunjuk logik
pada jawaban yang benar.
Pilihan jawaban yang benar adalah frase atau
kalimat terpanjang.
Penggunaan kata-kata yang diulang atau memiliki
makna sama baik pada soal maupuin pilihan
jawaban.
Strategi konvergens pada pilihan jawaban.
Item Development
Item writer Local Item Bank Administrator Regional Item Bank Administrator National Item Review Item Bank 1 National Item Bank Administrator Panel expert Item Bank 2 Non scoreditems Item Bank 3
Item Selection
process
Question book
A 60-year-old man is brought to the emergency department by the police, who found him lying unconscious on the sidewalk. After ascertaining that the airway is open, the first step in management should be intravenous administration of
A. Examination of cerebrospinal fluid B. Glucose with vitamin B1(thiamine)
C. CT scan of the head D. Phenytoin
E. diazepam
Grammatical cues: one or more distracters don’t follow grammatically from the stem
A 60-year-old man is brought to the emergency department by the police, who found him lying unconscious on the sidewalk. Ascertaining that the airway is open, tfirst step in management should be intravenous administration of:
A. Examination of cerebrospinal fluid B. Glucose with vitamin B1(thiamine) C. CT scan of the head
D. Phenytoin E. diazepam
In patients with advanced dementia, Alzheimer’s type, the memory defect
A. can be treated adequately with phosphatidylcholine (lecithin) B. could be a sequela of early parkinsonism
C. is never seen in patients with neurofibrillary tangles at autopsy D. is never severe
E. possibly involves the cholinergic system
Absolute terms: terms such as “always” or “never” are used in options
In patients with advanced dementia, Alzheimer’s type, the memory defect
A. can be treated adequately with phosphatidylcholine (lecithin) B. could be a sequela of early parkinsonism
C. is never seen in patients with neurofibrillary tangles at autopsy D. is never severe
Secondary gain is
A. synonymous with malingering
B. a frequent problem in obsessive-compulsive disorder
C. a complication of a variety of illnesses and tends to prolong many of them
D. never seen in organic brain damage
Long correct answer: correct answer is longer, more specific, or more
complete than other options
Secondary gain is
A. synonymous with malingering
B. a frequent problem in obsessive-compulsive disorder
C. a complication of a variety of illnesses and tends to prolong many of them
A 58-year-old man with a history of heavy alcohol use and previous psychiatric hospitalization is confused and agitated. He speaks of experiencing the world as unreal. This symptom is called
A. depersonalization B. derailment C. derealization D. focal memory deficit E. signal anxiety
Word repeats : a word or phrase is included in the stem and in the correct
answer
A 58-year-old man with a history of heavy alcohol use and previous psychiatric hospitalization is confused and agitated. He speaks of experiencing the world as unreal. This symptom is called
A. depersonalization B. derailment C. derealization D. focal memory deficit E. signal anxiety
Local anesthetics are most effective in the
A. Anionic form, acting from inside the nerve membrane B. Cationic form, acting from inside the nerve membrane C. Cationic form, acting from outside the nerve membrane D. Uncharged form, acting from inside the nerve membrane E. Uncharged form, acting from outside the nerve membrane
Convergence strategy : the correct answer includes the most elements in
common with the other options
Local anesthetics are most effective in the
A. Anionic form, acting from inside the nerve membrane B. Cationic form, acting from inside the nerve membrane C. Cationic form, acting from outside the nerve membrane D. Uncharged form, acting from inside the nerve membrane E. Uncharged form, acting from outside the nerve membrane
Peer review committees in HMOs may move to take action against a physician‟s credentials to care for participants of the HMO. There is an associated requirement to assure that the physician receives due process in the course of these activities. Due process must include which of the following
A. Notice, an impartial forum, council, a chance to hear and confront evidence against him/her
B. Proper notice, a tribunal empowered to make decision, a chance to confront witnesses against him/her, and a chance to present evidence in defense
C. Reasonable and timely notice, impartial panel empowered to make a decision, a chance to hear evidence against himself/herself and to confront witnesses, and the ability to present evidence in defense
Options are long, complicated, or double
Peer review committees in HMOs may move to take action against a physician‟s credentials to care for participants of the HMO. There is an associated requirement to assure that the physician receives due process in the course of these activities. Due process must include which of the following
A. Notice, an impartial forum, council, a chance to hear and confront evidence against him/her
B. Proper notice, a tribunal empowered to make decision, a chance to confront witnesses against him/her, and a chance to present evidence in defense
C. Reasonable and timely notice, impartial panel empowered to make a decision, a chance to hear evidence against himself/herself and to confront witnesses, and the ability to present evidence in defense
Severe obesity in early adolescence
A. Usually responds dramatically to dietary regimens B. Often is related to endocrine disorders
C. Has a 75% chance of clearing spontaneously D. Shows a poor prognosis
E. Usually responds to pharmacotherapy and intensive psychotherapy
Frequency terms in the options are vague (e.g. rarely, usually)
Severe obesity in early adolescence
A. Usually responds dramatically to dietary regimens B. Often is related to endocrine disorders
C. Has a 75% chance of clearing spontaneously D. Shows a poor prognosis
E. Usually responds to pharmacotherapy and intensive psychotherapy
In a vaccine trial, 200 2-year-old boys were given a vaccine against a certain disease and the monitored for five years for occurrence of the disease. Of this group, 85% never contacted the disease. Which of the following statements concerning these results is correct?
A. No conclusion can be drawn, since no follow-up was made of nonvaccinated children
B. The number of cases (ie, 30 cases over five years) is too small for statistically meaningful conclusions
C. No conclusions can be drawn because the trial involved only boys D. Vaccine efficacy (%) is calculated as 85 – 15/100
Language in the options is not parallel; options are in non-logical order
In a vaccine trial, 200 2-year-old boys were given a vaccine against a certain disease and the monitored for five years for occurrence of the disease. Of this group, 85% never contacted the disease. Which of the following statements concerning these results is correct?
A. No conclusion can be drawn, since no follow-up was made of nonvaccinated children
B. The number of cases (ie, 30 cases over five years) is too small for statistically meaningful conclusions
C. No conclusions can be drawn because the trial involved only boys D. Vaccine efficacy (%) is calculated as 85 – 15/100
The diagnosis of a large ovarian cyst is most strongly suggested by
A. an anterior dullness, lateral tympany B. a decreased peristalsis
C. a fluid wave D. a shifting dullness E. none of the above
None of the above is used as an option
The diagnosis of a large ovarian cyst is most strongly suggested by
A. an anterior dullness, lateral tympany B. a decreased peristalsis
C. a fluid wave D. a shifting dullness E. none of the above
Arrange the parents of the following children with Down’s syndrome in order of highest to lowest risk of occurrence. Assume that the maternal age in all cases is 22 years and that a subsequent pregnancy occurs within 5 years. The karyotypes of the daughters are :
I. 46, XX, -14, +T (14q21q) pat II. 46, XX, -14, +T (14q21q) de novo III. 46, XX, -14, +T (14q21q) mat IV. 46, XX, -21, +T (14q21q) pat
V. 47, XX, -21, +T (21q21q) (parents not karyotyped)
A. III, IV, I, V, II B. IV, III, V, I, II C. III, I, IV, V, II D. IV, III, I, V, II E. III, IV, I, II, V
Stems are tricky or unnecessarily complicated
Arrange the parents of the following children with Down’s syndrome in order of highest to lowest risk of occurrence. Assume that the maternal age in all cases is 22 years and that a subsequent pregnancy occurs within 5 years. The karyotypes of the daughters are :
I. 46, XX, -14, +T (14q21q) pat II. 46, XX, -14, +T (14q21q) de novo III. 46, XX, -14, +T (14q21q) mat IV. 46, XX, -21, +T (14q21q) pat
V. 47, XX, -21, +T (21q21q) (parents not karyotyped)
A. III, IV, I, V, II B. IV, III, V, I, II C. III, I, IV, V, II D. IV, III, I, V, II
V
IGNETTE
: K
LINIK
, M
ASALAH
KEDOKTERAN KELUARGA
(1)
Sebuah vignette antara lain berisi deskripsi
kondisi pasien, dan skenario klinik dengan
beberapa informasi, yaitu:
Usia, Jenis kelamin (e.g. laki-laki 45 tahun)
Tempat pelayanan (IGD)
Keluhan utama
Durasi
Riwayat Pasien
Diagnostik fisik
• +/- hasil diagnostik
• +/- terapi awal, temuan selanjutnya
NBME, p.38 or p.56
V
IGNETTE
: K
LINIK
, M
ASALAH
KEDOKTERAN KELUARGA
(2)
R
EFERENCE
:
