Evaluasi Capaian
Pembelajaran dan
Perbaikan Berkelanjutan
Dr. Eng. Paramashanti
Tujuan Sesi
•
Memahami proses evaluasi capaian pembelajaran
(student outcomes/SO)
Outline
•
Umum
•
Evaluasi Capaian Pembelajaran/Student Outcomes
(SO)
•
Perbaikan Berkelanjutan/Continuous
Improvement
•
Masukan untuk Proses Persiapan Evaluasi SO
•
Contoh Simulasi
1. Umum
Institution
Mission
Program
Mission
PEO
SO
Constituent’s Input
Advisory Board
Alumni
Faculty
User
Standar Nasional & International
Accreditation Criteria
Curriculum
Courses
PEO: Program Educational Objectives
(Tujuan Program Studi)
SO : Student Outcomes
(Capaian Pembelajaran)
Outcomes Based Education
Referensi ABET
•
“Criteria for Accrediting Engineering Program”,
EAC ABET, 2015.
•
“ABET Self Study Questionnaire: Template for
a Self Study Report”, EAC ABET, 2015.
Rumusan PEO dan SO Prodi
Apakah PEO dan SO dari Prodi
sudah sesuai dengan definisi yang
tertera dalam ABET?
PEO menurut ABET
•
Program Educational Objectives (PEO)
•
are broad statements that describe what graduates are
expected to attain within a few years of graduation.
•
Catatan penting tentang PEO:
•
Perumusan harus melibatkan konstituen Prodi
(dosen, advisory board, user, alumni).
•
Harus disampaikan tentang masukan dari masing-masing
konstituen tentang PEO Prodi
Program
Behaviors
Skills
Knowledge
Student Outcomes
SO menurut ABET
•
Student Outcomes (SO)
•
describe what students are expected to know and be able to do
by the time of graduation.
•
Catatan penting tentang SO:
•
ABET General Criteria memiliki daftar SO.
•
SO Prodi minimal mencakup SO tersebut.
General Criteria
Criterion 1. Students
Criterion 2. Program Educational Objectives
Criterion 3. Student Outcomes
Criterion 4. Continuous Improvement
Criterion 5. Curriculum
Criterion 6. Faculty
Criterion 7. Facilities
Criterion 8. Institutional Support
Program Criteria
Self Study
Report
(SSR)
2. Evaluasi Capaian
Pembelajaran/Student
Outcomes (SO)
Tujuan Evaluasi SO
•
Menunjukkan pencapaian Student Outcomes
Assessment
Evaluation
Improvement
Terhadap
Student
Outcomes
Berkelanjutan:
Cycle
Prodi perlu merancang bentuk siklus
Continuous
Improvement
yang akan dijalankan.
Evaluasi SO sebagai dasar
Continuous Improvement
Identify
Collect
Prepare
Which Courses? Assessment tools (assignment? Quiz?
Midterm/Final Test? Presentation?)? Expectation Level? etc.
Data collection
Data processing, calculate the attainment
Definisi Assessment
menurut ABET
SO
Assessment
PEO
Assessment
ABET New
Provision:
Tidak perlu di-assess,
namun perlu direview
Direct Measurement
Course Assessments
(quantitative)
Indirect Measurement
Exit Survey
(graduation,
qualitative)
Student
questionnaire
(qualitative)
Jenis Assessment
Course-1
Assessment
Assessment
Course-2
Assessment
Course-3
Assessment
Course-n
SO-a
Assessment
Course-4
Assessment
SO-b
Assessment
Assessment
SO-c
Assessment
SO-k
Perlu
Indikator
Student
Outcomes
Perlu
Rubrik
Penilaian Capaian
Indikator
Student
Outcomes
Direct Measurement: From Course Assessment
Perlu
matriks
kuliah vs SO
Indirect Measurement: From Survey
How to Assess SO?
•
Exit Survey: survey kepada mahasiswa yang baru
saja lulus (sebelum wisuda) untuk menilai
pencapaian SO menurut dirinya sendiri.
•
Kuisioner mahasiswa: survey kepada mahasiswa
setelah penyelenggaraan kuliah dalam satu
semester, mencakup aspek penilaian terhadap
pemahaman dan penguasaan materi pembelajaran,
penyelenggaraan perkuliahan oleh dosen, dll.
Catatan penting:
Evaluasi SO tidak boleh hanya berdasarkan pada indirect
measurement.
Catatan Penting dalam
SO Assessment (1)
Harus didasarkan pada proses
yang dilakukan oleh seluruh
Catatan Penting dalam
SO Assessment (2)
SO Assessment
Capaian nilai A, B, C, D, E
dari perkuliahan
•
Karena setiap alat assessment
(tugas, quiz, UTS, UAS, dll)
mungkin memiliki SO yang berbeda.
Catatan penting: Perlu menyiapkan data nilai yang lebih detail
(sebaiknya untuk setiap nomor soal pada setiap jenis alat
assessment).
Catatan Penting dalam
SO Assessment (3)
•
Perlu merancang prosedur evaluasi dan
assessment:
•
Kapan assessment dilakukan?
•
Kapan evaluasi dilakukan?
•
Kapan action perbaikan diterapkan?
•
Bagaimana bentuk tampilan hasil assessment?
Institution Mission Program
Mission PEO SO Stakeholders Input IAB Alumni Faculty User ABET Criteria (General & Program)
Curriculum DeliveryCourse
Faculty Facility ResourcesSupport &
Students Graduation SO Indicator Attainment Course Evaluations Yearly Improvements Exit Survey Course Assessment Course Assessment Course Assessment Student Semester questionnaire
5 year Loop: Program Assessment 1 year Loop: Course Assessment
A
A A A
2-3 year Loop: SO Assessment
Evaluation of SO Attainment
SO Program Improvement
A
Rancangan Evaluasi dalam Siklus
Continuous Improvement Prodi KL-ITB
Siklus Continuous Improvement
Prodi KL-ITB
•
Siklus 1 tahun-an:
•
Evaluasi dilakukan pada level matakuliah
•
Upaya perbaikan difokuskan pada course delivery oleh dosen
pengampu matakuliah dan melalui masukan prodi jika
dibutuhkan.
•
Siklus 2-3 tahun-an:
•
Evaluasi dilakukan pada level capaian Student Outcomes
•
Upaya perbaikan membutuhkan kebijakan program studi yang
lebih besar, dukungan institusi, ataupun konstituen Prodi.
•
Siklus 5 tahun-an:
•
Evaluasi untuk mereview PEO, SO, Kurikulum dengan
Contoh 1:
SO
Tahun Ajaran
2016/2017
2017/2018
2018/2019
a
√
√
√
b
√
√
√
c
√
√
√
d
√
√
√
e
√
√
√
f
√
√
√
g
√
√
√
h
√
√
√
i
√
√
√
j
√
√
√
k
√
√
√
Rancangan Waktu
Pelaksanaan Assessment
Contoh 2:
SO
Tahun Ajaran
2016/2017
2017/2018
2018/2019
a
√
√
b
√
√
c
√
√
d
√
e
√
f
√
g
√
h
√
i
√
√
j
√
√
Rancangan Waktu
Rancangan Pelaksanaan Evaluasi
SO Prodi KL-ITB
Contoh 2:
Cumulative Effect of Student Learning Over Time
SO-C
RC-1 RC-2 RC-3 RC-4 RC-5 RC-6 RC-7 RC-8
Semester
Rata-rata Capaian (RC)
LKK-1 LKK-2 LKK-3 LKK-4 LKK-5 LKK-6 LKK-7 LKK-8
Level Kognitif Kumulatif (LKK) C ap ai an x LKK
70% 70% 70% 70% 70% 70% 70% 70%
Target Capaian (TC)
×
×
×
×
×
×
×
×
O
O
O
O
O
O
O
O
O: Capaian
×: Target
Rancangan Tampilan
Hasil Assessment
Tahapan Evaluasi SO
Prodi KL-ITB
•
Tentukan SO dari Prodi
•
Tentukan indikator setiap SO
•
Tentukan matriks keterkaitan antara SO (indikator SO) dan
matakuliah
•
Susun rubrik penilaian pencapaian indikator SO
•
Kumpulkan data (hasil penilaian)
•
Lakukan course assessment untuk menentukan capaian SO dari
setiap matakuliah
•
Pilih matakuliah-matakuliah untuk proses evaluasi setiap SO
•
Tentukan target capaian setiap SO
•
Lakukan SO Assessment: rekap hasil untuk setiap capaian SO
Student Outcomes
Indicators
a
an ability to apply
knowledge of
mathematics, science,
and engineering
a1 an ability to apply knowledge of mathematics
a2 an ability to apply knowledge of science
a3 an ability to apply knowledge of basic
engineering
b
an ability to design and
conduct experiments, as
well as to analyze and
interpret data
b1 an ability to identify the type and objective of
experiment/field survey
b2 an ability to prepare and carry out an
experiment/field survey
b3 an ability to interpret result data from
experiment/field survey
c
an ability to design a
system, component, or
process to meet desired
needs within realistic
constraints such as
economic, environmental,
social, political, ethical,
health and safety,
manufacturability, and
sustainability
c1 an ability to design a component to meet
desired needs within realistic constraints
c2 an ability to design a system or process to meet
desired needs within realistic constraints
SO dan Indikator SO
dari Prodi KL-ITB
Student Outcomes
Indicators
d
an ability to function
and corporation on
multi-disciplinary teams
d1 an ability to explain the role of each member of
a multi-disciplinary team
d2 an ability toparticipate in active discussion within
a multi-disciplinary team
d3 an ability to summarize discussion results within
multi-disciplinary team
e
an ability to identify,
formulate, and solve
engineering problems
e1 an ability to identify engineering problems
e2 an ability to formulate engineering problems
e3 an ability to solve engineering problems
f
an understanding of
professional and ethical
responsibility
f1
an ability to explain roles, responsibilities and
professional ethics of an engineer.
f2
an ability to use codes or standards in Ocean
Engineering area.
f3
an ability to make decision based on professional
and ethical responsibility
g an ability to
communicate effectively
g1 an ability to prepare and do a presentation
g2 an ability to make a written report
g3 an ability to effectively discuss
Student Outcomes
Indicators
h
the broad education
necessary to understand
the impact of engineering
solutions in a global,
economic, environmental,
and societal context
h1 an ability to understand the impact of
engineering solutions in a global, eonomic,
environmental, and social context
h2 an ability to explain the impact of engineering
solutions to global-environmental and
social-economic context
i
a recognition of the need
for, and an ability to
engage in life-long
learning;
i1
an ability to understand on the importance and
needs of life-long learning;
i2
an ability to explain various career choices for
the Ocean Engineering graduate and
professional associations related to Ocean
Engineering area.
i3
an ability to collect information from many
sources in an efficient and effective manner.
j a knowledge of
contemporary issues;
j1
j2
an ability to understand current global issues
an ability to discuss the contemporary issues
related to Ocean Engineering field
SO dan Indikator SO
dari Prodi KL-ITB
Student Outcomes
Indicators
k
an ability to use the
techniques, skills, and
modern engineering tools
necessary for engineering
practice;
k1 an ability to develop a computational program
to solve engineering problem.
k2 an ability to make algorithm based on certain
numerical method to solve engineering
problem
k3 an ability to use up-to-date software
application to solve engineering problem
l
knowledge and the skills
to apply the principles of
fluid and solid mechanics,
dynamics, hydrostatics,
probability and applied
statistics to engineering
problems;
l1
an ability to apply principle of mechanics (fluid,
solid and dynamics) to engineering problems
l2
an ability to apply probability and applied
statistics to engineering problems;
m
knowledge and the skills
to apply the principles of
oceanography, water
waves, and underwater
acoustics to engineering
m1 an ability to apply the principle of
oceanography to engineering problems
m2 an ability to apply the principle of water waves
to engineering problems
m3 an ability to apply the principle of underwater
Student Outcomes
Indicators
n
the ability to work in
groups to perform
engineering design at the
system level, integrating
multiple technical areas
and addressing design
optimization
n1 an ability to perform system level design
integrating area of structure, foundation and
fluid-hydrodynamic
n2 an ability to include constraints and perform
design optimization.
SO dan Indikator SO
dari Prodi KL-ITB
Contoh Indikator SO-a
lebih detail pada level perkuliahan.
Menjadi dasar bagi rubrik indikator yang
Rubrik Penilaian
Kuliah yang dipilih untuk evaluasi SO
dalam siklus 2-3 tahun-an
Contoh Cuplikan
Matriks SO-Matakuliah
Contoh Tahapan
A1 #2 Q2 #2 MTE #3 FE#3
Contoh Tahapan
Course Assessment
Contoh Tahapan
(c.2)
Contoh Tahapan SO Assessment
Contoh Tahapan SO Assessment
KL-2101, TA 2012/2013
Tugas
Quiz
Contoh Course Assessment
Prodi KL-ITB
UTS
UAS
Contoh Course Assessment
Prodi KL-ITB
SO-a Assessment
Daftar Kuliah dan
Assessment Tools
Contoh SO Assessment
Prodi KL-ITB
Workshop Nasional Strategi Meraih Akreditasi Internasional, 18 - 19 April 2017 47
KL2101: Mekanika Fluida
KL2201: Mekanika Gelombang
KL3100: Hidrodinamika
Capaian SO-a untuk Setiap Kuliah
Capaian Indikator SO-a
Contoh SO Assessment
Prodi KL-ITB
Evaluasi Capaian SO-a
Contoh Evaluasi
SO Prodi KL-ITB
•
Seluruh indikator SO-a telah melebihi target yang
telah ditentukan (70%). Untuk siklus evaluasi SO-a
yang akan datang, target capaian akan ditingkatkan
menjadi 80%.
No. Student
Outcome
Indicator
1
a
apply knowledge of engineering
2
f
Use codes and standards, make decision based on
professional responsibility
3
j
understand current global issues
4
k
develop computational program, make algorithm
5
l
apply principle of mechanics
6
m
apply principle of water wave
Fokus Capaian Indikator SO yang perlu ditingkatkan
Rekap Hasil Evaluasi
SO Prodi KL-ITB
3. Perbaikan
Berkelanjutan
Improvement Plan berdasarkan
Hasil Evaluasi SO
Indicator SO yang
perlu diperkuat
Action
Required Resources
Responsible
Party
1
Apply knowledge of
engineering including
mechanics and water
wave
Mandatory structured
recitation / help session
with teaching assistant
Recitation material
Instructor
Teaching assistant
OE Program
Schedule and classroom OE Program/FCEE
Rules and regulation
OE Program
2
Use codes and standards,
make decision based on
professional
responsibility
Structured design
assignment
Assignment material
Assessment tools
Instructor
OE Program (task
force)
Assistant
OE Program/FCEE
Rules and regulation
OE Program
3
Understand global issues More planned guest
lectures, field studies,
topics of students seminars
Planned topics
OE Program (task
force)/Constituents
Cost
OE Program/FCEE
Rules and regulation
OE Program
4
Algorithm and
computational
programming
Structured programming
assignment in KL2203,
scheduled tutorial with
teaching assistant
Assignment material
Instructor
Teaching assistant
OE Program
Classroom / computer
lab
OE Program/FCEE
Rules and regulation
OE Program
Contoh Perbaikan pada
Siklus 1-tahun
•
Peningkatan kegiatan responsi tugas oleh asisten
matakuliah untuk lebih memberikan mahasiswa latihan.
•
Pembuatan alat peraga perkuliahan yang mendukung
pemahaman mahasiswa akan perkuliahan.
•
Penambahan contoh-contoh kasus masalah ekonomi
rekayasa dalam bidang Teknik Kelautan.
•
Penambahan contoh aplikasi dari setiap materi perkuliahan
dalam bidang kerja Teknik Kelautan.
•
Penambahan tugas membaca dan mereview issue seputar
Teknik Kelautan dikaitkan dengan materi kuliah.
Contoh Perbaikan
pada Siklus 3-tahun
•
Perubahan silabus matakuliah
•
Perubahan dosen pengampu kuliah
•
Perubahan indikator SO
4. Masukan untuk Proses
Persiapan Evaluasi SO
Untuk Dosen Pengampu Kuliah
•
Setiap awal semester:
•
Dosen diingatkan tentang SO apa saja yang dititipkan
pada matakuliahnya.
•
Dosen diingatkan untuk merancang alat assessment dari
SO pada matakuliahnya (quiz, tugas, tugas besar
kelompok, presentasi perorangan, ujian, dll).
•
Prodi melengkapi berkas absensi dan berita acara
perkuliahan dengan lembar berisi informasi: tujuan
perkuliahan, SO terkait, dan silabus matakuliah.
•
Prodi menyampaikan file yang berisi format untuk
melakukan course assessment.
Untuk Dosen Pengampu Kuliah
•
Setiap masa sebelum UTS dan UAS:
•
Dosen diingatkan untuk membuat soal dilengkapi dengan
identifikasi SO terkait.
•
Setiap akhir semester:
•
Dosen diingatkan untuk menyusun course assessment,
beserta evaluasi dan rencana upaya perbaikan.
Untuk Asisten Matakuliah
•
Setiap awal semester:
•
Prodi melakukan sosialisasi bagi seluruh asisten matakuliah
mengenai tugasnya dalam mendokumentasikan Course
Assessment baik dalam bentuk softcopy maupun hardcopy.
•
Prodi menyiapkan binder bagi penyimpanan dokumen course
assessment.
•
Setiap pelaksanaan assessment dan pada masa
sebelum UTS dan UAS:
•
Asisten diminta untuk berkoordinasi dengan dosen pengampu
untuk identifikasi SO-terkait.
•
Asisten diminta untuk langsung menyimpan soal, copy
pekerjaan mahasiswa dengan nilai terbaik, terburuk, dan
rata-rata.
Untuk Asisten Matakuliah
•
Setiap akhir semester:
•
Asisten diminta untuk membantu dosen dalam
melengkapi dokumen dan file course assessment.
•
Asisten menyerahkan dokumen dan file course
Untuk Staf Administrasi dan
Admin Website
•
Membantu mengumpulkan berkas dan file course
assessment dalam folder yang terstruktur untuk
setiap semester.
•
Mendokumentasikan setiap kegiatan rapat dalam
folder komputer yang berisi:
•
File undangan rapat
(berisi tanggal, daftar undangan, dan agenda)
•
File scan daftar hadir
•
File presentasi
•
Foto dokumentasi
•
Minutes of meeting
•
Mendokumentasikan setiap kegiatan dan agenda
penting prodi dalam website Prodi.
(website Prodi KL-ITB: www.ocean.itb.ac.id)
5. Simulasi
Simulasi
•
Penyusunan course assessment untuk salah satu
matakuliah
•
Evaluasi capaian salah satu Student Outcome
Terima kasih
