Assessment of Teaching and Learning

Assessment of Teaching and Learning

3r

^d

Asia Pacific Pharmacy Education Workshop

Nabishah Mohamad 20^th November 2012

The Task

“...for this 3rd PharmED Workshop we focus on students assessment : course learning

outcome in the area of Domains-Knowledge,

Skills and Attitude/Behaviour”

Outline

1. Assessment in Outcome-based education Learning outcome and competencies

2. Assessment of Learning

Basic Principles in assessment Who, where, how…….

3. Validity and Reliability

INTRODUCTION

Outcome-Based Education

An education approach that focuses on the graduate attributes or outcomes after

completing an academic programme (Barr et

al., 2006)

Programme outcomes

Course learning outcomes Teaching-learning activities

Assessment

The seven broad areas of competency - New Zealand

3. Provide Primary Health Care 4. Apply Management and

Organisation Skills

5. Research and Provide Information 6. Dispense Medicines

7. Prepare Pharmaceutical Products

1. Practise Pharmacy in a Professional and Culturally Competent Manner

2. Contribute to the Quality Use of Medicines

Systematic Approach in Student Assessment

1. Define the purpose

2. Determine what is to be assessed 3. Decide on appropriate methods

of assessment

4. Set standards of acceptable performance

5. Make a judgment with reference to the purpose

Purpose of Student Assessment

FORMATIVE: The scores are used to provide information on students learning.

SUMMATIVE: scores are used in making decisions on student standing (e.g.

pass or fail)

 Determines progression

 Certify competence

When?

FORMATIVE: The scores are used to provide information on students learning.

SUMMATIVE: scores are used in making

decisions on student standing (e.g. pass or fail)

 Determines progression

 Certify competence

In training

Mid sem,/ End sem, End of course

What is to be assessed?

Programme/ Module/ posting/

course Learning Outcomes

Domains- Knowledge, Skills and Attitude/behaviour

Bloom’s Taxonomy

Original

Knowledge Comprehension Application Analysis Synthesis Evaluation Perception Set Guided responses Mechanism Complex overt responses Adaptation Origination Receive Respond evaluate Organisation Characterization

C1 C2 C3 C4 C5 C6 P1 P2 P3 P4 P5 P6 P7 A1 A2 A3 A4 A5

The seven broad areas of competency - New Zealand

1. Practise Pharmacy in a Professional and Culturally Competent Manner

2. Contribute to the Quality Use of Medicines 3. Provide Primary Health Care

4. Apply Management and Organisation Skills 5. Research and Provide Information

6. Dispense Medicines

7. Prepare Pharmaceutical Products

The seven broad areas of competency

1. Practise Pharmacy in a Professional and Culturally Competent Manner (K

³

, S

^5,

A

⁴

)

2. Contribute to the Quality Use of Medicines (K, S) 3. Provide Primary Health Care (K, S, A)

4. Apply Management and Organisation Skills (K, S) 5. Research and Provide Information (S, K)

6. Dispense Medicines (K, S)

7. Prepare Pharmaceutical Products (K, S)

Does

Shows how Knows how

Knows

Levels of clinical competence

Miller’s Pyramid

HOW?

• Must match outcomes

• Valid

• Reliable

Levels of Clinical Competence

Knows 16

Know how Show how

Does

Real environment Portfolio, supervisor’s

report, logbook, 360^o

Context-based tests

MCQ, EMQ, KFQ, Oral.

Simulated environment

OSCE, OSPE Practical Exam

Factual tests

MCQ T/F, Essay

Outcomes _Assessment^{Method of}

Simulation SP

P-MEX, SBL

Written/oral tests Moral reasoning

Attitudes

Workplace assessments Critical incidents Routine performance

Action

Decision Making

Reasoning/

Attitude

Knowledge Knows How

Knows Shows How

Does

Written tests Traditional measures

Attitude/ Behaviour

Shows How-Decision Making

Real patients

Professionalism mini-evaluation

exercise or P-MEX (Cruess et al., 2005)

• Trainee is observed in a patient encounter

• Performance judged along the

dimensions of professionalism (eg- listen carefully, knows own limitation)

• Opportunity for feedback

Portfolio

Collection of a student’s works, that exhibits the student’s efforts progress and achievements in one or more areas.

Link to Learning Outcomes of the course or programme

WHO?

• Self

• Peer

• Teacher

• Supervisor

• Other health professionals

• Patients

Peer and self-assessment

The learner’s peers and the learners themselves

assess what they are able to do and how comfortably they are performing a task.

Nabishah/UKM 21

Self-assessment

1. Important element in adult learning:

Learner should be able to self-evaluate and take the appropriate action from that assessment.

2. An important attribute of lifelong learning

Nabishah/UKM 22

Self Assessment

Nabishah/UKM 23

Strengths Weaknesses

Improvement Personal

Growth

Helper

Self Assessment

Nabishah/UKM 24

Strengths Weaknesses

Improvement Evidence of

Personal Growth

Self Reflection

Helper

Self evaluation skill improve with time

Nabishah et al 2010

Key considerations for selecting assessment methods

How does the assessment method impact learning?

Is it relevant and authentic to clinical practice?

Is it valid and reliable?

Is it fair and transparent to students?

Does it provide valuable information that can facilitate teaching and learning?

Is it feasible in terms of implementation?

Why assessment is as the theme of this workshop?

• To signal its importance

• To identify/remediate problem pharmacists (students)

- Improves patient safety - Establishes accountability

“Students respect what you inspect.”

• To improve quality

- Assessment is the first step in QI cycle

Summary: Assessment of Students Learning

1. Reflect on the course learning outcomes 2. Design teaching-learning activities

3. Clear on the purpose of assessment

4. Use formative and summative assessments 5. Use a variety of assessment methods

6. Assessment tools- Valid and reliable

Source: R. Harden

References

• Accreditation Standards And Guidelines For The Professional Program In Pharmacy Leading To The Doctor Of Pharmacy

Degree. Accreditation Council For Pharmacy Education. 2011.

https://www.acpe-accredit.org/standards/

• Lambert W T Schuwirth, Cees P M van der Vleuten. 2003. ABC of learning and teaching in medicine- Written assessment.

BMJ , 326: 643-5.

• Nabishah, Kwang KC, Jiuan CS, Sue Ann L, Syazwina M, Phei KZ, Norhayati R and Mohd Nasri AB. 2010. Self-evaluation in Problem-based Learning. Asian Journal of Teaching and

Learning in Higher Education -AJTLHE Vol. 3, No.1, 50-57