Monitoring & Evalua/on
Learning Objec/ves
•
Students are able to explain the importance of
monitoring and evalua/on a program
management
•
Students are able to apply concepts of
What is Monitoring and Evalua/on
(M&E)?
Dimension Monitoring Evalua2on
Frequency Periodic, occurs regularly Episodic
Func2on Tracking/oversight Assessment
Purpose Improve efficiency, provide
informa/on for reprogramming to improve outcome
Improve effec/veness, impact, value for money, future programming, strategy and policy making
Focus Inputs, outputs, processes,
work-plans (opera/onal implementa/on)
Effec/veness, relevance, impact, cost-effec/veness (popula/on effect)
Methods Rou/ne review of reports,
registers, administra/ve
databases, field observa/ons
Scien/fic, rigorous research design, complex and intensive
Informa2on source
Rou/ne surveillance system, field observa/on reports, progress reports, rapid assessment, program review mee/ng
Same
Popula/on based surveys, vital registra/on, special studies
Cost Consistent, recurrent costs spread
across implementa/on period
Episodic, oSen at the midpoint and end of implementa/on period
Monitoring gives informa/on on
where
a policy,
program, or project is at any given /me (and
over /me) rela/ve to respec/ve targets and
outcomes. It is descrip/ve in intent. Evalua/on
gives evidence of
why
targets and outcomes are
or are not being achieved. It seeks to address
issues of causality.
M&E
Monitoring
•
Clarifies program objec/ves
•
Links ac/vi/es and their
resources to objec/ves
•
Translates objec/ves into
performance indicators and
sets targets
•
Rou/nely collects data on
these indicators, compares
actual results with targets
•
Reports progress to managers
and alerts them to problems
Evalua2on
•
Analyzes why intended results
were or were not achieved
•
Assesses specific causal
contribu/ons of ac/vi/es to
results
•
Examines implementa/on
process
•
Explores unintended results
•
Provides lessons, high- lights
significant accomplishment or
program poten/al, and offers
recommenda/ons for
improvement
Why M&E are needed?
•
To ensure that resources are used effec/ve
and efficient (input)
•
To ensure that the program is implemented as
planned (process)
•
To inform about the level of achieved output,
therefore correc/ons may be conducted if
needed (output/outcome)
How do we design M&E systems in a
program management?
Planning of M&E Systems
Indicators, baseline, target
Data source + Methods of data
Results based M&E
Long term, widespread improvement in
society
Intermediate effect of outputs on clients
Products and services produced
Task perssonel undertake to transform
inputs into outputs
Financial, human, and material resources
Example: Results based M&E
Increase the case no/fica/on rate annualy to achieve a minimun case detec/on rate of 70%, by 2019, as
compared to 33% in 2014
34 provincial level training
1 na/onal workshop on the guideline
Intensified Case Finding (ICF) is
implemented in all 34 provinces
Guidelines, training, implementa/on of ICF
Goal of
Program
Global
Context
Na/onal
Context
Other
Relevant
Indicators, Baseline, Target
Indikator yang Baik
C
lear
R
elevant
E
conomic
A
dequate
M
onitorable
M&E Data Source
Design an evalua/on
Ovretveit J. 2003. Evalua/ng Health Interven/on. Open University Press
Purpose
When the evalua/on is undertaken: before/
during/aSer?
Focus of evalua/on
Scope:
limited or comprehensive?
Methods Evaluator role: internal/external
evaluator?
Focus of evalua/on
Needs Demands Inputs Processes Outputs Outcome
Methods
Assessment of DOTS strategy implementa/on in
hospitals in Indonesia: 2005-2007
32
The Study
Burden of TB TB services in
hospital
DOTS strategy implementa/on
DOTS Centre Lab District wasor
Pa/ent Provider
ISTC Prescrip/on
The Perspec/ves
Focus of evalua/on
Needs Demands Inputs Processes Outputs Outcome
Methods
Process in external linkage
Free TB drugs and choices of
facili/es for treatment
Sputum collec/on and prac/ces in the
ward
37
“HIV pa/ents were put in the same room with TB,
38
Use of secondline drugs in TB treatment
•
Category I, II and Extra-pulmonary TB Cases
:
–
first line generic drugs (41-64%)
–
FDC provided by NTP (21-40%)
•
Children
:
–
first line generic (82%)
–
first-line branded (18%)
39
0%
Sec ondline drug
ISTC Standard Criteria
Hosp
Standard 2
Sputum microscopy for TB diagnosis:
Sputum test carried out
94.4
100.0
At least 2 specimens for diagnosis
94.4
100.0
Standard 3
Microscopy examina/on for EP-TB
66.7
50.0
Standard 4
Sputum examina/on for sugges/ve of TB
94.4
100.0
Standard 5
Diagnosis of SS (-) TB:
Repeat sputum examina/on
22.2
50.0
Repeat sputum examina/on&X Ray
22.2
0.0
Clinical judgement&X Ray
33.3
0.0
Standard 6
Scoring system for pediatrict TB
11.1
100.0
ISTC Standard
Criteria
Hosp
(%)
C.Clinic
(%)
Standard 7
Treatment monitoring
77.8
100.0
Standard 8
TB treatment:
First line drugs
94.4
100.0
Dossage conform to interna/onal
recommenda/on
94.4
100.0
Standard 9
Treatment adherence:
Treatment supporter
72.2
50.0
Referral for poor adherence
83.3
100.0
Standard 10
SS test for monitoring treatment
88.9
100.0
Standard 11
Recording all medica/ons
88.9
100.0
Standard 12
TB-HIV:
HIV risk assessed
50.0
50.0
ISTC Standard
Criteria
Hosp
(%)
C.Clinic
(%)
Standard 13
ARV treatment ini/a/on:
ARV and TB drugs given concurrently
55.6
50.0
Cotrimoxazole profilaxis
50.0
50.0
Standard 14
Monitoring of drug resistance
44.4
50.0
Standard 15
Drug resistance TB:
Consult to MDR expert
72.2
100.0
Effec/ve four drug regimen
44.4
50.0
Standard 16
Contact tracing
50.0
0.0
Standard 17
Repor/ng all TB cases to health
authori/es
94.4
100.0
44
ISTC Public Health Responsibili/es
ISTC Standard
Criteria
Hosp
(%)
Chest
Clinic
(%)
Standard 16
Contact tracing
50.0
0.0
Standard 17
Repor/ng all TB cases to health
authori/es
45
Findings
STRUCTURE
1. Ownership of hospitals
2. Quality assurance system in hospital 3. TB case load
4. Trained staffs 5. DOTS team 6. DOTS unit
7. NTP Guidelines 8. Financial incen/ves
from DHO to hospitals
PROCESS
1. Commitment from hospital
2. Adherence to standard of diagnosis and treatment
3. Conversion rate 4. Case holding
process
OUTCOME
1. Treatment success rate
2. Treatment
comple/on rate 3. Default rate
4. Coverage of DOTS within a hospital
Outpa/ent Unit
TB suspects
Outpa/ent Unit
Hospital Laboratory
DOTS unit
TB suspects do not perform sputum test
TB suspects perform sputum test
TB cases Not TB cases
Treated within the hospital not under DOTS
unit
Refer to other health service facili/es
Medical Record Report ICD X A. 15-A.19
DOTS paBent register Lab register
Referral rate from Irawa/ et. al. (2007)
20-53%
Loss of follow up among SS (+) TB cases: 8-18%
46
