Building Governance of Health
service delivery in Easternmost
Indonesia province
USAID Indonesia: Edhie S Rahmat, Ketty Kadarwaty, Sigid Sulistyo Kinerja/RTI: Marcia Soumokil, Elke Rapp
Presented by Edhie S Rahmat in 3rd INAHEA Congress
Kinerja Papua
•
Focus specifically on improving health
governance in Jayapura city & district,
Wamena and Mimika districts
•
Three main activities
–
Innovation: improving the
governance and management
structures within District Health
Offices (DHO) and
puskesmas
–
Incentives
:
engaging citizens to
demand better health services from
district governments and service
providers
–
Replication:
disseminating improved
practices
Program Assessment:
•
Three primary data collection: Organizational Capacity
Assessment (OCA), Customer Satisfaction Survey
(CSS), and Qualitative study
•
Purpose:
–
Identify capacity change among Key partner units 2 year
intervention
–
Changes in perception of unit management (has institutional
growth occurred?)
–
Changes in perception of service provided by unit (have clients
received better services?)
–
Document program performance, Determine efficacy of KP
approach
Methodology: Organizational Capacity Assessment (OCA)
•
Hypothesis
: Supply side intervention improves partner
health units’ organizational capacity
• Key partners focuses on the following four areas of
governance (Accountability, Responsiveness,
Community Participation & Transparency)
•
Instrument
: Custom Likert Scale Survey
•
Sample
: Random; 160 Organization Members (10 per
DHO & Puskesmas)
•
Timing:
Baseline April 2013, Endline March 2015
•
Analysis:
Descriptive statistics, T tests, and multiple
regressions
Methodology: Customer Satisfaction Survey (CSS)
•
Hypothesis: T
wo-pronged approach (supply & demand)
improves customer satisfaction with
puskesmas
services
•
Instrument
: Customer satisfaction survey developed by
Ministry for State Apparatus and Bureaucratic Reform (KemenPAN) specified 14 indicators to measure customer satisfaction with service units
•
Sample
: Random; 1,800 Service users in partner
puskesmas
•
Timing:
Baseline February 2013, Endline February – March
2015
•
Analysis:
Descriptive statistics, T tests, and multiple
regressions
Methodology: Qualitative
• Triangulate and augment quantitative data
•
Instrument
: Custom key informant interview (KII) and focus
group discussion (FGD) guides
•
Sample
: DHO and puskesmas staff, service users, and MSF
members (24 FGDs and 10 KIIs = 134 respondents)
•
Timing:
March – April 2015
•
Analysis:
Triangulate quantitative findings; coding of themes
Limitations: Quantitative & Qualitative
• Cannot make statements about causation
• Perception rather than direct measurement
• Limited sample size for qualitative
• Identifying respondents
• Data collection method (paper vs. electronic)
• Timing
• No qualitative baseline
8
Findings: Province & District-Wide
⁺Note: statistical significance is noted as follows:
Findings: OCA Qualitative
Social Impact
“The MSF reaches out to the community members and gets input from them. We [puskesmas staff] then receive community feedback via the MSF on how we
deliver services. For example, we received input on why some of the infrastructure in this puskesmas was broken, how we should use our vehicle, and why some of our staff were not friendly, etc. This is how the MSF works with this puskesmas.” –
Head of PKM Mapurujaya
“Studying while working is difficult because our workload is unpredictable. For instance, I remember I could only attend three video conference sessions. It
shows the fact that studying while learning is difficult, isn’t it?” – Head of DHO Kab Jayapura
“We have participated in the training and we were very happy because we could finally learn new things. However, the problem is that our boss did not follow-up [after training]. We are just staff, so we are just waiting for our boss.” –
12
Overall Findings: OCA
• Overall OCA scores increased in all KP districts
• Accountability and Responsiveness scores remain highest
in all districts, and Community Participation and
Transparency remain areas of weakness in KP districts.
However…
• 50% of units had biggest improvement in dimension
‘Community Participation’
• Change in OCA scores is higher in those units that had low
baseline scores, for example the three puskesmas in
Jayawijaya and Puskesmas Limau Asri in Kabupaten
Mimika
CSS Dimension Score
Based on the Ministry for State Apparatus and Bureaucratic Reform, Decree No.25/2004, regarding the Guidance of Developing Government Service Unit Costumer’s Satisfaction Index.
SOCIAL IMPACT
Ratings CSS Score
Very Good 3.26 – 4.00 or 81.26 – 100
Good 2.51 – 3.25 or 62.51 – 81.25
Fair 1.76 – 2.50 or 43.76 – 62.50
Not Good 1.00 – 1.75 or 25 – 43.75
Dimensions
1. Procedural Easiness
2. Conformity of service requirements
3. Clarity & Certainty
4. Discipline of staffs
5. Responsibility of staffs
6. Capability of staffs
7. Quickness of Service
8. Fairness of getting service
9. Friendliness of staffs
10.Cost Reasonableness
11.Fairness of Price
12.Adherence to hours of operations
13.Comfort of Environment
Findings:
District-wide
SOCIAL IMPACT 62.5 65.0 67.5 70.0 72.5 75.0 77.5 80.0 82.5Overall CSS Score⁺
Baseline Means Overall CSS Score Very Good Good Rankin g of change s Distric t Change (OLS Regression)
1 Jayapura 7.71***
2 Jayawija
ya 4.68**
3 JayapuraKota 4.51**
4 Mimika 4.46*
⁺Note: statistical significance is noted as follows: * p
SOCIAL IMPACT
Findings
: Qualitative - Quotes
“Before, they were indeed lazy to know but now that we have MSF they care more about discipline, 3S (sapa, senyum, sentuh) – greet, smile, touch. Those who are late are no longer late. Because they have made the promises. So, there are improvements. It’s not possible for me to say 100%...” - Member of MSF Puskesmas Abepantai, Kota Jayapura
“Some came late, puskesmas already closed, staff only comes at 9am, goes home at 10am, 11am. Finally with Kinerja they are continuously being pushed, the community repeatedly complained, so we finally addressed the issue and now there are very big changes.” – Member of MSF Puskesmas Dosay, Kab Jayapura
“On physical issues there are changes….Before patients could only
stand while waiting but now there are seats. This bench is new here.” –
Clients of Puskesmas Hom-Hom, Jayawijaya
“Quite a lot of changes. It is cleaner, they pay more attention to these. Yes, yes (there’s changes) only we ask that equipment for medical
examinations must be here, and the doctor must be here every evening as well…To say satisfied, in my opinion maybe not yet, because to be satisfied, well I think it’s when the doctors and
0.00 0.20 0.40 0.60 0.80 1.00 1.20 0.48 0.40.480.5 0.58 0.39 1.05 0.4 0.85 0.680.71 0.43
Most Changes: Kinerja Directly Related CSS Indicators*
Changes in
CSS Score KOTA JAYAWIJAYA MIMIKA JAYAPUR
A
Findings:
Overall
SOCIAL IMPACT
KAB JAYAPU
RA
*All changes are significant at the p < 0.01 level (99% confident)
Biggest Change:
#12 (Adherence to hours of operation): 67%
#3 (Clarity & Certainty): 16.7%
#13 (Comfort) : 16.7%
Highest Ranking:
#12 (Adherence): 33% #1 (Procedural
Easiness): 33%
SOCIAL IMPACT
Findings
: Overall
•
Despite positive changes (and for some units, ranked as
top 3 biggest changes), discipline of staff (#4) is still
relatively weak (in the lower/lowest range)
•
Conversely, adherence to hours of operation(#12) saw
quite significant changes where in some units it even
went from lowest to top 3 indicators in the endline
•
Weaker Puskesmas often had the biggest changes – Koya
Barat, Depapre, Hom-Hom and Limau Asri
•
Fairness of Price(#11) is consistently the indicator with
the biggest changes, often highest in the endline.
However, this could be due to the new BPJS scheme and
the availability of special funding (
Dana Otonomi
Qualitative Findings
19 Theme 1: Increased
satisfaction with health
services at partner puskesmas
Clients were mostly satisfied with simple demands
"...and the nurse or paramedic staff, they did not use to smile but now the changes are big, things that they did not change before, now have changed…" - Puskesmas staff in Mimika
“Before there was no afternoon service. To improve services in this puskesmas they must have afternoon service” – Puskesmas staff in Jayapura district
Theme 2: Increased
knowledge and motivation among health unit staff, but challenges observed with implementation
Health unit staff education levels affect satisfaction with training/program
“…there is personal motivation in the unit staff, …we finally have internal motivation…” – Puskesmas staff in Jayapura city
“…well, the first year was to prepare…second year was to familiarize ourselves. Year three is where we see whether it works or not. So, it did work but definitely with challenges. Year four and five should run by itself. ” – Head of
Puskesmas in Jayawijaya
Theme 3: Increased public engagement in provision of health services
MSFs in Kota and Kab Jayapura described themselves as “overseer” MSFs in Jayawijaya and Mimika described themselves as “helper”
“We feel that MSF is intimidating us, when they should have been facilitators. This problem actually happens in]….” – Puskesmas staff in Jayapura district
“..this forum is not ending
because Kinerja is no longer here, but it will continue to meet as a friend that helps them to advocate and not just as a
Conslusions:
1. Improved capacity of partner units in 4 governance areas (OCA) & level of satisfaction in Puskesmas (CSS) 2. Challenges in implementing new
knowledge/skills: biggest
improvements are still on basic needs (operating hours,
infrastructure)
3. Improvements shown in weaker areas & among lower capacity districts
4. Improved knowledge but with limited institutional change
5. Public engagement mechanism is successful at improving services
20
Recommendations:
1. Intervention should align with DHO’s needs, priorities & schedules
2. Knowledge transfer and
institutional change are more difficult in units that have many competing priorities
3. Utilizing OCA/CSS findings to address Puskesmas weakness 4. Capacitate DHO to monitor
Puskesmas & motivate improvements
5. Strengthen public engagement via MSFs on further subject
