Introduction to Implementation research
Key questions
•
What do we mean by implementation research?
•
Why do we need implementation research?
•
How to formulate implementation research objectives and questions?
•
What constructs/variables are relevant for implementation research?
•
What are common methods for implementation research?
What do e ea y …….
Health system research?
Health policy research?
Operational research?
Health services research?
I ple e tatio resear h is that
subset
of HSR that focuses on how to promote
the uptake and successful
implementation of evidence-based
interventions and policies
….
…..usefully defi ed as s ie tifi
inquiry
into questions concerning
implementation - the act of fulfilling or
carrying out an
i te tio
The ultiple defi itio s of operatio al
research, implementation research, and
health systems research creates confusion
and negatively affects the credibility
and progress of the
resear h…
Defining research to improve health systems
(Remme et al 2010)
Research Domain Primary characteristics
Focus
Users
Utility
Operational
Operational issues of specific
health programmes
Health care providers,
programme managers
Local
Implementation
Implementation strategies for
specific products or services
Programme managers, R&D
managers
Local/Broad
Health system
Issues affecting some or all of
the building blocks of a health
system
Health system managers,
policy makers
Defining research to improve health systems
Examples of research questions for the three
research domains
(Remme et al 2010)
Domain Question
Operational Which locations should be targeted for delivering HIV prevention services in Kawempe district, Uganda?
Which of the current ART payment strategies in use in Nairobi should be retained for the new integrated programme?
Implementation How to improve access to vaccination among children who are currently not reached by immunisation services?
How to deliver ivermectin for onchocerciasis control and ensure sustained high treatment coverage in isolated rural communities?
Health system How effective are different policies for attracting nurses to rural areas?
Common interpretations of IR*
•
Systematic approach to understanding and addressing barriers to
effective and quality implementation of health interventions,
strategies and policies.
•
IR is demand-driven and the research questions are framed based on
needs identified together with relevant stakeholders/implementers in
the health system
I ple e tatio resear h…*
•
addresses implementation bottlenecks,
•
identifies optimal approaches for a particular setting,
•
promotes the uptake of research findings
•
leads to improved health care and its delivery.
Characteristics of Implementation Research*
Systematic
Multidisciplinary
Contextual
Complex
Characteristics of Implementation Research
(WHO 2013)
Systematic • The systematic study of how evidence-based public health interventions are integrated and provided in specific settings, and how resulting health outcomes vary across communities.
• Balances relevance to real life situations with rigor, strictly adhering to norms of scientific inquiry. Multidisciplinary • Analysis of biological, social, economic, political, system and environmental factors that impact
implementation of specific health interventions.
• Interdisciplinary collaborations between behavioural and social scientists, clinicians,
epidemiologists, statisticians, engineers, business analysts, policy makers, and key stakeholders. Contextual • Demand driven. Framing of research questions is based on needs identified by implementers in
the health system.
• Research is relevant to local specifics and needs, and aims to improve health care delivery in a given context.
• Generates generalizable knowledge and insights that can be applied across various settings.
• Mindful of cultural and community-based influences. Complex • Dynamic and adaptive.
• Multi-scale: occurs at multiple levels of health systems and communities.
• Analyses multi-component programmes and policies.
E e he i ter e tio s are desig ed
in similar ways, implementation occurs
differently in different contexts, and
ith a y differe t effe ts.….
The importance of implementation
(WHO 2014)
Efficacious tool
Accessibility
Provider
compliance
Patient
adherence
The Prevention of Mother-To-Child-Transmission
(PMTCT) cascade in Zambia (2007-2008)*
There
is growing funding for intervention
research, particularly for drugs,
a i es….. Ho e er
, there is still little
funding for, and, therefore, a relative
dearth of implementation
resear h….
“o iety’s huge i est e t
in technological
innovations that only modestly improve efficacy,
by consuming resources needed for improved
delivery of care, may cost more lives than it saves.
……Health
, economic, and moral arguments make
the case for spending less on technological
advances and more on improving systems for
delivering care
.
Neglecting implementation
challenges costs lives and
money
….
Implementation research objectives and questions*
Research objective
Illustration of research question
Explore
What are the possible factors and agents responsible for good
implementation of a health intervention? For enhancing or expanding a
health intervention
Describe
What describes the context in which implementation occurs? What describes
the main factors influencing implementation in a given context?
Influence
Is coverage of a health intervention changing among beneficiaries of the
intervention?
Explain
How and why does implementation of the intervention lead to effects on
health behaviour, services, or status in all its variations?
Predict
What is the likely course of future implementation?
IR Traditions, Research Questions, and Initial
Core Disciplines
(Peters et al 2013)
Implementation Research Tradition
Typical research questions Core disciplines at origin
Dissemination and
implementation of evidence-based Medicine
What promotes the integration of research findings and
evidence on interventions into healthcare policy and practice?
Epidemiology, behavior change (psychology, sociology, education) Participatory action research
How can we (community members as research participants) learn and be empowered to take action?
Social psychology, education, and anthropology Management improvement How are the right services delivered to the right clients while
meeting the right standards for quality?
Management, engineering Operational research Which solution provides the most rational basis for a decision
concerning the optimal performance of a system?
Management, engineering, math Policy implementation • How was a policy implemented, and what contributed to
reaching the objectives or other effects?
• Which actors are involved in programme delivery in specific locations, how do they understand the problem of
implementation, and what influences their behaviour?
(proliferating) theoretical frameworks for IR?
• Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012). Bridging research and practice: models for dissemination and implementation research. American Journal of Preventive Medicine, 43(3), 337-350. doi: 10.1016/j.amepre.2012.05.024
• Flottorp, S. A., Oxman, A. D., Krause, J., Musila, N. R., Wensing, M., Godycki-Cwirko, M., . . . Eccles, M. P.
(2013). A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice.
Implement Sci, 8, 35. doi: 10.1186/1748-5908-8-35
• Meyers, D. C., Durlak, J. A., & Wandersman, A. (2012). The quality implementation framework: a synthesis of critical steps in the implementation process. American Journal of Community Psychology, 50(3-4), 462- 480. doi: 10.1007/s10464-012-9522-x
• Cane, J., O'Connor, D., & Michie, S. (2012). Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci, 7(1), 37. doi: 10.1186/1748-5908-7-37
• Hendriks, A.-M., Jansen, M., Gubbels, J., De Vries, N., Paulussen, T., & Kremers, S. (2013). Proposing a
conceptual framework for integrated local public health policy, applied to childhood obesity - the behavior change ball. Implementation Science, 8(1), 46.
• Chaudoir S.R., et al. Measuring factors affecting implementation of health innovations: A systematic review of structural, organizational, provider, patient, and innovation level measures. Implementation Science 2013; 8:22
Consolidated Framework for
Implementation Research (CFIR)*
Inner setting
Outer setting
Individual
Process
Intervention
Consolidated Framework for
Implementation Research (CFIR)*
Intervention Outer setting Inner setting Individuals Process
Intervention source Patient needs & resources
Structural characteristics
Knowledge & beliefs Planning Evidence strength &
quality
Cosmopolitanism Networks & communications
Self-efficacy Engaging Relative advantage Peer pressure Culture Individual stage of
change
Executing Adaptability External policy &
incentive Implementation climate Identification with organization Reflecting & evaluating Trialability Readiness for
implementation
Other personal attributes
Complexity
Design quality & packaging
Cost
Implementation outcome variables
(Peters et al 2013; WHO 2014)
Variables Working definition
Acceptability The perception among stakeholders (for example, consumers, providers, managers, policy makers) that an intervention is agreeable
Adoption The intention, initial decision, or action to try to employ a new intervention
Appropriateness The perceived fit or relevance of the intervention in a particular setting or for a particular target audience (for example, provider or consumer) or problem
Feasibility The extent to which an intervention can be carried out in a particular setting or organization
Fidelity The degree to which an intervention was implemented as it was designed in an original protocol, plan, or policy
Implementation cost
The incremental cost of the implementation strategy (for example, how the services are delivered in a particular setting). The total cost of implementation would also include the cost of the intervention itself
Coverage The degree to which the population that is eligible to benefit from an intervention actually receives it.
Methods particularly suitable for IR
(Peters et al 2013)
Mixed methods
Participatory
action research
Quality
improvement
studies
Effectiveness-implementation
hybrid trials
Spectrum of implementation research relevance
(Peters et al 2013)
Implementation
not relevant
Implementation
relevant but not
considered
Implementation
relevant but
effects reduced
Implementation
studied as
contributing
factors
Implementation
as primary focus
Context
Controlled
Largely controlled Real world &
partial control
Real world &
population
Real world &
population
IR Variables
Not relevant
Assumed to be
controlled or not
relevant
Controlled
Independent
variables
Primary
outcome/
determinants
Examples
Basic science,
Phase I and II
clinical trials
Efficacy study,
Phase III RCT
Key questions to assess research designs or
reports on implementation research
(Peters et al 2013)
•
Does the research clearly aim to answer a question concerning implementation?
•
Does the research clearly identify the primary audiences for the research and how they
would use the research?
•
Is there a clear description of what is being implemented (for example, details of the
practice, programme, or policy)?
•
Does the research involve an implementation strategy? If so, is it described and
examined in its fullness?
•
Is the research co ducted i a real world setti g? If so, is the co text a d sa ple
population described in sufficient detail?
•
Does the research appropriately consider implementation outcome variables?
•
Does the research appropriately consider context and other factors that influence
implementation?
On-going developments
•
Postgraduate scholarships for Implementation research
•
Implementation research workshops
•
Implementation research Massive Open Online Course
•
Ethics in implementation research
•
Reporting guidelines for implementation research
•
Implementation research grants
•
Updating of implementation research toolkit
