The main objectives of the evaluation were (1) to develop a balanced overall picture of our partners' satisfaction with the Accelerator and their perception of its impact and (2) to identify areas for improvement. The current report provides an impact assessment of the Accelerator, our flagship program now in its tenth year of operation. In early 2016, Atma conducted a thorough exit process with five soon-to-be-graduated partners that included a survey on their satisfaction with the Accelerator.
The planned project quickly developed into a full impact assessment of the first nine years of the Accelerator. On behalf of Atma, she conducted the evaluation over five months and completed a comprehensive report that was shared internally and became the basis for conversations about the next steps in the development of the Accelerator. We would also like to thank Pooja Warier, Amit Murugkar, Karen Braganza, and Paige Walker for their incisive and thoughtful feedback on the structure and design of the current review.
Since its inception in 2007, Atma has significantly supplemented and improved the Accelerator program in a constant effort to provide the most relevant and effective support to our partner organizations. Evaluate the different components of the acceleration program: assess their relevance and effectiveness; highlight areas of improvement; and offer recommendations. To evaluate the impact of the accelerator program: Assess the extent to which the program was able to build capacity and drive growth in our partner organizations.
Minor changes and additions to the surveys in the early stages of the data collection process meant that a few pieces of quantitative data could not be obtained from all partners.
How effectively are we delivering our model?
How do our partners benefit from the model?
The Accelerator: is it working?
HOW EFFECTIVELY ARE WE DELIVERING OUR MODEL?
- PARTNERSHIP MANAGERS
- VOLUNTEERS
- RESOURCE LINKAGES
- PROJECT IMPLEMENTATION15
They may not know the organization, but they are always well-prepared for what the job is and what the scope of it is.” However, only 27% indicated that support was very helpful or critical, indicating that more support is needed to connect organizations with potential donors. The Accelerator program impacted organizational leaders in two broad areas: second-line management and leadership skills.
Many founder-led organizations have come to appreciate the importance of second-line leadership and have acquired the skills to define organizational structure, hire skilled professionals, and improve accountability. Leaders honed their own strategic thinking toward a broad vision, practiced effective communication and team leadership, and learned to trust their teams.
HOW DO OUR PARTNERS BENEFIT FROM THE MODEL?
- LEADERSHIP
- NETWORK AND VISIBILITY
- x Budget 2.6 x Locations
- REACH AND SCALE
- MONITORING AND EVALUATION PROTOCOLS
- SYSTEMS AND PROCESSES19
So one of the very first strategic things that Atma helped us do was sit back and review what it is that we want to do. The first thing I thought about working with Atma was, [it] gives me visibility, access to. Better management of the fundraising pipeline (eg when and how to apply for funds, how to spend funds, etc.).
By supporting our partners in clarifying strategy and overall vision, establishing strong systems and processes that enable program iteration, and systematic long-term follow-up. Several partners mentioned that their goal in joining the accelerator program was to strengthen and develop organizational capacity – importantly, some also indicated that the program's scaling effect may occur after graduation rather than during the partnership; this should be taken into account in further evaluations of the impact of the programme.
The following figures reflect the growth from graduation to date of the partners surveyed who graduated before 2016. Interviewees emphasized that Atma was able to: However, a number of respondents reported that they would have appreciated more conscientious efforts to ensure planned full use. M&E systems. Furthermore, some partners would like better coaching in how to use data collected by the new M&E systems, especially to show long-term outcomes and impact.
Raising awareness of the importance of M&E processes (eg, the benefits of M&E data for donor engagement and iterative program improvements). Develop skills to later build the organization's own M&E tools (skills include identifying success indicators, measuring results, record keeping, etc.). In some circumstances, the systems developed did not actually have an effect because the organizations were not prepared to implement them.
Systems and processes include calendars, internal trackers, manuals/guides that promote organizational structure, and any other protocols that reduce repetition and automate operations. Overall, the accelerator program is able to support partner organizations to become stronger, better able to tackle challenges and more likely to survive. Of the partners interviewed who graduated before 2016, almost all reported continued use of systems and processes designed with Atma in the year(s) since graduation.
THE ACCELERATOR: IS IT WORKING?
- OVERALL SATISFACTION
- CAPACITY-BUILDING AND LONGEVITY OF CHANGE
- PARTNERSHIP DURATION
- VOLUNTEERSThoroughly
- PROJECT IMPLEMENTATION
- LEADERSHIPHire an
- MONITORING AND EVALUATION PROTOCOLS
Specifically, we describe seven key challenges identified in the review and describe Atma's plans to address them. Atma's programming has changed since the respondents' graduation; where these changes have already addressed the highlighted challenges, we take note of our progress. Due to the length of the program, PMs may be required to leave Atma before all associated 3-year partnerships are completed.
To the extent possible, transitions should not detract from partners' experience with the program nor significantly slow down the momentum of partner achievements, especially given that the relationship with the project manager is a central component of the Atma model. Atma has implemented this SOP once so far and in a subsequent quarterly evaluation (conducted every quarter to track partner satisfaction and project completion), the respective partners reported smooth transitions and no knowledge gaps with their new project managers. In the event that a volunteer leaves early or before project completion, offer plenty of support to transition the project to a new volunteer or project manager.
Atma has seen many organizations in the sector use similar models and struggle with a number of common challenges. In the last two years, marketing projects have been standard in the first quarter of the partnership. However, some partners requested or expected more formal training beyond that provided in the normal course of partnership.
As most of our current partners operate in the education space for individuals with disabilities, Atma particularly recognizes the need for in-house expertise in this sector. In the corresponding 'Findings' section (see pages 19-21), we asked whether, in general, the Accelerator is working. Against this part of the evaluation, there are no particular challenges that we want to highlight.
Ensure M&E support starts early in the partnership to allow sufficient time to design, implement and modify M&E systems. Within two months of the completion of the evaluation, we used its findings as a basis for discussions about the next steps in building the accelerator. Overall, we were pleased to note that many of the challenges highlighted in the assessment were ones that Atma is already familiar with and that we have been making progress towards in the time since our earliest cohorts graduated.
In this sense, the evaluation validated much of the strategic development that Atma has implemented in the past two years. We are grateful to all contributors for the opportunity to take stock of the work we have done so far and to carry forward lessons we have learned as we advance the vision of Quality Education for All with our vibrant partners.
