What is it?
A Midterm Evaluation Report (MTE) that reflects findings from data collection and analysis during A360’s design phase and the first few quarters of implementation (Aug. 2016-Sept. 2018). A360’s Management Response is also included.
Among the report’s highlights:
- The approach: The A360 approach is characterized by its six disciplines, which have had different levels of influence on the solutions. The disciplines’ level of influence differed based on the level of effort and resources allocated to each discipline representative or organization. Staff considered HCD and youth engagement the most influential of the disciplines, although there have been some tensions.
- Partnerships: Solidifying external partnerships for scaling the solutions was a challenge throughout the design process, in part due to the competitive environment of adolescent sexual and reproductive health (ASRH) programming, and the challenges of pursuing partnership while also undergoing an intensive design process and promoting adoption and replication.
- Cost: $9.9 million was spent designing and implementing A360 up to and including the end of the pilot phase. The cost-effective analysis suggested that if the A360 approach were being applied in a similar fashion again, $800,000 could potentially be saved across these categories.
- Impact: Monitoring data suggested that A360 was (and remains) on track to meet its target of 244,738 adopters by the end of the program. But a focus on achieving a minimal viable product, using adoption and conversion figures as benchmarks for success, reduced incentives to implement aspects of the solutions that were designed to address some of the gender and social norms challenges.
- Framing: While linking contraception to aspirations has been key to achieving these results, ‘flying under the radar’ may have bypassed rather than addressed community myths, misconceptions and stigma.
- Provider bias: A360 addresses the attitudes of service providers implementing the solutions. However, A360 faced multiple obstacles that hindered service providers from actively counseling adolescent girls on all available methods, including weaknesses in health systems, and entrenched service provider bias towards and against particular methods. A360 operates in complex policy environments, with health system limitations and entrenched social norms governing the acceptability of contraception for adolescent girls.
- Premature promotion: A360 actively promoted its approach and solutions to encourage others to adopt or replicate them. However, some of these promotion activities were before rigorous evidence on the effectiveness of solutions was available.
- Team structure: The organizations and people behind A360 were (and remain) critical to its success to date. However, the large consortium required significant management oversight and communication, which was not always adequately resourced.
How did we use it?
A360 welcomed, and agreed with the majority of the MTE findings; the report allowed A360 to shift from the project’s 2018 focus on reducing cost per adopter and increasing adoption rates to the 2019 goal of reaching a recommended viable product by easing the focus on cost and adoption rates and doubling down on intervention components that influenced intervention quality and sustainability. A360 secured approval to pivot to the 2019 focus in March 2019.
Consider using it to:
The MTE’s takeaways are useful for evaluators of HCD and adaptive programs (see page 10), and for donors and future ASRH implementers (see page 49).
