Research Advisor at Founders Pledge, formerly on the Policy team at J-PAL. Focused on Global Health and Development.
samcart
Karma: 9
Improving the Impact per Dollar of USAID Programming: The Power of Cost-Effectiveness Evidence to Improve Lives
Initially, I had a similar thought while reading this—my understanding is that, at the outset, a large part of TaRL Africa’s work focused on advocating for the TaRL approach and identifying partners who would be interested in adapting and implementing the approach in their contexts. However, TaRL Africa’s work does seem to include technical assistance to local implementers who have committed to using the approach. Their work is described as “supporting policymakers and practitioners to set achievable goals, use teaching-learning practices that are at the level of the child rather than being dictated by a rigid age-grade curriculum, set up hands-on, on-site mentoring systems to support teachers to deliver effectively, and promote measurement strategies that lead to action” (from the Co-Impact page on their grant to TaRL Africa).
From reading about it in more detail, it seems that TaRL Africa’s approach is substantially different from generic policy advocacy. It seems that TaRL Africa is working with a defined set of committed partners to achieve shared goals as effectively as possible. Their work in Zambia has already been rolled out to 1800 schools, and their partnerships elsewhere suggest that their probability of success is relatively high.
Of course, the cost-effectiveness of any donation will depend on the new activities that it enables, so it would be a good idea to check in with TaRL Africa (if you can) about what a donation of this size/timing would allow them to achieve.
Disclaimer: I work with Stephen at Founders Pledge and previously worked at J-PAL, including work on government partnerships that included the TaRL Zambia project.
Hi! Writing as a Research Advisor at Founders Pledge and formerly staff at J-PAL. I really enjoyed reading this report and was very impressed by the detail, clarity, and brevity (30 pages isn’t short, of course, but this could’ve been much wordier—it’s very to-the-point). Some specific things I found particularly exciting:
(1) The purpose of this report and the goals/values of the donors were made very clear at the beginning and referenced throughout the document. This is surprisingly easy to leave out, especially when writing for a familiar audience (like Effective Altruists) whom we may expect to have identical worldviews to our own.
(2) It was great to see your rationale for selecting your top interventions, as well as the reasons you chose not to select others from Table 1. It helped provide a complete picture of how and why you arrived at your conclusions.
(3) It was interesting (with no positive nor negative connotation—just truly interesting!) to see the GiveWell-evaluated health interventions included in the category of early childhood development. I’ve never seen them framed in that way, but it made sense to me while reading.
Other organizations you could consider looking into:
(1) BOMA Project, Trickle Up, and BRAC all implement the Graduation approach on the African continent.
(1a) BRAC is, of course, the creator of the approach and a very impressive and evidence-focused organization. I think it would be worth getting in touch with BRAC to see if donors of this size could earmark support to the Ultra-Poor Graduation Program in Uganda.
(1b) BOMA Project implements a version of the Graduation approach in a few countries, with most of its work in Kenya. I don’t know them well, but they’re often speakers at conferences with sessions on the Graduation approach and related social protection programs.
(1c) Trickle Up also implements the Graduation approach on the African continent and elsewhere. I have a positive impression of them from a few meetings with their M&E team, but don’t have deep knowledge about their programs so can’t recommend for or against.
(2) BRAC’s Empowerment and Livelihoods for Adolescents program is also something that might be of interest. It’s been studied with RCTs in Uganda and Sierra Leone (the latter study is really interesting as it took place during the Ebola outbreak) and found to have positive results on income and education levels. I’ve never seen a cost-effectiveness analysis, but the SL RCT notes that the program “has proved to be scalable and cost-effective across countries” (p. 11).
(3) J-PAL’s Innovation in Government Initiative (IGI). I’d guess this was left out because it fits into the “evidence-based policy” category, but the competitive nature of the fund means that the projects it supports are more like TaRL Africa’s work (specific, discrete technical assistance to meet policymaker demand) than generic advocacy. IGI is the recommended charity from Founders Pledge’s report on evidence-based policy, and fits the “transformative” criterion in a slightly different way—enabling the transformation of entire government programs that are designed to improve lives.
A few notes and questions I had while reading:
(1) Did you assess try to assess organizations’ track record/strength when making these recommendations? From my experience, both TaRL Africa and Village Enterprise are exceptionally strong, evidence-based organizations with strong leadership and implementation on the ground. It could strengthen your recommendation to include a note about their commitment to evidence use and generation, local partnerships, etc.
(2) For Village Enterprise in particular, it would be good to mention the Sedlmayr, Shah, and Sulaiman (2018) study that directly compared VE’s program with a cost-equivalent cash transfer in Uganda. This can help bolster your reasoning for selecting them over GiveDirectly (although the cash only results were really noisy and hard to interpret—I hope an updated paper will be released sometime soon).
(3) I was confused about your cost-effectiveness estimates for TaRL until I looked at the linked spreadsheet. The intervention can vary along a number of axes, and both the costs and impacts will vary based on the local context and population served, so I wasn’t sure what a cost-effectiveness estimate for the TaRL approach meant. I see from the spreadsheet that your cost estimates came from the studies with Pratham in India—I think it would help to clarify that in the document itself.