Nice work! This seems like a pretty great overview of our current understanding of a whole range of international development interventions, at least on the micro end of the spectrum. Useful not just for your donors, but the community as a whole.
Two quick points. First, in your appendix you write that it would be interesting to see a rigorous evaluation of Jeff Sachs’ Millenium Village Project. DFID did fund a big evaluation of that project which returned pretty negative results. Marginal Revolution discusses that here, plus there’s a paper by Gelman et al. cited at the end of the MR post.
Second, I do think there’s a bit of tension between the part where you write “the donors are comfortable with impact within the next 1 to 20 years or so, but not keen on options like R&D and policy advocacy which may or may not have an impact at all” and your recommendation of TaRL. As you note, it’s completely possible that J-PAL’s work to scale up TaRL across Sub Saharan African will fail. The line between policy advocacy and policymaking support is pretty blurry.
I don’t want to discourage you from recommending TaRL because I think it’s great! And of course you can’t evalute everything. But I’d be interested in your thinking about what differentiates TaRL advocacy from advocacy for better health or macro policies, which you’ve largely excluded.
@smclare: I’m glad you liked the report :) I definitely hope it can be helpful to others, since alot of work went into it! If it can save others some time, that would be great!
I’ll check out the Marginal Revolution post on Millenium Villages and see if I can include a few sentences about that in the report.
As for TaRL Africa, alot of what they are doing is directly implementing TaRL in different African countries with partners and trying to ensure that the scale up there is a success. So I don’t think of it as being mainly advocacy. You’re right that there’s no guarantee that it will work. I think there is a tension between the donors’ goal of backing things that have been shown to work and having a long term and transformative impact, since the evidence on long term impacts is generally lacking (beyond say 7 years or so at most, in the case of the graduation approach). I tried to find a middle ground between their different goals. One of the reasons I included various charities in Table 1 was a recognition that different interpretations of their goals might lead to different recommendations.
Initially, I had a similar thought while reading this—my understanding is that, at the outset, a large part of TaRL Africa’s work focused on advocating for the TaRL approach and identifying partners who would be interested in adapting and implementing the approach in their contexts. However, TaRL Africa’s work does seem to include technical assistance to local implementers who have committed to using the approach. Their work is described as “supporting policymakers and practitioners to set achievable goals, use teaching-learning practices that are at the level of the child rather than being dictated by a rigid age-grade curriculum, set up hands-on, on-site mentoring systems to support teachers to deliver effectively, and promote measurement strategies that lead to action” (from the Co-Impact page on their grant to TaRL Africa).
From reading about it in more detail, it seems that TaRL Africa’s approach is substantially different from generic policy advocacy. It seems that TaRL Africa is working with a defined set of committed partners to achieve shared goals as effectively as possible. Their work in Zambia has already been rolled out to 1800 schools, and their partnerships elsewhere suggest that their probability of success is relatively high.
Of course, the cost-effectiveness of any donation will depend on the new activities that it enables, so it would be a good idea to check in with TaRL Africa (if you can) about what a donation of this size/timing would allow them to achieve.
Disclaimer: I work with Stephen at Founders Pledge and previously worked at J-PAL, including work on government partnerships that included the TaRL Zambia project.
Nice work! This seems like a pretty great overview of our current understanding of a whole range of international development interventions, at least on the micro end of the spectrum. Useful not just for your donors, but the community as a whole.
Two quick points. First, in your appendix you write that it would be interesting to see a rigorous evaluation of Jeff Sachs’ Millenium Village Project. DFID did fund a big evaluation of that project which returned pretty negative results. Marginal Revolution discusses that here, plus there’s a paper by Gelman et al. cited at the end of the MR post.
Second, I do think there’s a bit of tension between the part where you write “the donors are comfortable with impact within the next 1 to 20 years or so, but not keen on options like R&D and policy advocacy which may or may not have an impact at all” and your recommendation of TaRL. As you note, it’s completely possible that J-PAL’s work to scale up TaRL across Sub Saharan African will fail. The line between policy advocacy and policymaking support is pretty blurry.
I don’t want to discourage you from recommending TaRL because I think it’s great! And of course you can’t evalute everything. But I’d be interested in your thinking about what differentiates TaRL advocacy from advocacy for better health or macro policies, which you’ve largely excluded.
@smclare: I’m glad you liked the report :) I definitely hope it can be helpful to others, since alot of work went into it! If it can save others some time, that would be great!
I’ll check out the Marginal Revolution post on Millenium Villages and see if I can include a few sentences about that in the report.
As for TaRL Africa, alot of what they are doing is directly implementing TaRL in different African countries with partners and trying to ensure that the scale up there is a success. So I don’t think of it as being mainly advocacy. You’re right that there’s no guarantee that it will work. I think there is a tension between the donors’ goal of backing things that have been shown to work and having a long term and transformative impact, since the evidence on long term impacts is generally lacking (beyond say 7 years or so at most, in the case of the graduation approach). I tried to find a middle ground between their different goals. One of the reasons I included various charities in Table 1 was a recognition that different interpretations of their goals might lead to different recommendations.
Initially, I had a similar thought while reading this—my understanding is that, at the outset, a large part of TaRL Africa’s work focused on advocating for the TaRL approach and identifying partners who would be interested in adapting and implementing the approach in their contexts. However, TaRL Africa’s work does seem to include technical assistance to local implementers who have committed to using the approach. Their work is described as “supporting policymakers and practitioners to set achievable goals, use teaching-learning practices that are at the level of the child rather than being dictated by a rigid age-grade curriculum, set up hands-on, on-site mentoring systems to support teachers to deliver effectively, and promote measurement strategies that lead to action” (from the Co-Impact page on their grant to TaRL Africa).
From reading about it in more detail, it seems that TaRL Africa’s approach is substantially different from generic policy advocacy. It seems that TaRL Africa is working with a defined set of committed partners to achieve shared goals as effectively as possible. Their work in Zambia has already been rolled out to 1800 schools, and their partnerships elsewhere suggest that their probability of success is relatively high.
Of course, the cost-effectiveness of any donation will depend on the new activities that it enables, so it would be a good idea to check in with TaRL Africa (if you can) about what a donation of this size/timing would allow them to achieve.
Disclaimer: I work with Stephen at Founders Pledge and previously worked at J-PAL, including work on government partnerships that included the TaRL Zambia project.