Huwelium

Million dol­lar dona­tion: penny for your thoughts?

Re­view of Ed­u­ca­tion In­ter­ven­tions and Char­i­ties in Sub-Sa­haran Africa

• Huwelium 25 Jan 2021 18:15 UTC

  7 points

  0 ∶ 0

  on: Million dol­lar dona­tion: penny for your thoughts?

  Epilogue: I am happy to report that the donors have decided today to donate 1 million dollars (CAD) over the next five years to support Teaching at the Right Level, Africa. This initiative is helping to change education systems in African countries so that students in primary school can learn to read, write and count (which is sadly not the case right now, for the most part). Since writing my report, Teaching at the Right Level, Africa has also been reviewed and recommended by Founders Pledge.

  I thank all of you for your thoughts and comments and am very excited to have been able to contribute to this :)

• Huwelium 20 Jun 2020 12:48 UTC

  3 points

  0 ∶ 0

  in reply to: samcart’s comment on: Million dol­lar dona­tion: penny for your thoughts?

  Hi samcart,

  I’m glad you liked the report :) Thanks for the detailed and constructive feedback! Note: One reason that it isn’t too long is that other reports preceded it (on education, early childhood development, small and medium enterprise support, rural electrification and long-term effects of cash transfers). This final report tries to summarize things.

  Regarding the other organizations that implement the graduation approach in Africa, I did indeed look into them (see the report on cash transfers for details). BRAC might have been a good choice, but they run a very wide range of programs, and it was alot harder to figure out what would be the net impact of even a targeted donation to their graduation approach program (see e.g. this GiveWell post on BRAC). I was also influenced by discussions with Founders Pledge who also looked into these different charities and ended up recommending Village Enterprise.

  As for J-PAL’s Innovation in Government Initiative, I personally find it very interesting. I don’t know if it would be a good fit with the donors’ preferences. They will be going over the comments here, though, so they’ll get a chance to see some of the other options that have been proposed.

  Responses to the questions you posted:

  1)Yes, I did try to assess organizations’ strength and track record, though admittedly that was harder to do than going over RCTs and literature. It was part of the motivation for recommending TaRL Africa and Village Enterprise. I’ll look over the text again and see if I can make that clearer. Mind you, all the other organizations that made it to Table 1 also seem to have a strong track record. I partly relied on the fact that many of these had been vetted by GiveWell and Founders Pledge.

  2) and 3) I’ll do that, thanks for the suggestion :)

• Huwelium 16 Jun 2020 21:54 UTC

  3 points

  0 ∶ 0

  in reply to: CarlShulman’s comment on: Million dol­lar dona­tion: penny for your thoughts?

  Thanks for your feedback Carl, and for the links to the Open Phil reports on Center for Global Development and Target Malaria. I will check them out. One thing I have to keep in mind though is the donor’s preferences/​mandate. While these are somewhat fuzzy/​open-ended, they did not seem very interested in policy advocacy or interventions where there isn’t yet clear evidence of impact.

• Huwelium 6 May 2019 15:19 UTC

  3 points

  0 ∶ 0

  in reply to: Wayne_Chang’s comment on: Re­view of Ed­u­ca­tion In­ter­ven­tions and Char­i­ties in Sub-Sa­haran Africa

  Hi Wayne,

  Thanks for your post. I would love to get in touch and compare notes on research for advising donors. I’ll try to reach you via this site’s messaging.

  Sorry for the very late reply (I don’t get alerts when someone posts here). I believe the difference comes simply from the wide range of cost effectiveness of education interventions. As mentioned in the Google doc, “Rachel Glennerster mentions in an 80000 Hours podcast that good interventions typically deliver at least 1 learning adjusted year of schooling (LAYS) per 100 USD spent, with some interventions delivering about 10-30 LAYS per 100 USD, and the best delivering up to 460 LAYS per 100 USD.”

  For Pratham, the info I found suggested roughly 1.7 to 27.6 extra years per 100 USD. Assuming an increase in income of 8.8% for each extra year of schooling, this means an increase in income of about 15% to 243% per 100 USD donated. Comparing to DDK 2017, GiveWell cites a 24% increase in income for 541$ spent, so 4.4% increase in income per 100$ spent.

  I don’t know if this helps? I think the basic explanation is that there is a very wide range in effectiveness of education interventions, and that Pratham seems to be higher in this range than DDK, say.

• Huwelium 27 Feb 2019 12:52 UTC

  3 points

  0 ∶ 0

  in reply to: Kirsten’s comment on: Re­view of Ed­u­ca­tion In­ter­ven­tions and Char­i­ties in Sub-Sa­haran Africa

  We’re planning to look into early childhood interventions (including things like deworming, improved nutrition, etc.) separately. Having said that, we had put deworming aside since it’s basically presented by GiveWell as “low probability of high impact” which didn’t appeal to the donors. But as you say, we should review what the evidence is in terms of education impact. I’ll add that to my to do list.

• Huwelium 29 Jun 2020 19:06 UTC

  2 points

  0 ∶ 0

  in reply to: HStencil’s comment on: Million dol­lar dona­tion: penny for your thoughts?

  @HStencil, TechnoServe does seem like an interesting organization. There is definitely some similarity to Village Enterprise, but Village Enterprise is focused on the graduation approach while TechnoServe runs a broader range of programs including support to small and medium enterprises. I was also very interested in small and medium enterprise support, but my review of the evidence on this suggests that it’s inconclusive whether such programs have had success at creating jobs or reducing poverty. Of course, it’s entirely possible that TechnoServe’s programs do indeed do this. If so, that would be great! ImpactMatter’s evaluation does suggest this, but as they point out the evidence is not very strong: it’s based on a pre-post analysis conducted by TechnoServe themselves, response rates post-intervention were low and enterprises that did not experience increased revenues were excluded from the analysis (see excerpt from ImpactMatter’s evaluation below).

  Excerpt from ImpactMatter’s evaluation report:

  “Impulsa Tu Empresa’s beneft/​cost ratio is based on data collected by TechnoServe on businesses’ gross revenues before and after participating in the program. We subtract from these figures the growth we assume businesses would have experienced had they not in fact participated in the program. However, only a small number of firms responded to TechnoServe’s surveys conducted two to three years after they completed the program. TechnoServe also did not count businesses that had lower revenues after the intervention than before the intervention, biasing upward the benefit/​cost ratio”

• Huwelium 20 Jun 2020 12:57 UTC

  2 points

  0 ∶ 0

  in reply to: Stephen Clare’s comment on: Million dol­lar dona­tion: penny for your thoughts?

  @smclare: I’m glad you liked the report :) I definitely hope it can be helpful to others, since alot of work went into it! If it can save others some time, that would be great!

  I’ll check out the Marginal Revolution post on Millenium Villages and see if I can include a few sentences about that in the report.

  As for TaRL Africa, alot of what they are doing is directly implementing TaRL in different African countries with partners and trying to ensure that the scale up there is a success. So I don’t think of it as being mainly advocacy. You’re right that there’s no guarantee that it will work. I think there is a tension between the donors’ goal of backing things that have been shown to work and having a long term and transformative impact, since the evidence on long term impacts is generally lacking (beyond say 7 years or so at most, in the case of the graduation approach). I tried to find a middle ground between their different goals. One of the reasons I included various charities in Table 1 was a recognition that different interpretations of their goals might lead to different recommendations.

• Huwelium 25 Mar 2019 18:32 UTC

  2 points

  0 ∶ 0

  in reply to: Sanjay’s comment on: Re­view of Ed­u­ca­tion In­ter­ven­tions and Char­i­ties in Sub-Sa­haran Africa

  Yes, you can check out this webpage: https://​​www.givewell.org/​​international/​​technical/​​programs/​​education

  Here’s a relevant excerpt:

  ″ Evidence of effect on outcomes (such as income, health, or social outcomes) rather than outputs (such as increased time in school or improved test scores). There are a number of variables that can be used to measure the effects of education interventions, and we place significantly more emphasis on the effects on some variables than others. We distinguish between ‘outputs’ of education interventions, namely whether they increase time in school or student learning (measured by test scores),3 and the effect of education interventions on people’s life ‘outcomes’, including employment, earnings, health outcomes, fertility, and marriage. We do not place much intrinsic value on increasing time in school or test scores, although we think that such improvements may have instrumental value.4 The majority of experimental studies of the effects of education interventions focus on the effects on time in school and test scores. However, we place far more emphasis on a few recent studies that estimate the effects of education programs on life outcomes, such as earnings and rates of fertility and marriage among young women and girls.”

• Huwelium 2 Mar 2019 19:22 UTC

  2 points

  0 ∶ 0

  in reply to: saulius’s comment on: Re­view of Ed­u­ca­tion In­ter­ven­tions and Char­i­ties in Sub-Sa­haran Africa

  I hadn’t seen it. Will definitely check it out. Thanks Saulius!

• Huwelium 27 Feb 2019 0:38 UTC

  2 points

  0 ∶ 0

  on: Re­view of Ed­u­ca­tion In­ter­ven­tions and Char­i­ties in Sub-Sa­haran Africa

  Thanks Aidan, Aaron and Khorton for your comments—much appreciated!

  I definitely agree that GiveWell does excellent work, and we are indeed thinking of including a GiveWell charity (or more) in our final recommendation to the donors, which will probably include a few charities rather than just one. As Aidan mentioned, GiveDirectly seems like it might be the best fit with the donors’ criteria, among GiveWell’s top charities (some of their standout charities might also be a good fit).

  Regarding education interventions, GiveWell did not have any recommendations, so using their recommendations was not an option. We could have recommended against donating in this sector based on GiveWell’s review, but we didn’t do that primarily because the donors assign intrinsic value to better education, while GiveWell does not (so here it’s more a question of values rather than expertise).

• Huwelium 12 Aug 2018 19:24 UTC

  2 points

  0 ∶ 0

  on: Im­pact In­vest­ing—A Vi­able Op­tion for EAs?

  Thanks Naryan for starting this interesting discussion! My own two cents:

  -I attended a presentation organized by Valeurs Mobilières Desjardins about two years back on socially responsible investing (SRI). The upshot was that the research on this indicates that SRI investments perform comparably (neither better nor worse) to non-SRI investments in general. Assuming this applies to impact investment, and that the vast majority of EAs will invest part of the money they earn, it seems to me that as long as there are any positive social impacts from SRI and impact investment, these should be the default mode of investment of EAs, all else being equal… If so, this seems well worth discussing.

  -I didn’t fully get the distinction you make between SRI and impact investment. For instance, I’m part of the Ethical Investment Group http://​​www.gie-eig.ca in Montreal. We’ve long been investing in individual companies we think of as having beneficial impact, as well as in community loans, etc (some of which are listed in the links you gave). I always thought of this as ethical or socially responsible investment. Which would you say it is?

• Huwelium 29 Jun 2020 18:22 UTC

  1 point

  0 ∶ 0

  in reply to: tomwein’s comment on: Million dol­lar dona­tion: penny for your thoughts?

  tomwein, I just looked through the RCT. The reduction in violence is indeed significant and promising. In terms of test scores, though, they report “There was no evidence that the intervention had an impact on any educational test scores”. Test scores and income were the main metrics we were focusing on for education interventions, so this intervention doesn’t seem like a good match to the donors’ preferences at this stage. From reading the RCT, it seems that studies about violence prevention in school are quite novel and that the evidence base is still thin, so again this wouldn’t match well with the donors’ wish to support interventions with strong evidence of effectiveness. Though obviously it’s very valuable to be gathering evidence given the lack of previous studies.

• Huwelium 29 Jun 2020 17:24 UTC

  1 point

  0 ∶ 0

  in reply to: samcart’s comment on: Million dol­lar dona­tion: penny for your thoughts?

  samcart, just a quick note that TaRL Africa is (or was) part of J-PAL’s Innovation in Government Initiative (or its predecessor, the Government Partnership Initiative). See these links:

  https://​​www.povertyactionlab.org/​​blog/​​1-16-18/​​increasing-use-data-and-evidence-real-world-policy-stories-j-pals-government

  https://​​www.povertyactionlab.org/​​blog/​​8-8-17/​​zambia-scale-teaching-right-level-program-1800-schools

• Huwelium 20 Jun 2020 13:03 UTC

  1 point

  0 ∶ 0

  in reply to: Stephen Clare’s comment on: Million dol­lar dona­tion: penny for your thoughts?

  @smclare Thanks for giving some background on the Founders Pledge cost-effectiveness scenarios. For TaRL, I’m surprised that you say that the optimistic scenario is the unfathomably best case scenario. Even in that scenario, impacts are assumed to last 20 years, and the impact of test scores improvements on earnings does not use the most optimistic cases mentioned in the Founders Pledge education report. It seems fathomable impacts could last a whole career (say 40 years). As you can see from my cost-effectiveness estimates for TaRL, my unfathomably best case scenario is significantly more optimistic than the one from Founders Pledge (I included in the cost-effectiveness spreadsheet a worksheet using the worksheet from Founders Pledge as a starting point, but with my own scenarios in there). And in both cases, we only include the impact on income. It seems quite plausible that education would have impacts beyond that which aren’t taken into account.

• Huwelium 16 Jun 2020 22:03 UTC

  1 point

  0 ∶ 0

  in reply to: tomwein’s comment on: Million dol­lar dona­tion: penny for your thoughts?

  I think the donors do indeed intend to commit for 5 years, for the reason tomwein invokes. But of course if new evidence suggests an intervention really isn’t having the impact that we expected, or something else that seems much more promising comes along, presumably they could still revisit their commitment on an annual basis.

  Regarding TaRL, the intervention has been studied extensively. The main uncertainty is whether and to what extent gains in test scores translate to long-term outcomes like higher income. But since the donors also care about improvements in learning outcomes per se, there is a bit of a hedge here. It just isn’t captured in the cost-effectiveness analyses, which only incorporate effects on income.

• Huwelium 16 Jun 2020 21:58 UTC

  1 point

  0 ∶ 0

  in reply to: tomwein’s comment on: Million dol­lar dona­tion: penny for your thoughts?

  Thanks for the recommendation, tomwein. Could you provide a link to the RCTs or other experimental evidence?

• Huwelium 25 Feb 2019 16:56 UTC

  1 point

  0 ∶ 0

  on: An­nounc­ing Pri­or­i­tyWiki: A Cause Pri­ori­ti­za­tion Wiki

  Hello Marcus,

  I’ve just posted today about a review that was performed by a member of Effective Altruism Québec on education interventions and charities in Sub-Saharan Africa. The review is here: https://​​docs.google.com/​​document/​​d/​​1-JzmsKJFHPq3j1vAypy8yZM7NbGRco6e_S5con3TOTI/​​edit?usp=sharing

  Thought I would let you know in case it’s relevant for the Priority Wiki.