$500 prize for anybody who can change our current top choice of intervention
We have completed our second phase of research which was to narrow a list of about thirty charity ideas to a more concise list of five top prospects worth further exploration. Below is a chart comparing these five possibilities. The rankings are relative to each other, not absolute and based on a primary and time limited review of the data on these causes.
We have written a detailed summary for each of these options elaborating on their strengths and weaknesses:
It is difficult to compare charities at this level, especially when the metrics we use to measure their respective impacts are so different (e.g. research vs. direct benefit). However, we still feel as though we have a front runner among these possibilities. SMS reminders to encourage vaccinations is currently our top pick for an intervention to pilot.
Though we are tentatively feel that our front-runner (SMS reminders) is the most promising intervention to proceed with, we have not closed the book on the other four options. We recognize that the difference in impact between the best and second best options could be very large, and thus feel that there can be “no stone unturned” with regards to making the right choice. To help make this the case, we are offering a one-time $500 prize to anyone who can significantly sway our decision by providing material that may challenge our conclusion thus far. How significant? We challenge you to change our top option by either weakening the case for SMS reminders or strengthening the case for another option. Why are we doing this? We need to make sure our decision is as airtight as possible before moving forward. Consider this an opportunity not only to get you (or a charity of your choice) $500 richer, but to make a hugely impactful contribution in the fight for global health. Email joey@charityscience.com for more information.
- Debrief: “cash prizes for the best arguments against psychedelics” by (14 Jul 2019 17:04 UTC; 47 points)
- How should we prioritize cause prioritization? by (13 Jun 2016 17:03 UTC; 25 points)
- 's comment on NunoSempere’s Quick takes by (8 Nov 2020 11:42 UTC; 18 points)
- 's comment on EA is vetting-constrained by (11 Mar 2019 22:08 UTC; 9 points)
- 's comment on EA Hotel with free accommodation and board for two years by (9 Jun 2018 0:00 UTC; 7 points)
- 's comment on EA Forum Prize: Winners for April 2019 by (5 Jun 2019 17:40 UTC; 3 points)
I’m concerned about the SMS reminder thing for a weird reason: it looks too easy.
A number of reasons this is concerning—not sure which apply in this case since I don’t fully understand the process by which this list was created, but off the top of my head here they are:
The idea is so simple and “obvious” (yes, I know, retrospect) that there are probably lots of people who have tried various forms of it.
You may be subject to biases which cause it to look better than it is because you want the best interventions to be easy to execute.
There are no (obvious) schleps, which may be a corollary of #1 or #2, but it still seems concerning. From Paul Graham’s schlep blindness essay: “Most hackers who start startups wish they could do it by just writing some clever software, putting it on a server somewhere, and watching the money roll in—without ever having to talk to users, or negotiate with other companies, or deal with other people’s broken code. Maybe that’s possible, but I haven’t seen it.” Swap “money” for “QALYs” and “startups” for “organizations”. (http://paulgraham.com/schlep.html)
Again, this may indeed be a good idea despite the above. But my alarm bells are going off.
Edit: Further thoughts—unsure if this ever could become a GiveWell top charity. It seems like the “room for more funding” isn’t that high, because the impact of this project doesn’t seem blocked on money.
Very briefly, the process was starting with GiveWell’s list of charities they might like to see (this was mostly broad cause areas) and doing further research into each area to determine a short list of promising specific charities.
1) I think my main thoughts around this is that the charity market is a lot less efficient than the for profit market so there are more low hanging opportunities. And even the for profit market isn’t perfect. There’s a specific brand of vegan sausages in the UK that is impossible to get in Canada, despite it being better than anything Canadian-made. Additionally, as we mentioned in our report, there are already charities doing this, it’s just it’s often easy to forget how large the world is. AMF and all the other enormous bed net charities still haven’t filled all the gaps for bednets because there are just so many people. Likewise with SMS. On top of that, mobile health is a relatively new field due to the nature of the technology, so a lot of obvious things are still just starting to permeate the space.
2) This could be true. We do have an explicit category for logistical difficulty, so we certainly have considered that as an important factor. That being said, I think even if we held logistical difficulty consistent across interventions, our team would still favour SMS based on the other criteria.
3) I can see this being tempting, but we do not think of SMS as something that we simply program and wait for the DALYs to roll in. We suspect there will be constant work for further expansion, tweaking of messaging and code, and testing to see what could improve the system.
Chobani was one of the world’s fastest growing companies for a while, based mainly just on bringing Turkish yogurt-making practices to the US.
there’s some competition in the vaccine reminder space:
http://immunizeindia.org/ http://healthmarketinnovations.org/program/m-chanjo http://healthmarketinnovations.org/program/m-vaccine http://www.path.org/publications/files/TS_mhealth_mobile_messaging_toolkit.pdf
also see: http://www.who.int/reproductivehealth/topics/mhealth/maps-toolkit/en/
Thanks for the links.
There is indeed some competition although we found this to be true in basically every area. We are in contact with a few these folks about possible partnerships.
I would recommend against doing ‘Poverty Research’. Like GiveWell I think most of the value comes from doing very high quality studies with proper samples. These are much harder to do than most people realise even within the EA community—small, hard to identify errors made early on can invalidate the results.
Most people with PhDs in quantitative research methods, who have trained with experts in the field, will still regularly conduct RCTs that are not very useful (though their work pushes the field forward by seeing what has to be done better next time).
A properly powered study also costs millions of dollars, so the minimum useful product here is quite substantial.
I could imagine most of the value of studies coming from very high quality studies, although I think there is also quite a lot of value in some replication and exploratory studies. I agree high quality studies are both hard to do and generally have their difficulty underestimated. We’re also on the same page when it comes to small problems at the beginning making the research effectively useless later on, which is why we would definitely hire people with a history of running exceptionally rigorous studies if we were to run this intervention. We are generally in favor of hiring capacity when it’s crucial.
The estimate of millions of dollars is a bit higher than the estimates I have seen although I could imagine it being the case. The estimate we used was the higher end of of the $50,000 and $500,000 range that IPA gave for its costs per studies. Do you have any information about the costs of running studies or more specifically the cost of the gold standard type studies?
Dear Joey,
I’ve recently written the following paper you might find interesting. It’s the first essay I’ve written for my PhD and, as the title suggests, I try to understand what a billionaire EA should do to maximise happiness. In brief, my conclusion is that mental health and happiness interventions look a lot more promising that our present anti-poverty and anti-malaria ones.
https://www.academia.edu/25088361/What_Should_A_Utilitarian_Billionaire_Do_To_Maximise_Happiness
I’ve been meaning to put something on the EA forum on this topic for a while but haven’t got around to it (because I don’t really use/understand the EA Forum and saw this on facebook).
Regarding the conclusions you reach:
-I’m actually sceptical that Conditional Cash Transfers increase happiness at all and it’s therefore questionable how useful they are. I don’t think we have enough evidence yet to know: we’d need to track people’s experiences of happiness over at least 6 months. (I’ve previously discussed this with Rob Wiblin and there might be an argument about the long-run economic benefits of CCTs that I’d like to see in made in writing).
-So, I think research in the effectiveness of poverty looks very good!
-I didn’t consider tobacco taxation or micro-nutrient fortification but my guess is that these are unlikely to change people’s experiences of happiness by much either. I’d suppose that a smoking reduction would help people live longer but there’s a further value question of how much you weight helping people live longer (I take the epicurean view on death, which means I’m uncertain there’s any value in prolonging life, but feel free to ignore this).
-I didn’t consider anything about SMS immunisation reminders. Intuitively that does look quite promising from the modules I took into behavioural econ a couple of years back, but I’m sure your knowledge is much more up to do that mine and you’ve harvested everything you can from the Behavioural Insights Team and so on. If you would like to talk to some (more) behavioural economists I have at least one friend working in the area I could put in touch with.
I hope that’s helpful. If you have any questions I’m best reached via the facebook or michael.plant@philosophy.ox.ac.uk
Thanks for the thoughtful response! Some thoughts on these that I will also email you but thought that other folks on the EA Forum might like being kept in the loop.
With regards to your essay, I really like it and found a lot of parts very interesting. This is in fact an area we looked into a little bit, although did not write much on it. I strongly agree that ‘internal happiness interventions’ (‘IHIs’) are extremely neglected in contrast to the ‘external happiness interventions’ (‘EHIs’) compared to what a rational utilitarian should value.
I did end up with a different conclusion on the issue, although I still think the case for this is strong, I just do not see it as quite as strong as the listed poverty options. In general I think it’s very hard to compare the possible expected value of research where it is very hard to get an estimate (e.g. IHI research) to the expected value of a more direct intervention (e.g. AMF). I think we mainly differed in how we weighted the Easterlin paradox and the expected value difference. I see how you came up with the 10-18 number but think that it was generated using a pretty favourable example given the DALY weightings I have seen for different things. For more info, check out Jeff Kauffman’s handy listing of disabilities by weighting https://www.jefftk.com/p/disability-weights.
Although the endline idea of online CBT/mindfulness is very interesting, I also have major concerns about evidence about even the offline versions of these and would expect the effect on happiness to be smaller than is currently stated in many studies if they were stronger studies/less publication bias etc. For example, the following review on mindfulness research found that of 21 pre-registered RCTs on mindfulness meditation, only 38% were published 30 months after registration, showing worrying signs of publication bias, and the median sample size of 124 RCTs was 54, including controls (http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0153220).
I have a lot of sympathy for your argument about death being over-valued (and we are aiming to publishing a blog post on this at some point), but it’s worth taking into account the happiness effects on a parent of losing a child (which can still make reducing death very significantly net positive.) This is even without taking into account the economics arguments mentioned.
I think a combination of all the points made above make me think that while this is a promising area for research, I would give it a smaller chance of revealing something more cost effective than AMF than I would give to doing more direct poverty research (on say SMS or CCT).
I would be very interested in talking more about this as I can imagine my views changing quite a bit. Particularly it would be great to talk about what research you plan on doing going forward on it.
Re: conclusions
CCTs not increasing happiness at all would surprise me. The cash component of the disagreement might come from different general views about the evidence that increased income leads to happiness. But a huge part of the benefit would be from the activity that is being CCTed. For example, even if the cash component of CCT for vaccines only lead to short term happiness, the vaccines might lead to ongoing happiness (details would largely dependent on the condition). There is substantial evidence that parents mourn the passing of a child, and thanks for medical advances, even in the developing world losing a child is a relatively rare event, so it’s unlikely there’s as strong a hedonic adaptation effect.
I do agree that happiness is not nearly as tracked as I would like and I would want to see long term happiness data in conjunction with health/wealth data for any intervention/RCT we end up doing. I think this is particularly true for CCTs, but I would love to see more happiness data almost everywhere.
Research was hard. Some people seemed to think it was much more effective than any direct options and others thought the opposite. I still do not have very confident views despite the time I have put into researching it. Would be super interested if you or anyone else had a cost effectiveness estimate for poverty research based on historical data that might be able to shed some light on the question. The biggest question mark is to what extent research affects where resources are spent counterfactually, rather than simply providing a justification for existing programs to continue.
On tobacco, even if one takes the epicurean view on death there seems to be considerable suffering caused to relatives from premature death (both from sadness and lost income if the person is still working age or due to widows being treated terribly in many cultures). When thinking about how much to value someone’s life we both took the above into account and the average happiness of the population we would be helping.
With regards to iron, this is in fact one of the ones we are more confident is connected directly to happiness. Anemia is quite correlated with negative emotional states relative to other health interventions we considered. It is so cost-effective than even small benefits in happiness really add up.
For SMS would be happy to talk to any behavioural economists who you think might be informed in the area.
I wonder if you should add a criteria for ‘personal fit’? Something can be good in general, but not good for you. I’m not sure how that would change the results though.
I agree personal fit is important. Although we do not have an explicit separate category for this, we definitely took it into account when evaluating options.
I guess I’m surprised to see tobacco taxation up there on personal fit grounds. While I think it’s a great cause in general, I think it would be a many-year effort to gain the credibility and local knowledge to be able to influence Indian politics, especially when big money is going to be running against you.
For even the easiest to execute interventions we would have to hire domain and country experts. Any operation that involves government lobbying (tobacco and fortification) would require hiring a lobbyist who already has established connections and local knowledge. I do not feel like hiring a tobacco expert would be substantially more difficult than hiring a lobbyist for any other topic or hiring a very qualified expert for another intervention.
OK if you’re planning to get in external experts it makes a lot more sense.
On SMS immunization reminders In case it is useful (it may not be) this reminded me of the UK Behavioural Insights Team’s work on: what’s the best messages to send through SMS. On the one hand this at least shows that varying the messages is both important and testable, on the other hand maybe this may be too far removed from developing world health to be useful. One such example: http://www.behaviouralinsights.co.uk/trial-results/reducing-missed-appointments/ (PS. If you wanted to get in touch with them at least 1EA works there)
I am wondering if there has been any research into reducing suicide rates. These may include:
Measuring effectiveness of current suicide hotlines
Training suicide hotline volunteers in more effective forms of communication
Decreasing the over-head of calling a hotline
(long-term) Research into co-relations with suicide and early warning signs.
Hi pasha,
Suicide prevention is an extremely neglected area and I believe has many high impact opportunities that are not yet taken up. 85% of suicides are in the developing world, little of which are covered by helplines, so I would think proliferating helplines would be high impact, especially when you factor in the low opportunity cost of this volunteer-based activity. India, in particular, is desperate for more helplines. One way of reducing cost is by having calls to the helpline automatically directed to volunteers’ phones, so that they don’t need to have a call centre.
Means restriction has by far the best evidence behind it for reducing suicide rates, particularly prohibition of highly toxic pesticides (currently the second most common means of suicide). Pesticide bans have decreased Sri Lanka’s suicide rate by three quarters and China’s suicide rate has decreased by more than half due to reduced pesticide access from bans and urbanization. My NGO, the Global Initiative for Pesticide Poisoning Prevention is currently being formed to advocate for more bans of highly poisonous pesticides around the world. Our website should be live in a few weeks at www.pesticidepoisoning.org.
Pasha & Austen, fantastic suggestions. Pasha, I’d be happy to do some research on effectiveness of hotlines or brainstorm proxies with you. Austen, I’m really excited to hear more about what you’re doing with PPP. I’m quite interested in suicide prevention and I think the neglectedness and avertable suffering and death make it an essential focus for our time and effort. I think that most in the EA community (as well as global health more broadly) question tractability, which is all the more reasons to demonstrate pathways to effective reduction in suicide. I would love to hear more about what you’re doing, and potentially get involved. I’m hoping to get together folks in EA who are interested in global mental health. Please let me know if you’d be willing to chat!
I do not know of any research in this area offhand, but I would expect there to be some online. I would be interested in seeing a cost-effectiveness estimate on this if you found one.