Error
Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?
It’s good that you are sharing the research effort you put into this so that others can critique it, use/reference it, and build on it.
I have assorted comments below with quotes they are responding to.
Some context for my view: while I think there is a strong case for this in terms of additive total utilitarianism, I think the dominance is far weaker when one takes value pluralism and normative uncertainty into account. GiveWell staff have sometimes talked about whether decisions would be recommended if one valued the entire future of civilization at a ‘mere’ 5 or 10 times the absolute value of a century of the world as it is today. Value pluralism is one reason to apply such a heuristic.
The argument there being that for most risks GCR versions are much more likely than direct existential risk versions, and GCRs have some chance of knock-on existential harms. Note that AI risk was excepted there, and has been noted as unusual in having a closer link between GCR and existential risk than others.
New staff members and entrepreneurs are very important in many cases. E.g. the EA movement has supplied a lot of GiveWell/OpenPhil staff, and founders for things like Charity Science and ACE which you mention.
Some of this is definitely the recent surge of progress in AI, e.g. former AAAI president Eric Horvitz mentions that this was important for him and others.
For MIRI’s causal influence some key elements I would highlight are:
The Singularity Summit playing a key causal role in getting Max Tegmark interested and the FLI created.
Bringing the issue to Stuart Russell’s attention, resulting in Stuart’s activity on the issue, including discussion in the most popular AI textbook, his involvement with the FLI grant program, etc.
Contributing substantially to Nick Bostrom’s publication of Superintelligence, which played a key role in getting Elon Musk involved (and thus funding the FLI grant program), and eliciting favorable reviews from various others (e.g. Stephen Hawking, Bill Gates, etc.
The technical agenda helping to demonstrate some approaches that could work.
Drawing attention to the issue by a number of the academic researchers who have taken FLI grants, and some of OpenPhil’s advisors.
Causing OpenPhil to be quite familiar with the issues, and ultimately to enter the area after seeing the results of the FLI conference, getting a sense of expert opinion, etc, as discussed on their website.
ETA: OpenPhilanthropy has now just put up a detailed summary of the reasoning behind the FLI grant which may be helpful. They also talk about why they have raised their priority for work on AI in this post.
This is an issue that will recur on any area where OpenPhil/GiveWell is active (which will shortly include factory farming with the new hire and grant program). Here are two of my posts discussing the issues, (the first has important comments from Holden Karnofsky about their efforts to manage ‘fungibility problems)’.
One quote from a GiveWell piece:
Also, you likely won’t have zero effect, but would likely shift the budget constraint, so you could think of your donation as expanding all of Good Ventures’ grants roughly in proportion to their size, which will be diversified and heavy on GiveDirectly. Or at least you could do that if they all had similar diminishing returns curves. If some have flatter curves (perhaps GiveDirectly) in Good Ventures’ calculus then marginal funds would go disproportionately to those.
That’s a surprising claim. Probably it would recommend an existing charity. Maybe what you mean is that your expected value for any given GCR charity given what you know now is less than your expectation would be for the charities OpenPhil will recommend, given knowledge of those recommendations?
Or maybe you mean that OpenPhil’s recommendations are likely to be charities that exist but that you currently don’t know of?
My comment was too long to fit in the 1000 word limit, so the remainder is below.
My comment was too long, so here’s the rest:
Is this something like unweighted QALYs per dollar? If you are analyzing in terms of long-run effects on the animal population, as elsewhere in the piece, those QALYs are a red herring. E.g. a tiny increase in economic activity very faintly expediting economic growth will overwhelm the direct QALYs involved with future populations. From the long-run point of view things like the changes in economic output, human populations, carbon emissions, human attitudes about other animals, and such would be the relevant metrics and don’t scale with QALYs (this is made blatantly clear if one consider things like flies and ants). From the tiny-animal focus (with no accounting for differences in nervous system scale), the large farm animals will be neglible compared to various effects on tiny wild animals. If one considers neural processes within and across animals, then the numbers will be far less extreme.
Now, as I said at the start of this comment, normative pluralism and such would suggest not allowing complete dominance of long-run QALYs over current ones, but comparisons in terms of QALYs here don’t track the purported long-run impacts, and if one focused only on unweighted animal QALYs without worrying about long-run consequences it would lead one away from farm animals towards wild animals.
Not true. Previously GiveWell had capped Good Ventures contributions at 20% of GiveWell’s budget. Recently they changed it to 20% for GiveWell’s top charities work, and 50% for the Open Philanthropy Project (reasoning that Good Ventures is the main customer of the latter at this time, so it is reasonable for it to bear a larger share).
Well nothing is going to force Good Ventures to hand over billions of dollars if it disagrees with the OpenPhil recommendations (and last year there was some disagreement between GW and GV about allocations to the different global poverty charities). But this does seem like a serious consideration to support outside donation to OpenPhil, and I think you may be underrating this donation option.
You only consider the case where it finds that all the current popular animal interventions are very poor. If many or most but not all are, then it could support productive reallocation from the ones that don’t work as well to the ones that work better, potentially multiplying effectiveness severalfold. That’s in fact the usual justification given by people in the animal charity community for doing this kind of research, but doesn’t appear at all here. So I think the whole discussion of #3 has gone awry. Also the ‘several orders of magnitude’ claim appears again here, and the issues with QALYs vs metrics that better track long-run changes (e.g. attitude changes, population changes, legal changes) recur.
Although note that that is valuing staff time at below minimum wage. If you valued it at closer to opportunity cost (or salaries at other orgs) the ratio would be far lower. I still think Charity Science is promising and deserving of support because of the knowledge it has produced, and I suspect its fundraising ratios will improve, but at the moment the ratio of EA resources put in to fundraising success is still on the lower end. See this discussion on the EA facebook group.
Shouldn’t the same caveat apply to your suggestions earlier in the post about the future being 1000+ times more important than present beings?
I will make a few edits to the document based on your suggestions, thanks.
I have a few more points worth discussing that I didn’t want to put into the doc, so I’ll comment here:
Why does value pluralism justify reducing the importance of the far future? It seems unreasonable to me to discount the far future and I find it very implausible that beings in the far future don’t have moral value.
On the flow through effects of global poverty over animal charities: Flow through effects probably do outweigh short-term effects, which means economic growth, etc. may be more impactful than preventing factory farming. But flow-through effects are hard to predict. I meant that effective factory farming interventions probably have much better demonstrable short-term effects than human-focused interventions. Actions that affect wild animals probably have much bigger effects, but again we run into the problem of not knowing whether our actions are net positive or negative. I’d certainly love to see some robust evidence that doing X will have a huge positive effect on wild animals so I can seriously consider supporting X.
I didn’t say ACE recommendations are two to three orders of magnitude better than GiveWell Classic, I said Open Phil’s factory farming grant is plausibly two to three orders of magnitude better than GV’s grant to GiveDirectly. There are three distinctions here. First, I’m fairly uncertain about this. Second, I expect Open Phil’s grant to be based on more robust evidence than ACE-recommended charities, so I can feel more confident about its impact. Third, GD has similar strength of evidence to AMF and is probably about an order of magnitude less impactful. So the difference between a factory farming grant and AMF may be more like one or two orders of magnitude.
I weighted ACE recs as 2x as impactful as GiveWell recs; Open Phil hasn’t produced anything on factory farming but my best guess is its results will have an expected value of maybe 2-5x that of current ACE recs (although I expect that the EV of ACE recs will get better if ACE gets substantially more funding), largely because a lot of ACE top charities’ activities are probably useless—all their activities look reasonable, but the evidence supporting them is pretty weak so it’s reasonable to expect that some will turn out not to be effective.
(EDIT: I’d also add that even if I’m fairly confident about a 100-1000x effect size difference from inside an argument, when weighting donations I should take the outside view and not let these big effect sizes carry too much weight.)
After further consideration I’m thinking I rated GiveWell recs too highly; their weighting should be more like 0.05 instead of 0.1. Most of REG-raised money for GiveWell top charities went to AMF, although this might shift more toward GiveDirectly in the future in which case I should give GW recs a lower weight. I would probably rate Open Phil factory farming grants at maybe 0.3-0.5, which is an order of magnitude higher than for GiveWell top charities.
When I change the GW rec weighting from 0.1 to 0.05, the weighted donations drop by about $0.1 per $1 to REG. That’s enough to make REG look a little weaker, although not enough to make me want to give to an object-level charity instead.
EDIT 2: Actually I’m not sure I should downweight GW recs from 0.1 to 0.05 because I don’t know that I have strong enough outside-the-argument confidence that MIRI is 20x better than AMF in expectation. This sort of thing is really hard to put explicit numbers on since my brain can’t really tell the difference between MIRI being 10x better and 100x better in expectation. My subjective perception of the probabilities of MIRI being 10x better versus 100x better feel about the same.
Others think that we have special obligations to those with whom we have relationships or reciprocity, who we have harmed or been benefited by, or adopt person-affecting views although those are hard to make coherent. Others adopt value holism of various kinds, caring about other features of populations like the average and distribution, although for many parameterizations and empirical beliefs those still favor strong focus on the long-run.
Right, sounds good.
I find all those views really implausible so I don’t do anything to account for them. On the other hand, you seem to have a better grasp of utilitarianism than I do but you’re less confident about its truth, which makes me think I should be less confident.
On his old blog Scott talks about how there are some people who can argue circles around him on certain subjects. I feel like you can do this to me on cause prioritization. Like no matter what position I take, you can poke tons of holes in it and convince me that I’m wrong.
The fact that Carl points out flaws with arguments on all sides makes him more trustworthy!
These comments are copied from some of the original ones I made when reviewing Michael’s post. My views are my own, not GiveWell’s.
I think the case for values spreading is quite a bit better. Reducing global catastrophic risks is pretty bimodel. Either the catastrophe happens, or it doesn’t. You can try to measure the risk being reduced, sometimes, but doing so isn’t straightforward, obvious, or something we have experience in.
We have lots of experience tracking value change. We can see it happen in incremental parts in the near-future. You don’t need special tools or access to confidential information to do a decent poll on values changing.
The strongest objection to this, I think, is that values changing in the short term won’t necessarily affect the long-term trajectory of our values, or at least not in a predictable way. In contrast, preventing an x-risk in the short term at least allows for the possibility of doing stuff in the far future (and it seems plausible that GCRs might also change far-future trajectory).
Another consideration is that values may become vastly more or less mutable if we develop technology that allows of certain types of self-modification, or an AI that enforces values that are programmed into it. Depending on how you believe this might happen, you might believe spreading good values before those technologies develop is vastly more or less important, exactly because then the likelihood of those values affecting the far future increases.
I think a lot of GCRs could be more tractable than AI risk (possibly by a large margin) if someone went through the work of identifying more opportunities to fund risk reduction for those GCRs, then made it available to small donors.
This is definitely an important point. I think that if someone did identify opportunities like this, that’s one of the most likely reasons why I might change where I donate. Right now it doesn’t look like any GCR is substantially more important/tractable/neglected than AI risk (biosecurity is probably a bigger risk but not by a huge margin, geoengineering might be more tractable but not for small donors), but this could change in the future.
Thanks for writing this, Michael. More people should write up documents like these. I’ve been thinking of doing something similar, but haven’t found the time yet.
I realized reading this that I haven’t thought much about REG. It sounds like they do good things, but I’m a bit skeptical re: their ability to make good use of the marginal donation they get. I don’t think a small budget, by itself, is strong evidence that they could make good use of more money. Can you talk more about what convinced you that they’re a good giving opportunity on the margin? (I’m thinking out loud here, don’t mean this paragraph to be a criticism.)
Re: ACE’s recommended charities. I know you know I think this, but I think it’s better for the health of ACE if their supporters divide their money between ACE and its recommended charities, even if the evidence for its recommended charities isn’t currently as strong as I’d like. But I admit this is based on a fuzzy heuristic, not a knock-down argument.
Re: MIRI. Setting aside what I think of Yudkowsky, I think you may be overlooking the fact that that “competence” is relative to what you’re trying to accomplish. Luke Muehlhauser accomplished a lot in terms of getting MIRI to follow nonprofit best practices, and from what I’ve read of his writing, I expect he’ll do very well in his new role as an analyst for GiveWell. But there’s a huge gulf between being competent in that sense, and being able to do (or supervise other people doing) ground breaking math and CS research.
Nate Soares seems as smart as you’d expect a former Google engineer to be, but would I expect him to do anything really ground breaking? No. Would I expect even the couple actual PhDs MIRI hired recently to do anything really ground breaking? They might, but I don’t see why you’d think it likely.
In a way, it was easier to make a case for MIRI back when they did a lot of advocacy work. Now that they’re billing themselves as a research institute, I think they’ve set a much higher bar for themselves, and when it comes to doing research (as opposed to advocacy) they’ve got much less of a track record to go on.
Thanks for bringing this up, Topher!
As Michael said, there are various things we would do if we had more funding.
1) REG’s ongoing operations need to be funded. Currently, we have around 6 months of reserves (at the current level of expenses), but ideally we would like to have 12 months. This would enable us to make use of more (sometimes unexpected) opportunities and to try things because we wouldn’t have to constantly be focused on our own funding situation.
2) We could potentially achieve (much) better results with REG by having additional people working on it. The best illustration of this is probably one person that we met (by going to poker stops) with a strong PR & marketing background who’s been working in the poker industry for 10 years now (there are not that many people with a level of expertise and network about the poker world like this person). This person woud like to work with us, but we had to decline her for the moment, even though we think that it would (clearly) be worth it to hire her. Another thing we would like to do is hiring someone to organise more charity tournaments and establish partnerships with industry leading organisations or strengthen existing ones, improve member communications and do social media. There are already several candidates who could do this, but we are hesitant to make this investment since we lack the appropriate funding.
3) Another way we would use additional funds is by working on various REG “extensions”. We are about to set up two REG expansions, but we won’t have enough resources to make the most out of even these two – and there are many more potentially really promising REG expansions that could be done. (The first of the two REG expansions that is likely going to be spread among the respective community in a few days is “DFS Charity”, a REG for Daily Fantasy Sports, an industry that is currently growing substantially and with a fair share of people with a similar (quantitative) mindset as poker players have. The preliminary website can be found at dfscharity.org – please don’t share it widely yet.)
I hope this helped!
To put this in context, the emerging consensus is that publicly advocating for x risk reduction in the area of AI is counterproductive, and it is better to network with researchers directly, something that may be best done by performing relevant research.
What are the best groups that are specifically doing advocacy for (against?) AI risk, or existential risks in general?
If I had to guess, I would guess FLI, given their ability to at least theoretically use the money for grant-making. Though after Elon Musk’s $10 million, donation this cause area seems to be short on room for more funding.
Although FLI were only able to grant a very small fraction of the funds that researchers applied for, and many organisations have scope for expansion beyond the grants they recieved.
I asked Tobias Pulver about this specifically. He told me about their future plans and how they’d like to use marginal funds. They have things that they would have done if they’d had more money but couldn’t do. I don’t know if they’re okay with me speaking about this publicly but I invite Tobias or anyone else at REG to comment on this.
If ACE thought this was best, couldn’t it direct some of the funds I donate to its top charities? (This is something I probably should have considered and investigated, although it’s moot since I’m not planning on donating directly to ACE.)
AI safety is such a new field that I don’t expect you need to be a genius to do anything groundbreaking. MIRI researchers are probably about as intelligent as most FLI grantees. But I expect them to be better at AI safety research because MIRI has been working on it for longer and has a stronger grasp of the technical challenges.
They claim to be working on areas like game theory, decision theory, and mathematical logic, which are all well-developed fields of study. I see no reason to think those fields have lots of low-hanging fruit that would allow average researchers to make huge breakthroughs. Sure, they have a new angle on those fields, but does a new angle really overcome their lack of an impressive research track-record?
Do they have a stronger grasp of the technical challenges? They’re certainly opinionated about what it will take to make AI safe, but their (public) justifications for those opinions look pretty flimsy.
I heard—all be it second hand, and last year—of two people involved, Lukas Gloor and Tobias Pulver, saying that thought that the minimal share of GBS/EAF manpower − 1.5 FTEs—that was being invested in REG was sufficient.
Thanks so much for writing this. I agree with your arguments and I find your conclusion fairly persuasive.
I wanted to say thank you for this. There’s always a tradeoff between reporting on what you’re doing and getting on and doing more stuff, but this was a good reminder to look at whether we’re getting the balance right, and I think we’re going to devote a bit more effort to transparency.
That’s good to hear! GPP looks promising and I’d like to see it talk more publicly about its activities.
Great post! Just a quick clarification, I definitely think AR research is worth doing but it would be better under a different organization/brand/startup . I think its valuable to keep an organization fairly focused on doing a few things well, and AR research is definitely not in the CS scope.
Really quick question: I was wondering why the 1.5:1 ratio is enough to outweigh your uncertainty about REG’s impact?
Well that’s certainly a concern. I’m made more confident by the fact that REG directs funding to multiple charities that are good candidates for top charity, and I believe their model has reasonably good learning value. Plus 1.5:1 is sufficiently higher than 1:1 that I believe it’s more likely to have a positive multiplicative effect from outside view.
I’m not sure I understand. I would think that in the face of uncertainty it would be better to divide donations in accordance to how likely we find each model.
Surely that depends on the level of uncertainty?
I read this post today after first reading a significant portion of it on ~December 2nd, 2019. I’m not sure my main takeaways are from reading it, but wanted to comment to say that it’s the best example I currently am aware of someone explaining their cause prioritization reasoning when deciding where to donate. Can anyone point me to more or better examples of people explaining their cause prioritization reasoning?
Some other related links I found helpful:
Vipul Naik’s “My 2018 donations”: https://forum.effectivealtruism.org/posts/dznyZNkAQMNq6HtXf/my-2018-donations
Adam Gleave’s “2017 Donor Lotter Report”: https://forum.effectivealtruism.org/posts/SYeJnv9vYzq9oQMbQ/2017-donor-lottery-report
Brian Tomasik’s “My Donation Recommendations”: https://reducing-suffering.org/donation-recommendations/
https://forum.effectivealtruism.org/posts/Z6FoocxsPfQdyNX3P/where-some-people-donated-in-2017
What was your final decision on this?
I made the donation to REG about a week ago.
Another reason to like REG: I expect bringing more poker players in to the EA movement will be good for our culture if poker is effective rationality training. (But I still think a profession where people are paid to make accurate predictions, say successful stock pickers, could be even better.)
How big of a difference do you think this makes? I don’t expect that bringing in high-rationality people is a particularly big consideration (I wouldn’t fund it over something like MIRI or even AMF) although I agree that it’s a small bonus.
“I don’t expect that bringing in high-rationality people is a particularly big consideration”—this is probably a point where we disagree. I’ve previously written about this here, here, and, in relation to the idea of effective altruists pursuing systemic change, in the comments of this post.
Let’s contrast top gamblers like competitive poker players with bottom tier gamblers: people who play the lottery, even though it’s negative expected value, and happen to win. Let’s say the same amount will be donated either way, so the difference is just whether it’s going to be directed by top tier or bottom tier gamblers. Imagine yourself reading over a cause selection piece from a top poker player vs a lottery winner… which cause selection piece do you anticipate learning something from? Being persuaded by? Which type of donor are you more confident will actually improve the world with their money vs doing something that sounds nice but isn’t actually very effective, or worse, amounts to shooting ourselves in the foot in the long run?
I sometimes wish people in the EA movement would taboo the concept of EA. EA isn’t some magic pixie dust you can sprinkle on someone such that they are automatically effective at doing good for the world. There’s a sense in which the wisdom of the EA movement as a whole is the sum of collective wisdom of the people who are in the movement. Adding wise people has the potential to improve the movement’s wisdom on the margin.
That’s actually a really good point. I had been considering that most rational people don’t do much good, so you need more than just rationality. But for something like REG where you’re drawing in charitable and altruistic people, it’s extremely valuable for those people to be as rational as possible.
Providing such an in depth writeup is really useful, thanks. At the risk of derailing into an academic philosophy discussion, here are some clarificatory questions about what you value (which I’m particularly interested in because I think your values are relatively common among EAs):
Why do you think that these are the only things of value?
Leaving aside (presumably hypothetical) computer simulations and artificial biological beings, do you think non-humans like chickens and fish have equally bad experiences in a month in a factory farm as a human would? If not, roughly how much worse or less bad would you guess they are? (I’m talking about a similar equivalence to that described in this Facebook poll, but focusing purely on morally relevant attributes of experiences.)
Can you give an example of the ideal form of joy? Would an intense, simple experience of physical pleasure be a decent candidate? (Picking an example of such an experience could be left as an exercise for the reader.)
What’s the most unintuitive result that you’re prepared to accept, and which gives you most pause?
The great thing about nested comments is derailments are easy to isolate. :)
I don’t understand what it would mean for anything other than positive and negative experiences to have value. I believe that when people say they inherently value art (or something along those lines), the reason they say this is because the thought of art existing makes them happy and the thought of art not existing makes them unhappy, and it’s the happy or unhappy feelings that have actual value, not the existence of art itself. If people thought art existed but it actually didn’t, that would be just as good as if art existed. Of course, when I say that you might react negatively to the idea of art not existing even if people don’t know it exists; but now you know that it doesn’t exist so you still experience the negative feelings associated with art not existing. If you didn’t experience those feelings, it wouldn’t matter.
I expect there’s a high probability (maybe 50%) that factory farms are just as bad for chickens as they are for humans, and a somewhat lower probability (maybe 25%) that they are just as bad for fish. I expect it’s more likely that factory farms are worse for humans than that they’re worse for chickens/fish, so in expectation, they’re worse for humans, but not much worse.
I don’t know how consciousness works, although I believe it’s fundamentally an empirical question. My best guess is that certain types of mental structures produce heightened consciousness in a way that gives a being greater moral value, but that most of the additional neurons that humans have do not contribute at all to heightened consciousness. For example, humans have tons of brain space devoted to facial recognition, but I don’t expect that we can feel greater levels of pleasure or pain as a result of having this brain space.
The best I can do is introspect about what types of pleasure I enjoy most and how I’m willing to trade them off against each other. I expect that the happiest possible being can be much happier than any animal; I also expect that it’s possible in principle to make interpersonal utility comparisons, so we could know what a super-happy being looks like. We’re still a long way away from being able to do this in practice.
There are a lot of results that used to make me feel uncomfortable, but I didn’t consider this good evidence that utilitarianism is false. They don’t make me uncomfortable anymore because I’ve gotten used to them. Whichever result gives me the most pause is one that I haven’t heard of before, so I haven’t gotten used to it. I predict that the next time I hear a novel thought experiment where utilitarianism leads to some unintuitive conclusion, it will make me feel uncomfortable but I won’t change my mind because I don’t consider discomfort to be good evidence. Our intuitions are often wrong about how the physical world works, so why should we expect them to always be right about how the moral world works?
At some point we have to use intuition to make moral decisions—I have a strong intuition that nothing matters other than happiness or suffering, and I apply this. But anti-utilitarian thought experiments usually prey on some identifiable cognitive bias. For example, the repugnant conclusion takes advantage of people’s scope insensitivity and inability to aggregate value across separate individuals.
Woaha, I didn’t realize that anyone thought that, it would make me change my views greatly if I did.
Impressive analysis. But what about the Global Catastrophic Risk Institute or the Copenhagen Consensus Center? Disclosure: I am an associate at GCRI.
GCRI is probably worth looking into. My first impression is it’s pretty similar to FHI and CSER and there’s nothing that make GCRI look clearly better than FHI/CSER. I do think it’s pretty unlikely that I would end up preferring GCRI to REG/MIRI/ACE, so I haven’t prioritized investigating it.
It hadn’t occurred to me to look into the Copenhagen Consensus Center. Based on what I know about it, there are few factors working against it:
It doesn’t appear to be funding constrained.
It does prioritization work on global poverty only, which is probably not the most effective cause area.
Its prioritization work on global poverty is probably not as useful as GiveWell’s.
It recommends interventions instead of specific charities. Implementation matters a lot—you shouldn’t support a poor implementation of a good intervention. GiveWell is more useful for this reason.
The big factor in CCC’s favor is it could move a lot of money (it has potentially moved about $5 billion, although this is probably optimistic). This might actually be sufficiently valuable to make CCC worth supporting. CCC is seeking public donations, but there’s still the big question of how donations translate into better recommendations.
Here’s a few questions I’d need to answer before feeling comfortable donating to CCC:
How do donations translate into better recommendations or more money moved?
How much money does it move?
How much better is the money moved compared to the counterfactual?
How effective is CCC’s money moved compared to GiveWell top charities, or compared to my favorite charities?
Right now it looks sufficiently unlikely that CCC is the best donation target that I don’t think it’s worth it for me to look into more.
Well, GCRI is much more funding constrained than FHI or CSER.
Great job writing this up Michael. I’d like to see many more people explaining their reasoning like this.
I was a bit surprised, however, to see 80,000 Hours as listed as “unclear has positive effective” when the charity you conclude is best, REG, likely wouldn’t exist if it weren’t for 80,000 Hours.
Similarly, your other finalist, ACE, is a spin-off of 80,000 Hours.
https://80000hours.org/about/impact/new-organisations/
A number of other organisations on your shortlist have also been boosted by us, including: CSER (received seed funding from someone etg in part due to us), Charity Science (Joey and Xio are 80k plan changes, they also received seed funding from etg donors, they recently hired someone who decided to work in EA orgs due in part to 80k), GWWC (recently hired someone who switched to EA orgs in part due to us), FHI (now managed by Niel Bowerman, an 80k plan change)...
This sounds worryingly close to claiming credit for all “etg donors”, all EAs’ careers and all EA organisations that have had some contact with EA organizations. Of course people like Jonas Vollmer are going to say nice things about 80,000 Hours when asked, and it would be impolitic for any organisation to challenge this, so I’ll say it: I don’t think all of GBS Switzerland’s activities can be classed as counterfactually dependent on 80,000 Hours getting funding. Likewise the volunteers who founded Effective Animal Activism (the predecessor of ACE) or CSER or Effective Fundraising (the predecessor of Charity Science) might have done so at some point anyway, for all I know, and it’s hard to buy their saying otherwise as unbiased.
This isn’t to single out 80,000 Hours as the only organisation with these murky counterfactuals, it’s only jumping off your comment. I’ve likewise heard people say that people were running fundraisers before Charity Science started recruiting people to do so and that people were giving (or, if students, planning to) before signing up to Giving What We Can’s list, and that neither organisation can claim credit for everything these people then go on to do.
I agree the counterfactuals are murky, so I’d never say it was 100% due to us. Nevertheless, I think we played a significant role.
We also certainly don’t claim credit for all etg donors, only those who say they were influenced by us and made a significant plan change (something like 25-50% of the total).
Thanks for writing this Ben! I would like to see more representatives from orgs giving cases like this one where you go beyond saying “we’re high impact” and explain why you believe you’re the most high impact.
Here’s the main reasons why I didn’t consider 80K further:
Based on my prior knowledge of 80K and the brief time I spent investigating it, I didn’t see good evidence that 80K played an important causal role in pushing people toward substantially better careers. Similarly, I don’t see much reason to believe that those organizations you listed wouldn’t have happened without 80K.
Some of 80K’s recommendations confuse me and seem wrong. I agree with Peter Hurford’s recent post about the importance of earning to give. I’m concerned because 80K’s current stance on etg looks fairly obviously wrong, and everyone I’ve talked to about this whose opinion I highly respect has agreed that it looks fairly obviously wrong. More generally, 80K’s public info on career recommendations focus more on individual fit and don’t say much about how much good different careers do or how to do maximal good through those careers.
It seems dubious that 80K could continue to have as large an impact as you claim it has had in the past.
80K is funded by YC and does not have clear room for more funding. I don’t know what 80K could do if they had more money from me that it can’t do now, and my donations may displace the donations of other donors.
I am open to considering donating to 80K more seriously if you can address these concerns and also give good reason to believe that the way 80K directs people’s careers is likely to have a larger positive impact on the long term future than e.g. reducing AI risk. It’s not obvious to me that 80K has a multiplicative effect in the same way REG does.
Hi Michael,
On 1) have you seen our evaluation documents? https://80000hours.org/about/impact/ Why don’t you think we’re moving people towards higher impact careers?
On organisations in particular you say “I don’t see much reason to believe that those organisations wouldn’t have happened without 80k”. The founders of those organisations say they likely wouldn’t have existed otherwise, why do you think the founders are wrong?
With ACE in particular: we came up with the idea, it was started by an intern working at 80k and initially housed within 80k, we introduced them to their first seed donors, and an 80k team member continues to play a role on the board.
2) I’m not sure Peter Hurford and 80k actually disagree on the proportion of ppl who should do etg. We say 15-25% in the long-run. He says 50% or perhaps higher, but then in the comments he clarifies (in the reply to AGB) that he means 50% of those choosing between etg and direct work, and not counting those going into academic, policy, grant making etc. If you suppose 50% of people will do that, then Peter thinks 25% of people should etg all considered, in line with our estimate.
There’s also a few other differences in how we each frame the question and define etg which could easily explain remaining differences (see Will’s comments). I also listed a bunch of problems with Peter’s arguments on the thread which he didn’t yet address.
Our career research in general is highly focused on which careers do the most good (in the past we’ve mainly received criticism that we focus on personal fit too little—it’s quite hard to say what’s best to focus on if you want to maximise long-run impact https://80000hours.org/2014/10/interview-holden-karnofsky-on-the-importance-of-personal-fit/). We only list personal fit as one factor in our key principles: https://80000hours.org/career-guide/basics/ Our career reviews discuss impact just as much as personal fit: https://80000hours.org/career-guide/profiles/
On 3), that’s a very big claim. Why? I expect the vast majority of 80k’s impact lies in the future. There’s the potential to develop a GiveWell-analogue but for career choice for all socially motivated graduates.
On 4), YC only provides $100,000 of funding once, so being YC-funded doesn’t mean we never need to fundraise again.
However, it’s true we haven’t publicly said we have room for more funding, so you have no way to know. I think we do have a large room for more funding though.
I think we have a multiplicative effect exactly like REG does, except we direct people to better careers rather than directing money.
On 1)
That’s not exactly what I said. What I said is that I don’t think there’s strong evidence that 80K is moving people toward higher impact careers.
80K’s impact page lists a bunch of career changes that people made after talking to 80K. But it’s not clear how many of these changes would have happened anyway or how much value 80K provided in the process. You also have to consider things that aren’t happening. If 80K claims credit for money donated by people who are now earning to give, then it should also subtract money not donated by people who now aren’t earning to give. The value of a career change isn’t from the value of the person’s career but from the marginal difference between their current career and their counterfactual career.
80K has moved people in lots of different directions and there’s no clear pattern I can see from public data. I’d expect that some careers are considerably more important and neglected than others, and 80K should be pushing people toward these careers in general, but I don’t see this happening.
On 3), if you believe most of 80K’s impact comes from helping start new effective charities (which you sort of imply I should believe when you claim that ACE and REG would not exist without 80K), then we should expect this effect to get a lot weaker in the future. I don’t think 80K played as big a role in creating ACE and REG as you say it did (there was a lot of demand for something like ACE when it came about so something similar probably would have arisen soon), but even if it did, creating new effective charities has rapidly diminishing marginal returns. The space of possible highly-effective charities (i.e. ones that are much more effective than top global poverty charities) is not that big.
On 4), there’s a huge gulf between “We don’t yet have all the money we could ever use” and “Giving us more funding would let us continue to be as effective as we have been with current funds.” You really only claim the former, but you have to establish the latter for 80K to be the best place to donate.
Hey Michael,
It’s better to look at the evaluations rather than the list of studies if you want to get a systematic picture of career changes.
e.g. here: https://80000hours.org/2014/05/plan-change-analysis-and-cost-effectiveness/#what-were-the-changes
The most common changes are:
More people earning to give
More people setting up or working in effective altruist charities
More people building career capital
More people working on xrisk
These are things people very unusually do otherwise, so it’s very unlikely they would happen without effective altruism or 80,000 Hours. Of course, it’s hard to untangle 80k’s impact on career choices from the rest of the EA movement, but it seems likely that 80k gets a substantial fraction of the impact. First, we’re the main group doing career stuff within the movement. Second, we’ve done a huge amount to boost the EA movement (e.g. we were the first org to use the term publicly), so if the EA movement has a lot of impact, then a significant fraction is due to us.
Note that a similar objection applies to the other charities you propose: e.g. if REG / Charity Science / GWWC / GiveWell didn’t exist, much of the impact would happen anyway eventually because the other groups would eventually step in to compensate. But that doesn’t mean none of them are having much impact.
Of course, we address this in the evaluation.
In short, I think in many of the cases the effectiveness boosts are very large, so when you subtract the impact they would have had anyway, it’s less than 10%. It depends on your view of how good “standard career choice” is.
I’d say our impact comes from giving people better information about how to have a social impact in their career, and so redirecting them into higher impact career paths.
You can try to quantify a component of that by looking at additional donations due to our members, number of new organisations founded, or other measures.
So, new organisations founded is just one component of our impact that’s relatively tractable to analyse. More often, people assess us in terms of extra donations for charity raised from more people pursuing earning to give. Our estimate is that those earning to give will donate an extra $10m+ of counterfactually-adjusted funds to high-impact charities within the next 3 years because of us. I think either of these methods mean we’ve been very cost-effective in the past (historical financial costs are under $500k), and that’s ignoring over half the plan changes.
https://80000hours.org/2015/07/update-how-many-extra-donations-have-we-caused/
It seems really unclear to me how close we are to that margin. Bear in mind explicitly effective altruist funding is under 0.04% of total US philanthropy. It seems like we could expand the number of organisations a great deal before hitting substantially diminishing returns. In particular when you consider how little research has been done, relatively speaking, it’s unlikely we discovered the best things already.
If we did run out of ideas for new organisations, 80k could move its focus to scaling up existing orgs. (Many people who’ve changed plans due to 80k have gone to work at existing EA orgs rather than found new ones). Or we could just encourage everyone to earn to give and donate to top global poverty charities.
Also, why you do you expect entrepreneurial-talent in EA to hit diminishing returns faster than donations? If anything, I expect we’ll hit diminishing returns to additional donations faster than with talent, because funding gaps are so much easier to resolve than talent-gaps (e.g. one billionaire could flood EA with money tomorrow). And that means REG also doesn’t have as much upside as it looks because in the future they won’t be able to direct the money as effectively as well.
Of course. I actually think we’re going to be more effective with future funds because we’re getting better and better at changing plans, so our cost per plan change is falling. This is because our main focus in the past was learning and research, which is only just starting to pay off. There’s a lot more to say here though!
“More people building career capital” … “These are things people very unusually do otherwise”
Why do you think it’s unusual for people to build career capital?
True, that one’s an exception. The other 3⁄4 are unusual otherwise though.
It feels like we’re getting off track here. You originally claimed that 80K played a large role in creating REG and ACE (the implication presumably being that I should donate to 80K). Now we’re talking about the strength of evidence on how 80K has changed people’s career paths.
Although its evidence is weaker than I’d like, I’m still fairly confident that 80K has a positive impact, and I’m glad it exists. I just don’t see that it’s the best place to donate. Are you trying to convince me that 80K’s activities are valuable, or that I should donate to it? If it’s the former, I already believe that. If the latter, you need to:
show why 80K has a higher impact than anything else
do more to make the strength of evidence supporting 80K more robust
demonstrate that 80K can effectively use marginal funds
Now that’s a pretty high bar, but I’m donating a lot of money and I want to make sure I direct it well.
Nitpick: robust evidence doesn’t seem necessary—weak evidence of high upside potential should also count.
Hi Michael,
In the original document you say next to 80k “unclear whether it has a positive effect”. So I was starting there.
REG and ACE are relevant because they’re examples of the value of the plan changes we cause. If you think 80k is changing plans such that more high impact organisations are created, then it’s likely 80k is also effective. (Though may of course still not be worth funding due to a lack of RFMF, but that’s not what you said initially).
The closest we’ve got recently to publicly arguing the case for 80k is here: https://80000hours.org/2015/08/plans-for-the-coming-year-may-2015/
Of course there’s a lot more to talk about. I’m always happy to answer more questions or share details about how marginal funds would be used via email.
We have to debate back and forth and figure out this EtG thing properly.
I think Hurford’s points about EtG are obviously really wrong. I find it baffling so many people are convinced.
See my comment here: http://effective-altruism.com/ea/mk/peter_hurford_thinks_that_a_large_proportion_of/515.
That smart people who have thought about this can have such different views is worrying.
Since it’s so sensitive to what “from a good university” or “altruistically motivated” mean, it would make more sense to argue over a few hypothetical marginal case studies.
I’m not too worried about this; it’s just a hard problem. That means we should perhaps invest more into solving it.
Another interesting consideration is that not all of the funds raised by fundraising organisations are attributable to the existence of that organisation per se. A lot of the funds would have been raised by the organisation’s founders, who would be enthusiastic networkers and fundraisers even if their organisation did not exist (and even moreso if it someday ceased to exist). Moreover, the ease of fundraising for an organisation like AMF is easier if they are better-funded, such that they use these funds to give themselves a good reputation and deliver positive results. I wonder how sensitive the results of your analysis would be to considerations about i) direct funding improving the ease of fundraising and ii) some funds raised being attributable to the existence of fundraising individuals rather than the organisations they establish.
It’s pretty harsh to defund people’s organisations because they make carefully reasoned arguments that disagree with your conclusions! I’m a vegetarian and thought the arguments were strong, so it’s hard to write that off as motivated reasoning. If you want to make a balanced judgement of what their blogs posts say about their values, mightn’t you want to do a more balanced survey of what the key players have written on a wider range of topics, rather than the one that reached your newsfeed because its claims were seemingly outrageous? It’d feel similarly unfair if people tried to discredit whatever outreach efforts I was performing because I’d made (quite good—or so I thought) arguments that organ donation was ineffective.
I assume you’re referring to my discussion of MIRI.
I’m NOT saying that some MIRI employees don’t care about animals, therefore they’re bad at reasoning. That’s NOT what I’m saying, and frankly that would be silly. Eliezer doesn’t care about animals but I believe he’s much smarter and probably more rational than I am.
What I AM saying is this:
MIRI/FAI researchers may have a large influence on what values end up shaping the far future.
Some sorts of FAI research are more likely to work out well for non-human animals than others. (I discuss this in OP.)
Therefore, I should want FAI researchers to have good values, and in particular, to assign appropriate worth to non-human animals because I think this is by far the biggest potential failure mode. I want to trust that they will choose to do the sorts of research that will work out well for non-human animals.
So I will attempt to assess how much value MIRI researchers assign to non-human animals, because this question is relevant to how much good I think they will produce for the far future.
This has nothing to do with my meta-level assessment of MIRI employees’ reasoning abilities and everything to do with their object-level beliefs on an issue that could be critically important for the shape of the far future.
I find this consideration less important than I used to because I’m more confident that preventing human extinction is net positive, but I still thought it was worth discussing.
You’re sceptical of their organisation because you disagree with them about the object-level topic of animals, which they assign less importance than you, right?
From the reader’s point of view, this kind of argument shouldn’t get much weight.
Why would the future welfare of animals be important in a future world with AIs? It’d make more sense for computing resources to be used to create things that people want (like fun virtual worlds?) and that they’d optimise their use of it, rather than putting animals there, which are unlikely to be useful for any specific human purpose, except perhaps as pets. Moreover, the activities of animals themselves are not going to have any long-run impacts. For reasons related to these two, it seems to me that those who argue that being vegetarian now is not useful in the long-run are closer to the mark than those like Rob (who nonetheless are well-represented in MRI), who argue that it is morally obligatory.
And at the bottom of all of this, the reader will note that you have converged toward MIRI’s views on other topics like the importance of AI research and existential risk reduction, and there’s little reason that you couldn’t update your views to be closer to the average of reasonable positions around this topic.
The argumentation ‘i won’t fund this because they criticised an endeavour that i value’ also gives a bad incentive, but at any rate, it seems like it is appopriate to downweight it.
I still feel like you’re misunderstanding my position but I don’t know how to explain it any differently than I already have, so I’ll just address some things I haven’t talked about yet.
A lot of what you’re talking about here is how I should change my beliefs when other smart people have different beliefs from me, which is a really complex question that I don’t know how to answer in a way that makes sense. I get the impression that you think I should put more weight on the fact that some MIRI researchers don’t think animals are important for the far future; and I don’t think I should do that.
I already agree that wild animals probably won’t exist in the far future, assuming humans survive. I also generally agree with Nate’s beliefs on non-human animals and I expect that he does a good job of considering their interests when he makes decisions. And my current best guess is that MIRI is the strongest object-level charity in the world. I don’t think I disagree with MIRI as much as you think I do.
Edited to add: I have seen evidence that Nate is asking questions like, “What makes a being conscious?” “How do we ensure that an AI makes all these beings well off and not just humans?” AI safety researchers need to be asking these questions.
EDIT: It looks like you heavily edited your comment so my reply here doesn’t make much sense anymore.
Well different people at MIRI have different opinions so I don’t want to treat them like a monolith. Nate explicitly agrees with me that extrapolating from human values could be really bad for non-human animals; Rob things vegetarianism is morally obligatory; Katja thinks animals matter but vegetarianism is probably not useful; Eliezer doesn’t think animals matter.
I agree, as I explain here. But I’m not that confident, and given that in expectation non-human animals currently account for maybe 99.9%[^1] of the utility of the world, it’s pretty important that we get this right. I’m not remotely comfortable saying “Well, according to this wild speculation that seems prima facie reasonable, wild animals won’t exist in the future, so we can safely ignore these beings that currently account for 99.9% of the utility.”
I don’t know what you mean by “less wrong about animals.” Less wrong about what, exactly? Do you mean about how morally valuable animals are? About the probability that wild animal suffering will dominate the far future?
It’s plausible that a lot of AI researchers have explicitly reasoned about why they expect safety research to be good for the far future even when you don’t massively discount the value of animals. The only person I’ve seen discuss this publicly is Carl Shulman, and I’ve talked to Nate Soares about it privately so I know he’s thought about it. But all of MIRI’s public materials are entirely focused on why AI safety is important for humans, and make no mention of non-humans (i.e. almost all the beings that matter). Nate has adequately convinced me that he has thought about these issues but I haven’t seen evidence that anyone else at MIRI has thought about them. I’m sure some of them have but I’m in the dark about it. Since hardly anyone talks publicly about this, I used “cares about animals/is veg*an” as a proxy for “will try to make sure that an AI produces a future that’s good for all beings, not just humans.” This is an imperfect metric but it’s the best I could do in some cases. I did speak to Nate about this directly though and I felt good about his response.
Of course I did still come out strongly in favor of MIRI, and I’m supporting REG because I expect REG to product a lot of donations to MIRI in the future.
As Carl points out, it’s not the case that non-human animals account for 99.9% of utility if you’re using brain mass as a heuristic for the importance of each animal.
About how important valuing animals is to the future? Though Katja and Robin are on a different side of the spectrum to you on this question, epistemic modesty means better to avoid penalizing them for their views.
It sounds like you and Michael just have different values. It’s pretty clear that you’d only find Michael’s argument viable if you share his opinion on animals. If you don’t share his value, you’d place different weights on the importance of the risk of MIRI doing a lot of bad things to animals.
I disagree that “[f]rom the reader’s point of view, this kind of argument shouldn’t get much weight.” It should get weight for readers that agree with the value, and shouldn’t get weight for readers that disagree with the value.
No, that’s exactly the issue—I want as much as the next person to see animals have better lives. I just don’t see why the ratio of humans to animals would be high in the future, especially if you weight the moral consideration to brain mass or information states.
I’m just wary of making confident predictions of the far future. A lot can change in a million years...
I agree with you that it probably won’t be high. But I would have to be >99% confident that animals won’t comprise much of the utility of the far future for me to be willing to just ignore this factor, and I’m nowhere near that confident. Maybe you’re just a lot more confident than I am.
That’s a good point. I’d like to see what the numbers look like when you include wild animals too.
Most of the neural mass will be wild animals, but I think more like 90% than 99.9% (the ratio has changed by orders of magnitude in recent thousands of years, and only needs to go a bit further on a log scale for human brain mass to dominate). Unless you very confidently think that a set of neurons being incorporated into a larger structure destroys almost all of their expected value, the ‘small animals are dominant’ logic can likewise be used to say ’small neural systems are dominant, within and between animals.” If sapient populations grow rapidly (e.g. AI) then wild animals (including simulated ones) would be absolutely dwarfed on this measure. However, non-sapient artificial life might or might not use more computation than sapient artificial beings.
Also, there can be utility monsters both above and below. The number of states a brain can be in goes up exponentially as you add bits. The finite numbers it can represent (for pleasure, pain, preferences) go up super-exponentially. If you think a simple reinforcement learning Pac-Man program isn’t enough for much moral value, that one needs more sensory or processing complexity, then one is allowing that the values of preferences and reward can scale depending on other features of the system. And once you allow that, it is plausible that parallel reinforcement/decision processes in a large mind will get a higher multiplier (i.e. not only will there be more neural-equivalent processes doing reinforcement updating, but each individual one will get a larger multiplier due to the system it is embedded in).
The conclusion that no existing animal will be maximally efficient at producing welfare according to a fairly impartial hedonistic utilitarianism is on much firmer ground than the conclusion that the maximally efficient production system on that ethical theory would involve exceedingly tiny minds rather than vast ones or enhanced medium-size ones, or complex systems overlapping these scales.
Small insects (the most common) have a order 10,000 neurons. One estimate is 10^18 insects, implying 10^22 neurons. In humans it is 10^21 neurons total. However, smaller organisms tend to have smaller cells, so if you go by mass, humans might actually be dominant. Of course there are other groups of wild and domestic animals, but it gives you some idea.