I lead the DeepMind mechanistic interpretability team
Neel Nanda
Karma: 5,079
Great post! I highly recommend using LLM assistance (especially Claude) here, eg drafting emails, preparing a script for phone calls, etc. Personally I find this all super awkward, and LLMs are much better than me at wording things gracefully. (Though you want to edit it enough that it doesn’t feel like LLM written slop)
I think you are conflating your specific cause prioritisation and a general question of how people who care about impact should think. If someone held your course prioritisation then they should clearly work at one of those top organisations, otherwise help with the issues, or earn the highest salary they can and donate that. I.E earning to give. Working at other impact-focused organisations not focused on those top causes wouldn’t make sense. I think that generally you should optimise for one thing rather than half-hardly optimising for several.
However, many people do not share your cause participation which leads to quite different conclusions. I have no regrets about doing direct work myself
I disagree. I think that if a government causes great harm by accident or great harm intentionally, either is evidence that it will cause great harm by accident or intentionally in future respectively and I just care about the great harm part
This is quite different from the case I would make for donor lotteries. The argument I would make is just that figuring out what to do with my money takes a bunch of time and effort. If I had 10 times the amount of money I could just scale up all of my donations by 10 times and the marginal utility would probably be about the same. So I would happily take a 10% chance to 10x my money and a 90% chance to be zero and otherwise follow the same strategy because in expectation the total good done is the same but the effort invested has 10% the cost, as I won’t bother doing it if I lose.
Further, it now makes more sense to invest way more effort, but that’s just a fun bonus. I can still just give the money to EA funds or whatever if that beats my personal judgement, but I can take a bit more time to look into this, maybe make some other grants if I prefer etc. And so likewise, being 100 or 1000x leveraged is helpful and justifies even more efforts in the world where I win.
Notably this argument works regardless of who else is participating in the lottery. if I just went to Vegas and bet a bunch of my money on roulette that gets a similar effect. Donor lotteries are just a useful way of doing this where everyone gets this benefit of a small chance of massively increasing their money and a high chance of losing it all, and it’s zero expected value unlike roulette
If people said all these things without the word marginal, would you be happy?
This feels less like a disagreement over what the word marginal means and more that you just disagree with people’s theory of impact and what they think the expected value of different actions are?
The GDM AGI Safety+Alignment Team is Hiring for Applied Interpretability Research
To me, it seems like it would be much more valuable to have a fully virtual event rather than one where all the in-person people want to prioritise other in-person people. I don’t know how the costs of organising an additional virtual event could compare to hybridizing an in-person event that would happen anyway would be, however
Human rights abuses seem much worse in China—this alone is basically sufficient for me
Seems pretty clear that he meant your second statement
There is publicly available and sufficient evidence indicating Charity X did not provide 10K meals to homeless individuals.
Yeah thi. In particular anytime you criticise an organisation and they are only able to respond a few weeks later, many readers will see your criticism but will not see the organization’s response. This inherently will give a misleading impression, so you must be incredibly confident that there is no mitigating context that this organisation would give lest you do their reputation undue damage empirically. I think it is obviously the case empirically that when an organisation gets critiqued including your two previous examples that there is valuable additional context they’re able to provide when given notice
I think the typical member of the EA community has more than enough technical skill to understand evidence that a web page has been edited to be different from an archived page, if pointed to both copies from a reliable source
Quoting your post:
It is not acceptable for charities to make public and important claims (such as claims intended to convince people to donate), but not provide sufficient and publicly stated evidence that justifies their important claims.
If a charity has done this, they should not be given the benefit of the doubt, because it is their own fault that there is not sufficient publicly stated evidence to justify their important claims; they had the opportunity to state this evidence but did not. Additionally, giving a charity the benefit of the doubt in this situation incentivizes not publicly stating evidence in situations where sufficient evidence does not exist, since the charity will simply be given the benefit of the doubt.
Note that I interpret this standard as “provide sufficient evidence to support their claims in the eyes of any skeptical outside observer”, as that’s the role you’re placing yourself in here
I pretty strongly disagree with the case that you make care and think that you should obviously give charities a heads up and it is bad for everyone’s interests if you don’t. The crucial reason is that it is very easy to misunderstand things or be missing key context and you want to give them a chance to clarify the situation
Regarding your concerns, just use archive.org or archive.is to make archives or relevant web pages. This is a standard and widely used third-party service, and if the charity changes things secretly but the archived evidence remains, they look bad.
Regarding unconscious biases. I don’t actually think this is a big deal, but if you are concerned it’s a big deal then you can just email the charity and be like: Here is the document, we will publish it, but you have a chance to comment in this Google doc to point out factual inaccuracies and provide us evidence which we will attempt to take into account. And we will give you the opportunity to prepare a written response that can either be a comments immediately after this goes live or that we will include at the end of our piece. Readers can make up their own mind.
Regarding charities being held accountable for making mistakes, I don’t think there’s a big difference between the charity being publicly called out for a mistake and then they quickly fix it or the charity fixed it just before it was publicly called out. They still made the mistake and it’s still obvious it was fixed because of you
Re your point about charities should be incentivised to provide sufficient public evidence. I think this is an extremely unreasonably high standard. Charities Should try to provide as much evidence as they can, but each person will have different objections and confusions and edge cases and it is just completely impractical to provide enough evidence to address all possible cases an adversarial reviewer might ask about. You can criticise charities for providing significantly insufficient rather than just mildly insufficient evidence. But ultimately there is always going to be the potential for misunderstanding and details that feel important to you that the charity did not predict would be important
Calendar syncing is so helpful
This got me curious so I had deep research make me a report on my probability of dying from different causes. It estimates that in the next 20 years I’ve maybe a 1 and 1⁄2 to 3% Chance of death, of which 0.5-1% is chronic illness where it’ll probably help a lot. Infectious diseases is less than .1%, Doesn’t really matter. Accidents are .5 to 1%, AI probably helps but kind of unclear. .5 to 1% on other, mostly suicide. Plausibly AI also leads to substantially improved mental health treatments which helps there? So yeah, I buy that having AGI today Vs in twenty years has small but non trivial costs to my chances of being alive when it happens
Thanks for clarifying. In that case I think that we broadly agree
I think that even the association between functional agency and preferences in a morally valuable sense is an open philosophical question that I am not happy taking as a given.
Regardless, it seems like our underlying crux is that we assign utility to different things. I somewhat object to you saying that your version of this is utilitarianism and notions of assigning utility that privilege things humans value are not
4% is higher than I thought! Presumably much of that is people who had pre-existing conditions which I don’t or people who got into eg a car accidents which AI probably somewhat reduces, but this seems a lot more complicated and indirect to me.
But this isn’t really engaging with my cruxes. it seems pretty unlikely to me that we will pause until we have pretty capable and impressive AIs and to me much of the non-doom scenarios comes from uncertainty about when we will get powerful ai and how capable it will be. And I expect this to be much clearer the closer we get to these systems, or at the very least the empirical uncertainty about whether it’ll happen will be a lot clearer. I would be very surprised if there was the political will to do anything about this before we got a fair bit closer to the really scary systems.
And yep, I totally put more than 4% chance that I get killed by AI in the next 20 years. But I can see this is a more controversial belief and one that requires higher standards of evidence to argue for. If I imagine a hypothetical world where I know that in 2 years we could have aligned super intelligent AI with 98% probability and it would kill us all with 2% probability. Or we could pause for 20 years and that would get it from 98 to 99%, then I guess from a selfish perspective I can kind of see your point. But I know I do value humanity not going extinct a fair amount even if I think that total utilitarianism is silly. But I observe that I’m finding this debate kind of slippery and I’m afraid that I’m maybe moving the goalposts here because I disagree on many counts so it’s not clear what exactly my cruxes are, or where I’m just attacking points in what you say that seem off
I do think that the title of your post is broadly reasonable though. I’m an advocate for making AI x-risk cases that are premised on common sense morality like “human extinction would be really really bad”, and utilitarianism in the true philosophical sense is weird and messy and has pathological edge cases and isn’t something that I fully trust in extreme situations
This was a very helpful post, thanks! Do you know of any way for UK donors to give to the rapid response fund? If not, has GWWC considered trying to set that up? (Like I think you have with a bunch of other charities)