I lead the DeepMind mechanistic interpretability team
Neel Nanda
Karma: 5,444
Are you assuming some kind of moral realism here? That there’s some deep moral truth, humans may or may not have insight into it, so any other intelligent entity is equally likely to?
If so, idk, I just reject your premise. I value what I chose to value, which is obviously related to human values, and an arbitrary sampled entity is not likely to be better on that front
Fascinating, I’ve never heard of this before, thanks! If anyone’s curious, I had Deep Research [take a stab at writing this] (https://chatgpt.com/share/67ac150e-ac90-800a-9f49-f02489dee8d0) which I found pretty interesting (but have totally not fact checked for accuracy)
I think you’re using the world utilitarian in a very non standard way here. “AI civilization has comparable moral value to human civilization” is a very strong claim that you don’t provide evidence for. You can’t just call this speciesism and shift the burden of proof! At the very least, we should have wide error bars over the ratio of moral value between AIs and humans, and I would argue also whether AIs have moral value at all.
I personally am happy to bite the bullet and say that I morally value human civilization continuing over an AI civilization that killed all of humanity, and that this is a significant term in my utility function.
Note that the UI is atrocious. You’re not using o1/o3-mini/o1-pro etc. It’s all the same model, a variant of o3, and the model in the bar at the top is completely irrelevant once you click the deep research button. I am very confused why they did it like this https://openai.com/index/introducing-deep-research/
I guess my issue is that this all seems strictly worse than “pledge to give 10% for the first 1-2 years after graduation, and then decide whether to commit for life”. Even “you commit for life, but with the option to withdraw 1-2 years after graduation”, ie with the default to continue. Your arguments about not getting used to a full salary apply just as well to those imo
More broadly, I think it’s bad to justify getting young people without much life experience to make a lifetime pledge, based on a controversial belief (that it should be normal to give 10%), by saying that you personally believe that belief is true. In this specific case I agree with your belief! I took the pledge (shortly after graduating I think). But there are all kinds of beliefs I disagree with that I do not want people using here. Lots of young people make choices that they regret later—I’m not saying they should be stopped from making these choices, but it’s bad to encourage them. I agree with Buck, at least to the extent of saying that undergrads who’ve been in EA for less than a year should not be encouraged to sign a lifetime pledge.
(On a meta level, the pledge can obviously be broken if someone really regrets it, it’s not legally binding. But I think arguments shouldn’t rely on the pledge being breakable)
I personally think it’s quite bad to try to get people to sign a lifetime giving pledge before they’ve ever had a real job, and think this is overemphasized in EA.
I think it’s much better to eg make a pledge for the next 1-5 years, or the first year of your career, or something, and re-evaluate at the end of that, which I think mitigates some of your concerns
Member of Technical Staff is often a catchall term for “we don’t want to pigeonhole you into a specific role, you do useful stuff in whatever way seems to add the most value”, I wouldn’t read much into it
Speaking as an IMO medalist who partially got into AI safety because of reading HPMOR 10 years ago, I think this plan is extremely reasonable
I disagree. I think it’s an important principle of EA that it’s socially acceptable to explore the implications of weird ideas, even if they feel uncomfortable, and to try to understand the perspective of those you disagree with. I want this forum to be a place where posts like this can exist.
The EA community still donates far more to global health causes than animal welfare—I think the meat eater problem discourse seems like a much bigger deal than it actually is in the community. I personally think it’s all kinda silly and significantly prioritise saving human lives
I strong downvoted because the title is unnecessarily provocative and in my opinion gives a misleading impression. I would rather not have this kind of thing on my forum feed
Interesting idea!
-
I recommend a different name, when I saw this I assumed it was about pledging around left wing causes
-
I feel like the spirit of the pledge would be to increase the 10% part with inflation? If you get a pay raise in line with inflation it seems silly to have to give half of that, since your real take home pay is unchanged. Even the further pledge is inflation linked
-
Would value drift be mitigated by donating to a DAF and investing there? Or are you afraid your views on where to donate might also shift
I feel pretty ok with a very mild and bounded commitment? Especially with an awareness that forcing yourself to be miserable is rarely the way to be just effective yourself. I think it’s pretty valid for someone’s college age self to say that impact does matter to them, and they do care about this, and don’t want to totally forget about it even if it becomes inconvenient, so long as they avoid ways this is psychological even by light of those values
I’ve only upvoted Habryka , to reward good formatting
It seems that we’re even afraid of them. I will never forget that just a week before I arrived at an org I was to be the manager of, they turned away an Economist reporter at their door...
Fwiw, I think being afraid of journalists is extremely healthy and correct, unless you really know what you’re doing or have very good reason to believe they’re friendly. The Economist is probably better than most, but I think being wary is still very reasonable.
Glad to hear it!
I commit to using my skills, time, and opportunities to maximize my ability to make a meaningful difference
I find the word maximise pretty scary here, for similar reasons to here. Analogous how GWWC is about giving 10%, a bounded amount, not “as much as you can possibly spare while surviving and earning money”
To me, taking a pledge to maximise seriously (especially in a naive conception where “I will get sick of this and break the pledge” or “I will burn out” aren’t considerations) is a terrible idea, and I recommend that people take pledges with something more like “heavily prioritise” or “keep as one of my top prioritise” or “actually put a sincere, consistent effort into, eg by spending at least an hour per month reflecting on whether I’m having the impact I want”. Of course, in practice, a pledge to maximise generally means one of those things, since people always have multiple priorities, but I like pledges to be something that could be realistically kept.
Thanks for sharing the list!
I notice most of these don’t have arguments for why individual donations are better than OpenPhil just funding the org for now (beyond the implicit argument that diverse donor base is good maybe). I’m curious if any of them have good arguments there? Without it, it feels like a donor’s money is just funging with OpenPhil’s last dollar—this is great, but I strive to do better.
I appreciated the clear discussion of this in the AI governance section and find opportunities there particularly compelling
Huh? This argument only goes through if you have a sufficiently low probability of existential risk or an extremely low change in your probability of existential risk, conditioned on things moving slower. I disagree with both of these assumptions. Which part of your post are you referring to?