Thanks for writing this! That overall seems pretty reasonable, and from a marketing perspective I am much more excited about promoting “weak” longtermism than strong longtermism.
A few points of pushback:
I think that to work on AI Risk, you need to buy into AI Risk arguments. I’m unconvinced that buying longtermism first really shifts the difficulty of figuring this point out. And I think that if you buy AI Risk, longtermism isn’t really that cruxy. So if our goal is to get people working on AI Risk, marketing longtermism first is strictly harder (even if it may be much easier)
I think that very few people say “I buy the standard AI X-Risk arguments and that this is a pressing thing, but I don’t care about future people so I’m going to rationally work on a more pressing problem”—if someone genuinely goes through that reasoning then more power to them!
I also expect that people have done much more message testing + refinement on longtermism than AI Risk, and that good framings could do much better—I basically buy the claim that it’s a harder sell though
Caveat: This reasoning applies more to “can we get people working on AI X-Risk with their careers” more so than things like broad societal value shifting
Caveat: Plausibly there’s enough social proof that people who care about longtermism start hanging out with EAs and are exposed to a lot of AI Safety memes and get there eventually? And it’s a good gateway thing?
I want AI Risk to be a broad tent where people who don’t buy longtermism feel welcome. I’m concerned about a mood affiliation problem where people who don’t buy longtermism but hear it phrased it as an abstract philosophical problem that requires you to care about the 10^30 future people won’t want to work on it, even though they buy the object level. This kind of thing shouldn’t hinge on your conclusions in contentious questions in moral philosophy!
More speculatively: It’s much less clear to me that pushing on things like general awareness of longtermism or longterm value change matter in a world with <20 year AI Timelines? I expect the world to get super weird after that, where more diffuse forms of longtermism don’t matter much. Are you arguing that this kind of value change over the next 20 years makes it more likely that the correct values are loaded into the AGI, and that’s how it affects the future?
Thanks for writing this! That overall seems pretty reasonable, and from a marketing perspective I am much more excited about promoting “weak” longtermism than strong longtermism.
A few points of pushback:
I think that to work on AI Risk, you need to buy into AI Risk arguments. I’m unconvinced that buying longtermism first really shifts the difficulty of figuring this point out. And I think that if you buy AI Risk, longtermism isn’t really that cruxy. So if our goal is to get people working on AI Risk, marketing longtermism first is strictly harder (even if it may be much easier)
I think that very few people say “I buy the standard AI X-Risk arguments and that this is a pressing thing, but I don’t care about future people so I’m going to rationally work on a more pressing problem”—if someone genuinely goes through that reasoning then more power to them!
I also expect that people have done much more message testing + refinement on longtermism than AI Risk, and that good framings could do much better—I basically buy the claim that it’s a harder sell though
Caveat: This reasoning applies more to “can we get people working on AI X-Risk with their careers” more so than things like broad societal value shifting
Caveat: Plausibly there’s enough social proof that people who care about longtermism start hanging out with EAs and are exposed to a lot of AI Safety memes and get there eventually? And it’s a good gateway thing?
I want AI Risk to be a broad tent where people who don’t buy longtermism feel welcome. I’m concerned about a mood affiliation problem where people who don’t buy longtermism but hear it phrased it as an abstract philosophical problem that requires you to care about the 10^30 future people won’t want to work on it, even though they buy the object level. This kind of thing shouldn’t hinge on your conclusions in contentious questions in moral philosophy!
More speculatively: It’s much less clear to me that pushing on things like general awareness of longtermism or longterm value change matter in a world with <20 year AI Timelines? I expect the world to get super weird after that, where more diffuse forms of longtermism don’t matter much. Are you arguing that this kind of value change over the next 20 years makes it more likely that the correct values are loaded into the AGI, and that’s how it affects the future?