Yes, I had this exact quote in mind when I said in Sect 5 that “Religions can withstand persecution by totalitarian governments, and some feel just about as strongly about EA.”
People would believe them, want to co-ordinate on it. Then they’d want to organise to help make their own ideas more efficient and boom, we’re just back to an EA movement all over again.
One of my main theses is supposed to be that people can and should coordinate their activities without acting like a movement.
I still want concern about reducing the suffering on non-human animals to grow, I still want humanity to expand its moral circle beyond the parochial, I still want us to find the actions individually and collectively that will lead to humanity flourishing. Apologies if I’m misinterpreting, but this sentance really seems to come out of left-field from me given the rest of your post.
This feels like the same misunderstanding. Spreading EA ideas and values seems fine and good to me. It’s the collectivism, branding, identity-based reasoning, and other “movement-like” characteristics that concern me.
I think again, given the ideas of EA exist, these cause-specific communities would find themselves connected
This seems like black and white thinking to me. Of course these people will connect over their shared interests in consequentialism, RCTs, and so on. But this is different from branding and recruiting together, regulating this area as one community, hosting student chapters, etc.
Thanks for explaining your viewpoints Ryan. I think I have a better understanding, but I’m still not sure I grok it intuitively. Let me try to repeat what I think is your view here (with the help of looking at some of your other quick takes)
note for readers, this is my understanding of Ryan’s thoughts, not what he’s said
1 > The EA movement was directly/significantly causally responsible for the FTX disaster, despite being at a small scale (e.g. “there are only ~10k effective altruists”)
2 > We should believe that without reform, similar catastrophes will contain to happen as the movement grows, which would lead to many more such FTXs catastrophes (i.e. if we get to a movement of 1 million EAs in the current movement, we should expect ~100 FTX-scale disasters)
3 > This outcome is morally unacceptable, so the EA movement shouldn’t grow in its current form
4 > An alternative way to grow would be to continue to grow as a set of different, interconnected movements focused around direct action (e.g. The xRisk community, The animal welfare community, the Global Development community, The Longtermist community etc...)
5 > This would allow EA values to spread without the harms that we see occurring with EA as a movement
I’m think I follow along. I’m not sure about the extrapolation of FTX (would it scale linearly or logarithmically? Does it actually make any sense to extrapolate as if EA will continue the same way at all?) But that aside I think my main disagreement is to think why a set of separate fields/communities that co-ordinate would be better at avoiding the failure modes you see in EA than the current one. I feel like “collectivism, branding, identity-based reasoning, and other “movement-like” characteristics” are going to occur whenever humans organise themselves into groups.
I think perhaps an underlying disagreement we have is about the power of ideas. I just don’t think you can cleanly separate the EA movement from EA values. Ideas are powerful things which have logical and empirical consequences. The EA movement has grown so much so quickly, in my view, because its ideas and values are true[1] and convincing. That causes movements and not the other way around. I guess I’m finding it difficult to picture what a movement-less EA would look like?
As an intuition pump, it’d be like a reformer saying Christians so just go to church on Sunday and listen to sermon, follow the commandments, read the bible, tithe, and do good works, but not bother with all of the Father/Son/Holy Ghost stuff. But that belief is the reason why they’re doing the former. In a world where that was attempted to be removed, I think people would either stop doing the activities or reinvent them.
I’m also worried about Leverage and various other cults and disasters, not just FTX.
I wouldn’t think of the separate communities as “movements” per se. Rather, each cause area would have a professional network of nonprofits and companies.
Basically, why do mid-sized companies usually not spawn cults and socially harm their members like movements like EA and the animal welfare community sometimes do? I think it’s because movements by their nature try to motivate members towards their goals, using social pressures. This attracts young idealists, some of whom will be impressionable. People will try radical stuff like traveling to locations where they’re unsupported, going on intensive retreats, circling, drugs, polyamory, etc. These things benefit some people in some situations, but in they also can put people in vulnerable situations. My hypothesis is that predators detect this vulnerability and then start even crazier and more cultish projects, arguably including Leverage and FTX, under the guise of advancing the movement’s goals.
Companies rarely put junior staff in such vulnerable positions. People generally know not to sleep with subordinates, and better manage conflicts of interest. They don’t usually give staff a pass for misbehaviour due to being value-aligned.
We don’t need to lose our goals, or our social network, but we could strip away a lot of risk-increasing behaviour that “movements” do, and take on some risk-reducing “professionalising” measures that’s more typical of companies..
I agree that ideas are powerful things, and that people will continue to want to follow those ideas to their conclusions, in collaboration with others. But I’m suggesting to be faithful to those ideas might be to shape up a little bit and practice them somewhat differently. For the case of Christianity, it’s not like telling Christians to disavow the holy Trinity. It’s more like noticing abuse in a branch of Christianity, and thinking “we’ve got to do some things differently”. Except that EA is smaller and thousands of years younger, so can be more ambitious in the ways we try to reform.
JWS, do you think EA could work as a professional network of “impact analysts” or “impact engineers” rather than as a “movement”?
Ryan, do you have a sense of what that would concretely look like?
If we look at other professionals, for example, engineers have in common some key ideas, values, and broad goals (like ‘build things that work’). Senior engineers recruit young engineers and go to professional conferences to advance their engineering skills and ideas. Some engineers work in policy or politics, but they clearly aren’t a political movement. They don’t assume engineering is a complete ethos for all major life decisions, and they don’t assume that other engineers are trustworthy just because they are engineers.
I share your appreciation for EA ideas and think they’ll have longevity. I don’t know that there is a way to push back against the pitfallls of being a social movement instead of just being a collection of professionals. But I agree with Ryan that if there were a way to just be a group of skilled colleagues rather than “brethren”, it would be better. Social movements have the pitfalls of religions, tribes, and cults that most professions do not and fall prey to more demagogues as a result.
JWS, do you think EA could work as a professional network of “impact analysts” or “impact engineers” rather than as a “movement”?
I guess I still don’t have a clear idea of what Ryan’s ‘network of networks’ approach would look like without the ‘movement’ aspect broadly defined. How definitely would that be practically from current EA but with more decentralisation of money and power, and more professional norms?
But would this be a set of rigid internal norms that prevent people from the philanthropy space connecting with those in specific cause areas? Are we going to split AI technical and governance fields strictly? Is nobody meant to notice the common philosophical ideas which underline the similar approaches to all these cause areas? It’s especially the latter I’m having trouble getting my head around.
Some engineers work in policy or politics, but they clearly aren’t a political movement. They don’t assume engineering is a complete ethos for all major life decisions, and they don’t assume that other engineers are trustworthy just because they are engineers.
I don’t think that ‘field of engineering’ is the right level of analogy here. I think the best analogies for EA are other movements, like ‘Environmentalism’ or ‘Feminism’ or ‘The Enlightenment’.
Social movements have the pitfalls of religions, tribes, and cults that most professions do not and fall prey to more demagogues as a result.
Social movements have had a lot of consequences in the human history, some of them very positive and some very negative. It seems to me that you and Ryan think that there’s a way to structure EA so that we can cleanly excise the negative parts of a movement and keep the positive parts without being a movement, and I’m not sure that’s really possible or even a coherent idea.
***
[to @RyanCarey I think you updated your other comment as I was thinking of my response, so folding in my thoughts on that here]
We don’t need to lose our goals, or our social network, but we could strip away a lot of risk-increasing behaviour that “movements” do, and take on some risk-reducing “professionalising” measures that’s more typical of companies.
I’m completely with you here, but to me this is something that ends up miles away from ‘winding down EA’, or EA being ‘not a movement’.
But I’m suggesting to be faithful to those ideas might be to shape up a little bit and practice them somewhat differently. For the case of Christianity, it’s not like telling Christians to disavow the holy Trinity. It’s more like noticing abuse in a branch of Christianity, and thinking “we’ve got to do some things differently”.
I think abuse might be a bit strong as an analogy but directionally I think this is correct, and I’d agree we need to do things differently. But in this analogy I don’t think the answer is end ‘Christianity’ as a movement and set up an overlapping network of tithing, volunteering, Sunday schools etc, which is what I take you to be suggesting. I feel like we’re closer to agreement here, but on reflection the details of your plan here don’t sum up to ‘end EA as a movement’ at all.
JWS, do you think EA could work as a professional network of “impact analysts” or “impact engineers” rather than as a “movement”? Ryan, do you have a sense of what that would concretely look like?
Well I’m not sure it makes sense to try to fit all EAs into one professional community that is labelled as such, since we often have quite different jobs and work in quite different fields. My model would be a patchwork of overlapping fields, and a professional network that often extends between them.
It could make sense for there to be a community focused on “effective philanthropy”, which would include OpenPhil, Longview, philanthropists, and grant evaluators. That would be as close to “impact analysis” as you would get, in my proposal.
There would be an effective policymaking community too.
And then a bevy of cause-specific research communities: evidence-based policy, AI safety research, AI governance research, global priorities research, in vitro meat, global catastrophic biorisk research, global catastrophic risk analysis, global health and development, and so on.
Lab heads and organisation leaders in these research communities would still know that they ought to apply to the “effective philanthropy” orgs to fund their activities. And they would still give talks at universities to try to attract top talent. But there wouldn’t be a common brand or cultural identity, and we would frown upon the risk-increasing factors that come from the social movement aspect.
Yes, I had this exact quote in mind when I said in Sect 5 that “Religions can withstand persecution by totalitarian governments, and some feel just about as strongly about EA.”
One of my main theses is supposed to be that people can and should coordinate their activities without acting like a movement.
This feels like the same misunderstanding. Spreading EA ideas and values seems fine and good to me. It’s the collectivism, branding, identity-based reasoning, and other “movement-like” characteristics that concern me.
This seems like black and white thinking to me. Of course these people will connect over their shared interests in consequentialism, RCTs, and so on. But this is different from branding and recruiting together, regulating this area as one community, hosting student chapters, etc.
Thanks for explaining your viewpoints Ryan. I think I have a better understanding, but I’m still not sure I grok it intuitively. Let me try to repeat what I think is your view here (with the help of looking at some of your other quick takes)
I’m think I follow along. I’m not sure about the extrapolation of FTX (would it scale linearly or logarithmically? Does it actually make any sense to extrapolate as if EA will continue the same way at all?) But that aside I think my main disagreement is to think why a set of separate fields/communities that co-ordinate would be better at avoiding the failure modes you see in EA than the current one. I feel like “collectivism, branding, identity-based reasoning, and other “movement-like” characteristics” are going to occur whenever humans organise themselves into groups.
I think perhaps an underlying disagreement we have is about the power of ideas. I just don’t think you can cleanly separate the EA movement from EA values. Ideas are powerful things which have logical and empirical consequences. The EA movement has grown so much so quickly, in my view, because its ideas and values are true[1] and convincing. That causes movements and not the other way around. I guess I’m finding it difficult to picture what a movement-less EA would look like?
As an intuition pump, it’d be like a reformer saying Christians so just go to church on Sunday and listen to sermon, follow the commandments, read the bible, tithe, and do good works, but not bother with all of the Father/Son/Holy Ghost stuff. But that belief is the reason why they’re doing the former. In a world where that was attempted to be removed, I think people would either stop doing the activities or reinvent them.
or true-enough, or seem true enough. Not claiming EA has anywhere near ‘ultimate truth’ on any issue
Roughly yes, with some differences:
I think the disasters would scale sublinearly
I’m also worried about Leverage and various other cults and disasters, not just FTX.
I wouldn’t think of the separate communities as “movements” per se. Rather, each cause area would have a professional network of nonprofits and companies.
Basically, why do mid-sized companies usually not spawn cults and socially harm their members like movements like EA and the animal welfare community sometimes do? I think it’s because movements by their nature try to motivate members towards their goals, using social pressures. This attracts young idealists, some of whom will be impressionable. People will try radical stuff like traveling to locations where they’re unsupported, going on intensive retreats, circling, drugs, polyamory, etc. These things benefit some people in some situations, but in they also can put people in vulnerable situations. My hypothesis is that predators detect this vulnerability and then start even crazier and more cultish projects, arguably including Leverage and FTX, under the guise of advancing the movement’s goals.
Companies rarely put junior staff in such vulnerable positions. People generally know not to sleep with subordinates, and better manage conflicts of interest. They don’t usually give staff a pass for misbehaviour due to being value-aligned.
We don’t need to lose our goals, or our social network, but we could strip away a lot of risk-increasing behaviour that “movements” do, and take on some risk-reducing “professionalising” measures that’s more typical of companies..
I agree that ideas are powerful things, and that people will continue to want to follow those ideas to their conclusions, in collaboration with others. But I’m suggesting to be faithful to those ideas might be to shape up a little bit and practice them somewhat differently. For the case of Christianity, it’s not like telling Christians to disavow the holy Trinity. It’s more like noticing abuse in a branch of Christianity, and thinking “we’ve got to do some things differently”. Except that EA is smaller and thousands of years younger, so can be more ambitious in the ways we try to reform.
JWS, do you think EA could work as a professional network of “impact analysts” or “impact engineers” rather than as a “movement”?
Ryan, do you have a sense of what that would concretely look like?
If we look at other professionals, for example, engineers have in common some key ideas, values, and broad goals (like ‘build things that work’). Senior engineers recruit young engineers and go to professional conferences to advance their engineering skills and ideas. Some engineers work in policy or politics, but they clearly aren’t a political movement. They don’t assume engineering is a complete ethos for all major life decisions, and they don’t assume that other engineers are trustworthy just because they are engineers.
I share your appreciation for EA ideas and think they’ll have longevity. I don’t know that there is a way to push back against the pitfallls of being a social movement instead of just being a collection of professionals. But I agree with Ryan that if there were a way to just be a group of skilled colleagues rather than “brethren”, it would be better. Social movements have the pitfalls of religions, tribes, and cults that most professions do not and fall prey to more demagogues as a result.
I guess I still don’t have a clear idea of what Ryan’s ‘network of networks’ approach would look like without the ‘movement’ aspect broadly defined. How definitely would that be practically from current EA but with more decentralisation of money and power, and more professional norms?
But would this be a set of rigid internal norms that prevent people from the philanthropy space connecting with those in specific cause areas? Are we going to split AI technical and governance fields strictly? Is nobody meant to notice the common philosophical ideas which underline the similar approaches to all these cause areas? It’s especially the latter I’m having trouble getting my head around.
I don’t think that ‘field of engineering’ is the right level of analogy here. I think the best analogies for EA are other movements, like ‘Environmentalism’ or ‘Feminism’ or ‘The Enlightenment’.
Social movements have had a lot of consequences in the human history, some of them very positive and some very negative. It seems to me that you and Ryan think that there’s a way to structure EA so that we can cleanly excise the negative parts of a movement and keep the positive parts without being a movement, and I’m not sure that’s really possible or even a coherent idea.
***
[to @RyanCarey I think you updated your other comment as I was thinking of my response, so folding in my thoughts on that here]
I’m completely with you here, but to me this is something that ends up miles away from ‘winding down EA’, or EA being ‘not a movement’.
I think abuse might be a bit strong as an analogy but directionally I think this is correct, and I’d agree we need to do things differently. But in this analogy I don’t think the answer is end ‘Christianity’ as a movement and set up an overlapping network of tithing, volunteering, Sunday schools etc, which is what I take you to be suggesting. I feel like we’re closer to agreement here, but on reflection the details of your plan here don’t sum up to ‘end EA as a movement’ at all.
To be clear, winding down EA is something I was arguing we shouldn’t be doing.
At a certain point it becomes semantic, but I guess readers can decide, when you put together:
the changes in sec 11 of the main post
ideas about splitting into profession-oriented subgroups, and
shifting whether we “motivate members re social pressures” and expose junior members to risk
whether or not it counts as changing from being a “movement” to something else.
Fair.
Having run through the analogy, EA becoming more like an academic field or a profession rather than a movement seems very improbable.
I agree that “try to reduce abuses common within the church” seems a better analogy.
Well I’m not sure it makes sense to try to fit all EAs into one professional community that is labelled as such, since we often have quite different jobs and work in quite different fields. My model would be a patchwork of overlapping fields, and a professional network that often extends between them.
It could make sense for there to be a community focused on “effective philanthropy”, which would include OpenPhil, Longview, philanthropists, and grant evaluators. That would be as close to “impact analysis” as you would get, in my proposal.
There would be an effective policymaking community too.
And then a bevy of cause-specific research communities: evidence-based policy, AI safety research, AI governance research, global priorities research, in vitro meat, global catastrophic biorisk research, global catastrophic risk analysis, global health and development, and so on.
Lab heads and organisation leaders in these research communities would still know that they ought to apply to the “effective philanthropy” orgs to fund their activities. And they would still give talks at universities to try to attract top talent. But there wouldn’t be a common brand or cultural identity, and we would frown upon the risk-increasing factors that come from the social movement aspect.