As someone who is extremely pro investing in big-tent EA, my question is, “what does it look like, in practice, to implement ‘AI safety...should have its own movement, separate from EA’?”
I do think it is extremely important to maintain EA as a movement centered on the general idea of doing as much good as we can with limited resources. There is serious risk of AIS eating EA, but the answer to that cannot be to carve AIS out of EA. If people come to prioritize AIS from EA principles, as I do, I think it would be anathema to the movement to try to push their actions and movement building outside the EA umbrella. In addition, EA being ahead of the curve on AIS is, in my opinion, a fact to embrace and treat as evidence of the value of EA principles, individuals, and movement building methodology.
To avoid AIS eating EA, we have to keep reinvesting in EA fundamentals. I am so grateful and impressed that Dave published this post, because it’s exactly the kind of effort that I think is necessary to keep EA EA. I think he highlights specific failures in exploiting known methods of inducing epistemic … untetheredness?
For example, I worked with CFAR where the workshops deliberately employed the same intensive atmosphere to get people to be receptive to new ways of thinking and being actually open to changing their minds. I recognized that this was inherently risky, and was always impressed that the ideas introduced in this state were always about how to think better rather than convince workshop participants of any conclusion. Despite many of the staff and mentors being extremely convinced of the necessity of x-risk mitigation, I never once encountered discussion of how the rationality techniques should be applied to AIS.
To hear that this type of environment is de facto being used to sway people towards a cause prioritization, rather than how to do cause prio makes me update significantly away from continuing the university pipeline as it currently exists. The comments on the funding situation are also new to me and seem to represent obvious errors. Thanks again Dave for opening my eyes to what’s currently happening.
“what does it look like, in practice, to implement ‘AI safety...should have its own movement, separate from EA’?”
Creating AI Safety focused Conferences, AI Safety university groups and AI Safety local meet-up groups? Obviously attendees will initially overlap very heavily with EA conferences and groups but having them separated out will lead to a bit of divergence over time
Wouldn’t this run the risk of worsening the lack of intellectual diversity and epistemic health that the post mentions? The growing divide between long/neartermism might have led to tensions, but I’m happy that at least there’s still conferences, groups and meet-ups where these different people are still talking to each other!
There might be an important trade-off here, and it’s not clear to me what direction makes more sense.
I am all for efforts to do AIS movement building distinct from EA movement building by people who are convinced by AIS reasoning and not swayed by EA principles. There’s all kinds of discussion about AIS in academic/professional/media circles that never reference EA at all. And while I’d love for everyone involved to learn about and embrace EA, I’m not expecting that. So I’m just glad they’re doing their thing and hope they’re doing it well.
I could probably have asked the question better and made it, “what should EAs do (if anything), in practice to implement a separate AIS movement?” Because then it sounds like we’re talking about making a choice to divert movement building dollars and hours away from EA movement building to distinct AI safety movement building, under the theoretical guise of trying to bolster the EA movement against getting eaten by AIS? Seems obviously backwards to me. I think EA movement building is already under-resourced, and owning our relationship with AIS is the best strategic choice to achieve broad EA goals and AIS goals.
As someone who is extremely pro investing in big-tent EA, my question is, “what does it look like, in practice, to implement ‘AI safety...should have its own movement, separate from EA’?”
I do think it is extremely important to maintain EA as a movement centered on the general idea of doing as much good as we can with limited resources. There is serious risk of AIS eating EA, but the answer to that cannot be to carve AIS out of EA. If people come to prioritize AIS from EA principles, as I do, I think it would be anathema to the movement to try to push their actions and movement building outside the EA umbrella. In addition, EA being ahead of the curve on AIS is, in my opinion, a fact to embrace and treat as evidence of the value of EA principles, individuals, and movement building methodology.
To avoid AIS eating EA, we have to keep reinvesting in EA fundamentals. I am so grateful and impressed that Dave published this post, because it’s exactly the kind of effort that I think is necessary to keep EA EA. I think he highlights specific failures in exploiting known methods of inducing epistemic … untetheredness?
For example, I worked with CFAR where the workshops deliberately employed the same intensive atmosphere to get people to be receptive to new ways of thinking and being actually open to changing their minds. I recognized that this was inherently risky, and was always impressed that the ideas introduced in this state were always about how to think better rather than convince workshop participants of any conclusion. Despite many of the staff and mentors being extremely convinced of the necessity of x-risk mitigation, I never once encountered discussion of how the rationality techniques should be applied to AIS.
To hear that this type of environment is de facto being used to sway people towards a cause prioritization, rather than how to do cause prio makes me update significantly away from continuing the university pipeline as it currently exists. The comments on the funding situation are also new to me and seem to represent obvious errors. Thanks again Dave for opening my eyes to what’s currently happening.
“what does it look like, in practice, to implement ‘AI safety...should have its own movement, separate from EA’?”
Creating AI Safety focused Conferences, AI Safety university groups and AI Safety local meet-up groups? Obviously attendees will initially overlap very heavily with EA conferences and groups but having them separated out will lead to a bit of divergence over time
Wouldn’t this run the risk of worsening the lack of intellectual diversity and epistemic health that the post mentions? The growing divide between long/neartermism might have led to tensions, but I’m happy that at least there’s still conferences, groups and meet-ups where these different people are still talking to each other!
There might be an important trade-off here, and it’s not clear to me what direction makes more sense.
I don’t think there’s much of a trade-off, I’d expect a decent proportion of AI Safety people to still be coming to EA conferences
I am all for efforts to do AIS movement building distinct from EA movement building by people who are convinced by AIS reasoning and not swayed by EA principles. There’s all kinds of discussion about AIS in academic/professional/media circles that never reference EA at all. And while I’d love for everyone involved to learn about and embrace EA, I’m not expecting that. So I’m just glad they’re doing their thing and hope they’re doing it well.
I could probably have asked the question better and made it, “what should EAs do (if anything), in practice to implement a separate AIS movement?” Because then it sounds like we’re talking about making a choice to divert movement building dollars and hours away from EA movement building to distinct AI safety movement building, under the theoretical guise of trying to bolster the EA movement against getting eaten by AIS? Seems obviously backwards to me. I think EA movement building is already under-resourced, and owning our relationship with AIS is the best strategic choice to achieve broad EA goals and AIS goals.