I do agree that there is some risk, and it’s certainly worth some thought and research. However, in the EA context, the cause areas should have effective interventions. Due to all this uncertainty, AI risk seems a very low-priority cause, since we cannot be sure if the research and other projects funded have any real impact. It would seem more beneficial to use the money for interventions that have been proved effective. That is why I think that EA is a wrong platform for AI risk discussion.
On the standard “importance, tractability, neglectedness” framework, I agree that tractability is AI risk’s worst feature if that’s what you mean. I think there is some consensus on this amongst people worried about the issue, as stated in 80k’s recently updated profile on the issue:
“Making progress on preventing an AI-related catastrophe seems hard, but there are a lot of avenues for more research and the field is very young. So we think it’s moderately tractable, though we’re highly uncertain — again, assessments of the tractability of making AI safe vary enormously.”
I think these other two aspects, importance and neglectedness, just matter a great deal and it would be a bad idea to disqualify cause areas just for moderately weak tractability. In terms of importance, transformative AI seems like it could easily be the most powerful technology we’ve ever made, for roughly the same reasons that humans are the most transformative “technology” on Earth right now. But even if you think this is overrated, consider the relatively meager funds and tiny field as it exists today. I think many people who find the risk a bit out there would at least agree with you that it’s “worth some thought and research”, but because of the rarity of the type of marginal thinking about good and willingness to take weird-sounding ideas seriously found in EA, practically no one else is ensuring that there is some thought and research. The field would, arguably, almost entirely dry up if EA stopped routing resources and people towards it.
Again though, I think maybe some of the disagreement is bound up in the “some risk” idea. My vague impression, and correct me if this doesn’t describe you, is that people who are weirded out by EA working on this as a cause area think that it’s a bit like if EA was getting people, right now, to work on risks from alien invasions (and then a big question is why isn’t it?), whereas people like me who are worried about it think that it is closer to working on risks from alien invasions if NASA discovered an alien spaceship parked five lightyears away from us. The risks here would still be very uncertain, the timelines, what we might be able to do to help, what sorts of things these aliens would be able to or want to do, but I think it would still look crazy if almost no one was looking into it, and I would be very wary of telling one of the only groups that was trying to look into it that they should let someone else handle it.
If you would like I would be happy to chat more about this, either by DMs, or email, or voice/video call. I’m probably not the most qualified person since I’m not in the field, but in a way that might give you a better sense of why the typical EA who is worried about this is. I guess I would like to make this an open invitation for anyone this post resonates with. Feel absolutely no pressure to though, and if you prefer I could just link some resources I think are helpful.
I’m just in the awkward position of both being very worried about this risk, and being very worried about how EA talking about this risk might put potential EAs off. I think it would be a real shame if you felt unwelcome or uncomfortable in the movement because you disagree about this risk, and if there’s something I can do to try to at least persuade you that those of us who are worried are worth sharing the movement with at least, I would like to try to do that.
I do agree that there is some risk, and it’s certainly worth some thought and research. However, in the EA context, the cause areas should have effective interventions. Due to all this uncertainty, AI risk seems a very low-priority cause, since we cannot be sure if the research and other projects funded have any real impact. It would seem more beneficial to use the money for interventions that have been proved effective. That is why I think that EA is a wrong platform for AI risk discussion.
On the standard “importance, tractability, neglectedness” framework, I agree that tractability is AI risk’s worst feature if that’s what you mean. I think there is some consensus on this amongst people worried about the issue, as stated in 80k’s recently updated profile on the issue:
“Making progress on preventing an AI-related catastrophe seems hard, but there are a lot of avenues for more research and the field is very young. So we think it’s moderately tractable, though we’re highly uncertain — again, assessments of the tractability of making AI safe vary enormously.”
I think these other two aspects, importance and neglectedness, just matter a great deal and it would be a bad idea to disqualify cause areas just for moderately weak tractability. In terms of importance, transformative AI seems like it could easily be the most powerful technology we’ve ever made, for roughly the same reasons that humans are the most transformative “technology” on Earth right now. But even if you think this is overrated, consider the relatively meager funds and tiny field as it exists today. I think many people who find the risk a bit out there would at least agree with you that it’s “worth some thought and research”, but because of the rarity of the type of marginal thinking about good and willingness to take weird-sounding ideas seriously found in EA, practically no one else is ensuring that there is some thought and research. The field would, arguably, almost entirely dry up if EA stopped routing resources and people towards it.
Again though, I think maybe some of the disagreement is bound up in the “some risk” idea. My vague impression, and correct me if this doesn’t describe you, is that people who are weirded out by EA working on this as a cause area think that it’s a bit like if EA was getting people, right now, to work on risks from alien invasions (and then a big question is why isn’t it?), whereas people like me who are worried about it think that it is closer to working on risks from alien invasions if NASA discovered an alien spaceship parked five lightyears away from us. The risks here would still be very uncertain, the timelines, what we might be able to do to help, what sorts of things these aliens would be able to or want to do, but I think it would still look crazy if almost no one was looking into it, and I would be very wary of telling one of the only groups that was trying to look into it that they should let someone else handle it.
If you would like I would be happy to chat more about this, either by DMs, or email, or voice/video call. I’m probably not the most qualified person since I’m not in the field, but in a way that might give you a better sense of why the typical EA who is worried about this is. I guess I would like to make this an open invitation for anyone this post resonates with. Feel absolutely no pressure to though, and if you prefer I could just link some resources I think are helpful.
I’m just in the awkward position of both being very worried about this risk, and being very worried about how EA talking about this risk might put potential EAs off. I think it would be a real shame if you felt unwelcome or uncomfortable in the movement because you disagree about this risk, and if there’s something I can do to try to at least persuade you that those of us who are worried are worth sharing the movement with at least, I would like to try to do that.