Some thoughts I had while reading that I expect you’d agree with:
There is probably a lot of overlap in the kinds of interventions that (some) AI safety folks would be on board with and the kinds of interventions that (some) AI ethics folks would be on board with. For example, it seems like (many people in) both groups have concerns about the rate of AI progress and would endorse regulations/policies that promote safe/responsible AI development.
Given recent developments in AI, and apparent interest in regulation that promotes safety, it seems like now might be a particularly good time for people to think seriously about how the AIS community and the AI ethics community could work together.
Despite differences, it would be surprising if there was rather little that the “two” communities could learn from each other.
I appreciate the links and examples. I’ll probably go through them at some point soon and possibly DM you. I think a lot of people are interested in this topic, but few have the time/background to actually “do research” and “compile resources”. It seems plausible to me that more “lists of resources/examples/case studies” could improve reasoning on this topic (even moreso than high-level argumentation, and I say that as someone who’s often advocating for more high-level argumentation!)
Some thoughts I had while reading that you might disagree with (or at least I didn’t see acknowledged much in the post):
The differences between the two groups are not trivial, and they’ll often lead to different recommendations. For example, if you brought ARC Evals together with (hypothetical) AI Ethics Evals, I imagine they would both agree “evals are important” but they would have strong and serious disagreements about what kinds of evals should be implemented.
In general, when two groups with different worldviews/priorities join coalitions, a major risk is that one (or both) of the groups’ goals get diluted.
It’s harder to maintain good epistemics and strong reasoning + reasoning transparency in large coalitions of groups who have different worldviews/goals. (“We shouldn’t say X because our allies in AI ethics will think it’s weird.”) I don’t think “X is bad for epistemics” means “we definitely shouldn’t consider X”, but I think it’s a pretty high cost that often goes underappreciated/underacknowledged (Holden made a similar point recently).
In general, I think the piece could have benefitted from expressing more uncertainty around certain claims, acknowledging counterarguments more, and trying to get an ITT of people who disagree with you.
’It’s harder to maintain good epistemics and strong reasoning + reasoning transparency in large coalitions of groups who have different worldviews/goals. (“We shouldn’t say X because our allies in AI ethics will think it’s weird.”) I don’t think “X is bad for epistemics” means “we definitely shouldn’t consider X”, but I think it’s a pretty high cost that often goes underappreciated/underacknowledged’
This is probably a real epistemic cost in my view, but it takes more than identifying a cost to establish that forming a coalition with people with different goals/beliefs is overall epistemically costly, given that doing so also has positive effects like bringing in knowledge that we don’t have because no group knows everything.
Just quickly on that last point: I recognise there is a lot of uncertainty (hence the disclaimer at the beginning). I didn’t go through the possible counterarguments because the piece was already so long! Thanks for your comment though, and I will get to the rest of it later!
Interesting that you don’t think the post acknowledged your second collection of points. I thought it mostly did. 1. The post did say it was not suggesting to shut down existing initiatives. So where people disagree on (for example) which evals to do, they can just do the ones they think are important and then both kinds get done. I think the post was identifying a third set of things we can do together, and this was not specific evals, but more about big narrative alliance when influencing large/important audiences. The post also suggested some other areas of collaboration, on policy and regulation, and some of these may relate to evals so there could be room for collaboration there, but I’d guess that more demand, funding, infrastructure for evals helps both kinds of evals. 2. Again I think the post addresses this issue: it talks about how there is this specific set of things the two groups can work on together that is both in their interest to do. It doesn’t mean that all people from each group will only work on this new third thing (coalition building), but if a substantial number do, it’ll help. I don’t think the OP was suggesting a full merger of the groups. They acknowledge the ‘personal and ethical problems with one another; [and say] that needn’t translate to political issues’. The call is specifically for political coalition building. 3. Again I don’t think the OP is calling for a merger of the groups. They are calling for collaborating on something. 4. OK the post didn’t do this that much, but I don’t think every post needs to and I personally really liked that this one made its point so clearly. I would read a post which responds to this with some counterarguments with interest so maybe that implies I think it’d benefit from one too, but I wouldn’t want a rule/social expectation that every post lists counterarguments as that can raise the barrier to entry for posting and people are free to comment in disagreements and write counter posts.
On evals, I think it is good for us to be doing as much evals as possible, firstly because both sorts of evaluations are important, but also more (even self imposed) regulatory hurdles to jump through, the better. Slow it down and bring the companies under control.
Indeed, the call is a broader political coalition building. Not everyone, not all the time, not on everything. But on substantially more than we currently are.
Yes
There are a number of counterarguments to this post, but I didn’t include them because a) I probably can’t give the strongest counterarguments to my own beliefs b) This post was already very long, and I had to cut out sections already on Actor-Network Theory and Agency and something else I can’t remember c) I felt it might muddle the case I’m trying to make here if it was intersperced with counterarguments. One quick point on counterarguments is I think a counterargument would need to be strong enough to not just prove that the extreme end result is bad ( a lot more coalition building would be bad ) , but probably that the post is directionally bad (some more coalition building would be bad).
Some thoughts I had while reading that I expect you’d agree with:
There is probably a lot of overlap in the kinds of interventions that (some) AI safety folks would be on board with and the kinds of interventions that (some) AI ethics folks would be on board with. For example, it seems like (many people in) both groups have concerns about the rate of AI progress and would endorse regulations/policies that promote safe/responsible AI development.
Given recent developments in AI, and apparent interest in regulation that promotes safety, it seems like now might be a particularly good time for people to think seriously about how the AIS community and the AI ethics community could work together.
Despite differences, it would be surprising if there was rather little that the “two” communities could learn from each other.
I appreciate the links and examples. I’ll probably go through them at some point soon and possibly DM you. I think a lot of people are interested in this topic, but few have the time/background to actually “do research” and “compile resources”. It seems plausible to me that more “lists of resources/examples/case studies” could improve reasoning on this topic (even moreso than high-level argumentation, and I say that as someone who’s often advocating for more high-level argumentation!)
Some thoughts I had while reading that you might disagree with (or at least I didn’t see acknowledged much in the post):
The differences between the two groups are not trivial, and they’ll often lead to different recommendations. For example, if you brought ARC Evals together with (hypothetical) AI Ethics Evals, I imagine they would both agree “evals are important” but they would have strong and serious disagreements about what kinds of evals should be implemented.
In general, when two groups with different worldviews/priorities join coalitions, a major risk is that one (or both) of the groups’ goals get diluted.
It’s harder to maintain good epistemics and strong reasoning + reasoning transparency in large coalitions of groups who have different worldviews/goals. (“We shouldn’t say X because our allies in AI ethics will think it’s weird.”) I don’t think “X is bad for epistemics” means “we definitely shouldn’t consider X”, but I think it’s a pretty high cost that often goes underappreciated/underacknowledged (Holden made a similar point recently).
In general, I think the piece could have benefitted from expressing more uncertainty around certain claims, acknowledging counterarguments more, and trying to get an ITT of people who disagree with you.
’It’s harder to maintain good epistemics and strong reasoning + reasoning transparency in large coalitions of groups who have different worldviews/goals. (“We shouldn’t say X because our allies in AI ethics will think it’s weird.”) I don’t think “X is bad for epistemics” means “we definitely shouldn’t consider X”, but I think it’s a pretty high cost that often goes underappreciated/underacknowledged’
This is probably a real epistemic cost in my view, but it takes more than identifying a cost to establish that forming a coalition with people with different goals/beliefs is overall epistemically costly, given that doing so also has positive effects like bringing in knowledge that we don’t have because no group knows everything.
Just quickly on that last point: I recognise there is a lot of uncertainty (hence the disclaimer at the beginning). I didn’t go through the possible counterarguments because the piece was already so long! Thanks for your comment though, and I will get to the rest of it later!
Interesting that you don’t think the post acknowledged your second collection of points. I thought it mostly did.
1. The post did say it was not suggesting to shut down existing initiatives. So where people disagree on (for example) which evals to do, they can just do the ones they think are important and then both kinds get done. I think the post was identifying a third set of things we can do together, and this was not specific evals, but more about big narrative alliance when influencing large/important audiences. The post also suggested some other areas of collaboration, on policy and regulation, and some of these may relate to evals so there could be room for collaboration there, but I’d guess that more demand, funding, infrastructure for evals helps both kinds of evals.
2. Again I think the post addresses this issue: it talks about how there is this specific set of things the two groups can work on together that is both in their interest to do. It doesn’t mean that all people from each group will only work on this new third thing (coalition building), but if a substantial number do, it’ll help. I don’t think the OP was suggesting a full merger of the groups. They acknowledge the ‘personal and ethical problems with one another; [and say] that needn’t translate to political issues’. The call is specifically for political coalition building.
3. Again I don’t think the OP is calling for a merger of the groups. They are calling for collaborating on something.
4. OK the post didn’t do this that much, but I don’t think every post needs to and I personally really liked that this one made its point so clearly. I would read a post which responds to this with some counterarguments with interest so maybe that implies I think it’d benefit from one too, but I wouldn’t want a rule/social expectation that every post lists counterarguments as that can raise the barrier to entry for posting and people are free to comment in disagreements and write counter posts.
Ye I basically agree with this.
On evals, I think it is good for us to be doing as much evals as possible, firstly because both sorts of evaluations are important, but also more (even self imposed) regulatory hurdles to jump through, the better. Slow it down and bring the companies under control.
Indeed, the call is a broader political coalition building. Not everyone, not all the time, not on everything. But on substantially more than we currently are.
Yes
There are a number of counterarguments to this post, but I didn’t include them because a) I probably can’t give the strongest counterarguments to my own beliefs b) This post was already very long, and I had to cut out sections already on Actor-Network Theory and Agency and something else I can’t remember c) I felt it might muddle the case I’m trying to make here if it was intersperced with counterarguments. One quick point on counterarguments is I think a counterargument would need to be strong enough to not just prove that the extreme end result is bad ( a lot more coalition building would be bad ) , but probably that the post is directionally bad (some more coalition building would be bad).