Saying thepaper is poorly argued is not particularly helpful or convincing. Could you highlight where and why Rubi? Breadth does not de-facto mean poorly argued. If that was the case then most of the key texts in x-risk would all be poorly argued.
Importantly, breadth was necessary to make a critique. There are simply many interrelated matters that are worth critical analysis.
Several times the case against the TUA was not actually argued, merely asserted to exist along with one or two citations for which it is hard to evaluate if they represent a consensus.
As David highlights in his response: we do not argue against the TUA, but point out the unanswered questions we observed. We do not argue against the TUA , but highlight assumptions that may be incorrect or smuggle in values. Interestingly, it’s hard to find how you believe the piece is both polemic but also not directly critiquing the TUA sufficiently. Those two criticisms are in tension.
If you check our references, you will see that we cite many published papers that treat common criticisms and open questions of the TUA (mostly by advancing the research).
You spoke to 20+ reviewers, half of which were sought out to disagree with you, and not a single one could provide a case for differential technology?
Of course there are arguments for it, some of which are discussed in the forum. Our argument is that there is a lack of peer-review evidence to support differential technological development as a cornerstone of a policy approach to x-risk. Asking that we articulate and address every hypothetical counterargument is an incredibly high-bar, and one that is not applied to any other literature in the field (certainly not the key articles of the TUA we focus on). It would also make the paper far longer and broader. Again, your points are in tension.
I think the paper would have been better served by focusing on a single section, leaving the rest to future work. The style of assertions rather than argument and skipping over potential responses comes across as more polemical than evidence-seeking.
Then it wouldn’t be a critique of the TUA. It would be a piece on differential tech development or hazard-centrism.
This is remarkably similar to a critique we got from Seán Ó hÉigeartaigh. He both said that we should zoom in an focus on one section and said that we should zoom out and compare the TUA against all (?) potential alternatives. The recommendations are in tension and obnly share the commonality of making sure we write a paper that isn’t a criqitue.
We see many remaining problems in x-risk. This paper is an attempt to list those issues and point out their weaknesses and areas for improvement. It should be read similar to a research agenda.
The abstract and conclusion clearly spell-out the areas where we take a clear position, such as the need for diversity in the field, taking lessons from complex risk assessments in other areas, and democratisting policy recommendations. We do not articulate a position on degrowth, differential tech development etc. We highlight that the existing evidence basis and arguments for them are weak.
We do not position ourselves in many cases, because we believe they require further detailed work and deliberation. In that sense I agree with you that we’re covering too much—but only if the goal was to present clear positions on all these points. Since this was not the goal, I think it’s fine to list many remaining questions and point out that indeed they still are questions that require answers. If you have strong opinion on any of the questions we mention, then go ahead write a paper that argues for one side, publish it, and let’s get on with the science.
Seán also called the paper polemic several times. (Per definition = strong verbal written attack, hostile, critical). This is not necessarily an insult (Orwell’s Animal Farm is considered a polemic against totalitarianism), but I’m guessing it’s not meant in that way.
We are somewhat disappointed that one of the most upvoted responses on the forum to our piece is so vague and unhelpful. We would expect a community that has such high epistemic standards to reward comments that articulate clear, specific criticisms grounded in evidence and capable of being acted on.
Finally, the ‘speaking abstractly’ about funding. It is hard not see to see this as an insinuation that this we have consistently produced such poor scholarship that it would justify withdrawn funding. Again, this does not signal anything positive aboput the epistemics, or just sheer civility, of the community.
Thanks for taking the time to engage with my reply. I’d like to engage with a few of the points you made.
First of all, my point prefaced with ‘speaking abstractly’ was genuinely that. I thought your paper was poorly argued, but certainly within acceptable limits that it should not result in withdrawn funding. On a sufficient timeframe, everybody will put out some duds, and your organizations certainly have a track record of producing excellent work. My point was about avoiding an overcorrection, where consistently low quality work is guaranteed some share of scarce funding merely out of fear that withdrawing such funding would be seen as censorship. It’s a sign of healthy epistemics (in a dimension orthogonal to the criticisms of your post) for a community to be able to jump from a specific discussion to the general case, but I’m sorry you saw my abstraction as a personal attack.
You saw “we do not argue against the TUA, but point out the unanswered questions we observed. .. but highlight assumptions that may be incorrect or smuggle in values”. Pointing out unanswered questions and incorrect assumptions is how you argue against something! What makes your paper polemical is that you do not sufficiently check whether the questions really are unanswered, or if the assumptions really are incorrect. There is no tension between calling your paper polemical and saying you do not sufficiently critique the TUA. A more thorough critique that took counterarguments seriously and tried to address them would not be a polemic, as it would more clearly be driven by truth-seeking than hostility.
I was not “asking that we [you] articulate and address every hypothetical counterargument”, I was asking that you address any, especially the most obvious ones. Don’t just state “it is unclear why” they are believed to skip over a counterargument.
I am disappointed that you used my original post to further attack the epistemics of this community, and doubly so for claiming it failed to articulate clear, specific criticisms. The post was clear that the main failing I saw in your paper was a lack of engagement with counterarguments, specifically the case for technological differentiation and the case for avoiding the disenfranchisement of future generations through a limited democracy. I do not believe that my criticism of the paper jumping around too much rather than engaging deeply on fewer issues was ambiguous either. Ignoring these clear, specific criticisms to use the post as evidence of poor epistemics in the EA community makes me think you may be interpreting any disagreement as evidence for your point.
You spoke to 20+ reviewers, half of which were sought out to disagree with you, and not a single one could provide a case for differential technology?
Personally I read this as a straightforward accusation of dishonesty—something I would expect moderators to object to if the comment was critical (rather than supportive) of EA orthodoxy.
This is remarkably similar to a critique we got from Seán Ó hÉigeartaigh
I also found it suspicious that Rubi felt the need to comment using an anonymous throwaway account despite speaking in favor of established power structures.
To clarify, that’s not to say Rubi is necessarily Seán Ó hÉigeartaigh. I have no idea and I don’t know Seán.
However, this situation is very strange. Almost everyone on the EAforum uses their real name or a very thin pseudonym.
I am anonymous because vocally disagreeing with the status quo would probably destroy any prospects of getting hired or funded by EA orgs (see my heavily downvoted comment about my experiences somewhere at the bottom of this thread).
Seán Ó hÉigeartaigh here. Since I have been named specifically, I would like to make it clear that when I write here, I do so under Sean_o_h, and have only ever done so. I am not Rubi, and I don’t know who Rubi is. I ask that the moderators check IP addresses, and reach out to me for any information that can help confirm this.
I am on leave and have not read the rest of this discussion, or the current paper (which I imagine is greatly improved from the draft I saw), so I will not participate further in this discussion at this time.
To clear up my identity, I am not Seán and do not know him. I go by Rubi in real life, although it is a nickname rather than my given name. I did not mean for my account to be an anonymous throwaway, and I intend to keep on using this account on the EA Forum. I can understand how that would not be obvious as this was my first post, but that is coincidental. The original post generated a lot of controversy, which is why I saw it and decided to comment.
You spoke to 20+ reviewers, half of which were sought out to disagree with you, and not a single one could provide a case for differential technology?
I would have genuinely liked an answer to this. If none of the reviewers made the case, that is useful information about the selection of the reviewers. If some reviewers did, but were ignored by the authors, then it reflects negatively on the authors not to address this and say that the case for differential technology is unclear.
I am anonymous because vocally disagreeing with the status quo would probably destroy any prospects of getting hired or funded by EA orgs (see my heavily downvoted comment about my experiences somewhere at the bottom of this thread).
This clearly doesn’t apply to Rubi, so what’s up?
There are many reasons for people to use pseudonyms on the Forum, and we allow it with few restrictions. It’s also fine to have multiple accounts.
To clarify, that’s not to say Rubi is necessarily Seán Ó hÉigeartaigh. I have no idea and I don’t know Seán.
However, this situation is very strange.
What exactly is “suspicious” or “strange” here? What is the thing you suspect, and is that thing against the Forum’s rules? If not, do you think it should be?
Using vague insinuations instead of straightforwardly accusing someone doesn’t change the result — which is that Seán understandably feels like he’s been called out and needs to deny the “non-accusation”. What were you trying to accomplish by talking about Seán here?
*****
You’ve now made several comments in this thread that were rude or insulting towards other users. That’s not okay, whether or not your position happens to align with any “status quo”. (See theseexamples of comments being moderated for exactly this reason despite their position on the “popular” side of whatever thread they were a part of.)
If you want to object to someone’s argument, state your objection. Explain why they’re wrong, or what they’ve missed. This is almost always better than “I find this user suspicious” or “this user is acting in bad faith”.
Several of your comments on this thread were good. I appreciated the links here and some of the questions here. But if you continue posting rude or insulting comments, the moderation team may take action.
Personally I read this as a straightforward accusation of dishonesty—something I would expect moderators to object to if the comment was critical (rather than supportive) of EA orthodoxy.
As a moderator, I wouldn’t object to this comment no matter who made it. I see it as a criticism of someone’s work, not an accusation that the person was dishonest.
If someone wrote a paper critiquing the differential technology paradigm and spoke to lots of reviewers about it — including many who were known to be pro-DT — but didn’t cite any pro-DT arguments, it would be fine for someone to ask: “Did you really not hear any cases for the DT paradigm?”
The question doesn’t have to mean “you deliberately acted like there were no good pro-DT arguments and hoped we would believe you”. That would frankly be a silly thing to say, since Carla and Luke are obviously familiar with these arguments and know that many of their readers would also be familiar with these arguments.
It could also imply:
“You didn’t ask the kinds of questions of reviewers that would lead them to spell out their cases for DT”
“You didn’t make room in your paper to discuss the pro-DT arguments you heard, and I think you should have”
Or, more straightforwardly, you could avoid assuming any particular implication and just read the question as a question: “Why were there no pro-DT arguments in your piece?”
I personally read implication (1), because of the statement ”...made it seem like you did not do your research”.
Carla’s response read to me as a response to implication (2): “We chose not to discuss pro-DT arguments, because trying to give that kind of space to counterarguments for all of our points would be beyond the scope of our paper.” Which is a fine, reasonable response.
I think Rubi’s comment should have been more clear; it’s more important for questioners to ask good questions than for respondents to correctly guess at what the questioner meant.
Overall, as a moderator, my response to this part of Rubi’s comment is “this is unclear and could mean many things — perhaps one of these things is uncivil, but Carla answered a civil version of the question, and I’m not going to deliberately choose to interpret the question as the most uncivil version of itself.”
*****
On the level of meta-moderation, these are the things I personally look for*, in rough priority order (other mods may differ):
Comments that clearly insult another user
Comments that include an information hazard or advocate for seriously harmful action
Comments that interfere with good discourse in other ways
If you say “Rubi’s comment is unclear, which means it’s in category (3)” — you’d be right, but there are a lot of comments that are unclear, and it isn’t realistic for moderators to respond to more than a tiny fraction of them, which means I focus on comments in the first two categories.
If you say “Rubi’s comment could be taken to imply an insult, which means it’s in category (1)” — I disagree, because I don’t see any insulting read as “clear”, and there are plenty of other ways to interpret the comment.
And of course, the specific position someone takes in a debate has no bearing on how we moderate, unless a particular position is in category (2) (“we should release a plague to kill everyone”).
*I should also mention that I’m a human with limited human attention. So I’m not going to see every comment on every post. That’s why every post comes with a “report” option, which people should really use if they think a comment should be moderated:
If you report a post or comment, one or more mods will definitely look at it and at least consider your argument for why it was reportable.
Something not being moderated doesn’t imply that it’s definitely fine — it could also mean the mods haven’t read it, or that the mods didn’t read it with “moderator vision” on. There have been times I read a comment in my off time, then saw the same comment reported later and said “oh, huh, this probably should be moderated”.
Saying thepaper is poorly argued is not particularly helpful or convincing. Could you highlight where and why Rubi? Breadth does not de-facto mean poorly argued. If that was the case then most of the key texts in x-risk would all be poorly argued.
Importantly, breadth was necessary to make a critique. There are simply many interrelated matters that are worth critical analysis.
Several times the case against the TUA was not actually argued, merely asserted to exist along with one or two citations for which it is hard to evaluate if they represent a consensus.
As David highlights in his response: we do not argue against the TUA, but point out the unanswered questions we observed. We do not argue against the TUA , but highlight assumptions that may be incorrect or smuggle in values. Interestingly, it’s hard to find how you believe the piece is both polemic but also not directly critiquing the TUA sufficiently. Those two criticisms are in tension.
If you check our references, you will see that we cite many published papers that treat common criticisms and open questions of the TUA (mostly by advancing the research).
You spoke to 20+ reviewers, half of which were sought out to disagree with you, and not a single one could provide a case for differential technology?
Of course there are arguments for it, some of which are discussed in the forum. Our argument is that there is a lack of peer-review evidence to support differential technological development as a cornerstone of a policy approach to x-risk. Asking that we articulate and address every hypothetical counterargument is an incredibly high-bar, and one that is not applied to any other literature in the field (certainly not the key articles of the TUA we focus on). It would also make the paper far longer and broader. Again, your points are in tension.
I think the paper would have been better served by focusing on a single section, leaving the rest to future work. The style of assertions rather than argument and skipping over potential responses comes across as more polemical than evidence-seeking.
Then it wouldn’t be a critique of the TUA. It would be a piece on differential tech development or hazard-centrism.
This is remarkably similar to a critique we got from Seán Ó hÉigeartaigh. He both said that we should zoom in an focus on one section and said that we should zoom out and compare the TUA against all (?) potential alternatives. The recommendations are in tension and obnly share the commonality of making sure we write a paper that isn’t a criqitue.
We see many remaining problems in x-risk. This paper is an attempt to list those issues and point out their weaknesses and areas for improvement. It should be read similar to a research agenda.
The abstract and conclusion clearly spell-out the areas where we take a clear position, such as the need for diversity in the field, taking lessons from complex risk assessments in other areas, and democratisting policy recommendations. We do not articulate a position on degrowth, differential tech development etc. We highlight that the existing evidence basis and arguments for them are weak.
We do not position ourselves in many cases, because we believe they require further detailed work and deliberation. In that sense I agree with you that we’re covering too much—but only if the goal was to present clear positions on all these points. Since this was not the goal, I think it’s fine to list many remaining questions and point out that indeed they still are questions that require answers. If you have strong opinion on any of the questions we mention, then go ahead write a paper that argues for one side, publish it, and let’s get on with the science.
Seán also called the paper polemic several times. (Per definition = strong verbal written attack, hostile, critical). This is not necessarily an insult (Orwell’s Animal Farm is considered a polemic against totalitarianism), but I’m guessing it’s not meant in that way.
We are somewhat disappointed that one of the most upvoted responses on the forum to our piece is so vague and unhelpful. We would expect a community that has such high epistemic standards to reward comments that articulate clear, specific criticisms grounded in evidence and capable of being acted on.
Finally, the ‘speaking abstractly’ about funding. It is hard not see to see this as an insinuation that this we have consistently produced such poor scholarship that it would justify withdrawn funding. Again, this does not signal anything positive aboput the epistemics, or just sheer civility, of the community.
Hi Carla,
Thanks for taking the time to engage with my reply. I’d like to engage with a few of the points you made.
First of all, my point prefaced with ‘speaking abstractly’ was genuinely that. I thought your paper was poorly argued, but certainly within acceptable limits that it should not result in withdrawn funding. On a sufficient timeframe, everybody will put out some duds, and your organizations certainly have a track record of producing excellent work. My point was about avoiding an overcorrection, where consistently low quality work is guaranteed some share of scarce funding merely out of fear that withdrawing such funding would be seen as censorship. It’s a sign of healthy epistemics (in a dimension orthogonal to the criticisms of your post) for a community to be able to jump from a specific discussion to the general case, but I’m sorry you saw my abstraction as a personal attack.
You saw “we do not argue against the TUA, but point out the unanswered questions we observed. .. but highlight assumptions that may be incorrect or smuggle in values”. Pointing out unanswered questions and incorrect assumptions is how you argue against something! What makes your paper polemical is that you do not sufficiently check whether the questions really are unanswered, or if the assumptions really are incorrect. There is no tension between calling your paper polemical and saying you do not sufficiently critique the TUA. A more thorough critique that took counterarguments seriously and tried to address them would not be a polemic, as it would more clearly be driven by truth-seeking than hostility.
I was not “asking that we [you] articulate and address every hypothetical counterargument”, I was asking that you address any, especially the most obvious ones. Don’t just state “it is unclear why” they are believed to skip over a counterargument.
I am disappointed that you used my original post to further attack the epistemics of this community, and doubly so for claiming it failed to articulate clear, specific criticisms. The post was clear that the main failing I saw in your paper was a lack of engagement with counterarguments, specifically the case for technological differentiation and the case for avoiding the disenfranchisement of future generations through a limited democracy. I do not believe that my criticism of the paper jumping around too much rather than engaging deeply on fewer issues was ambiguous either. Ignoring these clear, specific criticisms to use the post as evidence of poor epistemics in the EA community makes me think you may be interpreting any disagreement as evidence for your point.
Personally I read this as a straightforward accusation of dishonesty—something I would expect moderators to object to if the comment was critical (rather than supportive) of EA orthodoxy.
I also found it suspicious that Rubi felt the need to comment using an anonymous throwaway account despite speaking in favor of established power structures.
To clarify, that’s not to say Rubi is necessarily Seán Ó hÉigeartaigh. I have no idea and I don’t know Seán.
However, this situation is very strange. Almost everyone on the EAforum uses their real name or a very thin pseudonym.
I am anonymous because vocally disagreeing with the status quo would probably destroy any prospects of getting hired or funded by EA orgs (see my heavily downvoted comment about my experiences somewhere at the bottom of this thread).
This clearly doesn’t apply to Rubi, so what’s up?
Seán Ó hÉigeartaigh here. Since I have been named specifically, I would like to make it clear that when I write here, I do so under Sean_o_h, and have only ever done so. I am not Rubi, and I don’t know who Rubi is. I ask that the moderators check IP addresses, and reach out to me for any information that can help confirm this.
I am on leave and have not read the rest of this discussion, or the current paper (which I imagine is greatly improved from the draft I saw), so I will not participate further in this discussion at this time.
To clear up my identity, I am not Seán and do not know him. I go by Rubi in real life, although it is a nickname rather than my given name. I did not mean for my account to be an anonymous throwaway, and I intend to keep on using this account on the EA Forum. I can understand how that would not be obvious as this was my first post, but that is coincidental. The original post generated a lot of controversy, which is why I saw it and decided to comment.
I would have genuinely liked an answer to this. If none of the reviewers made the case, that is useful information about the selection of the reviewers. If some reviewers did, but were ignored by the authors, then it reflects negatively on the authors not to address this and say that the case for differential technology is unclear.
There are many reasons for people to use pseudonyms on the Forum, and we allow it with few restrictions. It’s also fine to have multiple accounts.
What exactly is “suspicious” or “strange” here? What is the thing you suspect, and is that thing against the Forum’s rules? If not, do you think it should be?
Using vague insinuations instead of straightforwardly accusing someone doesn’t change the result — which is that Seán understandably feels like he’s been called out and needs to deny the “non-accusation”. What were you trying to accomplish by talking about Seán here?
*****
You’ve now made several comments in this thread that were rude or insulting towards other users. That’s not okay, whether or not your position happens to align with any “status quo”. (See these examples of comments being moderated for exactly this reason despite their position on the “popular” side of whatever thread they were a part of.)
If you want to object to someone’s argument, state your objection. Explain why they’re wrong, or what they’ve missed. This is almost always better than “I find this user suspicious” or “this user is acting in bad faith”.
Several of your comments on this thread were good. I appreciated the links here and some of the questions here. But if you continue posting rude or insulting comments, the moderation team may take action.
As a moderator, I wouldn’t object to this comment no matter who made it. I see it as a criticism of someone’s work, not an accusation that the person was dishonest.
If someone wrote a paper critiquing the differential technology paradigm and spoke to lots of reviewers about it — including many who were known to be pro-DT — but didn’t cite any pro-DT arguments, it would be fine for someone to ask: “Did you really not hear any cases for the DT paradigm?”
The question doesn’t have to mean “you deliberately acted like there were no good pro-DT arguments and hoped we would believe you”. That would frankly be a silly thing to say, since Carla and Luke are obviously familiar with these arguments and know that many of their readers would also be familiar with these arguments.
It could also imply:
“You didn’t ask the kinds of questions of reviewers that would lead them to spell out their cases for DT”
“You didn’t make room in your paper to discuss the pro-DT arguments you heard, and I think you should have”
Or, more straightforwardly, you could avoid assuming any particular implication and just read the question as a question: “Why were there no pro-DT arguments in your piece?”
I personally read implication (1), because of the statement ”...made it seem like you did not do your research”.
Carla’s response read to me as a response to implication (2): “We chose not to discuss pro-DT arguments, because trying to give that kind of space to counterarguments for all of our points would be beyond the scope of our paper.” Which is a fine, reasonable response.
I think Rubi’s comment should have been more clear; it’s more important for questioners to ask good questions than for respondents to correctly guess at what the questioner meant.
Overall, as a moderator, my response to this part of Rubi’s comment is “this is unclear and could mean many things — perhaps one of these things is uncivil, but Carla answered a civil version of the question, and I’m not going to deliberately choose to interpret the question as the most uncivil version of itself.”
*****
On the level of meta-moderation, these are the things I personally look for*, in rough priority order (other mods may differ):
Comments that clearly insult another user
Comments that include an information hazard or advocate for seriously harmful action
Comments that interfere with good discourse in other ways
If you say “Rubi’s comment is unclear, which means it’s in category (3)” — you’d be right, but there are a lot of comments that are unclear, and it isn’t realistic for moderators to respond to more than a tiny fraction of them, which means I focus on comments in the first two categories.
If you say “Rubi’s comment could be taken to imply an insult, which means it’s in category (1)” — I disagree, because I don’t see any insulting read as “clear”, and there are plenty of other ways to interpret the comment.
And of course, the specific position someone takes in a debate has no bearing on how we moderate, unless a particular position is in category (2) (“we should release a plague to kill everyone”).
*I should also mention that I’m a human with limited human attention. So I’m not going to see every comment on every post. That’s why every post comes with a “report” option, which people should really use if they think a comment should be moderated:
If you report a post or comment, one or more mods will definitely look at it and at least consider your argument for why it was reportable.
Something not being moderated doesn’t imply that it’s definitely fine — it could also mean the mods haven’t read it, or that the mods didn’t read it with “moderator vision” on. There have been times I read a comment in my off time, then saw the same comment reported later and said “oh, huh, this probably should be moderated”.
Honestly, fair enough.