(warning: some parts contain sardonic tone. maybe de-snarkify through ChatGPT if you donāt like that)
Ok I have a lot of issues with this post/āwhole affair,[1] so after this Iām going to limit how much I respond (though Mikhail if you want to reach out via private message then please do so, but donāt feel obliged to)
This feels like a massive storm in a teacup over what honestly feels like quite a minor issue to me. I feel like a lot of this community heat and drama could have been avoided with better communication in private from the both of you, perhaps with the aid of a trusted third party involved.
I also get a large gut impression that this falls into the broad category of āBay Area shenanigans that I donāt care aboutā. I encourage everyone too caught up in it to take a breath, count to five, donate to AMF/āGiveDirectly/āconcrete-charity-of-your-choice, and then go about their day.
I donāt think you understand how communities function. They donāt function by dictat. Do you want CEA to scan every press release from every EA-related org? Do you want to disallow any unilateral action by people in the community? Thatās nonsense. Social norms are community enforcement mechanisms, and weāre arguing about norms here. I think the organisers made a mistake, you think she violated a deontological rule. I think this has already gone too far, you think it needs a stronger response. We argue about norms, persuade each other and/āor observers, and then norms and actions changes. This is the enforcement mechanism.[2]
In any case, Iām much more interested in norms/āsocial mechanisms for improving community error-correction than I am avoiding all mistakes from the community (at least, mistakes below a certain bar). And my impression[3] is that the organisers have tried to correct the mistake to the extent that they believes they made a mistake, and anything else is going to be a matter of public debate. Again, this is how communities work.
I also think you generally overestimate your confidence about what the consequences of the protest will be, which deontological norms were broken and if so how bad they were, and how it will affect peopleās impressions in general. I agree that I wouldnāt necessarily frame the protest in the way it was, but I think thatās going to end up being a lot less consequential in the grand scheme of things than a) the community drama this has caused and b) any community enforcement mechanisms you actually get set up
This doesnāt seem to be the first time Holly has clashed with rationalist norms, and in general when this has happened I tend to find myself generally siding with her perspective over whichever rationalist sheās questioning, fair warning.
Edit: I later discovered I explicitly warned Holly this messaging could be deceptive, which she understood, asked what could be done about it, but ended up not doing anything and leaving it as is.
5- I think the messaging around the protest is deceptive. I.e., it gives people a wrong impression of the world, in a way a part of the community assumed would be better suiting their goals. I think this is deontologically bad and we shouldnāt be even talking about consequences. This is a bad thing to do regardless of how likely and how badly it can backfire. If your calculations say that itās alright, that the EV for deceiving people is positive, ignore this calculations and maybe read https://āāwww.lesswrong.com/āāposts/āāK9ZaZXDnL3SEmYZqB/āāends-don-t-justify-means-among-humans.
This is the sort of thing that led to the FTX collapse. This is the sort of decision-making procedure that predictably leads to bad consequences, no matter what our brains calculate.
And in this area, things like that backfiring (e.g., creating an impression among the general public, experts, media, or policymakers that the community isnāt fully candid) can be increasing the chance of a permanent end of life in the lightcone.
4- Iām also interested in mechanisms for early error-correction. This is what my post calls for. I donāt even mention the names of the protest organisers or their orgs, because I care about the community preventing deceptive messaging in the future.
It doesnāt have to be a community-wide mechanism or a central org that checks press-releases; it can be more field-focused orgs and people, it can be practices like fact-checking final versions of public comms and edits to them with outside people who are good at that stuff and have a different background (see comments from @Jason), it can be any of lots of things. I want the community to think about those things and discuss them. Iād be optimistic about people spending time to figure this out and suggesting to the community.
my impression is that the organisers have tried to correct the mistake to the extent that they believes they made a mistake, and anything else is going to be a matter of public debate. Again, this is how communities work
My impression is that an organiser agreed they havenāt propagated the update, as I suggest in the post, which led to a message that was misleading and this hasnāt been understood or addressed at all until I published the post. Iām not confident this is what happened, though, as looking at the messages now, it seems fairly likely that an organiser understood the impression the messaging would create and decided to keep it that way.
3- I donāt think itās fair to, while no one of us spent a while thinking about potential mechanisms, suggest those that seem like they wouldnāt seem good/āwork. Current mechanisms are bad, as they donāt prevent deontologically dubious unilateral actions. I hope there can be better mechanisms.
I think the community as a whole acted deceptively. This seems really bad and the community should probably respond and improve. We donāt want to be the kind of community that misleads people when it suits our goals. I donāt think thereās been a reaction from the community where people tried to figure out how to address the problem. Instead, most of the comments are related to a protest organiser slowly realising what the issue people are talking about was.
2- I notice Iām confused and donāt really get your point. Like, yeah, in global health and development, deceptive messaging is obviously enough unlikely be beneficial for people to not do it and not have much reason to engage with another part of the community having issues around deceptive messaging, so this is not a post that seems relevant to all the community to the extent some parts of the community identify only with these parts and not with the whole. But this is not what youāre saying; and Iām failing to understand your point. If weāre discussing a cause area (x-risk) and deceptive messaging and potentially bad consequences of it, how are other cause area relevant here? I rarely go around shrimp welfare posts, saying that I donāt care about these deep sea shenanigans, and people should donate to MIRI instead.
1- I think better communication couldāve reduced the amount of public drama; but also thereās not a lot of it here, and Iām more concerned with the community not engaging enough with the problems raised in the post.
Thank you for linking the public tweets discussing deontology. If someone believes itās OK to violate deontological norms if they calculate the consequences to be positive, I want them to directly say that. If someone decided deceptive messaging can be fine and was ok in this case, I want them to directly say that and not be deceptive towards the community as well. Weād then be able to have a discussion about cruxes in strategies and norms and not about what happened.
Iām more concerned with the community not engaging enough with the problems raised in the post.
Experience teaches that there are (at least) two types of postmortem analyses when something goes wrong: a type that is more focused on questions of blame, and one that is focused on root cause analysis /ā lessons learned /ā how to improve /ā etc. These types of inquiry struggle to coexist in the same conversation, because the former creates an adversarial tone regarding the persons against whom blame is being considered.
Now, that is not to say that blame-focused inquiries are bad, and the other type is better as a matter of course. But the title and tone of your post placed this discussion clearly in the blame-focused camp, and itās hard for the non-blamey type of conservation to form out of such an environment.
I suspect most community members felt that the proffered evidence does not sufficiently make out a case of deception (i.e., intentional misrepresentation) and are thus disinclined to participate in discussion of downstream philosophical issues that only come into play if they reach a conclusion that deception was present (vs. making a mistake).
If youāre interested in learning more about the second type of analysis, Iād suggest reading more about just culture in fields like medicine and aviation.
To many community members, including protest participants, itās pretty clear the messaging was deceptive.
A protest organiser is saying it was the curse of knowledge, but I sent them messages directly pointing out how people will see the messaging. As I mentioned elsewhere in the comments, I want to have a third party look at the messages exchanged between me and the protest organiser, if they agree.
Also, I expect many people to only skim through the post, and look at of the protest organiserās initial engagement with it or a shortform post they made before I published the post; all of these make it seem like Iām saying the organiser intentionally made the mistake they then corrected:
āI ran a successful protest at [company] yesterday. Before the night was over, Mikhail Samin, who attended the protest, sent me a document to review that accused me of what sounds like a bait and switch and deceptive practices because I made an error in my original press release (which got copied as a description on other materials) and apparently didnāt address it to his satisfaction because I didnāt change the theme of the event more radically or cancel it.ā
These post and comments have not been corrected to show that this is not what Iām talking about, when the protest organiser understood what misleading messaging the post talks about.
(warning: some parts contain sardonic tone. maybe de-snarkify through ChatGPT if you donāt like that)
Ok I have a lot of issues with this post/āwhole affair,[1] so after this Iām going to limit how much I respond (though Mikhail if you want to reach out via private message then please do so, but donāt feel obliged to)
This feels like a massive storm in a teacup over what honestly feels like quite a minor issue to me. I feel like a lot of this community heat and drama could have been avoided with better communication in private from the both of you, perhaps with the aid of a trusted third party involved.
I also get a large gut impression that this falls into the broad category of āBay Area shenanigans that I donāt care aboutā. I encourage everyone too caught up in it to take a breath, count to five, donate to AMF/āGiveDirectly/āconcrete-charity-of-your-choice, and then go about their day.
I donāt think you understand how communities function. They donāt function by dictat. Do you want CEA to scan every press release from every EA-related org? Do you want to disallow any unilateral action by people in the community? Thatās nonsense. Social norms are community enforcement mechanisms, and weāre arguing about norms here. I think the organisers made a mistake, you think she violated a deontological rule. I think this has already gone too far, you think it needs a stronger response. We argue about norms, persuade each other and/āor observers, and then norms and actions changes. This is the enforcement mechanism.[2]
In any case, Iām much more interested in norms/āsocial mechanisms for improving community error-correction than I am avoiding all mistakes from the community (at least, mistakes below a certain bar). And my impression[3] is that the organisers have tried to correct the mistake to the extent that they believes they made a mistake, and anything else is going to be a matter of public debate. Again, this is how communities work.
I also think you generally overestimate your confidence about what the consequences of the protest will be, which deontological norms were broken and if so how bad they were, and how it will affect peopleās impressions in general. I agree that I wouldnāt necessarily frame the protest in the way it was, but I think thatās going to end up being a lot less consequential in the grand scheme of things than a) the community drama this has caused and b) any community enforcement mechanisms you actually get set up
This doesnāt seem to be the first time Holly has clashed with rationalist norms, and in general when this has happened I tend to find myself generally siding with her perspective over whichever rationalist sheās questioning, fair warning.
What did you think it looked like? Vibes? Papers? Essays?
Which could ofc be wrong, youāre in possession of private messages I donāt have
Edit: I later discovered I explicitly warned Holly this messaging could be deceptive, which she understood, asked what could be done about it, but ended up not doing anything and leaving it as is.
5- I think the messaging around the protest is deceptive. I.e., it gives people a wrong impression of the world, in a way a part of the community assumed would be better suiting their goals. I think this is deontologically bad and we shouldnāt be even talking about consequences. This is a bad thing to do regardless of how likely and how badly it can backfire. If your calculations say that itās alright, that the EV for deceiving people is positive, ignore this calculations and maybe read https://āāwww.lesswrong.com/āāposts/āāK9ZaZXDnL3SEmYZqB/āāends-don-t-justify-means-among-humans.
This is the sort of thing that led to the FTX collapse. This is the sort of decision-making procedure that predictably leads to bad consequences, no matter what our brains calculate.
And in this area, things like that backfiring (e.g., creating an impression among the general public, experts, media, or policymakers that the community isnāt fully candid) can be increasing the chance of a permanent end of life in the lightcone.
4- Iām also interested in mechanisms for early error-correction. This is what my post calls for. I donāt even mention the names of the protest organisers or their orgs, because I care about the community preventing deceptive messaging in the future.
It doesnāt have to be a community-wide mechanism or a central org that checks press-releases; it can be more field-focused orgs and people, it can be practices like fact-checking final versions of public comms and edits to them with outside people who are good at that stuff and have a different background (see comments from @Jason), it can be any of lots of things. I want the community to think about those things and discuss them. Iād be optimistic about people spending time to figure this out and suggesting to the community.
My impression is that an organiser agreed they havenāt propagated the update, as I suggest in the post, which led to a message that was misleading and this hasnāt been understood or addressed at all until I published the post. Iām not confident this is what happened, though, as looking at the messages now, it seems fairly likely that an organiser understood the impression the messaging would create and decided to keep it that way.
3- I donāt think itās fair to, while no one of us spent a while thinking about potential mechanisms, suggest those that seem like they wouldnāt seem good/āwork. Current mechanisms are bad, as they donāt prevent deontologically dubious unilateral actions. I hope there can be better mechanisms.
I think the community as a whole acted deceptively. This seems really bad and the community should probably respond and improve. We donāt want to be the kind of community that misleads people when it suits our goals. I donāt think thereās been a reaction from the community where people tried to figure out how to address the problem. Instead, most of the comments are related to a protest organiser slowly realising what the issue people are talking about was.
2- I notice Iām confused and donāt really get your point. Like, yeah, in global health and development, deceptive messaging is obviously enough unlikely be beneficial for people to not do it and not have much reason to engage with another part of the community having issues around deceptive messaging, so this is not a post that seems relevant to all the community to the extent some parts of the community identify only with these parts and not with the whole. But this is not what youāre saying; and Iām failing to understand your point. If weāre discussing a cause area (x-risk) and deceptive messaging and potentially bad consequences of it, how are other cause area relevant here? I rarely go around shrimp welfare posts, saying that I donāt care about these deep sea shenanigans, and people should donate to MIRI instead.
1- I think better communication couldāve reduced the amount of public drama; but also thereās not a lot of it here, and Iām more concerned with the community not engaging enough with the problems raised in the post.
Thank you for linking the public tweets discussing deontology. If someone believes itās OK to violate deontological norms if they calculate the consequences to be positive, I want them to directly say that. If someone decided deceptive messaging can be fine and was ok in this case, I want them to directly say that and not be deceptive towards the community as well. Weād then be able to have a discussion about cruxes in strategies and norms and not about what happened.
Experience teaches that there are (at least) two types of postmortem analyses when something goes wrong: a type that is more focused on questions of blame, and one that is focused on root cause analysis /ā lessons learned /ā how to improve /ā etc. These types of inquiry struggle to coexist in the same conversation, because the former creates an adversarial tone regarding the persons against whom blame is being considered.
Now, that is not to say that blame-focused inquiries are bad, and the other type is better as a matter of course. But the title and tone of your post placed this discussion clearly in the blame-focused camp, and itās hard for the non-blamey type of conservation to form out of such an environment.
I suspect most community members felt that the proffered evidence does not sufficiently make out a case of deception (i.e., intentional misrepresentation) and are thus disinclined to participate in discussion of downstream philosophical issues that only come into play if they reach a conclusion that deception was present (vs. making a mistake).
If youāre interested in learning more about the second type of analysis, Iād suggest reading more about just culture in fields like medicine and aviation.
To many community members, including protest participants, itās pretty clear the messaging was deceptive.
A protest organiser is saying it was the curse of knowledge, but I sent them messages directly pointing out how people will see the messaging. As I mentioned elsewhere in the comments, I want to have a third party look at the messages exchanged between me and the protest organiser, if they agree.
Also, I expect many people to only skim through the post, and look at of the protest organiserās initial engagement with it or a shortform post they made before I published the post; all of these make it seem like Iām saying the organiser intentionally made the mistake they then corrected: āI ran a successful protest at [company] yesterday. Before the night was over, Mikhail Samin, who attended the protest, sent me a document to review that accused me of what sounds like a bait and switch and deceptive practices because I made an error in my original press release (which got copied as a description on other materials) and apparently didnāt address it to his satisfaction because I didnāt change the theme of the event more radically or cancel it.ā
These post and comments have not been corrected to show that this is not what Iām talking about, when the protest organiser understood what misleading messaging the post talks about.