It seems fairly likely (25%) to me that had Kirsten not started this discussion (on Twitter) I would have pushed the button because:
actually preventing the destruction of the world is important to me.
doing so, especially as a “trusted community member”, would hammer home the danger of well intentioned unilateralists in the way an essay can’t, and I think that idea is important.
despite being aware of lesswrong and having co-authored one post there, I didn’t really understand how seriously some people took the game previously.
worse, I was in the dangerous position of having heard enough about Petrov day to, when I read the email, think “oh yeah I basically know what this is about”, and therefore not read the announcement post.
I decided not to launch, but this was primarily because it became apparent through this discussion how socially costly it would be. I find people being angry with me on the internet unusually hard, and expect that pushing the button using the reasoning above could quite easily have cost me a significant amount of productive work (my median is ~ 1 week).
Regardless of what you think of the unilateralist’s curse, I think Petrov Day is a uniquely bad time to lambast the foibles of being a well-intentioned unilateralist.
This wasn’t intended as a “you should have felt sorry for me if I’d done a unilateralist thing without thinking”. It was intended as a way of giving more information about the probability of unilateralist action than people would otherwise have had, which seems well within the spirit of the day.
I also think it’s noteworthy that in the situation being celebrated the ability to resist social pressure was pointing in the opposite direction to the way it goes here, which seems like a problem with the current structure, but I didn’t end up finding a good way to articulate it, and someone else said something similar already.
I think either you misinterpreted my comment or I misinterpreted yours.
This wasn’t intended as a “you should have felt sorry for me if I’d done a unilateralist thing without thinking”.
I’m genuinely confused how you could have gotten that interpretation from my comment.
It was intended as a way of giving more information about the probability of unilateralist action than people would otherwise have had, which seems well within the spirit of the day.
So to be clearer, Petrov here shouldnaively be read as a well-intentioned unilateralist. He happened to be right, and reasonable people can disagree about whether he was wrong but lucky or right all along. Regardless, I think it’s not very much in the spirit of the day to talk about or act out all the harms of being a well-intentioned unilateralist, though if you wish to do so, more power to you.
I also think it’s noteworthy that in the situation being celebrated the ability to resist social pressure was pointing in the opposite direction to the way it goes here, which seems like a problem with the current structure.
I agree, and have complained about this before. I’m also complaining about it now, in case that was not previously clear.
I interpreted your comment as saying that I was “lambasting the foibles of being a well intentioned unilateralist”, and that I should not be doing so. If that was not the intent I’m glad.
Doing so, especially as a “trusted community member”, would hammer home the danger of well intentioned unilateralists in the way an essay can’t, and I think that idea is important.
It seems to me that either the decision to push the button is net negative, and you shouldn’t do it, or it isn’t, and if you do it people should learn the lesson “people in my community will do helpful net-positive things”. There’s something strange about the reasoning of “if I do X, people will realize that people do things like X for reasons like Y, even tho I would not be doing it for reasons like Y” (compare e.g. “I will lie about Santa to my child because that will teach them that other people in the world aren’t careful about only communicating true things”, which I am similarly suspicious of).
Yeah I guess you could read what I’m saying as that I actually think I should have pressed it for these reasons, but my moral conviction is not strong enough to have borne the social cost of doing so.
One read of that is that the community is strong enough in its social pressure to quiet bad actors like me from doing stupid harmful stuff we think is right.
Another is that social pressure is often enough to stop people from doing the right thing, and that we should be extra grateful to Petrov, and others in similar situations, because of this.
But if you actually should press the button, and do so because you correctly understand why you should, then people shouldn’t learn the lesson “people will do wild crazy stuff out of misunderstandings or malice”, because that won’t be what happened.
Perhaps the idea is that it should be a symbolic reminder that trusted community members could do bad things, rather than evidence for that proposition?
This is closer, I think the framing I might have had in mind is closer to:
people underestimate the probability of tail risks.
I think one of the reasons why is that they don’t appreciate the size of the space of unknown unknowns (which in this case includes people pushing the button for reasons like this).
causing them to see something from the unknown unknown space is therefore useful.
I think last year’s phishing incident was actually a reasonable example of this. I don’t think many people would have put sufficiently high probability on it happening, even given the button getting pressed.
It seems fairly likely (25%) to me that had Kirsten not started this discussion (on Twitter) I would have pushed the button because:
actually preventing the destruction of the world is important to me.
doing so, especially as a “trusted community member”, would hammer home the danger of well intentioned unilateralists in the way an essay can’t, and I think that idea is important.
despite being aware of lesswrong and having co-authored one post there, I didn’t really understand how seriously some people took the game previously.
worse, I was in the dangerous position of having heard enough about Petrov day to, when I read the email, think “oh yeah I basically know what this is about”, and therefore not read the announcement post.
I decided not to launch, but this was primarily because it became apparent through this discussion how socially costly it would be. I find people being angry with me on the internet unusually hard, and expect that pushing the button using the reasoning above could quite easily have cost me a significant amount of productive work (my median is ~ 1 week).
Regardless of what you think of the unilateralist’s curse, I think Petrov Day is a uniquely bad time to lambast the foibles of being a well-intentioned unilateralist.
This wasn’t intended as a “you should have felt sorry for me if I’d done a unilateralist thing without thinking”. It was intended as a way of giving more information about the probability of unilateralist action than people would otherwise have had, which seems well within the spirit of the day.
I also think it’s noteworthy that in the situation being celebrated the ability to resist social pressure was pointing in the opposite direction to the way it goes here, which seems like a problem with the current structure, but I didn’t end up finding a good way to articulate it, and someone else said something similar already.
I think either you misinterpreted my comment or I misinterpreted yours.
I’m genuinely confused how you could have gotten that interpretation from my comment.
So to be clearer, Petrov here should naively be read as a well-intentioned unilateralist. He happened to be right, and reasonable people can disagree about whether he was wrong but lucky or right all along. Regardless, I think it’s not very much in the spirit of the day to talk about or act out all the harms of being a well-intentioned unilateralist, though if you wish to do so, more power to you.
I agree, and have complained about this before. I’m also complaining about it now, in case that was not previously clear.
I interpreted your comment as saying that I was “lambasting the foibles of being a well intentioned unilateralist”, and that I should not be doing so. If that was not the intent I’m glad.
I interpreted you as proposing doing an unilateralist action to demonstrate to others the harm of unilateralist actions. Apologies if I misread.
It seems to me that either the decision to push the button is net negative, and you shouldn’t do it, or it isn’t, and if you do it people should learn the lesson “people in my community will do helpful net-positive things”. There’s something strange about the reasoning of “if I do X, people will realize that people do things like X for reasons like Y, even tho I would not be doing it for reasons like Y” (compare e.g. “I will lie about Santa to my child because that will teach them that other people in the world aren’t careful about only communicating true things”, which I am similarly suspicious of).
Yeah I guess you could read what I’m saying as that I actually think I should have pressed it for these reasons, but my moral conviction is not strong enough to have borne the social cost of doing so.
One read of that is that the community is strong enough in its social pressure to quiet bad actors like me from doing stupid harmful stuff we think is right.
Another is that social pressure is often enough to stop people from doing the right thing, and that we should be extra grateful to Petrov, and others in similar situations, because of this.
Either reading seems reasonable to discuss today.
But if you actually should press the button, and do so because you correctly understand why you should, then people shouldn’t learn the lesson “people will do wild crazy stuff out of misunderstandings or malice”, because that won’t be what happened.
The lesson people I would want people to learn is “I might not have considered all the reasons people might do stuff”. See comment below.
Perhaps the idea is that it should be a symbolic reminder that trusted community members could do bad things, rather than evidence for that proposition?
This is closer, I think the framing I might have had in mind is closer to:
people underestimate the probability of tail risks.
I think one of the reasons why is that they don’t appreciate the size of the space of unknown unknowns (which in this case includes people pushing the button for reasons like this).
causing them to see something from the unknown unknown space is therefore useful.
I think last year’s phishing incident was actually a reasonable example of this. I don’t think many people would have put sufficiently high probability on it happening, even given the button getting pressed.