I don’t know how much we should update on this, but I’m now personally a bit less concerned about the “self-recommending” issues of EA resources being mostly recommended by people in the EA social community.[1]
I think this is a good sign for the effective giving ecosystem, and will make my relatives much less worried about how I spend my money.
Not that I was super concerned after digging deeper into things in the past year, but I remember being really concerned about it ~2 years ago, and most people don’t have that much time to look into things.
I don’t think his foundations really does marketing for donations. My guess is that from his point of view, he thinks it’s preferable—but that he thinks GiveWell could be a good fit as a donation target to many audience members.
Postmortem on mod action on the April fools post “A modest proposal for women in EA”
Summary
Last Sunday, a bad post (definitely violating multiple forum norms) from a new anonymous user was published on the Forum. It got a lot of views and we removed it 11 hours and 30 minutes after it went up. I think we made some mistakes and wanted to write a quick postmortem.
Timeline (all times are Sunday April 2, 2023 and in CEST, although the mod team is in different timezones):
10:49 AM
The post “A modest proposal for women in EA” from a new user “ozaiscometh” goes up
1:03 PM
I notice the first user report on the forum, “maybe this is well intentioned satire, but it misses on the rather sensitive issue of sexual assault”, and start a discussion in the moderation Slack
1:57 PM
I notice a second user report on the forum
4:56 PM
JP proposes banning ozaiscometh for the post
9:47 PM
We notice a lot of other reports (~6 from memory). We notice that the post (despite being heavily downvoted) is getting a lot more views than expected (~190 at the time)
9:55 PM
A user suggests deleting the post in a comment
10:06 PM
We ban ozaiscometh, and post a comment mentioning that
10:18 PM
We remove the post
Key mistake: I allowed the post to be visible on the Forum
I read it as a bad taste (satirical) criticism arguing something like “EA doesn’t listen to my reports on sexual assault”, which in hindsight is also not ok written like that, but I was worried about censoring criticism and didn’t examine enough whether it was just straightforwardly norm-violating. I didn’t realize other readers might interpret it very differently and didn’t consider enough how sensitive the topic is. We currently approve almost all new users, except in extreme cases (e.g. spam, calls to violence, etc.). Our general approach is to let karma take care of hiding low-quality content, and actively remove content very rarely.
Learnings
Before approving users whose first post/comment is dubious or in bad taste, I’ll default to double-checking with other moderators.
The moderation team is not synched up on which posts to approve, or our criteria for approving posts are wrong
We’ll think more about this in the future.
Other considerations
The post went up on a Sunday morning and was removed less than 12 hours later. The offender was banned at the same time. Overall, we think this is an ok response time, especially given that the post was so downvoted that it was not showing up for people who weren’t actively looking for it (at least 84% of views came from Twitter, and we think that many were after the post was removed).
We think the karma system did its job. The post was at −185 karma with 55 downvotes less than 12 hours after it was posted. If I remember correctly, it was at −10 karma after less than 10 minutes.
I’m particularly grateful to the (many) users who reported the post on the Forum.
We’re doing a quick check to see if this account is an alt of another user. Edit: the forum team checked and there is no other user using the same IP address.
I read it as a bad taste (satirical) criticism arguing something like “EA doesn’t listen to my reports on sexual assault”, which in hindsight is also not ok written like that, but I was worried about censoring criticism and didn’t examine enough whether it was just straightforwardly norm-violating.
I think this is a reasonable class of concerns for you to have had; there has been a lot of media criticism of schools and libraries for removing offensive books about sex or violence, including satire, though this post does seem to have been unusually bad. It seems like you guys have been focused on making sure the forum works well for people engaging with it organically, where as you pointed out the karma system worked well. People on twitter selectively highlighting content that would not be visible to ordinary users is a very different threat model that seems both less crucial and harder to address.
I agree that this is a reasonable class of concerns to have in general, and I agree that moderators probably shouldn’t optimize for the threat of people selectively highlighting content, though it does seem to be stretching the bounds of charitability for this specific post to count as a good-faith criticism/satire.
If the issue was that the post was left up longer than you feel it should have been left up in retrospect due to deliberation time, then I’d suggest adding a feature to temporarily remove a post. This would be similar to deleting and undeleting a post except that it would show a message saying that the post is currently under mod consideration.
I would generally hate to see situations where mods feel like they have to rush a decision as I suspect that this would lead to worse decisions.
Thanks for the suggestion. There is already a similar feature (we moved the post to the user’s drafts, and could theoretically undraft it), but it doesn’t show a special message.
I personally think that the response time, ~12 hours on a Sunday, was ok (but I might be wrong on this) and that the key mistake was allowing the post in the first place.
(Didn’t see this chain when I commented below, sorry!)
I agree 12 hours on a Sunday does seem reasonable and the main decision to let the post through seems less reasonable. I do think that whatever extent you expect to be less active with moderating decisions on a weekend, the bar for allowing the post should probably be higher as a result. I do recognize this was probably made much more difficult with April Fool’s though, thanks for all the work you do for the forum!
I didn’t see the post when it first came up, so I agree the karma system was working well. There might be some hindsight bias here, but I think the post clearly seemed to fall within what I would have hoped the mod check would screen out.
I also think another commentor’s suggestion in the original post about deanonymizing the main account of this user, if one exists, is also worth considering given how clearly egregious and malicious this post was. I’d also be happy for this person to not be at future EAGs or CEA-organized events, especially given the caliber of people I have met in EA spaces who have missed out.
Bill Gates just endorsed GiveWell!
I don’t know how much we should update on this, but I’m now personally a bit less concerned about the “self-recommending” issues of EA resources being mostly recommended by people in the EA social community.[1]
I think this is a good sign for the effective giving ecosystem, and will make my relatives much less worried about how I spend my money.
Not that I was super concerned after digging deeper into things in the past year, but I remember being really concerned about it ~2 years ago, and most people don’t have that much time to look into things.
I am confused as to why he doesn’t give $1bn a year to GiveWell, but I guess let’s wait a year and see if that happens.
I don’t think his foundations really does marketing for donations. My guess is that from his point of view, he thinks it’s preferable—but that he thinks GiveWell could be a good fit as a donation target to many audience members.
That’s fantastic!! Great to see, thanks for sharing!
Postmortem on mod action on the April fools post “A modest proposal for women in EA”
Summary
Last Sunday, a bad post (definitely violating multiple forum norms) from a new anonymous user was published on the Forum. It got a lot of views and we removed it 11 hours and 30 minutes after it went up. I think we made some mistakes and wanted to write a quick postmortem.
Timeline (all times are Sunday April 2, 2023 and in CEST, although the mod team is in different timezones):
Key mistake: I allowed the post to be visible on the Forum
I read it as a bad taste (satirical) criticism arguing something like “EA doesn’t listen to my reports on sexual assault”, which in hindsight is also not ok written like that, but I was worried about censoring criticism and didn’t examine enough whether it was just straightforwardly norm-violating. I didn’t realize other readers might interpret it very differently and didn’t consider enough how sensitive the topic is.
We currently approve almost all new users, except in extreme cases (e.g. spam, calls to violence, etc.). Our general approach is to let karma take care of hiding low-quality content, and actively remove content very rarely.
Learnings
Before approving users whose first post/comment is dubious or in bad taste, I’ll default to double-checking with other moderators.
The moderation team is not synched up on which posts to approve, or our criteria for approving posts are wrong
We’ll think more about this in the future.
Other considerations
The post went up on a Sunday morning and was removed less than 12 hours later. The offender was banned at the same time. Overall, we think this is an ok response time, especially given that the post was so downvoted that it was not showing up for people who weren’t actively looking for it (at least 84% of views came from Twitter, and we think that many were after the post was removed).
We think the karma system did its job. The post was at −185 karma with 55 downvotes less than 12 hours after it was posted. If I remember correctly, it was at −10 karma after less than 10 minutes.
I’m particularly grateful to the (many) users who reported the post on the Forum.
We’re doing a quick check to see if this account is an alt of another user.
Edit: the forum team checked and there is no other user using the same IP address.
I think this is a reasonable class of concerns for you to have had; there has been a lot of media criticism of schools and libraries for removing offensive books about sex or violence, including satire, though this post does seem to have been unusually bad. It seems like you guys have been focused on making sure the forum works well for people engaging with it organically, where as you pointed out the karma system worked well. People on twitter selectively highlighting content that would not be visible to ordinary users is a very different threat model that seems both less crucial and harder to address.
I agree that this is a reasonable class of concerns to have in general, and I agree that moderators probably shouldn’t optimize for the threat of people selectively highlighting content, though it does seem to be stretching the bounds of charitability for this specific post to count as a good-faith criticism/satire.
If the issue was that the post was left up longer than you feel it should have been left up in retrospect due to deliberation time, then I’d suggest adding a feature to temporarily remove a post. This would be similar to deleting and undeleting a post except that it would show a message saying that the post is currently under mod consideration.
I would generally hate to see situations where mods feel like they have to rush a decision as I suspect that this would lead to worse decisions.
Thanks for the suggestion.
There is already a similar feature (we moved the post to the user’s drafts, and could theoretically undraft it), but it doesn’t show a special message.
I personally think that the response time, ~12 hours on a Sunday, was ok (but I might be wrong on this) and that the key mistake was allowing the post in the first place.
(Didn’t see this chain when I commented below, sorry!)
I agree 12 hours on a Sunday does seem reasonable and the main decision to let the post through seems less reasonable. I do think that whatever extent you expect to be less active with moderating decisions on a weekend, the bar for allowing the post should probably be higher as a result. I do recognize this was probably made much more difficult with April Fool’s though, thanks for all the work you do for the forum!
Thanks for this postmortem.
I didn’t see the post when it first came up, so I agree the karma system was working well. There might be some hindsight bias here, but I think the post clearly seemed to fall within what I would have hoped the mod check would screen out.
I also think another commentor’s suggestion in the original post about deanonymizing the main account of this user, if one exists, is also worth considering given how clearly egregious and malicious this post was. I’d also be happy for this person to not be at future EAGs or CEA-organized events, especially given the caliber of people I have met in EA spaces who have missed out.
Added an update to the end of the postmortem, the forum team checked and there is no other user using the same IP address.
Agree, it was a mistake.