As of this writing, the suggestion “EA institutions should select for diversity with respect to hiring” has a karma of 17 upvotes, −21 disagreement (with 52 votes).
My suggestion “EA orgs should aim to be less politically and demographically homogenous” has 14 upvotes, +27 agreement (with 21 votes).
Why are these two statements so massively different in agreement score?
These suggestions, while not exactly equivalent, seem very similar. (How exactly will you become less demographically homogenous without aiming to be more diverse in hiring?)
My hypothesis is that either EA likes vaguer statements, but is allergic to more concrete proposals, or that people are reflexively downvoting anything that comes off as culture warrish or “woke”. I’d be interested in hearing from anyone that downvoted statement 1 and upvoted statement 2.
This also reveals the limitations of this method for actually making decisions: small changes in wording can have a huge effect on the result.
Statement 2 can be furthered by a number of methods—e.g., seeking new people and new hires in more/different places. Its easy to agree as long as you think there is at least one method of furthering the end goal you would support.
Statement 1 reads like a specific method with a specific tradeoff/cost. As I read it, it calls for sometimes hiring Person X for diversity reasons even though you think Person Y would have been a better choice otherwise (otherwise, “select for diversity” isn’t actually doing any work).
I don’t think this is just a small change in wording. It’s unsurprising to me that more people would endorse a goal like Statement 2 than a specific tradeoff like Statement 1.
I think that makes sense as a reason, if that’s how people interpreted the two statements. However, statement 1 was explicitly not referring to a narrow “hire a worse candidate” situtation. Statement 1 came from the megapost, which was linked along with statement 1. Heres a relevant passage:
Worryingly, EA institutions seem to select against diversity. Hiring and funding practices often select for highly value-aligned yet inexperienced individuals over outgroup experts, university recruitment drives are deliberately targeted at the Sam Demographic (at least by proxy) and EA organisations are advised to maintain a high level of internal value-alignment to maximise operational efficiency. The 80,000 Hours website seems purpose-written for Sam, and is noticeably uninterested in people with humanities or social sciences backgrounds.
They are advocating for the exact same things you are, eg “seeking new people and new hires in more/different places”, and that’s what they meant by selecting for diversity in hiring.
I think this makes it clearer what happened. Statement 1 resembles an existing culture war debate, so people assumed it was advocating for a side and position in said debate, and downvoted, whereas statement 2 appeared more neutral, so it was upvoted. I think this really just tells us to to be careful with interpreting these upvote/downvote polls.
People likely read it as a standalone statement without referring back to the megapost, and gave “select” its most common meaning in ordinary jargon. I agree that the wording of these items is tricky and can skew outcomes, I just feel the summary here did not accurately capture what the broader statement said. So I am not convinced that voters were actually inconsistent or that this finding represents a deep problem with this kind of sorting exercise.
To be clear, you’re saying that Nathan took the megapost out of context in a way that suggested a different interpretation of their words, which lead to a highly downvoted answer. (I’m not suggesting he did this on purpose). In other words, the framing of an answer has a large effect on the final result.
I think this does represent a problem with the sorting exercise. If it hadn’t been for my followup, the takeaway could have easily been “EA doesn’t like diversity”, when the actual takeaway is “EA likes diversity, but doesn’t like this one specific hiring tactic, which was never actually mentioned anywhere”.
Yes. We may not be that far apart on this one now. The validity of the results is only as good as the extent to which the answer stems accurately convey what you are trying to measure.
Although I understand why Nathan wrote it as he did, this answer stem isn’t (in my opinion) a good reflection of the underlying text because that text used “select” in a less common way that is only clear in context. Thus, the response to the stem only has validity, at most, for what the stem itself actually says.
I think the need for a summary to accurately reflect the idea in question is endemic to all attempts to gauge opinion, not just this method. Writing good summaries can be hard.
See my comment above on the political version—usually when people call for more diversity, they are not referring to adding political diversity. So I think the additional of political makes it significantly different.
As of this writing, the suggestion “EA institutions should select for diversity with respect to hiring” has a karma of 17 upvotes, −21 disagreement (with 52 votes).
My suggestion “EA orgs should aim to be less politically and demographically homogenous” has 14 upvotes, +27 agreement (with 21 votes).
Why are these two statements so massively different in agreement score?
These suggestions, while not exactly equivalent, seem very similar. (How exactly will you become less demographically homogenous without aiming to be more diverse in hiring?)
My hypothesis is that either EA likes vaguer statements, but is allergic to more concrete proposals, or that people are reflexively downvoting anything that comes off as culture warrish or “woke”. I’d be interested in hearing from anyone that downvoted statement 1 and upvoted statement 2.
This also reveals the limitations of this method for actually making decisions: small changes in wording can have a huge effect on the result.
Statement 2 can be furthered by a number of methods—e.g., seeking new people and new hires in more/different places. Its easy to agree as long as you think there is at least one method of furthering the end goal you would support.
Statement 1 reads like a specific method with a specific tradeoff/cost. As I read it, it calls for sometimes hiring Person X for diversity reasons even though you think Person Y would have been a better choice otherwise (otherwise, “select for diversity” isn’t actually doing any work).
I don’t think this is just a small change in wording. It’s unsurprising to me that more people would endorse a goal like Statement 2 than a specific tradeoff like Statement 1.
I think that makes sense as a reason, if that’s how people interpreted the two statements. However, statement 1 was explicitly not referring to a narrow “hire a worse candidate” situtation. Statement 1 came from the megapost, which was linked along with statement 1. Heres a relevant passage:
They are advocating for the exact same things you are, eg “seeking new people and new hires in more/different places”, and that’s what they meant by selecting for diversity in hiring.
I think this makes it clearer what happened. Statement 1 resembles an existing culture war debate, so people assumed it was advocating for a side and position in said debate, and downvoted, whereas statement 2 appeared more neutral, so it was upvoted. I think this really just tells us to to be careful with interpreting these upvote/downvote polls.
People likely read it as a standalone statement without referring back to the megapost, and gave “select” its most common meaning in ordinary jargon. I agree that the wording of these items is tricky and can skew outcomes, I just feel the summary here did not accurately capture what the broader statement said. So I am not convinced that voters were actually inconsistent or that this finding represents a deep problem with this kind of sorting exercise.
To be clear, you’re saying that Nathan took the megapost out of context in a way that suggested a different interpretation of their words, which lead to a highly downvoted answer. (I’m not suggesting he did this on purpose). In other words, the framing of an answer has a large effect on the final result.
I think this does represent a problem with the sorting exercise. If it hadn’t been for my followup, the takeaway could have easily been “EA doesn’t like diversity”, when the actual takeaway is “EA likes diversity, but doesn’t like this one specific hiring tactic, which was never actually mentioned anywhere”.
Yes. We may not be that far apart on this one now. The validity of the results is only as good as the extent to which the answer stems accurately convey what you are trying to measure.
Although I understand why Nathan wrote it as he did, this answer stem isn’t (in my opinion) a good reflection of the underlying text because that text used “select” in a less common way that is only clear in context. Thus, the response to the stem only has validity, at most, for what the stem itself actually says.
I think the need for a summary to accurately reflect the idea in question is endemic to all attempts to gauge opinion, not just this method. Writing good summaries can be hard.
See my comment above on the political version—usually when people call for more diversity, they are not referring to adding political diversity. So I think the additional of political makes it significantly different.