Some kind of anonymous survey mechanism that managed to capture people who had interacted with EA in a low-to-medium-intensity way (eg, through the Facebook group or one of the websites, through playing a giving game at a campus group, attending one meet up of a campus group, etc) and tracked a) whether they interacted at higher-intensity (eg, applying to EAG) later, and b) whether they internally felt it was welcoming.
My current belief is something like “EA is unwelcoming to people not in the standard EA demographic”. So based on that:
Weak evidence:
EA demographics have been moving towards “normal” (eg. less gender bias, more racial diversity, but probably still mostly relatively rich people). A priori, I would expect this to happen anyway, but at a pretty gradual rate, something like 1-5 percentage points per year for gender bias.
Moderate evidence:
Data that shows that people not in the standard EA demographic are just as likely to be committed EAs after a medium/high interaction that involved meeting other EAs in the real world (eg. physical meetups or EA Global).
Data that shows that people not in the standard EA demographic are just as likely to be committed EAs after a low/medium interaction (eg. a Giving Game). A priori, I would expect that they are just as likely to want to learn more, but are less likely to continue on the path after learning more and engaging with the community.
Something similar to the above two, like Ajeya’s suggestion.
Strong evidence:
The majority of other local EA group leaders disagree with me.
For this issue specifically, I trust observations made by local group organizers more than I would trust large scale observational/correlational data, just because I can imagine so many different factors at play here that even if the data supported the hypothesis that EA as a whole is welcoming, I would still expect there to be several subfactors where we could and should still improve. (Though I could be convinced that it may not be worthwhile to do so.)
Some kind of anonymous survey mechanism that managed to capture people who had interacted with EA in a low-to-medium-intensity way (eg, through the Facebook group or one of the websites, through playing a giving game at a campus group, attending one meet up of a campus group, etc) and tracked a) whether they interacted at higher-intensity (eg, applying to EAG) later, and b) whether they internally felt it was welcoming.
My current belief is something like “EA is unwelcoming to people not in the standard EA demographic”. So based on that:
Weak evidence:
EA demographics have been moving towards “normal” (eg. less gender bias, more racial diversity, but probably still mostly relatively rich people). A priori, I would expect this to happen anyway, but at a pretty gradual rate, something like 1-5 percentage points per year for gender bias.
Moderate evidence:
Data that shows that people not in the standard EA demographic are just as likely to be committed EAs after a medium/high interaction that involved meeting other EAs in the real world (eg. physical meetups or EA Global).
Data that shows that people not in the standard EA demographic are just as likely to be committed EAs after a low/medium interaction (eg. a Giving Game). A priori, I would expect that they are just as likely to want to learn more, but are less likely to continue on the path after learning more and engaging with the community.
Something similar to the above two, like Ajeya’s suggestion.
Strong evidence:
The majority of other local EA group leaders disagree with me.
For this issue specifically, I trust observations made by local group organizers more than I would trust large scale observational/correlational data, just because I can imagine so many different factors at play here that even if the data supported the hypothesis that EA as a whole is welcoming, I would still expect there to be several subfactors where we could and should still improve. (Though I could be convinced that it may not be worthwhile to do so.)
You could also ask these questions of EAG attendees who had relatively little contact with the community before attending.