I think there should be more questions under the ‘extra credit’ section. I was willing to spend more time on this, and I think there are other views I would be interested in understanding from the average EA.
A low effort attempt of listing a few things which come to mind:
We’d like to include more questions in the extra credit section, and I agree it would be useful to ask more about the topics you suggest.
Unfortunately, we don’t find that adding more questions to the extra credit section is completely ‘free’. Even though it’s explicitly optional, we still find people sometimes complain about the length including the optional extra credit section. And there’s still a tradeoff in terms of how many people complete all or part of the extra credit section. We’ll continue to keep track of how many people complete the survey (and different sections of it) over time to try to optimise the number of extra questions we can include. For example, last year about 50% of respondents started the extra credit section and about 25% finished it.
Notably we do have an opt-in extra survey, sent out some time after the main EA Survey. Previously we’ve used this to include questions requested by EA (usually academic) researchers, whose questions we couldn’t prioritise including in the main survey (even in extra credit). Because this is completely opt-in and separate from the EA Survey, we’re more liberal about including more questions, though there are still length constraints. Last year about 60% (900) people opted in to receive this, though a smaller number actually completed the survey when it was sent out.
We’ve previously included questions on some of the topics which you mention, though of course not all of them are exact matches:
Moral views: We previously asked about normative moral philosophy, metaethics, and population ethics
Identification with EA label: up until 2018, we had distinct questions asking whether people could be “described as “an effective altruist”″ and whether they “subscribe to the basic ideas behind effective altruism”. Now we just have the self-report engagement scale. I agree that more about self-identification with the EA label could be interesting.
Best and worst interactions with EA: we’ve definitely asked about negative interactions or experiences in a number of different questions over the years. We’ve not asked about best interactions, but we have asked people to name which individuals (if any) have been most helpful to them on their EA journey.
Community building preferences: we’ve asked a few different open-ended questions about ways in which people would like to see the community improve or suggestions for how it could be improved. I agree there’s more that would be interesting to do about this.
Thanks for the detailed response. It’s great hearing about the care and consideration when forming these surveys!
Given “last year about 50% of respondents started the extra credit section and about 25% finished it”, this still feels like free info even if people don’t finish. But I guess there are also reputation risks in becoming The Survey That None Can Finish.
I note that previous surveys had some of information I suggested as useful listed, and I think that’s why I’d be so excited to see it carried over across the years. Especially with rapid growth of EAs.
I don’t feel like any substantial change should be made off my views expressed here, but I did want to iron out a few points to make my feedback clearer. Your point about follow-up surveys probably catches most of my worries about sufficient information being collected.
Thanks again David and team :)
I think there should be more questions under the ‘extra credit’ section. I was willing to spend more time on this, and I think there are other views I would be interested in understanding from the average EA.
A low effort attempt of listing a few things which come to mind:
moral views
biggest current uncertainties with EA
community building preferences
identification with EA label
best and worst interactions with EA
Thanks! This is useful feedback.
We’d like to include more questions in the extra credit section, and I agree it would be useful to ask more about the topics you suggest.
Unfortunately, we don’t find that adding more questions to the extra credit section is completely ‘free’. Even though it’s explicitly optional, we still find people sometimes complain about the length including the optional extra credit section. And there’s still a tradeoff in terms of how many people complete all or part of the extra credit section. We’ll continue to keep track of how many people complete the survey (and different sections of it) over time to try to optimise the number of extra questions we can include. For example, last year about 50% of respondents started the extra credit section and about 25% finished it.
Notably we do have an opt-in extra survey, sent out some time after the main EA Survey. Previously we’ve used this to include questions requested by EA (usually academic) researchers, whose questions we couldn’t prioritise including in the main survey (even in extra credit). Because this is completely opt-in and separate from the EA Survey, we’re more liberal about including more questions, though there are still length constraints. Last year about 60% (900) people opted in to receive this, though a smaller number actually completed the survey when it was sent out.
We’ve previously included questions on some of the topics which you mention, though of course not all of them are exact matches:
Moral views: We previously asked about normative moral philosophy, metaethics, and population ethics
Identification with EA label: up until 2018, we had distinct questions asking whether people could be “described as “an effective altruist”″ and whether they “subscribe to the basic ideas behind effective altruism”. Now we just have the self-report engagement scale. I agree that more about self-identification with the EA label could be interesting.
Best and worst interactions with EA: we’ve definitely asked about negative interactions or experiences in a number of different questions over the years. We’ve not asked about best interactions, but we have asked people to name which individuals (if any) have been most helpful to them on their EA journey.
Community building preferences: we’ve asked a few different open-ended questions about ways in which people would like to see the community improve or suggestions for how it could be improved. I agree there’s more that would be interesting to do about this.
Thanks for the detailed response. It’s great hearing about the care and consideration when forming these surveys!
Given “last year about 50% of respondents started the extra credit section and about 25% finished it”, this still feels like free info even if people don’t finish. But I guess there are also reputation risks in becoming The Survey That None Can Finish.
I note that previous surveys had some of information I suggested as useful listed, and I think that’s why I’d be so excited to see it carried over across the years. Especially with rapid growth of EAs.
I don’t feel like any substantial change should be made off my views expressed here, but I did want to iron out a few points to make my feedback clearer. Your point about follow-up surveys probably catches most of my worries about sufficient information being collected. Thanks again David and team :)