In our first survey, we recruited 1,453 US adults from the Prolific.co platform, weighting responses to match the US general population across sex, age, education, income, race/ethnicity, region, and political party identification.
It would be interesting to know whether the results vary by demographic. If yes, I wonder whether a sample representative of the US general population is the most adequate. At least historically, the people getting involved with EA have been relatively younger, and had a higher income and level of education. CEA may want to be more broadly appealing, but arguably not to the whole US general population.
I think there are two distinct questions here: the sample and the analysis.
When resources allow, I think it is often better to draw a representative sample of the full population, and then analyse the effect of different traits / the effects within different sub-populations. Even if the people getting involved in EA have tended to be younger, more educated etc., I think there are still reasons to be concerned about the views of the broader population. Of course, if you are interested in a very niche population then this approach will either not be possible or will be very resources inefficient and you might have to sample more narrowly.
When looking for any interactions with demographics for these results, I found no significant demographic interactions, which is not uncommon. I focus here on the replication study in order to have a cleaner manipulation of the “doing good better” effect, though it’s a smaller sample, and we gathered fewer demographics than in the representative sample in the first study.
For age, we see a fairly consistent effect (the trends are fairly linear even if I allow them to be non-linear, fwiw).
For student status, we see no interaction effect and the same main effect.
For sex, we see no effect and a consistent main effect.[1]
If there’s a lot of interest this we could potentially look at education and income in the first survey.
Thanks for the interesting post, David.
It would be interesting to know whether the results vary by demographic. If yes, I wonder whether a sample representative of the US general population is the most adequate. At least historically, the people getting involved with EA have been relatively younger, and had a higher income and level of education. CEA may want to be more broadly appealing, but arguably not to the whole US general population.
Thanks Vasco.
I think there are two distinct questions here: the sample and the analysis.
When resources allow, I think it is often better to draw a representative sample of the full population, and then analyse the effect of different traits / the effects within different sub-populations. Even if the people getting involved in EA have tended to be younger, more educated etc., I think there are still reasons to be concerned about the views of the broader population. Of course, if you are interested in a very niche population then this approach will either not be possible or will be very resources inefficient and you might have to sample more narrowly.
When looking for any interactions with demographics for these results, I found no significant demographic interactions, which is not uncommon. I focus here on the replication study in order to have a cleaner manipulation of the “doing good better” effect, though it’s a smaller sample, and we gathered fewer demographics than in the representative sample in the first study.
For age, we see a fairly consistent effect (the trends are fairly linear even if I allow them to be non-linear, fwiw).
For student status, we see no interaction effect and the same main effect.
For sex, we see no effect and a consistent main effect.[1]
If there’s a lot of interest this we could potentially look at education and income in the first survey.
People sometimes ask why we use sex rather than gender in public surveys and it’s usually to match the census so that we can weight.
Thanks for clarifying, David!