are you sure this isnât just a function of the definition of highly engaged?
No, I think it it probably is partly explained by that.
For context for other readers: the highest level of engagement on the engagement scale is defined as âI am heavily involved in the effective altruism community, perhaps helping to lead an EA group or working at an EA-aligned organization. I make heavy use of the principles of effective altruism when I make decisions about my career or charitable donations.â The next highest category of engagement (âIâve engaged extensively with effective altruism content (e.g. attending an EA Global conference, applying for career coaching, or organizing an EA meetup). I often consider the principles of effective altruism when I make decisions about my career or charitable donations.â) is also included in the âhigh engagementâ group, when we apply a binary division between high and low engagement).
The definition of the highest level of engagement may skew against people who are earning to give, even if they are highly engaged (in the ordinary sense of the term), and donating a lot, because it includes the specific example of âperhaps helping to lead an EA group or working at an EA-aligned organizationâ (even though a high-donating earn-to-giver could judge that they counted as highly engaged anyway, they might think that the category doesnât best fit them).
That said, I would expect most engaged, high-donating earning-to-givers to fit at least the second highest category, which would still count them as âhighly engagedâ in our binary analyses. Note that the next category down (the highest category in the âlow engagementâ binary division) is quite a lot weaker âIâve engaged with multiple articles, videos, podcasts, discussions, or events on effective altruism (e.g. subscribing to the 80,000 Hours podcast or attending regular events at a local group).I sometimes consider the principles of effective altruism when I make decisions about my career or charitable donations.â
Given that, I donât expect these definitions to dramatically skew results related to earning-to-give, so long as weâre using the binary division, but the use of these specific examples is probably a part of the issue.
The engagement scale (which we didnât design ourselves, it was a requested question from 2019 onwards) definitely has some undesirable features. Itâs double or triple-barrelled (i.e. it asks people about multiple distinct dimensions in parallel (e.g. how far and/âor how often you consider EA principles and about different actions you could take)). And the characterisations of the different levels of the different dimensions are not clearly hierarchical and seem to jump between different dimensions (e.g. levels 2-4 specify engagement with different content, level 5 refers to engagement in groups or EA orgs). The ideal way to design the measures would be to have each measure specifying a distinct dimension (e.g. self-identified engagement, degree or frequency of considering EA principles, and engagement in different activities) and have each of these dimensions unambigiously hierarchical, and then see how these different dimensions are associated.
That said, I think the scale mostly works passably well for most purposes. When we examine the association between the self-report scale and different concrete measures and between different concrete measures themselves (2019, 2020), we find a fairly high level of consilience between different measures. So people who are less/âmore engaged on the scale are also less/âmore likely to engage in a suite of different activities. Granted, this is still a certain kind of EA engagement (namely engagement with EA content and activities), and so wonât capture some cases of people who are very dedicated and do high value work (e.g. perhaps a high powered policy maker or donor); but most cases where people have âengaged extensivelyâ with EA content, should still be captured as high engagement, regardless of their current activities.
Since we didnât gather donation data this year, in order to keep the survey shorter, we have to go back to data from earlier survey years.
In EAS 2020, we asked about current career (rather than career strategy). This is obviously appreciably different (and more vulnerable to just reflecting the fact that people in different current careers can afford to donate different amounts, rather than reflecting the different groups), but here are the results:
Here we can see that high donors are much more likely to be in for-profit (earning to give), though there is also a non-significant trend in the direction of higher donors being more likely to be in for-profit (not earning to give). A higher percentage of high donors also selected work at an EA non-profit (though the difference was small and not significant). Higher donors were less likely to select âstill deciding what to pursueâ and âbuilding flexible career capital and will decide laterâ (compared to the lowest donors), but I imagine that this likely reflect these categories being more often selected by early career/âstudent low-earning, low-donors.
In 2019, we asked a question which may be a more informative comparison to 2022, âIf you had to guess, which broad career path(s) are you planning to follow?â (this may be somewhat less vulnerable to simply reflecting the fact that people currently in earning to give can currently donate more, but probably still reflects this to a significant degree).
Here the highest donors are more likely to select earning to give, and they are also less likely to select academia (than the lowest donors).
As with the original graphs, these differences are likely at least partly explained by other confounding variables (e.g. how long people have been in their career etc.). If we wanted to assess what is ultimately causing the differences, weâd need to examine a more complex model.
That said, while I think this is interesting, looking at total donations as an operationalization of engagement seems less informative, since although the existing measures may be slightly skewed against E2G people counting (themselves) as highly engaged (due to giving EA orgs and community buildings as exemplars of the highest level of engagement), total donations seems very skewed towards counting E2G people as highly engaged simply in virtue of them earning more than people in other roles (or people who are students/âearly career).
My guess is that either a more neutral measure of engagement (e.g. simple self-report of low to high engagement) or some more complex ideal measure of engagement would probably find that (rightly or wrongly) higher engagement is associated with higher interest in research /â EA org research over earning to give. Itâs possible that thereâd be a different association with a good measure of EA dedication, which may be different from engagement (but that seems harder and more controversial to measure).
are you sure this isnât just a function of the definition of highly engaged?
No, I think it it probably is partly explained by that.
For context for other readers: the highest level of engagement on the engagement scale is defined as âI am heavily involved in the effective altruism community, perhaps helping to lead an EA group or working at an EA-aligned organization. I make heavy use of the principles of effective altruism when I make decisions about my career or charitable donations.â The next highest category of engagement (âIâve engaged extensively with effective altruism content (e.g. attending an EA Global conference, applying for career coaching, or organizing an EA meetup). I often consider the principles of effective altruism when I make decisions about my career or charitable donations.â) is also included in the âhigh engagementâ group, when we apply a binary division between high and low engagement).
The definition of the highest level of engagement may skew against people who are earning to give, even if they are highly engaged (in the ordinary sense of the term), and donating a lot, because it includes the specific example of âperhaps helping to lead an EA group or working at an EA-aligned organizationâ (even though a high-donating earn-to-giver could judge that they counted as highly engaged anyway, they might think that the category doesnât best fit them).
That said, I would expect most engaged, high-donating earning-to-givers to fit at least the second highest category, which would still count them as âhighly engagedâ in our binary analyses. Note that the next category down (the highest category in the âlow engagementâ binary division) is quite a lot weaker âIâve engaged with multiple articles, videos, podcasts, discussions, or events on effective altruism (e.g. subscribing to the 80,000 Hours podcast or attending regular events at a local group). I sometimes consider the principles of effective altruism when I make decisions about my career or charitable donations.â
Given that, I donât expect these definitions to dramatically skew results related to earning-to-give, so long as weâre using the binary division, but the use of these specific examples is probably a part of the issue.
The engagement scale (which we didnât design ourselves, it was a requested question from 2019 onwards) definitely has some undesirable features. Itâs double or triple-barrelled (i.e. it asks people about multiple distinct dimensions in parallel (e.g. how far and/âor how often you consider EA principles and about different actions you could take)). And the characterisations of the different levels of the different dimensions are not clearly hierarchical and seem to jump between different dimensions (e.g. levels 2-4 specify engagement with different content, level 5 refers to engagement in groups or EA orgs). The ideal way to design the measures would be to have each measure specifying a distinct dimension (e.g. self-identified engagement, degree or frequency of considering EA principles, and engagement in different activities) and have each of these dimensions unambigiously hierarchical, and then see how these different dimensions are associated.
That said, I think the scale mostly works passably well for most purposes. When we examine the association between the self-report scale and different concrete measures and between different concrete measures themselves (2019, 2020), we find a fairly high level of consilience between different measures. So people who are less/âmore engaged on the scale are also less/âmore likely to engage in a suite of different activities. Granted, this is still a certain kind of EA engagement (namely engagement with EA content and activities), and so wonât capture some cases of people who are very dedicated and do high value work (e.g. perhaps a high powered policy maker or donor); but most cases where people have âengaged extensivelyâ with EA content, should still be captured as high engagement, regardless of their current activities.
Should redefine engagement in terms of total $ donated to charity in the last year and see how the stats look.
Thanks. I agree this is interesting to look at.
Since we didnât gather donation data this year, in order to keep the survey shorter, we have to go back to data from earlier survey years.
In EAS 2020, we asked about current career (rather than career strategy). This is obviously appreciably different (and more vulnerable to just reflecting the fact that people in different current careers can afford to donate different amounts, rather than reflecting the different groups), but here are the results:
Here we can see that high donors are much more likely to be in for-profit (earning to give), though there is also a non-significant trend in the direction of higher donors being more likely to be in for-profit (not earning to give). A higher percentage of high donors also selected work at an EA non-profit (though the difference was small and not significant). Higher donors were less likely to select âstill deciding what to pursueâ and âbuilding flexible career capital and will decide laterâ (compared to the lowest donors), but I imagine that this likely reflect these categories being more often selected by early career/âstudent low-earning, low-donors.
In 2019, we asked a question which may be a more informative comparison to 2022, âIf you had to guess, which broad career path(s) are you planning to follow?â (this may be somewhat less vulnerable to simply reflecting the fact that people currently in earning to give can currently donate more, but probably still reflects this to a significant degree).
Here the highest donors are more likely to select earning to give, and they are also less likely to select academia (than the lowest donors).
As with the original graphs, these differences are likely at least partly explained by other confounding variables (e.g. how long people have been in their career etc.). If we wanted to assess what is ultimately causing the differences, weâd need to examine a more complex model.
That said, while I think this is interesting, looking at total donations as an operationalization of engagement seems less informative, since although the existing measures may be slightly skewed against E2G people counting (themselves) as highly engaged (due to giving EA orgs and community buildings as exemplars of the highest level of engagement), total donations seems very skewed towards counting E2G people as highly engaged simply in virtue of them earning more than people in other roles (or people who are students/âearly career).
My guess is that either a more neutral measure of engagement (e.g. simple self-report of low to high engagement) or some more complex ideal measure of engagement would probably find that (rightly or wrongly) higher engagement is associated with higher interest in research /â EA org research over earning to give. Itâs possible that thereâd be a different association with a good measure of EA dedication, which may be different from engagement (but that seems harder and more controversial to measure).
Wow, very detailed analysis, thank you!