it’s striking that this basket [Learning] outperforms Connections, despite the survey explicitly prompting attendees to report on connections. This updates me towards thinking that learning (about career options or ideas) isa large source of value from EA community-building events.
In the EA Survey, we also found that there was a surprisingly low gap between learning something important and making a new connection, in cases where one might have expected an influence to strongly favour one or the other. (For example, it was mentioned to me that the EA Forum producing so many connections as it did was surprising).
I think these results are probably partly due to measurement error (specifically, I think a general ‘positive feeling about the influence’ factor makes people more likely to attribute either learning or connection to the factor, but not entirely.
It’s perhaps also worth noting that in these results EAG and EAGx come out with more Connections than Learnings (not weighted for importance), though they also perform appreciably well compared to other sources for Learnings.
I think these earlier results also confirm the importance of scale noted in your post, i.e. the dominant factors just seem to be those operating at much larger scale, rather than smaller, more targeted interventions (this seems to be a recurring theme across our results) and the difference in scale between different EA projects (in terms of how many people they reach) varies enormously:
Hey David, your observation that ‘the dominant factors just seem to be those operating at much larger scale, rather than smaller, more targeted interventions’ is a recurring theme is very interesting!
To check I understand, are you saying that things like websites, podcasts, and other scalable things seem to be having much more impact than things such as 1-1s? I’m asking because we at EA Netherlands sometimes wonder how much we ought to be investing in our website vs smaller and more targeted interventions.
are you saying that things like websites, podcasts, and other scalable things seem to be having much more impact than things such as 1-1s?
Basically (though I’m thinking more about the scale of programs (i.e. whether the program actually reaches a large scale) than the scalability of kinds of program).
We observe that the factors which are cited by the largest numbers of people as being among the most important influences on them are those with very large reach, i.e. those which many EAs have been exposed to (e.g. 80K website), followed by those with smaller reach (e.g. EA Forum), and then by smaller programs.
Indeed, when we look at the association between the number of people in the EA Survey reporting having interacted with a given factor in the last 12 months and indicating that it was one of the most important influences on their ability to have a personal impact, we observe a very strong correlation (r=0.8, [0.5-0.9], p=0.001]. That might seem truistic, but I think (per the OP’s other post), many people expect some smaller, more targeted programs to be dramatically more influential than more ‘mass’ outreach.
Some important qualifiers:
Obviously this is looking at reported exposures among people who are taking the EA Survey. It should not be taken to capture all exposures (e.g. many people could view the 80K or GiveWell websites and never have any further interaction with EA), so this shouldn’t be taken to indicate a general ratio of impacts per exposure.
This data comes from two separate questions with different numbers of respondents (so it’s possible for an influence to have more impacts than exposures) which is another reason why it shouldn’t be taken as giving a literal exposures:impacts ratio.
This is looking at numbers of people citing the factor as among the most important influences on their ability to have an impact, but these are not weighted. So this is compatible with certain influences being much more important than others, though as noted, the post linked above may give some reasons counting against that.
This strong correlation is compatible with some large differences (visible on the plot) between influences which have similar numbers of exposures but which lead to very different numbers of impacts.
That said, I think this still serves as a somewhat useful illustration of the extent to which the factors with the largest number of impacts are those with the largest number of exposures among EAs. Given the very large differences in scale between programs, smaller programs needs to be getting a dramatically higher number (or higher value) of hits to compete with the larger influences.
I’m asking because we at EA Netherlands sometimes wonder how much we ought to be investing in our website vs smaller and more targeted interventions.
When it comes to thinking about EAN’s particular options, I would add some additional caveats:
The differences in scale between the largest programs (e.g. the most prominent EA websites / podcasts) and the smaller programs, discussed above, may differ when comparing the possible EAN website / podcast / smaller programs (e.g. the likely differences in scale may be a lot smaller. And in your particular situation, you might be dramatically better placed than others to run a website/podcast/other program.
In other data, we observe that the impact of different factors like websites and podcasts is very skewed e.g. the most effective podcast reaches many more EAs than any of the others. So even if podcasts as a category seems like it large scale, the typical new podcast/website might be expected to have much smaller reach, more comparable to a less scalable kind of program.
In the EA Survey, we also found that there was a surprisingly low gap between learning something important and making a new connection, in cases where one might have expected an influence to strongly favour one or the other. (For example, it was mentioned to me that the EA Forum producing so many connections as it did was surprising).
I think these results are probably partly due to measurement error (specifically, I think a general ‘positive feeling about the influence’ factor makes people more likely to attribute either learning or connection to the factor, but not entirely.
It’s perhaps also worth noting that in these results EAG and EAGx come out with more Connections than Learnings (not weighted for importance), though they also perform appreciably well compared to other sources for Learnings.
I think these earlier results also confirm the importance of scale noted in your post, i.e. the dominant factors just seem to be those operating at much larger scale, rather than smaller, more targeted interventions (this seems to be a recurring theme across our results) and the difference in scale between different EA projects (in terms of how many people they reach) varies enormously:
Thanks for sharing, David!
Hey David, your observation that ‘the dominant factors just seem to be those operating at much larger scale, rather than smaller, more targeted interventions’ is a recurring theme is very interesting!
To check I understand, are you saying that things like websites, podcasts, and other scalable things seem to be having much more impact than things such as 1-1s? I’m asking because we at EA Netherlands sometimes wonder how much we ought to be investing in our website vs smaller and more targeted interventions.
Basically (though I’m thinking more about the scale of programs (i.e. whether the program actually reaches a large scale) than the scalability of kinds of program).
We observe that the factors which are cited by the largest numbers of people as being among the most important influences on them are those with very large reach, i.e. those which many EAs have been exposed to (e.g. 80K website), followed by those with smaller reach (e.g. EA Forum), and then by smaller programs.
Indeed, when we look at the association between the number of people in the EA Survey reporting having interacted with a given factor in the last 12 months and indicating that it was one of the most important influences on their ability to have a personal impact, we observe a very strong correlation (r=0.8, [0.5-0.9], p=0.001]. That might seem truistic, but I think (per the OP’s other post), many people expect some smaller, more targeted programs to be dramatically more influential than more ‘mass’ outreach.
Obviously this is looking at reported exposures among people who are taking the EA Survey. It should not be taken to capture all exposures (e.g. many people could view the 80K or GiveWell websites and never have any further interaction with EA), so this shouldn’t be taken to indicate a general ratio of impacts per exposure.
This data comes from two separate questions with different numbers of respondents (so it’s possible for an influence to have more impacts than exposures) which is another reason why it shouldn’t be taken as giving a literal exposures:impacts ratio.
This is looking at numbers of people citing the factor as among the most important influences on their ability to have an impact, but these are not weighted. So this is compatible with certain influences being much more important than others, though as noted, the post linked above may give some reasons counting against that.
This strong correlation is compatible with some large differences (visible on the plot) between influences which have similar numbers of exposures but which lead to very different numbers of impacts.
That said, I think this still serves as a somewhat useful illustration of the extent to which the factors with the largest number of impacts are those with the largest number of exposures among EAs. Given the very large differences in scale between programs, smaller programs needs to be getting a dramatically higher number (or higher value) of hits to compete with the larger influences.
When it comes to thinking about EAN’s particular options, I would add some additional caveats:
The differences in scale between the largest programs (e.g. the most prominent EA websites / podcasts) and the smaller programs, discussed above, may differ when comparing the possible EAN website / podcast / smaller programs (e.g. the likely differences in scale may be a lot smaller. And in your particular situation, you might be dramatically better placed than others to run a website/podcast/other program.
In other data, we observe that the impact of different factors like websites and podcasts is very skewed e.g. the most effective podcast reaches many more EAs than any of the others. So even if podcasts as a category seems like it large scale, the typical new podcast/website might be expected to have much smaller reach, more comparable to a less scalable kind of program.
Thanks so much for the detailed reply! This is very helpful :)