My current sense (as someone who organized EAG in the past and has thought about the effects of EAG a lot) is that it would be better to increase the size of the event, and if that’s not financially viable, reduce the size of the subsidies for attendees to make that possible.
I don’t think the effect size of “some people felt like the event was too big” is comparable to the effect size of allowing up to 50% more people to participate in the event, and so I think 1000+ person EAG events are probably worth it.
My experience from finding venues is that it is quite doable to find 1000+ person sized venues for reasonable prices, and I didn’t share the impression that venue-price seems to increase drastically for more than 500-600 people. I do think the price-per-head might increase a bit, but I would be surprised if it increased by more than 15%.
So, I have a few considerations that tend to argue against that. Here are some of them:
1. Common knowledge is built better by having everyone actually in the same space
I think having common knowledge of norms, ideas and future plans is often very important, and is better achieved by having everyone in the same place. If you split up the event into multiple events, even if all the same people attend, the participants of those events can now no longer verify who else is at the event, and as such can no longer build common knowledge with those other people about the things that have been discussed.
2. EAGx events have historically been of lower quality
I have been to 3 EAGx events, all three of which seemed to me to be just generally much worse run than EAG, both in terms of content and operations. And to be clear, I don’t think this reflects particularly badly on the organizers, running a conference is just hard and requires a lot of time, which most EAGx organizers don’t tend to have. In general I am in favor of specialization here. Obviously you helping the organizers might be able to address this consideration, so this might be moot.
3. EAGx events should go deep, whereas EAG events should go wide
When I designed the original goal-document for EAGx together with the EAO team, the goal of EAGx was in large parts to allow the creation of more specialist conferences, in which participants could go significantly more in-depth into a topic, and overall feel more like researcher conferences. I think for a variety of reasons that never ended up happening, but one of the reasons is I think that we did try to compensate for the lack of space in the EAG events by encouraging people to go to EAGx events instead.
My current sense is that we do actually also want distributed intro events, and we might want a separate brand from EAGx for that. But for now, I think encouraging usual EAG attendees to go to EAGx events as a replacement will prevent more specialist EAGx-type events from happening, which seems sad to me.
4. The value of a conference does scale to a meaningful degree with n^2
I don’t think this fully applies to conferences, but I do think it applies to a large degree. The value of an event to me is somewhat proportional to the number of people at that event, so I think there are strong increasing returns to conference size, at least from that perspective.
5. Group membership is in significant parts determined by who attends EAG, and not by who attends EAGx, and I feel somewhat uncomfortable with the degree of control CEA has over that
I think there is a meaningful sense in which people’s intuitive sense of “who is an active member of the EA community” is closely related to who attended past EAG events, and so I think preventing people from attending EAG is actually something that has a pretty significant effect on people’s social standing. I think having smaller events introduces a lot of noise into that system, and I also don’t currently trust CEA to make a lot of the relevant decisions here, and would prefer CEA to on the margin have less control over EA group membership.
I have some more concerns, but these are the ones that I felt like I could write up easily.
I don’t feel like I get more value out of large conferences and I’d be curious about seeing more data on this question. For me, having more people at a conference makes it harder to physically find the people I actually want to talk to. They make up a smaller fraction of attendees and are more spread out. I have also had the impression that conversations at large conferences are shorter. In combination, I get much less value out of very large events compared to small or medium sized ones.
The event size was one of the main reasons I decided not to attend EAG London this year for the first time. It is too big for me to get sufficient value out of it.
5. Group membership is in significant parts determined by who attends EAG, and not by who attends EAGx, and I feel somewhat uncomfortable with the degree of control CEA has over that
Strongly agree. EAG attendance is a Schelling point for who is “an EA” and who isn’t, even if EAG organizers don’t endorse this, and even if “being an EA” isn’t an endorsed and/or fully coherent concept.
Agree wholeheartedly! Especially for those who fall under “You’ve been several times before and we want to give a spot to a first-timer”. I imagine if you go to EAG every year and were suddenly rejected you’d feel like you were kicked out of the club. A huge part of community building in professional associations is going to an annual conference and getting to catch up with your peers, EAG is that way for those of us who don’t live in largely populated EA cities.
A huge part of community building in professional associations is going to an annual conference and getting to catch up with your peers, EAG is that way for those of us who don’t live in largely populated EA cities.
+1 to the analogy of EA Global as a professional association’s annual conference.
I think having common knowledge of norms, ideas and future plans is often very important, and is better achieved by having everyone in the same place. If you split up the event into multiple events, even if all the same people attend, the participants of those events can now no longer verify who else is at the event, and as such can no longer build common knowledge with those other people about the things that have been discussed.
Interesting, this doesn’t fit with my experience for two reasons: a) attendance is so far past Dunbar’s number that I have a hard time knowing who attended any individual EA Global and b) even if I know that someone attended a given EA Global, I’m not sure whether they attended any individual talk/workshop/etc. (since many people don’t attend the same talks, or even any talks at all).
I’m curious if you have examples of “norms, ideas, or future plans” which were successfully shared in 2016 (when we had just the one large EA Global) that you think would not have successfully been shared if we had multiple events?
I have been to 3 EAGx events, all three of which seemed to me to be just generally much worse run than EAG, both in terms of content and operations
We have heard concerns similar to yours about logistics and content in the past, and we are providing more support for EAGx organizers this year, including creating a “playbook” to document best practices, having monthly check-in calls between the organizers and CEA’s events team, and hosting a training for the organizers (which is happening this week).
At least in recent years, the comparison of the Net Promoter Score of EAG and EAGx events indicate that the attendees themselves are positive about EAGx, though there are obviously lots of confounding factors:
The value of a conference does scale to a meaningful degree with n^2… I think there are strong increasing returns to conference size
Echoing Denise, I would be curious for evidence here. My intuition is that marginal returns are diminishing, not increasing, and I think this is a common view (e.g. ticket prices for conferences don’t seem to scale with the square of the number of attendees).
Group membership is in significant parts determined by who attends EAG, and not by who attends EAGx, and I feel somewhat uncomfortable with the degree of control CEA has over that
Do you have examples of groups (events, programs, etc.) which use EA Global attendance as a “significant” membership criterion?
My impression is that many people who are highly involved in EA do not attend EA Global (some EA organization staff do not attend, for example), so I would be pretty skeptical of using it.
Meta Note
To clarify my above responses: I (and the Events team, who are currently running a retreat with the EAGx organizers) believe that more people being able to attend EA Global is good, all other things being equal. Even though I’m less positive about the specific things you are pointing to here than you are, I generally agree that you are pointing to legitimate sources of value.
I’m curious if you have examples of “norms, ideas, or future plans” which were successfully shared in 2016 (when we had just the one large EA Global) that you think would not have successfully been shared if we had multiple events?
I think EAG 2016 was the last time that I felt like there was a strong shared EA culture. These days I feel quite isolated from the european EA culture, and feel like there is a significant amount of tension between the different cultural clusters (though this is probably worsened by me no longer visiting the UK very much, which I tended to do more during my time at CEA). I think that tension has always been there, but I feel like I am now much more disconnected from how EA is going in other places around the world (and more broadly, don’t see a path forward for cultural recombination and reconciliation) because the two clusters just have their own events. I also feel somewhat similar about east-coast and west-coast cultural differences.
More concrete examples would be propagating ongoing shifts in cause-priorities. Many surveys suggest there has been an ongoing shift to more long-term causes, and my sense is that there is a buildup of social tension associated with that, that I think is hard to resolve without building common knowledge.
I think EAG 2016 very concretely actually did a lot by creating common-knowledge of that shift in cause-priorities, as well as a broader shift towards more macro-scale modeling, instead of more narrow RCT-based thinking that I think many assumed to be “what EA is about”. I.e. I think EAG 2016 did a lot to establish that EA wasn’t just primarily GiveWell and GiveWell style approaches.
A lot of the information I expect to be exchanged here is not going to be straightforward facts, but much more related to attitudes and social expectations, so it’s hard to be very concrete about these things, which I regret.
Importantly, I think a lot of this information spreads even when not everyone is attending the same talk. At all EAGs I went to, basically everyone knew by the end what the main points of the opening talks were, because people talked to each other about the content of the opening talks (if they were well-delivered), even if they didn’t attend, so there is a lot of diffusion of information that makes literally everyone being in the same talk not fully necessary (and where probabilistic common-knowledge can still be built). The information flow of people who attended separate EA Globals is still present, just many orders of magnitude weaker.
At least in recent years, the comparison of the Net Promoter Score of EAG and EAGx events indicate that the attendees themselves are positive about EAGx, though there are obviously lots of confounding factors:
These graphs are great and surprising to me. I don’t yet have great models of how I expect the Net Promoter Score to vary for different types of events like this, so I am not sure yet how to update.
Echoing Denise, I would be curious for evidence here. My intuition is that marginal returns are diminishing, not increasing, and I think this is a common view
At this year’s EAG there were many core people in EA that I had hoped I could talk to, but that weren’t attending, and when I inquired about their presence, they said they were just planning to attend EAG London, since that was more convenient for them. I also heard other people say that they weren’t attending because they didn’t really expect a lot of the “best people” to be around, which is a negative feedback loop that I think is at least partially caused by having many events, without one clear Schelling event that everyone is expected to show up to.
(e.g. ticket prices for conferences don’t seem to scale with the square of the number of attendees).
This assumes a model of perfect monopoly for conferences. In a perfectly competitive conference landscape, you expect ticket prices to be equal to marginal costs, which would be decreasing with size. I expect the actual conference landscape to be somewhere in-between, with a curve that does increase in prize proportional to size for a bit, but definitely not completely. Because of that, I don’t think price is much evidence either way on this issue.
Do you have examples of groups (events, programs, etc.) which use EA Global attendance as a “significant” membership criterion?
I think I do to some significant extend. I definitely have a significantly different relationship to how I treat people who I met at EA Global. I also think that if someone tells me that they tried to get into EA Global but didn’t get in, then I do make a pretty significant update on the degree to which they are core to EA, though the post above has definitely changed that some for me (since it made it more clear that CEA was handling acceptances quite differently than I thought they were). But I don’t expect everyone to have read the post in as much detail as I have, and I expect people will continue to think that EAG attendance is in significant parts screening for involvement and knowledge about EA.
I have a variety of other thoughts, but probably won’t have time to engage much more. So this will most likely be my last comment on the thread (unless someone asks a question or makes a comment that ends up feeling particularly easy or fun to reply to).
My impression is that many people who are highly involved in EA do not attend EA Global (some EA organization staff do not attend, for example), so I would be pretty skeptical of using it.
On the “group membership” dimension, attending EAG is less important for EA org staff as they have other signifiers of membership in the group.
I agree with Denise that I prefer smaller conferences, but what I really want to attend is a small specialist conference (for example, an EAGx for the top 150 policy experts in EA). Your suggestion here that a large EAG could lead to smaller EAGx conferences has gotten my attention.
Is travel price ever taken into consideration? Non-US and/or east cost attendees can easily spend $1k+ just for flights/lodging which probably causes them to take a large discount on the ticket price.
Yes, we do take attendees’ travel costs into consideration. This is part of why we went back to having multiple EA Global conferences after trying out having only one in 2016, and why we have continued to have EAGx events, so that more people have something near them. We recognize that reduced travel costs would be one benefit to having more events.
My current sense (as someone who organized EAG in the past and has thought about the effects of EAG a lot) is that it would be better to increase the size of the event, and if that’s not financially viable, reduce the size of the subsidies for attendees to make that possible.
I don’t think the effect size of “some people felt like the event was too big” is comparable to the effect size of allowing up to 50% more people to participate in the event, and so I think 1000+ person EAG events are probably worth it.
My experience from finding venues is that it is quite doable to find 1000+ person sized venues for reasonable prices, and I didn’t share the impression that venue-price seems to increase drastically for more than 500-600 people. I do think the price-per-head might increase a bit, but I would be surprised if it increased by more than 15%.
Thanks for the feedback, Oliver. Do you have opinions on our hypothesis that we should focus on EAGx over more/bigger EA Globals?
So, I have a few considerations that tend to argue against that. Here are some of them:
1. Common knowledge is built better by having everyone actually in the same space
I think having common knowledge of norms, ideas and future plans is often very important, and is better achieved by having everyone in the same place. If you split up the event into multiple events, even if all the same people attend, the participants of those events can now no longer verify who else is at the event, and as such can no longer build common knowledge with those other people about the things that have been discussed.
2. EAGx events have historically been of lower quality
I have been to 3 EAGx events, all three of which seemed to me to be just generally much worse run than EAG, both in terms of content and operations. And to be clear, I don’t think this reflects particularly badly on the organizers, running a conference is just hard and requires a lot of time, which most EAGx organizers don’t tend to have. In general I am in favor of specialization here. Obviously you helping the organizers might be able to address this consideration, so this might be moot.
3. EAGx events should go deep, whereas EAG events should go wide
When I designed the original goal-document for EAGx together with the EAO team, the goal of EAGx was in large parts to allow the creation of more specialist conferences, in which participants could go significantly more in-depth into a topic, and overall feel more like researcher conferences. I think for a variety of reasons that never ended up happening, but one of the reasons is I think that we did try to compensate for the lack of space in the EAG events by encouraging people to go to EAGx events instead.
My current sense is that we do actually also want distributed intro events, and we might want a separate brand from EAGx for that. But for now, I think encouraging usual EAG attendees to go to EAGx events as a replacement will prevent more specialist EAGx-type events from happening, which seems sad to me.
4. The value of a conference does scale to a meaningful degree with n^2
Metcalfe’s Law states that
I don’t think this fully applies to conferences, but I do think it applies to a large degree. The value of an event to me is somewhat proportional to the number of people at that event, so I think there are strong increasing returns to conference size, at least from that perspective.
5. Group membership is in significant parts determined by who attends EAG, and not by who attends EAGx, and I feel somewhat uncomfortable with the degree of control CEA has over that
I think there is a meaningful sense in which people’s intuitive sense of “who is an active member of the EA community” is closely related to who attended past EAG events, and so I think preventing people from attending EAG is actually something that has a pretty significant effect on people’s social standing. I think having smaller events introduces a lot of noise into that system, and I also don’t currently trust CEA to make a lot of the relevant decisions here, and would prefer CEA to on the margin have less control over EA group membership.
I have some more concerns, but these are the ones that I felt like I could write up easily.
I don’t feel like I get more value out of large conferences and I’d be curious about seeing more data on this question. For me, having more people at a conference makes it harder to physically find the people I actually want to talk to. They make up a smaller fraction of attendees and are more spread out. I have also had the impression that conversations at large conferences are shorter. In combination, I get much less value out of very large events compared to small or medium sized ones.
The event size was one of the main reasons I decided not to attend EAG London this year for the first time. It is too big for me to get sufficient value out of it.
Thank you for this fantastic comment.
Strongly agree. EAG attendance is a Schelling point for who is “an EA” and who isn’t, even if EAG organizers don’t endorse this, and even if “being an EA” isn’t an endorsed and/or fully coherent concept.
Agree wholeheartedly! Especially for those who fall under “You’ve been several times before and we want to give a spot to a first-timer”. I imagine if you go to EAG every year and were suddenly rejected you’d feel like you were kicked out of the club. A huge part of community building in professional associations is going to an annual conference and getting to catch up with your peers, EAG is that way for those of us who don’t live in largely populated EA cities.
+1 to the analogy of EA Global as a professional association’s annual conference.
I also think this is a valuable analogy.
Thanks for the detailed thoughts Oli.
Interesting, this doesn’t fit with my experience for two reasons: a) attendance is so far past Dunbar’s number that I have a hard time knowing who attended any individual EA Global and b) even if I know that someone attended a given EA Global, I’m not sure whether they attended any individual talk/workshop/etc. (since many people don’t attend the same talks, or even any talks at all).
I’m curious if you have examples of “norms, ideas, or future plans” which were successfully shared in 2016 (when we had just the one large EA Global) that you think would not have successfully been shared if we had multiple events?
We have heard concerns similar to yours about logistics and content in the past, and we are providing more support for EAGx organizers this year, including creating a “playbook” to document best practices, having monthly check-in calls between the organizers and CEA’s events team, and hosting a training for the organizers (which is happening this week).
At least in recent years, the comparison of the Net Promoter Score of EAG and EAGx events indicate that the attendees themselves are positive about EAGx, though there are obviously lots of confounding factors:
(More information about EAGx can be found here.)
Echoing Denise, I would be curious for evidence here. My intuition is that marginal returns are diminishing, not increasing, and I think this is a common view (e.g. ticket prices for conferences don’t seem to scale with the square of the number of attendees).
Do you have examples of groups (events, programs, etc.) which use EA Global attendance as a “significant” membership criterion?
My impression is that many people who are highly involved in EA do not attend EA Global (some EA organization staff do not attend, for example), so I would be pretty skeptical of using it.
Meta Note
To clarify my above responses: I (and the Events team, who are currently running a retreat with the EAGx organizers) believe that more people being able to attend EA Global is good, all other things being equal. Even though I’m less positive about the specific things you are pointing to here than you are, I generally agree that you are pointing to legitimate sources of value.
I think EAG 2016 was the last time that I felt like there was a strong shared EA culture. These days I feel quite isolated from the european EA culture, and feel like there is a significant amount of tension between the different cultural clusters (though this is probably worsened by me no longer visiting the UK very much, which I tended to do more during my time at CEA). I think that tension has always been there, but I feel like I am now much more disconnected from how EA is going in other places around the world (and more broadly, don’t see a path forward for cultural recombination and reconciliation) because the two clusters just have their own events. I also feel somewhat similar about east-coast and west-coast cultural differences.
More concrete examples would be propagating ongoing shifts in cause-priorities. Many surveys suggest there has been an ongoing shift to more long-term causes, and my sense is that there is a buildup of social tension associated with that, that I think is hard to resolve without building common knowledge.
I think EAG 2016 very concretely actually did a lot by creating common-knowledge of that shift in cause-priorities, as well as a broader shift towards more macro-scale modeling, instead of more narrow RCT-based thinking that I think many assumed to be “what EA is about”. I.e. I think EAG 2016 did a lot to establish that EA wasn’t just primarily GiveWell and GiveWell style approaches.
A lot of the information I expect to be exchanged here is not going to be straightforward facts, but much more related to attitudes and social expectations, so it’s hard to be very concrete about these things, which I regret.
Importantly, I think a lot of this information spreads even when not everyone is attending the same talk. At all EAGs I went to, basically everyone knew by the end what the main points of the opening talks were, because people talked to each other about the content of the opening talks (if they were well-delivered), even if they didn’t attend, so there is a lot of diffusion of information that makes literally everyone being in the same talk not fully necessary (and where probabilistic common-knowledge can still be built). The information flow of people who attended separate EA Globals is still present, just many orders of magnitude weaker.
These graphs are great and surprising to me. I don’t yet have great models of how I expect the Net Promoter Score to vary for different types of events like this, so I am not sure yet how to update.
At this year’s EAG there were many core people in EA that I had hoped I could talk to, but that weren’t attending, and when I inquired about their presence, they said they were just planning to attend EAG London, since that was more convenient for them. I also heard other people say that they weren’t attending because they didn’t really expect a lot of the “best people” to be around, which is a negative feedback loop that I think is at least partially caused by having many events, without one clear Schelling event that everyone is expected to show up to.
This assumes a model of perfect monopoly for conferences. In a perfectly competitive conference landscape, you expect ticket prices to be equal to marginal costs, which would be decreasing with size. I expect the actual conference landscape to be somewhere in-between, with a curve that does increase in prize proportional to size for a bit, but definitely not completely. Because of that, I don’t think price is much evidence either way on this issue.
I think I do to some significant extend. I definitely have a significantly different relationship to how I treat people who I met at EA Global. I also think that if someone tells me that they tried to get into EA Global but didn’t get in, then I do make a pretty significant update on the degree to which they are core to EA, though the post above has definitely changed that some for me (since it made it more clear that CEA was handling acceptances quite differently than I thought they were). But I don’t expect everyone to have read the post in as much detail as I have, and I expect people will continue to think that EAG attendance is in significant parts screening for involvement and knowledge about EA.
I have a variety of other thoughts, but probably won’t have time to engage much more. So this will most likely be my last comment on the thread (unless someone asks a question or makes a comment that ends up feeling particularly easy or fun to reply to).
On the “group membership” dimension, attending EAG is less important for EA org staff as they have other signifiers of membership in the group.
I agree with Denise that I prefer smaller conferences, but what I really want to attend is a small specialist conference (for example, an EAGx for the top 150 policy experts in EA). Your suggestion here that a large EAG could lead to smaller EAGx conferences has gotten my attention.
Is travel price ever taken into consideration? Non-US and/or east cost attendees can easily spend $1k+ just for flights/lodging which probably causes them to take a large discount on the ticket price.
Yes, we do take attendees’ travel costs into consideration. This is part of why we went back to having multiple EA Global conferences after trying out having only one in 2016, and why we have continued to have EAGx events, so that more people have something near them. We recognize that reduced travel costs would be one benefit to having more events.