We chose brand and funding as themes in part because attendees flagged these as two of the biggest challenges facing the community. Sorry, I should’ve made that clearer.
OllieBase
Karma: 6,064
Thanks for resurfacing this take, Guy.
There’s a trade-off here, but I think some attendees who can provide valuable input wouldn’t attend if their name was shared publicly and that would make the event less valuable for the community.
That said, perhaps one thing we can do is emphasise the benefits of sharing their name (increases trust in the event/leadership, greater visibility for the community about direction/influence) when they RSVP for the event, I’ll note that for next time as an idea.
Thanks! I think this note explains the gap:
Note that we have a few attendees at this year’s event who are specialists in one of our focus areas rather than leaders of an EA meta organization or team (though some attendees are both).
We were not trying to optimise the attendee list for connectedness or historical engagement with the community, but rather who can contribute to making progress on our core themes; brand and funding. When you see what roles these attendees have, I think it’s fairly evident why we invited them, given this lens.
I’ll also note that I think it’s healthy for there to be people joining for this event who haven’t bene in the community as long as you have. They can bring new perspectives, and offer expertise the community / organisational leaders has been lacking.
Thanks, it wasn’t open at the time I posted this, but I’ve added the link now.
We’re not currently planning to run another GCR-focused EAG, but we do plan on continuing to investigate what other types of events we could run, including cause-specific (e.g. GCR-focused) events.
I think this post might benefit from some commentary, caveats and warnings about how to engage in politics sensibly e.g.
Campaign finance rules
How non-Americans can and can’t engage
PR risks to EA from doing this sort of thing
I don’t have any expertise here, but I don’t think this community will handle this all sensibly by default (see err 2022).
If you have some experience in a relevant field, you could also consider offering to speak at an event :) When I was organising my university group, I know I was pretty nervous about reaching out to people working in EA-aligned careers. I expect having alumni speak might make those career paths particularly salient (“I used to be exactly like you, and now I do this”).
Thanks Bryce!
Yes, we’re testing out another tier of event with Summits. Summits are typically smaller, operate on a smaller budget, are often 1 day instead of 2–3 days and don’t use CEA’s application platforms. The idea here is that we can see if a location might be suitable for an EAGx event before we invest significant resources in supporting an EAGx.We hope to share more about these events in a future post.
Thanks Cathy :)
Are there any concerns about nearby conferences cannibalizing each others’ potential audiences?
Yes, this is something we’re tracking. We try and coordinate the European conferences in particular so that there isn’t too much overlap. Most others serve very large regions so I’m less worried about those.
Is there a maximum number of conferences that we think will produce the most effective outcome?
We don’t have an answer to that but it’s also something we’re thinking about. We hope to run more EAGx events in the coming years, since a lot of people travel a long way to attend the nearest EAGx event, and the costs of that travel can sometimes exceed the costs of running another event. The bottlenecks are around our capacity to support more events and strong regional organising teams (please apply if you’re interested!). We aren’t currently planning to run more than three EA Globals a year.
A similar one was run last year, here :)
(Jumping in for our busy comms/exec team) Understanding the status of the EA brand and working to improve it is a top priority for CEA :) We hope to share more work on this in future.
First, 6 months is probably too short a timeframe to measure conference impact—many of my most valuable changes in behavior (e.g., starting new projects/collaborations, or providing (and getting) support/advice) occurred years after the conference where I met someone for the first time.
Possibly, though there is a trade-off here. We also hear in our 3–6 month follow-up surveys that people don’t really remember conversations from the event. Maybe that’s just a sign that nothing super valuable occurred but if you attend lots of events, questions about an event that occurred >6mo ago can be difficult to answer even if it was impactful. If we ask straight after the event, and 3–6 months later and a year later, I’d worry about survey fatigue.
Replying in full now!
Thanks once again for conducting this study and taking the time to write up the results so clearly. As Guy Raveh says, it takes courage to share your work publicly at all, let alone work that runs contrary to popular opinion on a forum full of people who make a living out of critiquing documents. The whole CEA events team really appreciates it!
Firstly, I want to concede and acknowledge that we haven’t done a great job at sharing how we measure and evaluate the impact of our programmes lately. There’s no complicated reason behind this; running large conferences around the world is very time-consuming, and analysing the impact of events is very difficult! That said, we’ve made more capacity on the team for this, and increasing our capacity for this work is a top priority. As a result, there’s been a lot more impact evaluation going on behind the scenes this year. We hope to share that work soon.
This null result is useful to us; while changes in behaviours and attitudes aren’t a key outcome we’re aiming for with EAGx events (more on this below), it is something we often allude to or claim as something that happens as a result of our events. This study is a data point against that, and that might mean we should redirect some efforts towards generating other kinds of outcomes. Thanks!
However, I don’t expect we’ll update our theory of change because of this study, which I don’t think will come as a surprise to you.
This was mentioned in the comments, and you acknowledge this too, but this survey is very likely underpowered, and I’d probably want to see a much larger sample size before reaching any firm conclusions.
Secondly, many of the behaviours and attitudes you ask about, particularly things like attending further events and engaging more in online EA spaces, aren’t the primary things we aim to influence via EAGx events. We’re typically aiming for concrete plan changes, such as finding new impactful roles, opportunities, and collaborators, and we think a lot of the value of events often comes from just a few cases.
You write:
A large portion of the impact may be concentrated in a small subset of attendees. It is plausible that the majority of conference attendees experience few long-term effects, while a small minority are led to major life or career changes as a result of connections made or motivation obtained at the conference.
This is our current best guess at what’s happening and what we target with our evaluations, but I don’t expect a study like this to pick up on this effect. I like your recommendation to find even more ways to actually identify these impacts!
That said, donating more to EA causes and creating more connections in the community are outcomes that we aim for, so it’s interesting that you didn’t see a long-term effect on these questions (though note the point about the survey being underpowered).
Thanks for writing up the recommendations. As mentioned, we’ve been investing more time in measuring our impact and hope to share some thoughts here soon. This post updated me a bit towards trying something more longitudinal too.
Thanks again!
The aforementioned Ollie from the CEA events team here. Thanks so much James and Miles for running this survey!
I reviewed this post before it went out, but I just wanted to quickly drop by to say that our team has seen this and that I intend to take some time soon to read the comments and provide some additional reactions and commentary here.
As someone on the team running the intervention, I strongly agree with this!
Thanks for writing this! I’m excited to see organisers and advisors really dive into theories of change for EAGx.
That said, I think the models here might benefit from looking at the existing data. Reading this post, you might think will be the first EA Global or EAGx event to take place—but there have been many, and we have lots of data on what people find useful.The most useful post is probably this one.
I also gathered data about what part of the event was most valuable from 2023 events (from follow-up surveys, so asking people what was valuable several months after the event). Sharing below.
We aren’t planning on having a GCR (or other cause area) focus for this event, but we’ll confirm that in due course.
EA Global: Bay Area 2025 will take place 21-23 February 2025 at the Oakland Marriott (the same venue as the past two years). Information on how to apply and other details to follow, just an FYI for now since we have the date.
Thanks for your comment.
We’ve mentioned elsewhere that we might revisit the decision not to claim Gift Aid after the event, and so we’re planning to send around Gift Aid Declaration forms to donors soon. We think this would be helpful so we can preserve the option of claiming Gift Aid on these donations in the future.
We think the HMRC guidance to individuals is not very clear, and thanks for your prompt to check this. We have looked into it, and consulted our external advisors and we do not think that it is a requirement of personal tax relief that the individual donor signs a Gift Aid Declaration form. However, as HMRC’s own guidance isn’t consistently clear on this, and as we’d like to keep the option of claiming the Gift Aid afterwards, we’d like to send around GAD forms and will be in touch with donors to do this.
This thread has prompted us to pay closer attention here, so thank you (everyone in this thread) for flagging it!
Really happy to see this! Thanks for getting it going again :)