(Jumping in for our busy comms/exec team) Understanding the status of the EA brand and working to improve it is a top priority for CEA :) We hope to share more work on this in future.
OllieBase
Karma: 6,212
First, 6 months is probably too short a timeframe to measure conference impact—many of my most valuable changes in behavior (e.g., starting new projects/collaborations, or providing (and getting) support/advice) occurred years after the conference where I met someone for the first time.
Possibly, though there is a trade-off here. We also hear in our 3–6 month follow-up surveys that people don’t really remember conversations from the event. Maybe that’s just a sign that nothing super valuable occurred but if you attend lots of events, questions about an event that occurred >6mo ago can be difficult to answer even if it was impactful. If we ask straight after the event, and 3–6 months later and a year later, I’d worry about survey fatigue.
Upcoming EA conferences in 2024 and 2025
Replying in full now!
Thanks once again for conducting this study and taking the time to write up the results so clearly. As Guy Raveh says, it takes courage to share your work publicly at all, let alone work that runs contrary to popular opinion on a forum full of people who make a living out of critiquing documents. The whole CEA events team really appreciates it!
Firstly, I want to concede and acknowledge that we haven’t done a great job at sharing how we measure and evaluate the impact of our programmes lately. There’s no complicated reason behind this; running large conferences around the world is very time-consuming, and analysing the impact of events is very difficult! That said, we’ve made more capacity on the team for this, and increasing our capacity for this work is a top priority. As a result, there’s been a lot more impact evaluation going on behind the scenes this year. We hope to share that work soon.
This null result is useful to us; while changes in behaviours and attitudes aren’t a key outcome we’re aiming for with EAGx events (more on this below), it is something we often allude to or claim as something that happens as a result of our events. This study is a data point against that, and that might mean we should redirect some efforts towards generating other kinds of outcomes. Thanks!
However, I don’t expect we’ll update our theory of change because of this study, which I don’t think will come as a surprise to you.
This was mentioned in the comments, and you acknowledge this too, but this survey is very likely underpowered, and I’d probably want to see a much larger sample size before reaching any firm conclusions.
Secondly, many of the behaviours and attitudes you ask about, particularly things like attending further events and engaging more in online EA spaces, aren’t the primary things we aim to influence via EAGx events. We’re typically aiming for concrete plan changes, such as finding new impactful roles, opportunities, and collaborators, and we think a lot of the value of events often comes from just a few cases.
You write:
A large portion of the impact may be concentrated in a small subset of attendees. It is plausible that the majority of conference attendees experience few long-term effects, while a small minority are led to major life or career changes as a result of connections made or motivation obtained at the conference.
This is our current best guess at what’s happening and what we target with our evaluations, but I don’t expect a study like this to pick up on this effect. I like your recommendation to find even more ways to actually identify these impacts!
That said, donating more to EA causes and creating more connections in the community are outcomes that we aim for, so it’s interesting that you didn’t see a long-term effect on these questions (though note the point about the survey being underpowered).
Thanks for writing up the recommendations. As mentioned, we’ve been investing more time in measuring our impact and hope to share some thoughts here soon. This post updated me a bit towards trying something more longitudinal too.
Thanks again!
The aforementioned Ollie from the CEA events team here. Thanks so much James and Miles for running this survey!
I reviewed this post before it went out, but I just wanted to quickly drop by to say that our team has seen this and that I intend to take some time soon to read the comments and provide some additional reactions and commentary here.
As someone on the team running the intervention, I strongly agree with this!
Thanks for writing this! I’m excited to see organisers and advisors really dive into theories of change for EAGx.
That said, I think the models here might benefit from looking at the existing data. Reading this post, you might think will be the first EA Global or EAGx event to take place—but there have been many, and we have lots of data on what people find useful.The most useful post is probably this one.
I also gathered data about what part of the event was most valuable from 2023 events (from follow-up surveys, so asking people what was valuable several months after the event). Sharing below.
We aren’t planning on having a GCR (or other cause area) focus for this event, but we’ll confirm that in due course.
EA Global: Bay Area 2025 will take place 21-23 February 2025 at the Oakland Marriott (the same venue as the past two years). Information on how to apply and other details to follow, just an FYI for now since we have the date.
Thanks for your comment.
We’ve mentioned elsewhere that we might revisit the decision not to claim Gift Aid after the event, and so we’re planning to send around Gift Aid Declaration forms to donors soon. We think this would be helpful so we can preserve the option of claiming Gift Aid on these donations in the future.
We think the HMRC guidance to individuals is not very clear, and thanks for your prompt to check this. We have looked into it, and consulted our external advisors and we do not think that it is a requirement of personal tax relief that the individual donor signs a Gift Aid Declaration form. However, as HMRC’s own guidance isn’t consistently clear on this, and as we’d like to keep the option of claiming the Gift Aid afterwards, we’d like to send around GAD forms and will be in touch with donors to do this.
This thread has prompted us to pay closer attention here, so thank you (everyone in this thread) for flagging it!
Thanks for your comment, Rasool.
We’re happy to provide as much info as we can here but I just want to make clear that, ultimately, it’s your call about whether you claim tax relief on this donation.
We do think we haven’t been sufficiently clear about how donations to EA Global work and we’ll be revisiting this language in due course to make things clearer. Addressing a few other points:
Our payment platform is set up to receive payments, and to generate receipts. A purchase receipt does not change the fact that this was a donation to a charity.
Re: whether tickets recoup costs: EA Global conferences cost >£1k per person per event (though that’s coming down) so the suggested donation amount wouldn’t cover the costs of one attendee. Our goal isn’t to maximise donations and break even, so while we’re very grateful for all donations, we want to ensure the event is accessible to a wide range of people.
Re: HMRC rules: As above, our goal isn’t to cover our costs through ticket sales—and we view giving free tickets as part of our wider charitable purposes—so this part of the HMRC guidance doesn’t apply to us.
RE: applications: Filtering all applications and then only inviting selected people to register for a free event is allowed within HMRC rules—provided the event is open to a sufficiently wide section of the public, it is fine to require attendees to meet some standards (e.g. demonstrating a genuine interest in the topics at the conference, before they are accepted and then invited to donate). CEA only asks people to make a donation once they have been accepted to attend the event.
Again, thanks so much for your comments here.
Sharing a piece of advice I’ve given to a few people about applying for (EA) funding.
I’ve heard various people working on early-stage projects express hesitancy about applying for EA funding because their plan isn’t “complete” enough. They don’t feel confident enough in their proposal, or think what they’re asking for is too small. They seem to assume that EA funders only want to look at proposals with a long time-horizons from applicants who will work full-time who are confident their plan will work.In my experience (I’ve done various bits of grantmaking and regularly talk to EA funders), grantmakers in EA spaces are generally happy to receive applications that don’t have these qualities. It’s okay to apply if you just want to test a project out for a few months; maybe you won’t be full-time, maybe you aren’t confident in some part of the theory of change, maybe it’s just a few months. You should apply and just explain your thinking, including all of your uncertainties.
Funders are uncertain too, and often prefer to fund tests for a few months than commit to multi-year projects with full-time staff because tests give them useful information about you and the theory of change. Ideally, funders eventually support long-term projects too.
I’m not super confident in this take, but I ran it past a few EA funders and they agreed. Note that I think this probably doesn’t apply outside of EA; I understand many grant applications require detailed plans.
You might like David Nash’s Monthly Overload of Effective Altruism.
It has a combination of forum/EA org updates and relevant news.
I can also recommend Shakeel’s Transformer newsletter for AI-specific news.
I agree that upvotes are an update that the content was useful and that might mean I’m just off here.
I guess I think relevancy or interest isn’t the only thing we should be tracking; I think the forum adds value because, unlike Twitter, the posts and replies here are usually more thoughtful and more carefully investigated. To put it another way, if I just wanted relevant news and updates, I’d go to Twitter. Here, I’d hope for more commentary (e.g. “Jan has joined Anthropic. Here are some thoughts I have about this and how they might be relevant for EAs deciding which lab to join”).
I’m not saying you personally should have done this and, again, I am grateful you made the effort to share this.
I’m grateful you shared this since it’s probably interesting to the community, but flagging I’m a little nervous about the EA Forum filling up with cross-posts from X. I think that would make it less interesting and unique.
Thanks AGB (and Rasool below),
I’m looking into this. Again, it seems our language here hasn’t been clear enough and I want to make sure I’m as clear as possible when I respond.
Update here: we’re supporting an EA Summit in Abuja, Nigeria on 6–7 September :)
https://forum.effectivealtruism.org/posts/8eWysFRNK9biZnECY/announcing-the-ea-nigeria-summit
Thanks for your comment AGB, and sorry I didn’t give enough detail initially. I’ve checked with relevant people internally, and our thoughts are below.
HMRC uses “Gift Aid” (in both the form and their Adjusted Net Income page) to mean that the donation was eligible for Gift Aid in the charity’s hands. We don’t have to claim Gift Aid for the donation to be eligible (and HMRC does not expect donors to confirm this).
If you donated £400 for EAG London and you’re filling out the self-assessment tax return, you would add £400 to boxes 5 and 6 of the Charitable Giving section (as a “one off” donation, because EAG London is not a regular monthly donation). The level of tax relief you receive will depend on your own income levels through the year, and this isn’t something we can comment on.
We aren’t currently planning on claiming Gift Aid on the donations to EAG London to reduce administrative overhead, but we might possibly revisit that in future years. We will take another look at our website language, as we’ve had a few people ask about Gift Aid, and so we may not have been clear enough that our decision on whether or not to claim Gift Aid does not actually impact an individual donor’s tax filing position.
Hi Lovkush, Ollie from the CEA events team here.
We are not claiming Gift Aid on the donations which relate to EAG London this year.
However, this is separate from your own personal tax reliefs, which you can claim on your donations in your personal tax return (and which does not depend on whether Effective Ventures, our parent organisation, claims Gift Aid).
A similar one was run last year, here :)