Thanks for writing this, itâs useful to have a summary in one place. And Iâm glad that CEA and others publishing these metrics has enabled you to do this research â kudos to Angelina in particular for the CEA dashboard!
We (CEA) have an internal draft of how engagement metrics have changed, and are working on publishing it. My understanding is that publication is currently bottlenecked on getting approval from the various orgs to share their data sources. [Edit: now published]
You also may or may not be aware that Rethink Priorities is doing a follow-up survey shortly; hopefully the results of that will give us an update about how people within EA are feeling about EA.
Iâve asked the Events team for data on applications, but received no response
Sorry to hear about this, could you forward me the email? The events team canât find a record of this, and Iâd like to figure out what happened.
Thanks Ben! I look forward to reading CEAâs report on this topic. While we wait for publication, are you able to share the big picture takeaway? Having seen the data in CEAâs internal report and the information Iâve presented here, do you still hold your June belief that: âit seems that the FTX crash hasnât, overall, impacted sentiments toward EA very muchâ?
I also look forward to seeing the results of the updated RP survey, though it sounds like it hasnât been conducted yet and then will take some time to analyze the results. Do you know roughly when we can expect these results to be published?
Re: the events team, I didnât see email addresses on the team page of CEAâs website, so I sent Amy Labenz a PM through the forum on October 3.
Do you know roughly when we can expect these results [from the followup RP survey] to be published?
Iâm hopeful that we can launch the survey sometime between the end of this month and the middle of November (the next 2-4 weeks). Like with Benâs report, weâre just waiting on input from a variety of different orgs (the survey is addressing multiple different aims, besides looking at community health/âFTX, so there are quite a few different stakeholders). Allowing another 2-4 weeks for the survey to run (taking us up to early-mid December), I would still aim report on the FTX/âcommunity health results before the end of the year.
Luke Freeman from GWWC has shared some excellentobservations around reduced willingness for community members to advocate for GWWC post-FTX (partially but not solely due to FTX). I think it would be quite valuable if the follow-up survey examined the dynamics he describes, as I doubt they are unique to GWWC.
Having seen the data in CEAâs internal report and the information Iâve presented here, do you still hold your June belief that: âit seems that the FTX crash hasnât, overall, impacted sentiments toward EA very muchâ?
Yes, my best understanding is still that people mostly donât know what EA is, the small fraction that do mostly have a mildly positive opinion, and that neither of these points were affected much by FTX.[1] This understanding was not affected by your post or the internal document I referred to very much, but I suspect we are talking past each other because I wouldnât have expected them to update me, since they seem to be targeted towards answering a different question.
I didnât see email addresses on the team page of CEAâs website, so I sent Amy Labenz a PM through the forum on October 3.
Ah ok, an email address is available on the FAQ page. We will look into making this more prominent, thanks for the feedback!
Yeah I think weâre in agreement that for people outside of EA, FTX didnât really have an impact because the vast majority of those people have never even heard of EA.
What I guess Iâm more interested in is your current thinking (and whether/âwhy it has changed) about how FTX has impacted people already in EA and the implications for EAâs future trajectory. I assume the internal report and my analysis are more relevant to the EA community rather than the general public.
Cool, in that case: A summary of our internal growth estimates shows more growth than your summary here (though there are a bunch of examples of things with decreased engagement, including stuff you point out here), and my guess is that RPâs survey will show mean reversion (i.e. peopleâs opinions about EA will be more similar to pre-FTX levels than they were immediately following the FTX collapse), though to be clear this is just a prediction because RP hasnât done the survey yet.[1]
This is a scenario where a prediction market might be action guiding for me, if someone wants to make it. If I thought the answers to RPâs survey were going to be the same or lower than they were in December, CEA might change some of its priorities, though Iâm not exactly sure how.
Iâve looked over Angelinaâs (excellent) report on EA growth rates, and it raised some questions about how you see top of funnel growth rates for the community.
In Julyâs mid-year update on CEA, you wrote: âour priorities havenât changed since we wrote about our work in [December] 2022: helping people who have heard about EA to deeply understand the ideas, and to find opportunities for making an impact in important fields. We continue to think that top-of-funnel growth is likely already at or above healthy levels, so rather than aiming to increase the rate any further, we want to make that growth go well.â (emphasis added).
Angelinaâs analysis suggests that in 2022 top-of-funnel growth was quite rapid (73% growth in 2022 vs. 2021) but has slowed significantly so far this year (30% in 2023 vs. 2022). Do you agree with her finding that top-of-funnel growth has slowed significantly (though IMO the extent of the drop might be a bit overstated by her analysis due to e.g. huge 2022 spikes in search activity for EA in the wake of FTX)?
Do you still think top-of-funnel growth is already at or above healthy levels; and if so, is that because your assessment of a healthy growth rate has dropped significantly similar to how actual growth has fallen? If not, is CEA planning any strategic shifts as a result? What do you consider a healthy level for top-of-funnel growth over the next year or two?
This is a great and well researched question, thanks for asking!
First: I think my personal opinion here might matter less than you would imagine: I am intentionally trying not to push for any major strategic changes to CEA while I am in the interim role.
That being said:
I agree that FTX causes the drop to be overstated. (And more importantly, Angelina, who collected this information, also agrees.)
Historically CEA has targeted a top of funnel growth of 30%, which, if you take these metrics at face value, means we are perfectly on target.
My personal low confidence guess is that our target growth rate should be a bit higher than it was historically, because:
CEA now has two full-time communications people, an increase of infinity percent over the zero people we had last year
EA (and particularly AI safety) have become mainstream/âprofessionalized (more people know what the causes are, why they are important, stable funding is available, etc.) which allows us to attract a broader demographic, specifically older/âmore senior people
Some details that seem worth considering:
Will is planning to âdistance myself from the idea that Iâm âthe face ofâ or âthe spokesperson forâ EAâ. I have not heard a compelling suggestions for an alternative person to replace him. (E.g. there are a few people writing books or doing interviews, but my sense is that none of them are likely to be cited as a major place people first heard about EA in the next EA survey.)
Peter Singer is doing a tour. Iâm interested in seeing if we can rope him into doing some stuff, he has historically been very successful at top of funnel outreach.
Some cruxes that would change my mind:
More employers (or things further down in the funnel) saying stuff like âthe difference between my top candidates is really small; getting more people into the hiring pool doesnât seem very useful.â I hear this occasionally, but not enough to make me currently worry about it too much.
More stories of harm from people getting involved in EA and then bouncing. I tried to do some investigation into this in the past, and itâs obviously by definition a hard population to interview, but my sense is that substantial harms are relatively rare.
More evidence behind the âkids these days arenât as good as the ones in my day wereâ complaints that Iâve heard every year since Iâve been in EA. I do worry that we might be accidentally losing all aspects of EA which are important by having too many new people come in without acculturation, but I havenât seen any persuasive arguments that this is actually happening.
More stories of harm from people getting involved in EA and then bouncing. I tried to do some investigation into this in the past, and itâs obviously by definition a hard population to interview, but my sense is that substantial harms are relatively rare.
Oops that was supposed to link to this sequence, updated now. (That sequence isnât a complete list of everything that I and others at CEA have done, but itâs the best I know of.)
People who are substantially harmed by a movement typically donât tell the community builders of that movement that theyâre leaving because they were substantially harmed. They give some other, less vulnerable reason. Some examples of this could be âlack of culture fit or interpersonal conflictâ or âburnout/âmental healthâ, two of the major cited factors in the linked sequence of why people leave.
Re: 2, very helpful to know CEAâs top of funnel target. To the best of my knowledge, this hasnât been shared before. Are there also targets for middle and bottom of funnel growth, and if so, would you mind sharing those?
Re: 3, I agree that both of your points suggest raising the target might make sense. But in the other direction, all else equal we should expect growth rates to slow over time (30% annual growth obviously isnât sustainable in perpetuity).
Re: 4, I would VERY much like to see EA develop growth channels that arenât dependent on a public figure (particularly a philosopher) releasing a book, going on a tour, publicizing his multibillion dollar crypto exchange, etc. More organic channels (e.g. campus outreach) seem more sustainable, more scalable, and less prone to the hero worship that often seems to be found in EA.
Thanks for writing this, itâs useful to have a summary in one place. And Iâm glad that CEA and others publishing these metrics has enabled you to do this research â kudos to Angelina in particular for the CEA dashboard!
We (CEA) have an internal draft of how engagement metrics have changed, and are working on publishing it. My understanding is that publication is currently bottlenecked on getting approval from the various orgs to share their data sources. [Edit: now published]
You also may or may not be aware that Rethink Priorities is doing a follow-up survey shortly; hopefully the results of that will give us an update about how people within EA are feeling about EA.
Sorry to hear about this, could you forward me the email? The events team canât find a record of this, and Iâd like to figure out what happened.
Thanks Ben! I look forward to reading CEAâs report on this topic. While we wait for publication, are you able to share the big picture takeaway? Having seen the data in CEAâs internal report and the information Iâve presented here, do you still hold your June belief that: âit seems that the FTX crash hasnât, overall, impacted sentiments toward EA very muchâ?
I also look forward to seeing the results of the updated RP survey, though it sounds like it hasnât been conducted yet and then will take some time to analyze the results. Do you know roughly when we can expect these results to be published?
Re: the events team, I didnât see email addresses on the team page of CEAâs website, so I sent Amy Labenz a PM through the forum on October 3.
Iâm hopeful that we can launch the survey sometime between the end of this month and the middle of November (the next 2-4 weeks). Like with Benâs report, weâre just waiting on input from a variety of different orgs (the survey is addressing multiple different aims, besides looking at community health/âFTX, so there are quite a few different stakeholders). Allowing another 2-4 weeks for the survey to run (taking us up to early-mid December), I would still aim report on the FTX/âcommunity health results before the end of the year.
Hi David! Any update on what (if anything) is going on with this survey and sharing its results? Was this part of the survey that was conducted in late December?
Thanks for the update David! These results should be very interesting.
Luke Freeman from GWWC has shared some excellentobservations around reduced willingness for community members to advocate for GWWC post-FTX (partially but not solely due to FTX). I think it would be quite valuable if the follow-up survey examined the dynamics he describes, as I doubt they are unique to GWWC.
Yes, my best understanding is still that people mostly donât know what EA is, the small fraction that do mostly have a mildly positive opinion, and that neither of these points were affected much by FTX.[1] This understanding was not affected by your post or the internal document I referred to very much, but I suspect we are talking past each other because I wouldnât have expected them to update me, since they seem to be targeted towards answering a different question.
Ah ok, an email address is available on the FAQ page. We will look into making this more prominent, thanks for the feedback!
This is a very brief summary; see our original post for more details
Yeah I think weâre in agreement that for people outside of EA, FTX didnât really have an impact because the vast majority of those people have never even heard of EA.
What I guess Iâm more interested in is your current thinking (and whether/âwhy it has changed) about how FTX has impacted people already in EA and the implications for EAâs future trajectory. I assume the internal report and my analysis are more relevant to the EA community rather than the general public.
Cool, in that case: A summary of our internal growth estimates shows more growth than your summary here (though there are a bunch of examples of things with decreased engagement, including stuff you point out here), and my guess is that RPâs survey will show mean reversion (i.e. peopleâs opinions about EA will be more similar to pre-FTX levels than they were immediately following the FTX collapse), though to be clear this is just a prediction because RP hasnât done the survey yet.[1]
This is a scenario where a prediction market might be action guiding for me, if someone wants to make it. If I thought the answers to RPâs survey were going to be the same or lower than they were in December, CEA might change some of its priorities, though Iâm not exactly sure how.
Iâve looked over Angelinaâs (excellent) report on EA growth rates, and it raised some questions about how you see top of funnel growth rates for the community.
In Julyâs mid-year update on CEA, you wrote: âour priorities havenât changed since we wrote about our work in [December] 2022: helping people who have heard about EA to deeply understand the ideas, and to find opportunities for making an impact in important fields. We continue to think that top-of-funnel growth is likely already at or above healthy levels, so rather than aiming to increase the rate any further, we want to make that growth go well.â (emphasis added).
Angelinaâs analysis suggests that in 2022 top-of-funnel growth was quite rapid (73% growth in 2022 vs. 2021) but has slowed significantly so far this year (30% in 2023 vs. 2022). Do you agree with her finding that top-of-funnel growth has slowed significantly (though IMO the extent of the drop might be a bit overstated by her analysis due to e.g. huge 2022 spikes in search activity for EA in the wake of FTX)?
Do you still think top-of-funnel growth is already at or above healthy levels; and if so, is that because your assessment of a healthy growth rate has dropped significantly similar to how actual growth has fallen? If not, is CEA planning any strategic shifts as a result? What do you consider a healthy level for top-of-funnel growth over the next year or two?
This is a great and well researched question, thanks for asking!
First: I think my personal opinion here might matter less than you would imagine: I am intentionally trying not to push for any major strategic changes to CEA while I am in the interim role.
That being said:
I agree that FTX causes the drop to be overstated. (And more importantly, Angelina, who collected this information, also agrees.)
Historically CEA has targeted a top of funnel growth of 30%, which, if you take these metrics at face value, means we are perfectly on target.
My personal low confidence guess is that our target growth rate should be a bit higher than it was historically, because:
CEA now has two full-time communications people, an increase of infinity percent over the zero people we had last year
EA (and particularly AI safety) have become mainstream/âprofessionalized (more people know what the causes are, why they are important, stable funding is available, etc.) which allows us to attract a broader demographic, specifically older/âmore senior people
Some details that seem worth considering:
Will is planning to âdistance myself from the idea that Iâm âthe face ofâ or âthe spokesperson forâ EAâ. I have not heard a compelling suggestions for an alternative person to replace him. (E.g. there are a few people writing books or doing interviews, but my sense is that none of them are likely to be cited as a major place people first heard about EA in the next EA survey.)
Peter Singer is doing a tour. Iâm interested in seeing if we can rope him into doing some stuff, he has historically been very successful at top of funnel outreach.
Some cruxes that would change my mind:
More employers (or things further down in the funnel) saying stuff like âthe difference between my top candidates is really small; getting more people into the hiring pool doesnât seem very useful.â I hear this occasionally, but not enough to make me currently worry about it too much.
More stories of harm from people getting involved in EA and then bouncing. I tried to do some investigation into this in the past, and itâs obviously by definition a hard population to interview, but my sense is that substantial harms are relatively rare.
More evidence behind the âkids these days arenât as good as the ones in my day wereâ complaints that Iâve heard every year since Iâve been in EA. I do worry that we might be accidentally losing all aspects of EA which are important by having too many new people come in without acculturation, but I havenât seen any persuasive arguments that this is actually happening.
Can you say more what investigation you did?
Oops that was supposed to link to this sequence, updated now. (That sequence isnât a complete list of everything that I and others at CEA have done, but itâs the best I know of.)
People who are substantially harmed by a movement typically donât tell the community builders of that movement that theyâre leaving because they were substantially harmed. They give some other, less vulnerable reason. Some examples of this could be âlack of culture fit or interpersonal conflictâ or âburnout/âmental healthâ, two of the major cited factors in the linked sequence of why people leave.
Agreed, it feels real hard to get clear data on this â I would be excited for other people to research and share what they can find.
Thanks Ben, I appreciate this detailed response!
Re: 2, very helpful to know CEAâs top of funnel target. To the best of my knowledge, this hasnât been shared before. Are there also targets for middle and bottom of funnel growth, and if so, would you mind sharing those?
Re: 3, I agree that both of your points suggest raising the target might make sense. But in the other direction, all else equal we should expect growth rates to slow over time (30% annual growth obviously isnât sustainable in perpetuity).
Re: 4, I would VERY much like to see EA develop growth channels that arenât dependent on a public figure (particularly a philosopher) releasing a book, going on a tour, publicizing his multibillion dollar crypto exchange, etc. More organic channels (e.g. campus outreach) seem more sustainable, more scalable, and less prone to the hero worship that often seems to be found in EA.
Thanks for sharing your thinking and predictions Ben. I look forward to the publication of the internal report and RPâs updated survey.