As you can see, some metrics take a dip in Q1 and Q2 2023: site visitors & engagement time, new newsletter subscribers, podcast listening time, and applications to advising.
I’d like to say four things about that data:
It seems pretty plausible to me that lower interest in EA due to the FTX crash is one (important) factor driving those metrics that took a dip. That said:
All of those seem to have “bounced back” in Q3
Our website (and to some extent podcast) metrics are very heavily driven by how much outreach & marketing we do. In Q4 2022, we spent very little on marketing compared to the rest of 2022 & 2023, which I think is a significant contributor to the trend.
It looks like the second half of 2022 was just an unusually high-growth period (for 80k, and I think EA more broadly), and falling from that peak is not particularly surprising due to regression to the mean. Maintaining a level of growth that high might have been possible, but it would be historically very unusual.
To make a more general point, I think a number of the metrics you cite in the post show returning to levels from early 2022, or 2021, which (at least to me) doesn’t seem like ‘worrisome deterioration of EA activity’ but rather something more like ‘a return to a historically normal level of growth for EA.’
Thanks so much for sharing this data Bella! I agree 80k’s metrics look a lot better than everything else I looked at. I’ll add an edit to the OP to that effect, and will link to your comment. To the extent 80k is doing something different from other orgs that could explain the better performance, it would be great to identify that (I wonder if your third point about advertising might be a differentiating factor).
To make a more general point, I think a number of the metrics you cite in the post show returning to levels from early 2022, or 2021, which (at least to me) doesn’t seem like ‘worrisome deterioration of EA activity’ but rather something more like ‘a return to a historically normal level of growth for EA.’
I disagree with this interpretation pretty strongly. The metrics that have returned to early 2022 levels are basically the EA Forum metrics, which had been experiencing extremely rapid growth since 2019/2020. Even if those fast growth rates were unsustainable, the baseline expectation should probably be that the high growth rates gradually slow down and flatten out, not that they abruptly shift to zero growth (which is basically what we see with the forum metrics). EA Funds metrics, where we also have long data history, are back to 2020 levels. Effectivealtruism.org Intro Page views are at the lowest levels ever, after many years of being rangebound. And the EA Newsletter had years of relatively steady subscriber growth that abruptly flipped to subscriber losses after FTX.
So I’d argue that for the publicly available data series with the longest histories, we’re not seeing a “normal level of growth”. While growth had generally been the norm pre-FTX, we’re now either seeing zero growth (EA forum metrics) or outright contraction (EA Funds donations/donors, EA.org intro page, EA newsletter).
Also, if you had asked for growth predictions in September 2022 -- and told the forecasters that 2023 would be a year in which public awareness of / interest in the nearness and risks of powerful AI greatly increased—one would likely factor this information into the prediction in addition to historical trends.
By rough analogy, if the stock market is unexpectedly up 25 percent over last year, your sector is up 35 percent, but your company’s stock is flat . . . the market seems to be saying something negative about your company. The flat stock value may well mean that positive macroeconomic trends and sector trends have been cancelled out by negative outlook on your company.
Thanks for replying & editing the OP — appreciate it! :D
I wonder if your third point about advertising might be a differentiating factor
Yeah — to my knowledge, we have the biggest team focused on outreach of any (single) EA organisation (3 FTE). I think this is probably a big part of it.
...we’re not seeing a “normal level of growth”. While growth had generally been the norm pre-FTX, we’re now either seeing zero growth (EA forum metrics) or outright contraction (EA Funds donations/donors, EA.org intro page, EA newsletter).
I think that’s fair enough — we’re talking about “telling a story” about a bunch of data points here, across several years, each of which might have a whole load of complex factors driving them. I think my view here is not very resilient to new info/new ways of looking at it, and I agree some of the data seems like it fits a lot better with your high-level story with mine.
That said, I’d point out that:
EA Forum engagement hours are still above early 2022 levels, if I’m reading the graph correctly, which was my claim above
The EA Newsletter shrinking is probably mostly driven by where the signup is advertised, and how much.
It also looks like the newsletter has barely grown since 2018, so I’d guess CEA are not actively trying to get many more signups.
Also, isn’t the main place it’s advertised the EA intro essay? So if traffic to that page has dropped, signups will drop → they’re not independent sources of data on interest in EA dropping.
Put another way, it’d be really remarkable if the newsletter could still grow when its signup page was being shown to fewer people!
Basically we’re grabbing analytics from Apple Podcasts, Spotify for Podcasters and Google Podcast Manager (which internally I call the ‘Big 3’), and adding them up.
But Spotify and Google Podcasts only became available around/after Nov 2019. Drop me an email if you’d like to discuss! :)
Hey! I work at 80k doing outreach.
Thanks for your work here!
I think the data from 80k overall tells a bit of a different story.
Here’s a copy of our programmes’ lead metrics, and our main funnel metrics (more detailed).
As you can see, some metrics take a dip in Q1 and Q2 2023: site visitors & engagement time, new newsletter subscribers, podcast listening time, and applications to advising.
I’d like to say four things about that data:
It seems pretty plausible to me that lower interest in EA due to the FTX crash is one (important) factor driving those metrics that took a dip. That said:
All of those seem to have “bounced back” in Q3
Our website (and to some extent podcast) metrics are very heavily driven by how much outreach & marketing we do. In Q4 2022, we spent very little on marketing compared to the rest of 2022 & 2023, which I think is a significant contributor to the trend.
It looks like the second half of 2022 was just an unusually high-growth period (for 80k, and I think EA more broadly), and falling from that peak is not particularly surprising due to regression to the mean. Maintaining a level of growth that high might have been possible, but it would be historically very unusual.
To make a more general point, I think a number of the metrics you cite in the post show returning to levels from early 2022, or 2021, which (at least to me) doesn’t seem like ‘worrisome deterioration of EA activity’ but rather something more like ‘a return to a historically normal level of growth for EA.’
Thanks so much for sharing this data Bella! I agree 80k’s metrics look a lot better than everything else I looked at. I’ll add an edit to the OP to that effect, and will link to your comment. To the extent 80k is doing something different from other orgs that could explain the better performance, it would be great to identify that (I wonder if your third point about advertising might be a differentiating factor).
I disagree with this interpretation pretty strongly. The metrics that have returned to early 2022 levels are basically the EA Forum metrics, which had been experiencing extremely rapid growth since 2019/2020. Even if those fast growth rates were unsustainable, the baseline expectation should probably be that the high growth rates gradually slow down and flatten out, not that they abruptly shift to zero growth (which is basically what we see with the forum metrics). EA Funds metrics, where we also have long data history, are back to 2020 levels. Effectivealtruism.org Intro Page views are at the lowest levels ever, after many years of being rangebound. And the EA Newsletter had years of relatively steady subscriber growth that abruptly flipped to subscriber losses after FTX.
So I’d argue that for the publicly available data series with the longest histories, we’re not seeing a “normal level of growth”. While growth had generally been the norm pre-FTX, we’re now either seeing zero growth (EA forum metrics) or outright contraction (EA Funds donations/donors, EA.org intro page, EA newsletter).
Also, if you had asked for growth predictions in September 2022 -- and told the forecasters that 2023 would be a year in which public awareness of / interest in the nearness and risks of powerful AI greatly increased—one would likely factor this information into the prediction in addition to historical trends.
By rough analogy, if the stock market is unexpectedly up 25 percent over last year, your sector is up 35 percent, but your company’s stock is flat . . . the market seems to be saying something negative about your company. The flat stock value may well mean that positive macroeconomic trends and sector trends have been cancelled out by negative outlook on your company.
Thanks for replying & editing the OP — appreciate it! :D
Yeah — to my knowledge, we have the biggest team focused on outreach of any (single) EA organisation (3 FTE). I think this is probably a big part of it.
I think that’s fair enough — we’re talking about “telling a story” about a bunch of data points here, across several years, each of which might have a whole load of complex factors driving them. I think my view here is not very resilient to new info/new ways of looking at it, and I agree some of the data seems like it fits a lot better with your high-level story with mine.
That said, I’d point out that:
EA Forum engagement hours are still above early 2022 levels, if I’m reading the graph correctly, which was my claim above
The EA Newsletter shrinking is probably mostly driven by where the signup is advertised, and how much.
It also looks like the newsletter has barely grown since 2018, so I’d guess CEA are not actively trying to get many more signups.
Also, isn’t the main place it’s advertised the EA intro essay? So if traffic to that page has dropped, signups will drop → they’re not independent sources of data on interest in EA dropping.
Put another way, it’d be really remarkable if the newsletter could still grow when its signup page was being shown to fewer people!
Off-topic but asking for personal interest:
Would you be up for explaining (briefly) how you calculate your podcast metrics? E.g.
Total listening time estimate using March 2022 method (40% Apple up to Nov 2019 then ’75% in the Big 3′)
Big 3 subscriber estimate at end of period
(I couldn’t figure out what Big 3 was in this context nor your March 2022 method)
Basically we’re grabbing analytics from Apple Podcasts, Spotify for Podcasters and Google Podcast Manager (which internally I call the ‘Big 3’), and adding them up.
But Spotify and Google Podcasts only became available around/after Nov 2019. Drop me an email if you’d like to discuss! :)