Hey Cillian — thanks so much for a really thoughtful/detailed question!
I’ll take this one since I was the only staff member on marketing last year :)
The short answer is:
Marketing ramped up considerably over the second half of 2022. Web engagement time grew a lot more in the second half of 2022 as well — if we just compare Q3 &Q4 2020 and 2022, engagement time grew 50% (rather than 10%).
But that still doesn’t look like web engagement time rising precisely in step with marketing investment, as you point out!
We don’t know all the reasons why, but the reasons you gave might be part of it.
There are five other reasons I want to mention:
marketing didn’t focus on engagement time
people we find via marketing seem less inclined to use our advice than other users
there’s a headwind on engagement time
site engagement time doesn’t count the book giveaway
2020 was a bumper year!
The long answer...
It’s true that there’s only a 10% rise in engagement time between 2020 and 2022.
The main thing going on here is that marketing investment didn’t rise until the second half of 2022 — I spent a while trying out smaller scale pilots. So the spend is very unevenly distributed.
If we compare just Q3 and Q4 (i.e. Q3 and Q4 2020 with Q3 and Q4 2021 and Q3 and Q4 2022), there was a 6% fall from 2020 to 2021, and then a 59% rise from 2021 to 2022, resulting in an overall 50% rise from 2020 to 2022!
But our marketing budget still rose by a lot more than 50% — so what’s going on there?
I’ll start out with the reasons you gave then add my own:
Maybe the price of acquiring new users / engagement hours increases geometrically or something
Yeah I don’t exactly know how the price of finding new users increases, but we should probably expect some diminishing returns from increased investment.
It looks like marketing drove a large increase in newsletter subs. Maybe they’re engaging with the content directly in their inbox instead?
I think this might be a smallish part of it — I’ve noticed an effect where if the email we send on the newsletter is itself full of content, people click through to the website less than if the newsletter doesn’t itself provide much value.
I don’t think this can account for tons of what we’re seeing, though, just cos I don’t think the emails work as a 1:1 replacement for 80k’s site (I can’t really imagine there being much of a ‘substitution effect’ here).
Maybe you expect a lag in time between initial reach & time spent on the 80k website for some reason (e.g. because people become more receptive to the ideas on 80k’s website over time, especially if they’re receiving regular emails with some info about it)
I think this might be a pretty big part of what’s going on.
There does seem to be a significant ‘lag time’ from people first hearing about us and people making an important change to their careers (about 2 years on average) and I think there’s often a lag before people get really engaged with site content, too.
Also, bear in mind that because of what I said about the ‘unevenness’ of growth from marketing, people who found out about us this year are mostly still really new.
Maybe marketing mainly promoted podcast / 1-1 service / job board (or people reached by marketing efforts mainly converted to users of these services)
Yep, I did put some resources directly towards promoting the podcast, and a much smaller amount of resources towards directly promoting 1-1 (about 7% of the budget as a whole, and probably more like 10% of my time). So this could be (a small) part of what’s going on.
There are five other main things I think explain this effect:
Marketing didn’t focus on engagement time
My focus was really on reaching new users, rather than trying to get people to spend time with our content.
For example, last year I spent almost zero effort and resources on trying to get people who already knew about 80,000 Hours to engage with stuff on the site.
People who find out about us via marketing are, on average, less interested in our advice
I think this is probably a pretty big factor here!
There’s a headwind on engagement time, i.e. engagement time by default seems to go down over time
We think this is because, e.g.:
Articles that were last updated longer ago are seen as less relevant by search engines and deprioritised
Things we wrote become out of date and less useful to people
There might be some broader internet trend where people are spending more time on social media and less time on individual websites
Our current metric doesn’t incorporate engagement time from reading the books in the book giveaway
However, we are looking into including it in future reporting!
My (still ongoing) analysis of survey results about the book giveaway suggest that this engagement time is at least tens of thousands of hours, possibly >100k hrs.
(For context, our average monthly engagement time since Jan 2021 is about 8,000 hours (ha!)).
2020 was weird
We saw a very large spike in traffic in early 2020 — partly because we had content on COVID when many places didn’t, and partly because of the broader trend where tons of sites got a lot more traffic as people were staying home and spending more time online.
So 2020 might be an “inappropriate benchmark” or something like that.
Okay I hope that gives you an insight into what I think is going on here! Sorry for length :)
Hey Cillian — thanks so much for a really thoughtful/detailed question!
I’ll take this one since I was the only staff member on marketing last year :)
The short answer is:
Marketing ramped up considerably over the second half of 2022. Web engagement time grew a lot more in the second half of 2022 as well — if we just compare Q3 &Q4 2020 and 2022, engagement time grew 50% (rather than 10%).
But that still doesn’t look like web engagement time rising precisely in step with marketing investment, as you point out!
We don’t know all the reasons why, but the reasons you gave might be part of it.
There are five other reasons I want to mention:
marketing didn’t focus on engagement time
people we find via marketing seem less inclined to use our advice than other users
there’s a headwind on engagement time
site engagement time doesn’t count the book giveaway
2020 was a bumper year!
The long answer...
It’s true that there’s only a 10% rise in engagement time between 2020 and 2022.
The main thing going on here is that marketing investment didn’t rise until the second half of 2022 — I spent a while trying out smaller scale pilots. So the spend is very unevenly distributed.
If we compare just Q3 and Q4 (i.e. Q3 and Q4 2020 with Q3 and Q4 2021 and Q3 and Q4 2022), there was a 6% fall from 2020 to 2021, and then a 59% rise from 2021 to 2022, resulting in an overall 50% rise from 2020 to 2022!
But our marketing budget still rose by a lot more than 50% — so what’s going on there?
I’ll start out with the reasons you gave then add my own:
Yeah I don’t exactly know how the price of finding new users increases, but we should probably expect some diminishing returns from increased investment.
I think this might be a smallish part of it — I’ve noticed an effect where if the email we send on the newsletter is itself full of content, people click through to the website less than if the newsletter doesn’t itself provide much value.
I don’t think this can account for tons of what we’re seeing, though, just cos I don’t think the emails work as a 1:1 replacement for 80k’s site (I can’t really imagine there being much of a ‘substitution effect’ here).
I think this might be a pretty big part of what’s going on.
There does seem to be a significant ‘lag time’ from people first hearing about us and people making an important change to their careers (about 2 years on average) and I think there’s often a lag before people get really engaged with site content, too.
Also, bear in mind that because of what I said about the ‘unevenness’ of growth from marketing, people who found out about us this year are mostly still really new.
Yep, I did put some resources directly towards promoting the podcast, and a much smaller amount of resources towards directly promoting 1-1 (about 7% of the budget as a whole, and probably more like 10% of my time). So this could be (a small) part of what’s going on.
There are five other main things I think explain this effect:
Marketing didn’t focus on engagement time
My focus was really on reaching new users, rather than trying to get people to spend time with our content.
For example, last year I spent almost zero effort and resources on trying to get people who already knew about 80,000 Hours to engage with stuff on the site.
People who find out about us via marketing are, on average, less interested in our advice
I talked about this a little bit in my post that you linked.
I think this is probably a pretty big factor here!
There’s a headwind on engagement time, i.e. engagement time by default seems to go down over time
We think this is because, e.g.:
Articles that were last updated longer ago are seen as less relevant by search engines and deprioritised
Things we wrote become out of date and less useful to people
There might be some broader internet trend where people are spending more time on social media and less time on individual websites
Our current metric doesn’t incorporate engagement time from reading the books in the book giveaway
However, we are looking into including it in future reporting!
My (still ongoing) analysis of survey results about the book giveaway suggest that this engagement time is at least tens of thousands of hours, possibly >100k hrs.
(For context, our average monthly engagement time since Jan 2021 is about 8,000 hours (ha!)).
2020 was weird
We saw a very large spike in traffic in early 2020 — partly because we had content on COVID when many places didn’t, and partly because of the broader trend where tons of sites got a lot more traffic as people were staying home and spending more time online.
So 2020 might be an “inappropriate benchmark” or something like that.
Okay I hope that gives you an insight into what I think is going on here! Sorry for length :)