80,000 Hours has been putting much more resources into growing our audience

[Bella]

Since the start of 2022, 80,000 Hours has been putting a lot more effort and money into getting more people to hear of us and engage with our advice.

This post aims to give some insight into what we’ve been up to and why.

Why invest more in outreach?

80,000 Hours has, we think, been historically cost-effective at causing more people to aim their careers at tackling pressing world problems. We’ve built a system of resources (website, podcast, job board, advising) that many people have found helpful for this end — and so we want more people to find them.

Also, 80,000 Hours has historically been the biggest single source of people learning about the EA community. If we want to grow the community, increasing the number of people reached by 80k seems like one of the best available tools for doing that.

Thirdly, outreach at the “top of the funnel” (i.e. getting people to subscribe after they hear about 80k’s ideas for the very first time) has unusually good feedback mechanisms & is particularly easy to measure. For the most part, we can tell if what we’re doing isn’t working, and change tack pretty quickly.^[1]

Another reason is that a lot of these activities take relatively little staff time, but can scale quite efficiently with more money.

Finally, based on our internal calculations, our outreach seems likely to be cost-effective as a means of getting more people into the kinds of careers we’re really excited about.

What did we do to invest more in outreach?

In 2020, 80k decided to invest more in outreach by moving one of their staff into a position focused on outreach, but it ended up not working out & that person left their role.

Then in mid-2021, 80k decided to hire someone new to work on outreach full-time. They hired 1 staff member (me!), and I started in mid-January 2022.

In mid-2022, we found that our initial pilots in this area looked pretty promising — by May we were on track to 4x our yearly rate of subscriber growth — and we decided to scale up the team and the resource investment. I ran a hiring round and made two hires, who started at the end of Nov 2022 and in Feb 2023; I now act as head of marketing.

We also decided to formalise a “marketing programme” for 80k, which is housed within the website team. Since this project spends money so differently from the rest of 80k, and in 2022 was a large proportion of our overall spending, last year we decided to approach funders specifically to support our marketing spend (rather than draw from our general funds). The marketing programme has a separate fundraising cycle and decisions are made on it somewhat independently from the rest of 80k.

In 2022, the marketing programme spent $2.65m (compared to ~$120k spent on marketing in 2021). The bulk of this spending was on sponsored placements with selected content creators ($910k), giving away free books to people who signed up to our newsletter ($1.04m), and digital ads ($338k). We expect to spend more in 2023, and are in conversation with funders about this.

As a result of our efforts, more than 5x as many people subscribed to our newsletter in 2022 (167k) than 2021 (30k), and we had more website visitors in Q4 2022 than any previous quarter (1.98m).^[2]

We can’t be sure how many additional people will change to a high-impact career as a result, in large part because we have found that “career plan changes” of this kind take, on average, about 2 years from first hearing about 80k.

Still, our current best guess is that these efforts will have been pretty effective at helping people switch careers to more impactful areas.

Partly this guess is based on the growth in new audience members that we’ve seen (plus 80k’s solid track record of getting new people to eventually switch to more impactful careers), and partly it’s based on a few “proof of concept” switches we’ve seen already.

For example, some small-scale social media ads which 80k ran in 2017 as an experiment led to at least one person switching to a career we’re especially excited about (and 70 people who reported changing their career plans due to 80k).^[3] We’ve also already encountered^[4] several people who found us via our marketing who seem likely to switch to a more impactful career.^[5]

(A bit more on audience inclination below.)

What specific projects have we completed in this area?

Advertising on social media

• We ran a series of ads on Facebook, Instagram, Google search, and YouTube.

• Most of the ads were targeted pretty narrowly at the audience for whom we believe our advice has historically been most useful — students and graduates at English-speaking top world universities — though some of them were much broader.

• We were pleased by the results and plan on continuing to run digital ads on these and other platforms.

Sponsored placements in other media, especially YouTube

• We worked with YouTube channels, podcasts, and newsletters to include mentions of 80k in their videos/​podcasts/​newsletters.

  • See some examples on YouTube:

    • Tom Scott

    • Ali Abdaal

    • SciShow

    • Adam Ragusea

    • Dr. Becky

    • Minute Earth

  • These were our single biggest source of growth, with one especially big sponsorship getting around 18,000 new subscribers!

Book giveaway

• We offer a free book to anyone who joins our newsletter.

• Readers get to choose from three books — Doing Good Better, The Precipice, and our own 80,000 Hours book.

Podcast advertising

• We paid to advertise The 80,000 Hours Podcast in a few places: on Facebook & Instagram, as above, and also on various podcast listening platforms like Podcast Addict, PlayerFM, Castro, etc.

Improvements to website “calls to action”

• Late in 2021, we made a suite of improvements to the site designed to increase the rate at which people sign up to our newsletter and apply for advising calls with us.

• We also made a few smaller adjustments and improvements throughout 2022 based on A/​B tests.

• We think there’s probably much more to do here in the future, in terms of optimising our readers’ experience with the website.

FAQ^[6]

Why ask people to sign up to your newsletter?

• I used newsletter signups in a lot of our outreach because there are both theoretical and empirical reasons to think it’s likely to make people engage with our advice.

• The theoretical reason is, well, when people join the newsletter they get our advice sent to their inbox!

  • Conventional wisdom says that regular prompts to engage with a service are better than one-off prompts, and we’ve also put a lot more effort behind making our newsletter full of consistent, high-quality releases this year.

    • (We now release them as blog posts as well as newsletters, for that reason!)

• The empirical reason is that being a newsletter subscriber correlates with having spent about twice the recorded time on our website, compared to non-subscribers. And every time we send out a newsletter, it gets many tens of thousands of opens and thousands of clicks.

• Newsletter signups are also easier to measure than some other outcomes, which makes it easier to see what’s working and what’s not.

• We’ve also experimented with some other “conversion” goals and may do more later (e.g. visits to site pages, podcast subscriptions).

Aren’t you going to get a lot of low-inclination traffic via these means (Where by ‘inclination’ I mean inclination to, eventually, change career plans to tackle one of the world’s most pressing problems)?

• Yes, I think we will.

  • The majority of the people who visit 80,000 Hours because of our outreach efforts likely won’t go on to make high impact career changes or otherwise contribute to tackling the world’s most pressing problems.

  • Our advice just isn’t that relevant or interesting to lots of people!

• We have various ways of measuring the inclinations of our new audience members, and almost all of them suggest that they are indeed significantly less inclined towards our advice, on average, than control groups of people that didn’t find us through our active outreach efforts.

• However, so long as we are still getting a good proportion of high-inclination traffic too (which it looks like we are), these efforts probably still look worth it.

• Our internal calculations of the value of this work take into account this expected decrease in inclination (and we think it looks good overall).

• All that said, the inclination of the new users we get from our outreach is a top priority, which we will continue to monitor.

Are you worried about any negative externalities from this work?

• Yes. I’m especially concerned about:

  • Accelerating the growth of the community beyond the optimal level

    • Lots (but not all) of the people who make career plan changes because of 80k do so via getting involved in the EA community at some point.

    • Many people are concerned that if the EA community “grows too fast,” we risk losing what’s important or distinctive about EA.

    • Or, we might not have enough suggestions for what the additional people might do, meaning the community growth isn’t useful and it’s demoralising for the new joiners.

      • I think these concerns are reasonable, and I think it’s important that we try and have systems to notice whether this is happening.

      • Currently, I think it’s likely that without additional efforts from initiatives like these, we would by default be far below the optimal growth level for the community — but that’s not obviously right, and I’m keen to learn more.

  • Annoying people by spamming them with ads, and putting them off EA ideas

    • We can use “frequency caps” on individual ad platforms, but we can’t necessarily control how many times somebody might see 80k mentioned across different platforms.

  • Idea inoculation

    • I think 80k’s offerings are really good, but they’re probably not the absolutely optimal way of presenting EA ideas for all types of people who might be interested in those ideas. Might we put off people from high-impact work, who we could have later convinced with better or different messaging?

    • Of course, we’ll never be sure we’ve arrived at the best presentation of our ideas, and there are costs to waiting around and not trying to get people on board, too.

    • Still, worth keeping an eye on & something I actively consider!

  • Entrenching demographic & ideological homogeneity in EA

    • The audiences that we’ve found most likely to sign up for 80,000 Hours’ newsletter have historically often also been quite similar demographically to the current EA community.^[7] This means that if we were to just optimise for number of signups, we might entrench existing demographic homogeneity in EA.

      • For example, many of our most successful outreach activities draw from heavily male audiences, such as STEM YouTube channels, and productivity/​entrepreneurship spaces.

      • We have weighed demographic diversity of a creator’s audience as a factor when deciding which creators to work with, and we have started out experimenting with more targeted approaches to reaching more diverse audiences, but so far our outreach efforts haven’t prioritised this concern above the number of new audience members we could attract.

    • This year I plan to investigate ways to increase demographic diversity among the people we reach, and indeed it’s possible that our work here could really do quite a lot to move the needle in a positive direction for the community as a whole.

Feel free to ask me any questions you have about this work in the comments. You can also email me at [e-mail], or leave a comment on my admonymous.

1. ^

   Of course, some kinds of outreach activities are harder to measure, such as “awareness marketing” which aims to get people to hear about 80k, without optimising for them taking any particular action.

2. ^

   Our internal metrics make us feel confident that the fact that our website traffic reached its highest point this quarter was due to our outreach efforts, rather than because of the heightened media coverage on EA at this time.

3. ^

   By this, I mean that, 18 months after the ads were run, 70 people who first found out about us via these social media ads answered ‘yes’ to the question ‘Have your career plans changed in any way as result of engaging with 80,000 Hours?’

4. ^

   e.g. at EAGs, in our annual user survey, and in some user interviews I recently conducted.

5. ^

   I haven’t given any detail about the kind of careers they’re in or hope to switch to because I don’t have permission from them to mention them publicly. 80k publicly posts some people’s stories with their permission, which might include some of these people in future.

6. ^

   I’m not sure these are actually ‘frequently asked’ but more ‘questions I anticipate you might ask,’ based on the above.

7. ^

   Historically, people who found out about EA via 80k skew slightly more male, but slightly less white, than average EAs.

[NunoSempere]

I like that you had the balls to scale 5-20x in a year, respect.

[Peter Wildeford]

I really appreciate how thoughtful you’ve been about this, including sensitivity to downside risks. Do you have any plans to monitor the downside risks? A lot of them seem quite verifiable/​testable.

[Bella]

Thanks Peter!

Yep; in the order I mentioned them in the post:

(Note: some of these are things I’ve done already, some are things I have vague plans to do at some point, and some are much firmer plans. You shouldn’t take this as a commitment from me to take any of these particular actions, but more a sense of the kind of stuff I’ve thought about here. I’d greatly appreciate suggestions you or others have about how I could do better!)

• Community growth rate —

  • Staying up-to-date on the views of others in community building;

  • Learning from data on the current growth rate of EA (much of which is gathered by Rethink, so thanks!);

  • User interviews.

• Annoying people —

  • This one seems the hardest to learn about because if you annoy someone and cause them to bounce off your stuff, they don’t want to talk to you and give you feedback on why! Aaargh.

  • I did do one survey (with the help of the Good Impressions team) to ask people if they got annoyed by some very high-frequency ads we ran, but I don’t trust the results too much (small, biased sample).

  • I ran a message testing survey last year which had a free text response for one question, so we found some evidence of things that annoyed people.

  • We also monitor discussion about 80k on social media (Twitter, Facebook, LinkedIn, Reddit) as well as the comments on any ads we run.

• Idea inoculation —

  • For this one I think I can mostly use our usual feedback mechanisms for 80k (experts/​advisors who read our drafts, our annual user survey, user interviews, some feedback forms on our site, message testing, feedback we get opportunistically via email or at EAGs, etc.)

• Demographic diversity —

  • We gather info on our audience’s demographics in our annual user survey, so that’s the most useful source here.

  • Also the EA Survey :D

[MHR]

I just want to second Peter’s comments—your discussion of downside risks was really thoughtful and cogent.

Of the risks you identified, I’m personally most concerned about 80k contributing to a negative feedback loop on demographic diversity (public perception of EA becomes that it is demographically homogenous → people who don’t match those demographic characteristics are rightfully more skeptical of EA → recruitment efforts target demographically skewed areas because engagement in them is higher → public perception of EA as demographically homogenous solidifies → …). I look forward to seeing what you find in your investigation of methods of improving 80k’s demographic reach!

[Madhav Malhotra]

I’m surprised to see how the book giveaway is more expensive than the costs of actually placing the ads to get eyes on the sites! Why did you decide to give away a physical book? What do you think the cost-effectiveness of that is compared to ebooks or not having a giveaway?

[Bella]

Nice, thanks for your question!

One relevant thing here is that I’m not thinking about the book giveaway as just (or even primarily) an incentive to get people to view our site — I think most of the value is probably in getting people in our target audience to read the books we give out, because I think the books contain a lot of important ideas. I think I’d be potentially excited about doing this without the connection to 80k at all (though the connection to 80k/​incentive to engage with us seems like an added bonus).

Re: physical books versus ebook:

• We do offer the ebook for the 80,000 Hours book (which is the most popular book of the three by a long way, taking up over half the orders).

• [A quick edit: also, sending ebooks other than the 80,000 Hours book isn’t free — it’d cost something like half to a quarter of the cost of sending a physical book]

• My guess is that people are just much more likely to read a physical book than an ebook — it’s cool and exciting to receive a thing in the post, rather than an infinitely replicable digital artefact. Just from personal experience, I’ve had a lot of organisations send me digital books I’ve never read, but I think it’d be much more ‘special’ if I got something in the mail.

  • This is also a bit more than a guess — I’ve looked at some (old, smaller scale) data from sites where you give out codes to redeem ebooks, and the vast majority of people never redeem them.^[1]

  • Whereas 89% of the people we surveyed who received the book from our book giveaway reported reading at least some of the book (and 38% reported reading the whole thing!)^[2]

Edited cos it occured to me that maybe I didn’t really answer your core question here, about cost-effectiveness —

I currently reckon the book giveaway is comparably cost-effective with other things we’ve done, but that’s based on some fairly rough estimates. I didn’t feel very satisfied with them, which is why we did the in-depth survey that I mentioned above. I haven’t finished analysing the results yet, as I said, so I can’t say anything more useful about it for now :)

1. ^

   I couldn’t quickly find this data to cite my sources so I can’t be more precise — so don’t put too much stock in my claim here!

2. ^

   You should of course adjust down for sampling bias and self-report bias. I haven’t finished analysing our data from this survey yet — it’s hot off the press — so I don’t have a view on how much to adjust down.

[DominikPeters]

Interesting that the 80k book is so popular. Is this the book that came out in late 2016, or has the content been updated? If not, it may be worth doing a second edition, since I assume that 80k’s thinking has evolved significantly since publication.

[Bella]

It’s the book that came out in late 2016.

Yeah, I agree a second edition could be really valuable, and this might be something we end up doing (It’s definitely high up on the list! There are some complications that make it not totally obvious that we’ll be able to do it soon, though).

[Madhav Malhotra]

I’d look forward to seeing you post the results of the in-depth survey on the forum :-)

[David M]

Pleased to hear this:

This year I plan to investigate ways to increase demographic diversity among the people we reach, and indeed it’s possible that our work here could really do quite a lot to move the needle in a positive direction for the community as a whole.

[geoffrey]

Do you have plans for increasing class diversity via 80k career advice /​ tailoring advice to those with less resources? If so, what are some strategies you have?

I’ve loved 80k career advice and have benefited a ton from it. But one frustration I’ve had (especially earlier in my career) is that it doesn’t offer much advice for people starting with less resources. For example, non-profit jobs can be out-of-reach without relevant /​ outstanding credentials or money to do a Masters degree if moving into policy.

I also suspect there’s working-class cultural factors some need to un-learn (this was true for me). Manual labor tends to reward putting your head down and doing the hard work. But professional + managerial jobs reward creativity, relationships, and questioning systems.

[Bella]

Thanks for your question & feedback about our advice!

Just to clarify, my responsibility is for outreach and promotion, and that’s all this post is intended to be about. The content of our advice/​website/​podcast is written & handled by others at 80k.

I passed along your question, and Arden from the content team asked me to share this:

We think a lot of our advice can be useful for people from a variety of backgrounds, but it’s true that most of our advice is geared toward people who have or are planning to get a college degree, because we think that’s our comparative advantage. There might also be cases where we’re unintentionally alienating people from working class backgrounds by not acknowledging the specific challenges they face — thanks for raising that.

If you have concrete ideas about how we could do better at these kinds of issues, we’d be really excited to hear it!

[quinn]

epistemic status: started reading the sequences while delivering food to people on a bike.

I never really felt excluded by problem profile pages. I felt like I could access the tone they take of “here’s some broken stuff why don’t you go fix it”, I just had to discount or not read any sections that talked about status, top universities, etc., kind of assumed I’d have to write my own theory of change and have a thick skin about not always being taken seriously. (and then what happened was the actually-existing EA movement contained tons of more sr people doing actual work saying “yeah here take a crack at it”, encouraging me to accept mentorship or jobs from them. If anyone’s ever assumed I’m low potential it’s been the more community builder types lol)

Age is a more likely vector of neglectedness for 80k than class. I’d be more excited about 80k targeting the frustration people older than 25 or 35 have with 80k, that seems more useful than thinking deeply about class. Related to both of these, we also have to talk about the risk intolerant: are people with obligations and responsibilities (family members with expensive diseases, childrearing projects) low EV? This again seems like a more potent surface for 80k criticism and improvement than class stuff. (But of course, both age and obligation load have some correlations with class!)

[geoffrey]

I never felt excluded either. And 80k does a lot of things right on this front. The messages of ambition and “here’s some broken stuff why don’t you go fix it” are good and certainly have pushed me to do things I wouldn’t have done otherwise. I genuinely feel people from underprivileged backgrounds need to hear more of it and I try to promote them as much as I can.

>I just had to discount or not read any sections that talked about status, top universities, etc., kind of assumed I’d have to write my own theory of change and have a thick skin about not always being taken seriously.

I think this is where our experiences diverge. I also discounted these sections… by charging straight ahead into applying for my dream jobs and thinking I had the similar chances to Ivy League graduates. But (1) college brand name does matter if comparing two people with no experience, (2) the Ivy League graduate probably had more opportunities, networks and internships than me, and (3) many of my dream jobs were magnitudes more competitive than I initially thought.

Maybe 80k advice worked too well and I flew too close to the sun? I don’t know. Messaging is tricky.

>I’d be more excited about 80k targeting the frustration people older than 25 or 35 have with 80k, that seems more useful than thinking deeply about class.

Agreed. To be clear, I’m not endorsing thinking deeply about class. The “business case” for class diversity doesn’t seem great to me. But I saw “demographic diversity” mentioned in the original post and I was wondering if class was included in that.

[geoffrey]

Noted! Sorry for the misinterpretation.

One concrete idea could be an article centered on “class migrants”. Perhaps it could be similar to the format of the anonymous interview series, or it could be like the imposter syndrome article where there’s one personal profile and a few mini-profiles attached.

Partly, this is to help people feel less alone. But also, I think the strategy for developing your career differs based on where you’re starting. Even between colleges, there’s variation.

Beyond that, I’m not sure. I get that 80k’s target audience is different from me and messaging is hard. So I’m hesitant to recommend huge strategic changes in content.

[Darren McKee]

Thanks for sharing. Do you happen to have the number of books you sent out (as you’ve already given the cost). Just wondering if my current estimate of around 30000 is close.

[Bella]

Edit: I’m sorry, I made a spreadsheet error in the place where I sourced the figure for my previous answer — the real answer is 46,000!

Sure thing! Yep your estimate is really good — it’s about 36,000 :)

[Aidan Alexander]

Hi Bella, thank you for writing this up! Are you willing to share more granular performance data for the different marketing efforts to help other orgs estimate the expected cost and performance of paid advertising?

[Bella]

Hey Aidan!

I’m not sure — I didn’t do this in this post/​didn’t have any plans to, mostly because I’m unsure how much our experiences would generalise to different contexts.

Performance of our ads within the same channel can vary by up to a couple orders of magnitude, so I’m just not sure how helpful it’d be for others.

That said, if you’re considering a specific project, I’d probably be happy for me or one of my team to chat to you about it based on our experience?

[Aidan Alexander]

Thanks for the prompt response! DM’d you

[Yonatan Cale]

I’d like to hear more about why you think this causes people to take high impact jobs (what you’re measuring, what you’re observing) (or more cliche, “what do you think you know and why do you think you know it?”).

I’m asking this in the spirit of “trust but verify”: I do assume you did a good job here, and at the same time this seems to me like the main place a project like this might break, so it seems healthy to ask.

For reference of others, here’s what was said in the post:

• We have various ways of measuring the inclinations of our new audience members, and almost all of them suggest that they are indeed significantly less inclined towards our advice, on average, than control groups of people that didn’t find us through our active outreach efforts.

• However, so long as we are still getting a good proportion of high-inclination traffic too (which it looks like we are), these efforts probably still look worth it.

• Our internal calculations of the value of this work take into account this expected decrease in inclination (and we think it looks good overall).

• All that said, the inclination of the new users we get from our outreach is a top priority, which we will continue to monitor.

Still, as I said elsewhere, seems really promising

[Bella]

Hey Yonatan —I think the more relevant part of my post is the following, which hopefully answers your question? Let me know if it doesn’t.

There are some details I can’t give because (as I said in the post) I don’t have permission from the relevant people to talk about it publicly.

We can’t be sure how many additional people will change to a high-impact career as a result, in large part because we have found that “career plan changes” of this kind take, on average, about 2 years from first hearing about 80k.

Still, our current best guess is that these efforts will have been pretty effective at helping people switch careers to more impactful areas.

Partly this guess is based on the growth in new audience members that we’ve seen (plus 80k’s solid track record of getting new people to eventually switch to more impactful careers), and partly it’s based on a few “proof of concept” switches we’ve seen already.

For example, some small-scale social media ads which 80k ran in 2017 as an experiment led to at least one person switching to a career we’re especially excited about (and 70 people who reported changing their career plans due to 80k).^[3] We’ve also already encountered^[4] several people who found us via our marketing who seem likely to switch to a more impactful career.^[5]

[Yadav]

Thanks for writing this! This is cool to see.

I may have missed this but do you have a sense of whether this marketing push is leading to more people working on pressing problems that 80k endorses? I am curious if there is a direct correlation between money spent on marketing and people working on these problems.

[Bella]

Glad you liked the post!

I talk a bit about this in this section – basically, we don’t know for sure how much this is causing people to work on pressing problems yet (because people changing their careers like this takes something like 2 years on average, some much longer — and we just haven’t been at it that long yet!)

That said, we do have some preliminary evidence (at the very least, some “proofs of concept”) that it’s working. We also did a small-scale experiment in 2017, where we ran some social media ads and found 70 people who told us they changed their career plans 18 months later.

[Yonatan Cale]

Any opinions about using this as a baseline to compare other EA outreach efforts?

I tend to be in favor, since this seems to be working, kind of tested, scalable, potentially high upside given some tweaks, and so on. I feel more reluctant to do my own outreach after reading this (but I’m saying this in a good way—as a compliment to you)

[Bella]

I’m not sure I understand what you mean by ‘using this as a baseline to compare other EA outreach efforts’? Is there some specific outcome metric you’d want to use as a baseline?

In general, I think what works best in outreach can be pretty context-specific, and I wouldn’t recommend everything I’ve done to people with different goals & constraints.

Being more reluctant to do your own outreach after learning about this makes sense if you think there’s some optimal growth rate in EA which we are at or nearly at. If you learn that I’m doing lots of outreach, then it decreases the value of additional outreach (unless we are not yet at or near the optimal rate of growth).

[Yonatan Cale]

As one example, I have a tiny project where I publicly measure the effectiveness of Israeli startups (in a Facebook group where people can comment and ask questions), and a big part of my goal is to explain the Israeli tech ecosystem how one even measures impact, and that not all med-tech startups are equal.

I might try to estimate how much it would “cost” me to get one person to a high impact job, and if that “cost” is way more than what you’re doing with the 80k outreach, I should maybe just dump the project. Or in other words, for my project to be worth while, it must be at least POTENTIALLY more cost effective than the 80k outreach. Similar to how if I’d want to run an intervention for global health and wellbeing, I’d want it to at least POTENTIALLY be better than AMF. (in both these examples, 80k and AMF are also tested and scalable, which are two advantages that new projects might not always have)

(Did this make more sense?)

[Bella]

This makes sense to me, but I don’t think I provided anything in this post which you could easily use to compare to your project here.

How would you go about guessing whether the cost of what you were doing was higher or lower than that of 80k’s outreach?

[JakubK]

The empirical reason is that being a newsletter subscriber correlates with having spent about twice the recorded time on our website, compared to non-subscribers.

I feel like the causality could be “people spend a lot of time on the website” --> “people subscribe to the newsletter” rather than the other way?

And how does 80k record people’s time spent on the website?

[Bella]

I totally agree the causality could go that way.

In fact, I think it’d be really weird if there wasn’t at least some effect in that direction, because people have to go on our website at some point in order to sign up to the newsletter!

I do think there is a significant effect in the direction ‘people subscribe to the newsletter → people spend time on the website,’ though.

Why? Mostly cos of the big spikes in engagement we see on pages that we link to from the newsletter — it just does look like people click on and then sometimes read stuff on our site, if we put it in the newsletter.

As for how we record people’s time spent on the website: we use the web analytics tools Google Analytics and Mixpanel. There are various caveats and systematic measurement errors that apply to both of them that we try to account for in our reporting on web engagement statistics.

[PeterSlattery]

I am excited to hear this. Thanks for your work.