Toby Tremlett🔹

• Toby Tremlett🔹May 22, 2025, 11:10 AM

  12 points

  0 ∶ 0

  on: State of the EA Newsletter

  Here is the very sketchy first salvo at measuring “Impact-weighted clicks” (not sure how much I endorse this anymore, but feel free to treat me as if I do endorse it in responses to the comment—that’ll help me figure out what I think).
  
  Was going to just share a doc with @Vasco Grilo🔸 but I figured there is no reason not to make it public. Very happy to get input on this as well, especially if you have experience trying to measure the value of content (outside of just “clicks on jobs”)

  Impact-weighted clicks

  Not all EA Newsletter clicks are created equal. The EA Newsletter contains links to impactful jobs, but also interesting podcasts, videos, and articles — where the only further call to action is subscribing. This content isn’t of no value, but it’s very unlikely to be the source of most of the newsletter’s value.

  The metric I’m making will give categories of clicks a value from 0-1, and then produce a total number of “impact-weighted” clicks, where the total amount of unique clicks each category is multiplied by the value assigned to that category.

  What is the value of content clicks?

  Firstly — retention. Without content (hypothesis 1), people wouldn’t be interested in the EA Newsletter.

  Secondly — content pipelines. If someone starts reading an EA friendly outlet, subscribes to the 80k podcast, buys a book, subscribes to rational animations, etc… They increase their chance of coming across EA content in the future.

  Third — Changing minds. People who are aware of big problems might decide to work on them.

  How should we categorise EANL clicks?

  Category	% of clicks for the march newsletter
  Content	69%
  Job ad (and job boards)	23%
  Fellowship/​ funding	7%
  Courses	1%
  Donation opportunities	0% (none included)

  What value should we give to each type of click?

  An obvious ranking is:

  1. Job ads (this person is already engaged and ready to act)

  2. Fellowship/​ funding (likewise ready to act, but funding is short term and fellowship is the step before the impactful job)

  3. Courses (intent to get more involved, learn more, but step before deeper engagement)

  4. Donation (signals strong interest, and likely to sign them up to marketing from an effective charity)

  5. Content (value described above, pretty loosely valuable)

  I would stand by the obvious ranking (though maybe more quibling should separate “fellowship/​ funding”). However, we also need to provide values for each step. In the ideal world, we could base this on the drop off rate between each step in our funnel (perhaps X% of people who do a fellowship don’t end up getting a relevant job, therefore fellowship clicks should be ranked as the value of job ads—X). We should come back to this, but for now, I’ll do some crude estimates.

  • Job ad clicks = 1

  • Let’s say 50% of fellowship graduates , or funding recipients get a relevant job. Fellowship = 0.5

  • 25% of courses graduates do something relevant afterwards = 0.25

  • Donation = 0.05

  • Content = 0

  Category	Value per click
  Content	0
  Job ad (and job boards)	1
  Fellowship/​ funding	0.5
  Courses	0.25
  Donation opportunities	0.05

  How could the metric be better?

  • If we knew more about our audience, we could multiply the impact-weighted total clicks by the “value” of our current email-opening audience (i.e. each click is more valuable in expectation if the audience is highly motivated, educated, etc…). Hopefully if we get a marketer involved we could learn more about this.

  Hypotheses to test:

  H1: “Without content, people wouldn’t be interested in reading the EA Newsletter”. I.e. without content, overall clicks go down.

  • This might be wrong—the content doesn’t seem to have led to more subscribers.

  • And yet—people come back to click every month.

• Toby Tremlett🔹May 22, 2025, 10:33 AM

  4 points

  0 ∶ 0

  in reply to: Toby Tremlett🔹’s comment on: State of the EA Newsletter

  Also also—this isn’t included in the post but 70% of clicks on the newsletter go towards content links, i.e. articles, podcasts, videos etc… which are great resources on something EAs care about, but don’t have clear CTAs. I’m not sure how to measure the value of this yet—but in surveys, many of the responses show people changing their minds/​ plans because of content they read in the newsletter. Seems likely this is a very important part of the newsletter’s impact, but currently it counts for 0 in my impact-weighted-clicks metric.

• Toby Tremlett🔹May 22, 2025, 10:27 AM

  10 points

  0 ∶ 0

  in reply to: Vasco Grilo🔸’s comment on: State of the EA Newsletter

  “I would ask current subscribers how much they would be willing to pay in order to maintain their subscription.”
  
  That’s interesting! I could definitely give that a go. I think that might be a bit off though because the more relevant thing is how much money I have access to in our budget/​ could access by making the right case to OP (and what it would be spent on otherwise). I’ve also never really understood how individual willingness to pay is a good signal when people have wildly different resources available to them. Could be a useful benchmark.

  Another point on that—I don’t trust myself when it comes to valuing things monetarily. I’d never pay for social media or a newsletter, because it just isn’t something I’m used to (I’m used to them being free), but if I actually calculate the time I’m spending on social media/​ newsletters I count as a valuable use of time, and use something like this to calculate the value of my time, I get surprisingly high figure.

  Fair point on click-rates!

  “How does your 0.167 impact-weighted-click/​$ (= ¹⁄₆) compare with the clicks per $ regarding 80 k’s, Animal Advocacy Career’s, and Probably Good’s job boards?”

  This is a good question—I should prioritise comparing these. I do expect that some of those programs (and possibly the EA opportunities board) are cheaper per click, and have a higher intent audience. The EA Newsletter hasn’t been optimised for job clicks (and probably shouldn’t be). I’ve been a bit bottlenecked by what I can most easily measure.

• Toby Tremlett🔹May 22, 2025, 10:17 AM

  3 points

  0 ∶ 0

  in reply to: Toby Tremlett🔹’s comment on: Re­quest for Feed­back: Scal­ing TaRL through Stu­dent-Teacher Place­ments in Nigeria

  Awesome—thanks!

• Toby Tremlett🔹May 22, 2025, 8:05 AM

  4 points

  3 ∶ 0

  on: Re­quest for Feed­back: Scal­ing TaRL through Stu­dent-Teacher Place­ments in Nigeria

  Hey! I think you’d get more of the right people engaging with this if you shared a bit more detail (people are very unlikely to click on a link without knowing what to expect).
  I.e. - share that you are in Nigeria, you’re talking about TaRL, include the 70% of kids fact (that was great), and then outline your proposed intervention, and how the reader can help.
  The title is also a little too vague- maybe title based on a more specific ask?

• Toby Tremlett🔹May 21, 2025, 12:25 PM

  2 points

  0 ∶ 0

  on: In­tro­duc­ing EA in a non-EA non­profit

  @Gemma 🔸 I think you mentioned doing something like this? Or was it for the GWWC pledge in particular?

State of the EA Newsletter

• Toby Tremlett🔹May 20, 2025, 10:43 AM

  2 points

  0 ∶ 0

  in reply to: AhmedRashid’s comment on: Open thread: April—June 2025

  Great to have you here Ahmed!
  I’d recommend Probably Good’s career advising, if you’ve got the time.

• Toby Tremlett🔹May 19, 2025, 12:05 PM

  4 points

  0 ∶ 0

  in reply to: Toby Tremlett🔹’s comment on: Pod­cast epi­sode with Michael St. Jules

  Just listened—really enjoyed it!
  I recommend for:

  • Elijah’s enthusiasm

  • Michael’s clear reasoning

  • A whistlestop overview of the development of Michael’s ethics (especially, scepticism of kneejerk utilitarian ways of measuring and comparing wellbeing).

  Also, seconding Elijah in the intro (though EAs doubtless need no excuse) speeding the podcast up to 1.5x was great.

EA Or­ga­ni­za­tion Up­dates: May 2025

• Toby Tremlett🔹May 9, 2025, 2:02 PM

  2 points

  1 ∶ 0

  in reply to: Clara Torres Latorre 🔶️’s comment on: Open thread: April—June 2025

  Yep quick takes are the best spot for more speculative, early stage, or short thoughts :)

• Toby Tremlett🔹May 9, 2025, 8:36 AM

  6 points

  2 ∶ 0

  on: Pod­cast epi­sode with Michael St. Jules

  Very cool, will listen. Have you considered posting a transcript? It’s super easy with AI now (use one AI to make a transcript, and another one to clean it up and make it more grammatical).

• Toby Tremlett🔹May 9, 2025, 8:25 AM

  4 points

  0 ∶ 0

  in reply to: Clara Torres Latorre 🔶️’s comment on: Open thread: April—June 2025

  Hey Clara, congrats on taking the pledge!
  My guess on this question is that you’d need a lot more people to give before you saw macro-economic effects. But people’s giving does differ greatly country to country, so I’m sure there is some economic work on the effects of that on savings, investment, wages etc!
  Also- welcome to the Forum! Let me know if you have any questions about how it works.

• Toby Tremlett🔹May 8, 2025, 4:08 PM

  4 points

  0 ∶ 0

  on: to­bytrem’s Quick takes

  This is just a quick note to let you know that polls-in-posts weren’t just for DIY debate week. You can use them at any time.

  More about the feature, and how to use it, in this post.

• Toby Tremlett🔹May 1, 2025, 2:42 PM

  2 points

  0 ∶ 0

  in reply to: Jason’s comment on: Crit­i­cism on the EA Forum

  Fantastic! I’ve linked it at the bottom of the post.

• Toby Tremlett🔹Apr 30, 2025, 12:17 PM

  6 points

  1 ∶ 0

  on: De­bate: should EA avoid us­ing AI art out­side of re­search?

  Should EA avoid using AI art for non-research purposes?

  Treating “agree” as “yes”

  I think the strongest reason against (using AI art for non-research purposes) for me is the idea of all the uncompensated art that made up the data. It’s a bit of an original sin for AI in general (including text generation), and not one which we’ve found a good response to.
  
  Reasons for (using AI art for non-research purposes):
  - It makes sense for EAs to adopt a place in the memetic space where we think that AI is and will be very powerful (and therefore it’s important to learn how to use it), and it’s likely to be very dangerous. I don’t think there is a contradiction there, and avoiding the use of AI would be increasingly hobbling. This is relevant because I don’t think we can make a clean distinction between AI generated art and AI generated text—both are likely built on an amount of stolen/​ un-compensated data.
  - Using AI images (like bulby above) is just a bit of fun, i.e. the scale of use is pretty small—this correlates with this not being a very big deal.

  - A lot of the concerns are hypothetical comms concerns, I’d take this more seriously if things played out that way, but right now I’d guess that the anti-AI-use camp is fairly loud but not strategically useful. And since I disagree with them for the other reasons above, I’d rather not pretend that I do for optics reasons.

  Overall: I’m definitely open to changing my mind on this. I especially don’t feel like I have a principled response to the un-compensated labour that went into creating AI, and it’d be great to have one.

• Toby Tremlett🔹 29 Apr 2025 15:23 UTC

  2 points

  0 ∶ 0

  in reply to: Will Aldred’s comment on: Three polls: on timelines and cause prio

  This is helpful. If this was actually for a debate week, I’d have made it ‘more than 5% extinction risk this century’ and (maybe) excluded risks from AI.

• Toby Tremlett🔹 29 Apr 2025 13:30 UTC

  4 points

  0 ∶ 0

  in reply to: Will Aldred’s comment on: Three polls: on timelines and cause prio

  I’m a pretty strong anti-realist but this is one of the strongest types of shoulds for me.
  I.e. ‘If you want to achieve the best consequences, then you should expect the majority of affectable consequences to be in the far future’ Seems like the kind of thing that could be true or false on non-normative grounds, and would normatively ground a ‘should’ if you are already committed to consequentialism. In the sense that believing “I should get to Rome as fast as possible” and “The fastest way to get to Rome is to take a flight” grounds a ‘should’ for “I should take a flight to Rome”.

• Toby Tremlett🔹 29 Apr 2025 13:27 UTC

  2 points

  0 ∶ 0

  in reply to: MichaelDickens’s comment on: Three polls: on timelines and cause prio

  However people interpret the question is how we should discuss it, but when I was writing it, I was wondering about whether bioweapons can cause extinction/​ existential risks or not per se. I.e. can bioweapons either:
  a) kill everyone

  b) Kill enough of the population, forever, such that we can never achieve much as a species.
  I’m not sure about the feasibility of either.

• Toby Tremlett🔹 29 Apr 2025 13:23 UTC

  4 points

  0 ∶ 0

  in reply to: TFD’s comment on: Crit­i­cism on the EA Forum

  Thanks for this comment! I think you’ve pointed out a few places where this post clearly isn’t comprehensive. I’m not sure how frequently asked these questions will be, but in case they are, some quickfire answers:

  If someone makes a post that criticizes someone on the forum but does not reach out to the target of their criticism first, would you consider that to be violating a norm of the forum, even if that violation won’t result in any enforcement?

  No—I mistakenly used the word norms in an ambiguous sentence in the second section. I’ve changed the word to practices. Reaching out to a critiqued organisation or person, or giving right of reply are ‘practices we’d like to encourage’ rather than new norms. In practice this means that we (the mods) will advise people to follow these practices in many cases, and in many cases, will help reduce friction (by doing the reaching out on the critics behalf for example).

  What is in scope for “criticism” in this context?

  This is a good question. I could cop out with a “I know it when I see it” which is partially true. But broadly I think the type of criticism we are more concerned about/​ would more strongly encourage to follow these practices is criticism which could damage the reputation of an organisation or individual if was read without a response. General disagreement/​ critical engagement with the ideas of an organisation could technically fall into this category, but is generally read as more collaborative than as an accusation of wrongdoing. Tone probably matters a bit here. Others on the mod team may have different views on this question.

  I think its not uncommon that critics and their targets have major disagreements about whether these types of beliefs are reasonable. When can one invoke this type of reasoning for not reaching out?

  When it’s reasonable to do so. I think the Forum is naturally quite sceptical and won’t let bad faith arguments stand for long, so in many cases, I don’t think it will matter if a bad faith response is published alongside a critique. But it’s a little hard to form a principle here (hence practices, not norms).