Toby Tremlett🔹

• Toby Tremlett🔹Mar 21, 2025, 11:36 AM

  6 points

  2 ∶ 0

  in reply to: Jason’s comment on: 80,000 Hours is shift­ing its strate­gic ap­proach to fo­cus more on AGI

  I love point 3 “to the extent that I see something as core infrastructure that is a natural near-monopoly (e.g., the Forum, university groups) [...] I think there’s an enhanced obligation to share the commons”—that’s a good articulation of something I feel about Forum stewardship.

• Toby Tremlett🔹Mar 21, 2025, 9:35 AM

  4 points

  0 ∶ 0

  in reply to: jackva’s comment on: jackva’s Quick takes

  Thanks for engaging on this as well! I do feel the responsibility involved in setting event topics, and it’s great to get constructive criticism like this.

  To respond to the points a bit (and this is just my view- quite quickly written because I’ve got a busy day today and I’m happy to come back and clarify/​change my mind in another reply):
  
  (a) - maybe, but I think the actual content of the events almost always contains some scepticism of the question itself, discussion of adjacent debates etc… The actual topic of the event doesn’t seem like a useful place to look for evidence on the community’s priorities. Also, I generally run events about topics I think people aren’t prioritising. However, I think this is the point I disagree with the least—I can see that if you are looking at the forum in a pretty low-res way, or hearing about the event from a friend, you might get an impression that ‘EA cares about X now’.
  
  (b) - The Forum does appear in EA-critical pieces, but I personally don’t think those pieces distinguish much between what one post on the Forum says and what the Forum team puts in a banner (and I don’t think readers who lack context would distinguish between those things either). So, I don’t worry too much about what I’m saying in the eyes of a very adversarial journalist (there are enough words on the forum that they can probably find whatever they’d like to find anyway).

  To clarify—for readers and adversarial journalists—I still have the rule of “I don’t post anything I wouldn’t want to see my name attached to in public” (and think others should too), but that’s a more general rule, not just for the Forum.

  (c)- I’m sure that it isn’t the optimum Forum week. However (1) I do think this topic is important and potentially action-relevant—there is increasing focus on ‘AI Safety’, but AI Safety is a possibly vast field with a range of challenges that a career or funding could address, and the topic of this debate is potentially an important distinction to have a take on when you are making those decisions. And (2) I’m pretty bullish on forum events, and I’d like to run more, and get the community involved more, so any suggestions for future events are always welcome.

• Toby Tremlett🔹Mar 20, 2025, 1:51 PM

  7 points

  0 ∶ 0

  in reply to: Maxime Riché 🔸’s comment on: Dis­cus­sion Thread: Ex­is­ten­tial Choices De­bate Week

  I think yes and for all the reasons. I’m a bit sceptical that we can change the values ASIs will have—we don’t understand present models that well, and there are good reasons not to treat how a model outputs text as representative of its goals (it could be hallucinating, it could be deceptive, it’s outputs might just not be isomorphic to a reward structure).
  
  And even if we could, I don’t know of any non-controversial value to instill in the ASI, that isn’t just included in basic attempts to control the ASI (which I’d be doing mostly for extinction related reasons).

• Toby Tremlett🔹Mar 19, 2025, 9:46 AM

  2 points

  0 ∶ 0

  on: The Value of a Statis­ti­cal Life is not a good metric

  I’ve had drafts of this take lying around for years—really glad to see it out in the open!
  I’d love to hear pushback from anyone who thinks it is still valuable.

• Toby Tremlett🔹Mar 18, 2025, 10:45 AM

  8 points

  1 ∶ 0

  in reply to: jackva’s comment on: jackva’s Quick takes

  I think this is a fair point—but it’s not the frame I’ve been using to consider debate week topics.

  My aim has been to generate useful discussion within the effective altruism community. I’d like to choose topics which nudge people to examine assumptions they’ve been making, and might lead to them changing their minds, and perhaps their priorities, or the focus of their work. I haven’t been thinking about debate weeks as a piece of communications work/​ as a way of reaching out to a broader audience. This question in particular was chosen because the Forum audience wouldn’t necessarily have cached takes on it—an audience outside the Forum would need a lot of context to get what we are talking about.
  
  Perhaps I’m missing something though—do you think this is more public facing than I’m assuming? To be clear, I know that it is public, but it’s not directed at an outside audience in the way a book or podcast or op-ed might be.

  Edit: I’m also uncertain on the claim that “there are few interventions that are predictably differentiated along those lines”—I think Forethought would disagree, and though I’m not sure I agree with them, they’ve thought about it more than I have.

• Toby Tremlett🔹Mar 18, 2025, 10:17 AM

  11 points

  1 ∶ 0

  in reply to: Christopher Clay’s comment on: Ex­is­ten­tial Choices Sym­po­sium with Will MacAskill and other spe­cial guests (3-5pm GMT Mon­day)

  Yeah I’ve always been a bit sceptical of this as well. Surely it’s just a yardstick that a department uses to decide between which investments it should make, rather than a considered (or even descriptive) “value of a life” for the US Government.
  Descriptively—the US government would spend far more for a few lives if those lives were hostages of a foreign adversary, and probably has far less willingness to pay for cheap ways the US govt could save lives (idk what these are, probably there are examples in public health).
  Basically—I don’t think it’s a number that can be meaningfully extrapolated to figure out the value of avoiding extinction or catastrophe, because the number was designed with far smaller trade-offs in mind, and doesn’t really make sense outside of its intended purpose.

• Toby Tremlett🔹Mar 18, 2025, 9:48 AM

  2 points

  1 ∶ 0

  in reply to: WilliamKiely’s comment on: Dis­cus­sion Thread: Ex­is­ten­tial Choices De­bate Week

  Thanks—yep I think this is becoming a bit of an issue (it came up a couple times in the symposium as well). I might edit the footnote to clarify—worlds with morally valuable digital minds should be included as a non-extinction scenario, but worlds where an AI which could be called “intelligent life” but isn’t conscious/​ morally valuable takes over and humans become extinct should count as an extinction scenario.

• Toby Tremlett🔹Mar 17, 2025, 4:04 PM

  5 points

  1 ∶ 0

  in reply to: finm’s comment on: Ex­is­ten­tial Choices Sym­po­sium with Will MacAskill and other spe­cial guests (3-5pm GMT Mon­day)

  I think the “earth-originating intelligent life” term should probably include something that indicates sentience/​ moral value. Perhaps you could read that into “intelligent” but that feels like a stretch. But I didn’t want to imply that a world with no humans but many non-conscious AI systems would count as anything but an extinction scenario—that’s one of the key extinction scenarios.

• Toby Tremlett🔹Mar 17, 2025, 3:58 PM

  4 points

  0 ∶ 0

  in reply to: Toby Tremlett🔹’s comment on: Ex­is­ten­tial Choices Sym­po­sium with Will MacAskill and other spe­cial guests (3-5pm GMT Mon­day)

  Maybe another way to think about this (dropping the religion stuff—don’t want to cast aspersions on any particular religions) is that we could think of black-ball and white-ball ideologies (like the Bostrom thought experiment where black-balls = technologies which can cause extinction). Perhaps certain ideologies are just much more exclusive and expansion focused than others—black-balls. You can pick out as many white-balls as you like, but picking out a black-ball means you have to get rid of your white-balls. Even if there are few black-balls in the bag, you’d always end up holding one.

• Toby Tremlett🔹Mar 17, 2025, 3:49 PM

  2 points

  0 ∶ 0

  in reply to: Robi Rahman’s comment on: Dis­cus­sion Thread: Ex­is­ten­tial Choices De­bate Week

  No worries!

• Toby Tremlett🔹Mar 17, 2025, 3:37 PM

  4 points

  0 ∶ 0

  in reply to: Robi Rahman’s comment on: Dis­cus­sion Thread: Ex­is­ten­tial Choices De­bate Week

  I think Owen is voting correctly Robi—he disagrees that there should be more work on extinction reduction before there is more work on improving the value of the future. (to complicate this, he is understanding working on AI x-risk is mostly about increasing the value of the future, because, in his view, it isn’t likely to lead to extinction).
  
  Apologies if the “agree” “disagree” labelling is unclear—we’re thinking of ways to make it more parsable.

• Toby Tremlett🔹Mar 17, 2025, 3:20 PM

  4 points

  0 ∶ 0

  in reply to: William_MacAskill’s comment on: Ex­is­ten­tial Choices Sym­po­sium with Will MacAskill and other spe­cial guests (3-5pm GMT Mon­day)

  I’m wondering whether we should expect worlds which converge on moral views to converge on bad moral views.

  From the space of world religions—we’ve seen a trend where we converge over time (at least from a high level of abstraction where we can refer to “Christianity” and “Islam” rather than “mega-church Christians” or whatever). Is this because the religions that succeed are exclusive and expansionary? Of all religions that have existed, I know that many of them don’t much care if you also worship other gods. My empirical (ish) question is whether we should expect world in which a sizable fraction of the population follows the same religion to be one where the religion they follow is exclusive (you can’t follow others) and expansionary (other people should also follow this religion). PS- I know that not all Christians or Muslims are exclusionary about other religions, this is over-simplified.
  
  This is relevant because, if this is a mechanism, we might expect the same thing of morality or political organisation—beliefs which demand you don’t follow others, and that others follow the same beliefs as you, rather than tolerant beliefs. Perhaps this would make it more likely that futures which converge have converged on something extreme and closed, rather than exploratory and open.
  
  This is pretty vague—just wondering if others a) know more than me about the religion question and can speak to that or b) have had similar thoughts, or c) think that the existence of exclusive and expansionary (and wrong) ideologies might make convergence more likely.

• Toby Tremlett🔹Mar 17, 2025, 1:52 PM

  4 points

  0 ∶ 0

  on: to­bytrem’s Quick takes

  Heads up that the symposium event will be happening in the comments of this post, starting at 3pm (UK time).
  
  Now’s a good time to leave questions or key considerations in the comments of the post, for Will MacAskill, Tyler John, Andreas Mogensen, Greg Colbourn and Michael St Jules to respond to.

• Toby Tremlett🔹Mar 17, 2025, 9:56 AM

  5 points

  2 ∶ 0

  on: Dis­cus­sion Thread: Ex­is­ten­tial Choices De­bate Week

  Two major reasons/​ considerations:
  1- I’m unconvinced of the tractability of non-extinction-risk reducing longtermist interventions.
  2- Perhaps this is self-defeating—but I feel uncomfortable substantively shaping the future in ways that aren’t merely making sure it exists. Visions of the future that I would have found un-objectionable a century ago would probably seem bad to me today. In short—this consideration is basically “moral uncertainty”. I think extinction-risk reduction is, though not recommended on every moral framework, at least recommended on most. I haven’t seen other ideas for shaping the future which are as widely recommended.

• Toby Tremlett🔹Mar 17, 2025, 9:20 AM

  3 points

  0 ∶ 0

  in reply to: M Bennett’s comment on: Open thread: Jan­uary—March 2025

  Hey Mira! Sorry for such a late reply!
  Have you heard of the EA intro programme? The syllabus covers the areas that you mention.
  If you don’t have time, or would rather not do a zoom-based course, you can also read the syllabus here.

• Toby Tremlett🔹Mar 17, 2025, 9:15 AM

  2 points

  0 ∶ 0

  in reply to: Bella’s comment on: Dis­cus­sion Thread: Ex­is­ten­tial Choices De­bate Week

  That’s odd—to me I’m still just seeing your first vote. I’ll send this to Will to check out.

• Toby Tremlett🔹Mar 14, 2025, 6:01 PM

  4 points

  0 ∶ 0

  in reply to: lilly’s comment on: Ex­is­ten­tial Choices Sym­po­sium with Will MacAskill and other spe­cial guests (3-5pm GMT Mon­day)

  Yep it’ll all be in the comments, so if you aren’t around you can read it later (and I’m sure a bunch of the conversations will continue, just potentially without the guests)
  this was a good flag btw—I’ve changed the first sentence to be clearer!

Dis­cus­sion Thread: Ex­is­ten­tial Choices De­bate Week

• Toby Tremlett🔹Mar 14, 2025, 2:36 PM

  2 points

  0 ∶ 0

  in reply to: Toby Tremlett🔹’s comment on: An­nounc­ing: Ex­is­ten­tial Choices De­bate Week (March 17-24)

  I think I’ll stick with this current statement—partly because it’s now been announced for a while so people may be relying on its specific implications for their essays, but also because this new formulation (to me) doesn’t seem to avoid the problem you raise, that it isn’t clear what your vote would be if you think the same type of work is recommended for both. Perhaps the solution to that issue is in footnote 3 on the current banner—if you think that the value from working on AI takeover is mostly from avoiding extinction, then you should vote agree. If you think it is from increasing the value of the future by another means (such as more democratic control of the future by humans), then you should vote disagree.

Ex­is­ten­tial Choices Sym­po­sium with Will MacAskill and other spe­cial guests (3-5pm GMT Mon­day)