Benjamin_Todd

It looks like there are some good fund­ing op­por­tu­ni­ties in AI safety right now

• Benjamin_Todd 10 Dec 2024 4:21 UTC

  6 points

  0 ∶ 0

  on: AI stocks could crash. And that could have im­pli­ca­tions for AI safety

  Spitballing: EA entrepreneurs should be preparing for one of two worlds:

  i) Short timelines with 10x funding of today

  ii) Longer timelines with relatively scarce funding

• Benjamin_Todd 5 Sep 2024 11:46 UTC

  2 points

  0 ∶ 0

  in reply to: Ben_West🔸’s comment on: Are there dis­ec­onomies of scale in the rep­u­ta­tion of com­mu­ni­ties?

  The response to Michael is an interesting point, but it only concerns diminishing returns in individual capabilities of new members.

  Diminishing returns are mainly driven by the quality of opportunities being used up, rather than the capabilities.

  IIRC a 10x in resources to get a 3x in impact was a typical response in the old coordination forum survey responses.

  In the past at 80k I’d often assume a 3x increase in inputs (e.g. advising calls) to get a 2x increase in outputs (impact-adjusted plan changes), and that seemed to be roughly consistent with the data (though the data don’t tell us that much). In some cases, returns seem to diminish a lot faster than that. And you often face diminishing returns at several levels (e.g. 3x as much marketing to get 2x as many applicants to advising).

  I agree returns are more linear in areas where EA resources are a small fraction of the total, like global health, but that’s not the case in areas like AI safety, GCBRs, new causes like digital sentience, or promoting EA.

  And even in global health, if GiveWell only had $100m to allocate, average cost-effectiveness would be a lot higher (maybe 3-10x higher?) than where the marginal dollar goes today. If GiveWell had to allocate $10bn, I’d guess returns would be at least several fold lower again on the marginal spending.

• Benjamin_Todd 24 Aug 2024 8:34 UTC

  4 points

  0 ∶ 0

  in reply to: Benjamin_Todd’s comment on: Are there dis­ec­onomies of scale in the rep­u­ta­tion of com­mu­ni­ties?

  Aside: A more compelling argument against growth in this area to me is something like “EA should focus on improving its brand and comms skills, and on making reforms & changing its messaging to significantly reduce the chance of something like FTX happening again, before trying to grow aggressively again”; rather than “the possibility of scandals means it should never grow”.

  Another one is “it’s even more high priority to grow X others movements than EA” rather than “EA is net negative to grow”.

• Benjamin_Todd 24 Aug 2024 8:27 UTC

  2 points

  0 ∶ 0

  in reply to: Benjamin_Todd’s comment on: Are there dis­ec­onomies of scale in the rep­u­ta­tion of com­mu­ni­ties?

  Less importantly, I also feel less confident coordination benefits would mean impact per member goes up with the number of members.

  I understand that the value of a social network like Facebook grows with the number of members. But many forms of coordination become much harder with the number of members.

  As an analogy, it’s significantly easier for 2 people to decide where to go to dinner than for 3 people to decide. And 10 people in a group discussion can take ages to come to consensus.

  Or, it’s much harder to get a new policy adopted in an organisation of 100 than an organisation of 10, because there are more stakeholders to consult and compromise with, and then more people to train in the new policy etc. And large organisations are generally way more bureaucratic than smaller ones.

  I think these analogies might be closer than the analogy of Facebook.

  You also get effects like in a movement of under 1000, it’s possible to have met in person most of the people, and know many of them well; while in a movement of 10,000, coordination has to be based on institutional mechanisms, which tend to involve a lot of overhead and not be as good.

  Overall it seems to me that movement growth means more resources and skills, more shared knowledge, infrastructure and brand effects, but also many ways that it becomes harder to work together, and the movement becoming less nimble. I feel unsure which effect wins, but I put a fair bit of credence on the term decreasing rather than increasing.

  If it were decreasing, and you also add in diminishing returns, then impact per member could be going down quite fast.

• Benjamin_Todd 24 Aug 2024 8:14 UTC

  2 points

  0 ∶ 0

  on: Are there dis­ec­onomies of scale in the rep­u­ta­tion of com­mu­ni­ties?

  Thanks for the analysis! I think it makes sense to me, but I’m wondering if you’ve missed an important parameter: diminishing returns to resources.

  If there are 100 community members they can take the 100 most impactful opportunities (e.g. writing DGB, publicising that AI safety is even a thing), while if there are 1000 people, they will need to expand into opportunities 101-1000, which will probably be lower impact than the first 100 (e.g. becoming the 50th person working on AI safety).

  I’d guess a 10x increase to labour or funding working on EA things (even setting aside coordination and reputation issues) only increases impact by ~3x.

  It seems like that might make significant difference to the model—if I’ve understood, currently the impact of marginal members in the model is actually increasing due coordination benefits, whereas this could mean it’s decreasing.

  I’d still guess marginal growth is net positive, but I feel less confident than the post suggests.

• Benjamin_Todd 21 Aug 2024 17:31 UTC

  3 points

  0 ∶ 0

  in reply to: JWS 🔸’s comment on: JWS’s Shortform

  Thank you, I appreciate that.

• Benjamin_Todd 21 Aug 2024 12:17 UTC

  28 points

  1 ∶ 0

  in reply to: JWS 🔸’s comment on: JWS’s Shortform

  Hey JWS,
  
  These comments were off-hand and unconstructive, have been interpreted in ways I didn’t intend, and twitter isn’t the best venue for them, so I apologise for posting, and I’m going to delete them. My more considered takes are here. Hopefully I can write more in the future.

• Benjamin_Todd 13 Aug 2024 20:31 UTC

  2 points

  0 ∶ 0

  in reply to: Owen Cotton-Barratt’s comment on: On the Value of Ad­vanc­ing Progress

  Thank you!

• Benjamin_Todd 11 Aug 2024 9:05 UTC

  10 points

  1 ∶ 0

  on: On the Value of Ad­vanc­ing Progress

  I’d be interested to hear a short explanation of why this seems like a different result from Leopold’s paper, especially the idea that it could be better to accelerate through the time of perils.

• Benjamin_Todd 22 May 2024 20:11 UTC

  5 points

  0 ∶ 0

  in reply to: Geoffrey Miller’s comment on: AI stocks could crash. And that could have im­pli­ca­tions for AI safety

  I should maybe have been more cautious—how messaging will pan out is really unpredictable.

  However, the basic idea is that if you’re saying “X might be a big risk!” and then X turns out to be a damp squib, it looks like you cried wolf.

  If there’s a big AI crash, I expect there will be a lot of people rubbing their hands saying “wow those doomers were so wrong about AI being a big deal! so silly to worry about that!”

  That said, I agree if your messaging is just “let’s end AI!”, then there’s some circumstances under which you could look better after a crash e.g. especially if it looks like your efforts contributed to it, or it failed due to reasons you predicted /​ the things you were protesting about (e.g. accidents happening, causing it to get shut down).

  However, if the AI crash is for unrelated reasons (e.g. the scaling laws stop working, it takes longer to commercialise than people hope), then I think the Pause AI people could also look silly – why did we bother slowing down the mundane utility we could get from LLMs if there’s no big risk?

• Benjamin_Todd 20 May 2024 8:43 UTC

  4 points

  0 ∶ 0

  in reply to: Ozzie Gooen’s comment on: Pro­ject idea: AI for epistemics

  Thanks that’s helpful!

Pro­ject idea: AI for epistemics

• Benjamin_Todd 17 May 2024 8:57 UTC

  2 points

  0 ∶ 0

  in reply to: Nathan Young’s comment on: Nu­clear se­cu­rity seems like an in­ter­est­ing fund­ing gap

  https://​​www.macfound.org/​​media/​​article_pdfs/​​nuclear-challenges-synthesis-report_public-final-1.29.21.pdf[16].pdf

• Benjamin_Todd 17 May 2024 8:53 UTC

  8 points

  1 ∶ 0

  in reply to: tlevin’s comment on: Nu­clear se­cu­rity seems like an in­ter­est­ing fund­ing gap

  I agree people often overlook that (and also future resources).

  I think bio and climate change also have large cumulative resources.

  But I see this as a significant reason in favour of AI safety, which has become less neglected on an annual basis recently, but is a very new field compared to the others.

  Also a reason in favour of the post-TAI causes like digital sentience.

• Benjamin_Todd 15 May 2024 9:05 UTC

  3 points

  1 ∶ 0

  in reply to: Benjamin_Todd’s comment on: Nu­clear se­cu­rity seems like an in­ter­est­ing fund­ing gap

  Or you might like to look into Christian’s grantmaking at Founders Pledge: https://​​80000hours.org/​​after-hours-podcast/​​episodes/​​christian-ruhl-nuclear-catastrophic-risks-philanthropy/​​

• Benjamin_Todd 15 May 2024 8:14 UTC

  5 points

  1 ∶ 0

  in reply to: Sanjay’s comment on: Nu­clear se­cu­rity seems like an in­ter­est­ing fund­ing gap

  Thanks that’s helpful background!

  I agree tractability of the space is the main counterargument, and MacArthur might have had good reasons to leave. Like I say in the post, I’d suggest people think about this issue carefully if you’re interested in giving to this area.

• Benjamin_Todd 15 May 2024 8:12 UTC

  9 points

  4 ∶ 1

  in reply to: Vasco Grilo🔸’s comment on: Nu­clear se­cu­rity seems like an in­ter­est­ing fund­ing gap

  I don’t focus exclusively on philanthropic funding. I added these paragraphs to the post to clarify my position:

  I agree that a full accounting of neglectedness should consider all resources going towards the cause (not just philanthropic ones), and that ‘preventing nuclear war’ more broadly receives significant attention from defence departments. However, even considering those resources, it still seems similarly neglected as biorisk.

  And the amount of philanthropic funding still matters because certain important types of work in the space can only be funded by philanthropists (e.g. lobbying or other policy efforts you don’t want to originate within a certain national government).

  I’d add that if if there’s almost no EA-inspired funding in a space, there’s likely to be some promising gaps by someone applying that mindset.

  In general, it’s a useful approximation to think of neglectedness as a single number, but the ultimate goal is to find good grants, and to do that it’s also useful to break down neglectedness into different types of resources, and consider related heuristics (e.g. that there was a recent drop).

  --

  Causes vs. interventions more broadly is a big topic. The very short version is that I agree doing cost-effectiveness estimates of specific interventions is a useful input into cause selection. However, I also think the INT framework is very useful. One reason is it seems more robust. Another reason is that in many practical planning situations that involve accumulating expertise over years (e.g. choosing a career, building a large grantmaking programme) it seems better to focus on a broad cluster of related interventions.

  E.g. you could do a cost-effectiveness estimate of corporate campaigns and determine ending factory farming is most cost-effective. But once you’ve spent 5 years building career capital in that factory farming, the available interventions or your calculations about them will likely very different.

• Benjamin_Todd 14 May 2024 21:28 UTC

  6 points

  0 ∶ 0

  in reply to: Jason’s comment on: Up­dates on the EA catas­trophic risk land­scape

  It might take more than $1bn, but around that level, you could become a major funder of one of the causes like AI safety, so you’d already be getting significant benefits within a cause.

  Agree you’d need to average 2x for the last point to work.

  Though note the three pathways to impact—talent, intellectual diversity, OP gaps—are mostly independent, so you’d only need one of them to work.

  Also agree in practice there would be some funging between the two, which would limit the differences, that’s a good point.

• Benjamin_Todd 14 May 2024 18:52 UTC

  3 points

  1 ∶ 0

  in reply to: Chris Leong’s comment on: Nu­clear se­cu­rity seems like an in­ter­est­ing fund­ing gap

  I’d also be interested in that. Maybe worth adding that the other grantmaker, Matthew, is younger. He graduated in 2015 so is probably under 32.