I havenât yet decided, but itâs likely that a majority of my donations will go to this yearâs donor lottery. Iâm fairly convinced by the arguments in favour of donor lotteries [1, 2], and would encourage others to consider them if theyâre unsure where to give.
Having said that, lotteries have less fuzzies than donating directly, so I may separately give to some effective charities which Iâm personally excited about.
ThanksâI just saw RP put out this post, which makes much the same point. Good to be cautious about interpreting these results!
YouGov Poll on SBF and EA
I recently came across this article from YouGov (published last week), summarizing a survey of US citizens for their opinions on Sam Bankman-Fried, Cryptocurrency and Effective Altruism.
I half-expected the survey responses to be pretty negative about EA, given press coverage and potential priming effects associating SBF to EA. So I was positively surprised that:
(itâs worth noting that there were only ~1000 participants, and the survey was online only)
(FYI to othersâIâve just seen Ajeyaâs very helpful writeup, which has already partially answer this question!)
Whatâs the reason for the change from Longtermism to GCRs? How has/âwill this change strategy going forward?
It seems that OPâs AI safety & gov teams have both been historically capacity-constrained. Why the decision to hire for these roles now (rather than earlier)?
Ten Project Ideas for AI X-Risk Prioritization
I made a list of 10 ideas Iâd be excited for someone to tackle, within the broad problem of âhow to prioritize resources within AI X-risk?â I wonât claim these projects are more /â less valuable than other things people could be doing. However, Iâd be excited if someone gave a stab at them
10 Ideas:
Threat Model Prioritization
Country Prioritization
An Inside-view timelines model
Modelling AI Lab deployment decisions
How are resources currently allocated between risks /â interventions /â countries
How to allocate between âAI Safetyâ vs âAI Governanceâ
Timing of resources to reduce risks
Correlation between Safety and Capabilities
How should we reason about prioritization?
How should we model prioritization?
I wrote up a longer (but still scrappy) doc here
Thanks Ajeya, this is very helpful and clarifying!
I am the only person who is primarily focused on funding technical research projects ⌠I began making grants in November 2022
Does this mean that prior to November 2022 there were ~no full-time technical AI safety grantmakers at Open Philanthropy?
OP (prev. GiveWell labs) has been evaluating grants in the AI safety space for over 10 years. In that time the AI safety field and Open Philanthropy have both grown, with OP granting over $300m on AI risk. Open Phil has also done a lot of research on the problem. So, from someone on the outside, it seems surprising that the number of people making grants has been consistently low
Following the episode with Mustafa, it would be great to interview the founders of leading AI labsâperhaps Dario (Anthropic) [again], Sam (OpenAI), or Demis (DeepMind). Or alternatively, the companies that invest /â support themâSundar (Google) or Satya (Microsoft).
It seems valuable to elicit their honest opinions[1] about âp(doom)â, timelines, whether they believe theyâve been net-positive for the world, etc.
I think one risk here is either:
a) not challenging them firmly enoughâlending them undue credibility /â legitimacy in the minds of listeners
b) challenging them too stronglyâreducing willingness to engage, less goodwill, etc
For deception (not deceptive alignment) - AI Deception: A Survey of Examples, Risks, and Potential Solutions (section 2)
This looks very exciting, thanks for posting!
Iâll quickly mention a couple of things that stuck out to me that might make the CEA potentially overoptimistic:
IQ points lost per Îźg/âdl of leadâthis is likely a logarithmic relationship (as suggested by Bernard and Schukraft). For a BLL of 2.4 â 10 Îźg/âdl, IQ loss from 1 Îźg/âdl increase may be close to 0.5, but above 10, itâs closer to 0.2 per 1 Îźg/âdl increase, and above 20, closer to 0.1. Average BLL in Bangladesh seem to be around 6.8 Îźg/âdl, though amongst residents living near turmeric sources of lead, it could plausibly be (much) higher, and thus a lower IQ gain will be had from a 1 Îźg/âdl reduction in lead.
Income loss per IQ points lossâThe CEA assumes that 1 IQ point loss leads to a 2.1% reduction in income. However, some work by GiveWell (here) suggests this might be closer to 0.67% (and there might be some reasons to discount this further, e.g. due to replicability concerns)
Replicability of interventionâas noted in the text, itâs hard to estimate how much the Bangladesh program reduced lead exposure by. If Bangladeshâs average BLL level is around 6.8 Îźg/âdl, then a 1.64 reduction from the intervention implies this intervention cut BLL by 25% for half of children in Bangladesh. This is great, but I can see several reasons why this may not be informative of future programsâ cost-effectiveness
Maybe Turmeric is much more prevalent in rural Bangladesh than other regions
Maybe it was unusually easy to get regulators to agree to introduce /â enforce standards
Maybe it was unusually easy to get producers to switch away from lead chromate
Each of these reasons on their own is fairly weak, but the likelihood of at least one being true gives us reason to discount future cost-effectiveness analyses. More generally, we might expect some regression to the mean w.r.t reducing exposure from tulmericâmaybe everything went right for this particular program, but this is unlikely to be true in future programs. To be clear, there are likely also reasons that this analysis is too pessimistic, and thus on net it may be the case that cost-effectiveness remains at $1/â DALY (or even better). Nonetheless, I think itâs good to be cautious, since $1 /â DALY implies this program was >800x better than cash transfers and >80x better than GiveWellâs top charitiesâa strong claim to make (though still possible!)
My bad, thanks so much!
It would be great to have some way to filter for multiple topics.
Example: Suppose I want to find posts related to the cost-effectiveness of AI safety. Instead of just filtering for âAI safetyâ, or for just âForecasting and estimationâ, I might want to find posts only at the intersection of those two. I attempted to do this by customizing my frontpage feed, but this doesnât really work (since it heavily biases to new/âupvoted posts)
it relies primarily on heuristics like organiser track record and higher-level reasoning about plans.
I think this is mostly correct, with the caveat that we donât exclusively rely on qualitative factors and subjective judgement alone. The way Iâd frame it is more as a spectrum between
[Heuristics] <------> [GiveWell-style cost-effectiveness modelling]
I think Iâd place FPâs longtermist evaluation methodology somewhere between those two poles, with flexibility based on whatâs feasible in each cause
Iâll +1 everything Johannes has already said, and add that several people (including myself) have been chewing over the âhow to rate longtermist projectsâ question for quite some time. Iâm unsure when we will post something publicly, but I hope it wonât be too far in the future.
If anyone is curious for details feel free to reach out!
Quick take: renaming shortforms to Quick takes is a mistake
Mildly against the Longtermism --> GCR shift
Epistemic status: Pretty uncertain, somewhat rambly
TL;DR replacing longtermism with GCRs might get more resources to longtermist causes, but at the expense of non-GCR longtermist interventions and broader community epistemics
Over the last ~6 months Iâve noticed a general shift amongst EA orgs to focus less on reducing risks from AI, Bio, nukes, etc based on the logic of longtermism, and more based on Global Catastrophic Risks (GCRs) directly. Some data points on this:
Open Phil renaming itâs EA Community Growth (Longtermism) Team to GCR Capacity Building
This post from Claire Zabel (OP)
Giving What We Canâs new Cause Area Fund being named âRisk and Resilience,â with the goal of âReducing Global Catastrophic Risksâ
Longview-GWWCâs Longtermism Fund being renamed the âEmerging Challenges Fundâ
Anecdotal data from conversations with people working on GCRs /â X-risk /â Longtermist causes
My guess is these changes are (almost entirely) driven by PR concerns about longtermism. I would also guess these changes increase the number of people donation /â working on GCRs, which is (by longtermist lights) a positive thing. After all, no-one wants a GCR, even if only thinking about people alive today.
Yet, I canât help but feel something is off about this framing. Some concerns (no particular ordering):
From a longtermist (~totalist classical utilitarian) perspective, thereâs a huge difference between ~99% and 100% of the population dying, if humanity recovers in the former case, but not the latter. Just looking at GCRs on their own mostly misses this nuance.
(see Parfit Reasons and Persons for the full thought experiment)
From a longtermist (~totalist classical utilitarian) perspective, preventing a GCR doesnât differentiate between âhumanity prevents GCRs and realises 1% of itâs potentialâ and âhumanity prevents GCRs realises 99% of its potentialâ
Preventing an extinction-level GCR might move us from 0% to 1% of future potential, but thereâs 99x more value in focusing on going from the âokay (1%)â to âgreat (100%)â future.
See Aird 2020 for more nuances on this point
From a longtermist (~suffering focused) perspective, reducing GCRs might be net-negative if the future is (in expectation) net-negative
E.g. if factory farming continues indefinitely, or due to increasing the chance of an S-Risk
See Melchin 2021 or DiGiovanni 2021 for more
(Note this isnât just a concern for suffering-focused ethics people)
From a longtermist perspective, a focus on GCRs neglects non-GCR longtermist interventions (e.g. trajectory changes, broad longtermism, patient altruism/âphilanthropy, global priorities research, institutional reform, )
From a âcurrent generationsâ perspective, reducing GCRs is probably not more cost-effective than directly improving the welfare of people /â animals alive today
Iâm pretty uncertain about this, but my guess is that alleviating farmed animal suffering is more welfare-increasing than e.g. working to prevent an AI catastrophe, given the latter is pretty intractable (But I havenât done the numbers)
See discussion here
If GCRs actually are more cost-effective under a âcurrent generationsâ worldview, then I question why EAs would donate to global health /â animal charities (since this is no longer a question of âworldview diversificationâ, just raw cost-effectiveness)
More meta points
From a community-building perspective, pushing people straight into GCR-oriented careers might work short-term to get resources to GCRs, but could lose the long-run benefits of EA /â Longtermist ideas. I worry this might worsen community epistemics about the motivation behind working on GCRs:
If GCRs only go through on longtermist grounds, but longtermism is false, then impartial altruists should rationally switch towards current-generations opportunities. Without a grounding in cause impartiality, however, people wonât actually make that switch
From a general virtue ethics /â integrity perspective, making this change on PR /â marketing reasons aloneâwithout an underlying change in longtermist motivationâfeels somewhat deceptive.
As a general rule about integrity, I think itâs probably bad to sell people on doing something for reason X, when actually you want them to do it for Y, and youâre not transparent about that
Thereâs something fairly disorienting about the community switching so quickly from [quite aggressive] âyay longtermism!â (e.g. much hype around launch of WWOTF) to essentially disowning the word longtermism, with very little mention /â admission that this happened or why