Although I laud posts like the OP, Iâm not sure I understand this approach to uncertainty.
I think a lot turns on what you mean by the AI cause area being âPlausibly betterâ than global poverty or animal welfare on EV. The Gretchenfrage seems to be this conditional forecast: âIf I spent (lets say) 6 months looking at the AI cause area, would I expect to identify better uses of marginal funding in this cause area than those I find in animal welfare and global poverty?â
If the answer is âplausibly so, but probably notâ (either due to a lower âprima facieâ central estimate, or after pricing in regression to the mean etc.), then I understand the work uncertainty is doing here (modulo the usual points about VoI): one canât carefully look at everything, and one has to make some judgments on what cause areas look most promising to investigate on current margins.
Yet if the answer is âProbably, yesâ, then offering these recommendations simpliciter (i.e. âEA should fully fund thisâ) seems premature to me. The evaluation is valuable, but should be presented with caveats like, âConditional on thinking global poverty is the best cause area, fund X; conditional on thinking animal welfare is the best cause area, fund Y (but, FWIW, I believe AI is the best cause area, but I donât know what to fund within it).â It would also lean against making ones own donations to X, Y etc., rather than spending time thinking about it/âfollowing the recommendations of someone one trusts to make good picks in the AI cause area.
If the answer is âplausibly so, but probably notâ (either due to a lower âprima facieâ central estimate, or after pricing in regression to the mean etc.)
An additional point to take into account when it comes to examining the research on AI as possible space for donations: as a scientific domain the topic of AI risks and safety can easily fall under the public/âacademic funding, even under the assumption that it is currently underfunded. To this end, individual applicants (precisely those who would be conducting research by means of donations) can apply for individual PhD and postdoc grants. There are numerous opportunities of that kind across EU. Moreover, the funding agencies (e.g. in Germany, Belgium, Netherlands, etc.) will employ expert refereeing system (sometimes even asking the applicant to suggest suitable referees) to assess the project and its effectiveness (which I find very relevant from the perspective of EA). If we take this into account, then a number of other organizations that canât be so easily funded via already existing institutional channels becomes much more urgent.
: one canât carefully look at everything, and one has to make some judgments on what cause areas look most promising to investigate on current margins.
This is why EA rarely falls into what can accurately be described as a âstreetlight effectâ. We arenât looking for one set of keys, weâre looking for a bunch of keys (threats to human welfare) and theres a bunch of us drunkards, all with differing abilities and expertise. So Iâd argue if its dark somewhere, those with the expertise need to start building streetlights, but if the lights getting brighter in certain areas (RCTs in health) then we need people there too.
[Note: I work on existential risk reduction]
Although I laud posts like the OP, Iâm not sure I understand this approach to uncertainty.
I think a lot turns on what you mean by the AI cause area being âPlausibly betterâ than global poverty or animal welfare on EV. The Gretchenfrage seems to be this conditional forecast: âIf I spent (lets say) 6 months looking at the AI cause area, would I expect to identify better uses of marginal funding in this cause area than those I find in animal welfare and global poverty?â
If the answer is âplausibly so, but probably notâ (either due to a lower âprima facieâ central estimate, or after pricing in regression to the mean etc.), then I understand the work uncertainty is doing here (modulo the usual points about VoI): one canât carefully look at everything, and one has to make some judgments on what cause areas look most promising to investigate on current margins.
Yet if the answer is âProbably, yesâ, then offering these recommendations simpliciter (i.e. âEA should fully fund thisâ) seems premature to me. The evaluation is valuable, but should be presented with caveats like, âConditional on thinking global poverty is the best cause area, fund X; conditional on thinking animal welfare is the best cause area, fund Y (but, FWIW, I believe AI is the best cause area, but I donât know what to fund within it).â It would also lean against making ones own donations to X, Y etc., rather than spending time thinking about it/âfollowing the recommendations of someone one trusts to make good picks in the AI cause area.
This is what captures my views best right now.
An additional point to take into account when it comes to examining the research on AI as possible space for donations: as a scientific domain the topic of AI risks and safety can easily fall under the public/âacademic funding, even under the assumption that it is currently underfunded. To this end, individual applicants (precisely those who would be conducting research by means of donations) can apply for individual PhD and postdoc grants. There are numerous opportunities of that kind across EU. Moreover, the funding agencies (e.g. in Germany, Belgium, Netherlands, etc.) will employ expert refereeing system (sometimes even asking the applicant to suggest suitable referees) to assess the project and its effectiveness (which I find very relevant from the perspective of EA). If we take this into account, then a number of other organizations that canât be so easily funded via already existing institutional channels becomes much more urgent.
P.S. Great post, Peter, only now saw it.
To attempt to complement what Peter already said,
This is why EA rarely falls into what can accurately be described as a âstreetlight effectâ. We arenât looking for one set of keys, weâre looking for a bunch of keys (threats to human welfare) and theres a bunch of us drunkards, all with differing abilities and expertise. So Iâd argue if its dark somewhere, those with the expertise need to start building streetlights, but if the lights getting brighter in certain areas (RCTs in health) then we need people there too.