What % of your grants (either grantee- or $-weighted, but preferably specify which denominator you’re using) do you expect to be net negative to the world?
A heuristic I have for being less risk-averse is
If X (horrible thing) never happens, you spend too much resources on preventing X.
Obviously this isn’t true for everything (eg a world without any existential catastrophes seems like a world that has its priorities right), but I think it’s overall a good heuristic, as illustrated by Scott Aaronson’s Umeshisms and Mitchell and Webb’s “No One Drowned” episode.
My knee-jerk reaction is: If “net negative” means “ex-post counterfactual impact anywhere below zero, but including close-to-zero cases” then it’s close to 50% of grantees. Important here is that “impact” means “total impact on the universe as evaluated by some omniscient observer”. I think it’s much less likely that funded projects are net negative by the light of their own proxy goals or by any criterion we could evaluate in 20 years (assuming no AGI-powered omniscience or similar by then).
(I still think that the total value of the grantee portfolio would be significantly positive b/c I’d expect the absolute values to be systematically higher for positive than for negative grants.)
This is just a general view I have. It’s not specific to EA Funds, or the grants this round. It applies to basically any action. That view is somewhat considered but I think also at least somewhat controversial. I have discussed it a bit but not a lot with others, so I wouldn’t be very surprised if someone replied to this comment saying “but this can’t be right because of X”, and then I’d be like “oh ok, I think you’re right, the close-to-50% figure now seems massively off to me”.
--
If “net negative” means “significantly net negative” (though I’m not sure what the interesting bar for “significant” would be), then I’m not sure I have a strong prior. Glancing over the specific grants we made I feel that for very roughly 1⁄4 of them I have some vague sense that “there is a higher-than-baseline risk for this being significantly net negative”. But idk what that higher-than-baseline risk is as absolute probability, and realistically I think all that’s going on here is that for about 1⁄4 of grants I can easily generate some prototypical story for why they’d turn out to be significantly net negative. I don’t know how well this is correlated with the actual risk.
(NB I still think that the absolute values for ‘significantly net negative’ grants will be systematically smaller than for ‘significantly net positive’ ones. E.g., I’d guess that the 99th percentile ex-post impact grant much more than offsets the 1st percentile grant [which I’m fairly confident is significantly net negative].)
Thanks a lot for this answer! After asking this, I realize I’m also interested in asking the same question about what ratio of grants you almost funded would be ex post net-negative.
This isn’t what you asked, but out of all the applications that we receive (excluding desk rejections), 5-20% seem ex ante net-negative to me, in the sense that I expect someone giving funding to them to make the world worse. In general, worries about accidental harm do not play a major role in my decisions not to fund projects, and I don’t think we’re very risk-averse. Instead, a lot of rejections happen because I don’t believe the project will have a major positive impact.
I include the opportunity cost of the broader community (e.g., the project hires people from the community who’d otherwise be doing more impactful work), but not the opportunity cost of providing the funding. (This is what I meant to express with “someone giving funding to them”, though I think it wasn’t quite clear.)
[thinking/rambling aloud] I feel like an “ideal reasoner” or something should indeed have that heuristic, but I feel unsure whether boundedly rational people internalising it more or having it advocated for to them more would be net positive or net negative. (I feel close to 50⁄50 on this and haven’t thought about it much; “unsure” doesn’t mean “I suspect it’d probably be bad.)
If I had to choose whether to make most of the world closer to naive consequentialism than they are now, and I can’t instead choose sophisticated consequentialism, I’d probably do that. But I’m not sure for EA grantmakers. And of course sophisticated consequentialism seems better.
Maybe there’s a way we could pair this heuristic with some other heuristics or counter-examples such that the full package is quite useful. Or maybe adding more of this heuristic would already help “balance things out”, since grantmakers may already be focusing somewhat too much on downside risk. I really don’t know.
Hmm, I think this heuristic actually doesn’t make sense for ideal (Bayesian) reasoners, since ideal reasoners can just multiply the EVs out for all actions and don’t need weird approximations/heuristics.
I broadly think this heuristic makes sense in a loose way in situations where the downside risks are not disproportionately high. I’m not sure what you mean by “sophisticated consequentialism” here, but I guess I’d sort of expect sophisticated consequentialism (at least in situations where explicit EV calculations are less practical) to include a variant of this heuristic somewhere.
Consequentialists are supposed to estimate all of the effects of their actions, and then add them up appropriately. This means that they cannot just look at the direct and immediate effects of their actions, but also have to look at indirect and less immediate effects. Failing to do so amounts to applying naive consequentialism. That is to be contrasted with sophisticated consequentialism, which appropriately takes indirect and less immediate effects into account (cf. the discussion on “simplistic” vs. “correct” replaceability on 80,000 Hours’ blog (Todd 2015)).
As for a concrete example, a naive conception of consequentialism may lead one to believe that it is right to break rules if it seems that that would have net positive effects on the world. Such rule-breaking normally has negative side-effects, however—e.g. it can lower the degree of trust in society, and for the rule-breaker’s group in particular—which means that sophisticated consequentialism tends to be more opposed to rule-breaking than naive consequentialism.
I think maybe what I have in mind is actually “consequentialism that accounts appropriately for biases, model uncertainty, optimizer’s curse, unilateralist’s curse, etc.” (This seems like a natural fit for the words sophisticated consequentialism, but it sounds like that’s not what the term is meant to mean.)
I’d be much more comfortable with someone having your heuristic if they were aware of those reasons why your EV estimates (whether implicit or explicit, qualitative or quantitative) should often be quite uncertain and may be systematically biased towards too much optimism for whatever choice you’re most excited about. (That’s not the same as saying EV estimates are useless, just that they should often be adjusted in light of such considerations.)
What % of your grants (either grantee- or $-weighted, but preferably specify which denominator you’re using) do you expect to be net negative to the world?
A heuristic I have for being less risk-averse is
Obviously this isn’t true for everything (eg a world without any existential catastrophes seems like a world that has its priorities right), but I think it’s overall a good heuristic, as illustrated by Scott Aaronson’s Umeshisms and Mitchell and Webb’s “No One Drowned” episode.
My knee-jerk reaction is: If “net negative” means “ex-post counterfactual impact anywhere below zero, but including close-to-zero cases” then it’s close to 50% of grantees. Important here is that “impact” means “total impact on the universe as evaluated by some omniscient observer”. I think it’s much less likely that funded projects are net negative by the light of their own proxy goals or by any criterion we could evaluate in 20 years (assuming no AGI-powered omniscience or similar by then).
(I still think that the total value of the grantee portfolio would be significantly positive b/c I’d expect the absolute values to be systematically higher for positive than for negative grants.)
This is just a general view I have. It’s not specific to EA Funds, or the grants this round. It applies to basically any action. That view is somewhat considered but I think also at least somewhat controversial. I have discussed it a bit but not a lot with others, so I wouldn’t be very surprised if someone replied to this comment saying “but this can’t be right because of X”, and then I’d be like “oh ok, I think you’re right, the close-to-50% figure now seems massively off to me”.
--
If “net negative” means “significantly net negative” (though I’m not sure what the interesting bar for “significant” would be), then I’m not sure I have a strong prior. Glancing over the specific grants we made I feel that for very roughly 1⁄4 of them I have some vague sense that “there is a higher-than-baseline risk for this being significantly net negative”. But idk what that higher-than-baseline risk is as absolute probability, and realistically I think all that’s going on here is that for about 1⁄4 of grants I can easily generate some prototypical story for why they’d turn out to be significantly net negative. I don’t know how well this is correlated with the actual risk.
(NB I still think that the absolute values for ‘significantly net negative’ grants will be systematically smaller than for ‘significantly net positive’ ones. E.g., I’d guess that the 99th percentile ex-post impact grant much more than offsets the 1st percentile grant [which I’m fairly confident is significantly net negative].)
Thanks a lot for this answer! After asking this, I realize I’m also interested in asking the same question about what ratio of grants you almost funded would be ex post net-negative.
This isn’t what you asked, but out of all the applications that we receive (excluding desk rejections), 5-20% seem ex ante net-negative to me, in the sense that I expect someone giving funding to them to make the world worse. In general, worries about accidental harm do not play a major role in my decisions not to fund projects, and I don’t think we’re very risk-averse. Instead, a lot of rejections happen because I don’t believe the project will have a major positive impact.
are you including opportunity cost in the consideration of net harm?
I include the opportunity cost of the broader community (e.g., the project hires people from the community who’d otherwise be doing more impactful work), but not the opportunity cost of providing the funding. (This is what I meant to express with “someone giving funding to them”, though I think it wasn’t quite clear.)
As an aside, I think that’s an excellent heuristic, and I worry that many EAs (including myself) haven’t internalized it enough.
(Though I also worry that pushing too much for it could lead to people failing to notice the exceptions where it doesn’t apply.)
[thinking/rambling aloud] I feel like an “ideal reasoner” or something should indeed have that heuristic, but I feel unsure whether boundedly rational people internalising it more or having it advocated for to them more would be net positive or net negative. (I feel close to 50⁄50 on this and haven’t thought about it much; “unsure” doesn’t mean “I suspect it’d probably be bad.)
I think this intersects with concerns about naive consequentialism and (less so) potential downsides of using explicit probabilities.
If I had to choose whether to make most of the world closer to naive consequentialism than they are now, and I can’t instead choose sophisticated consequentialism, I’d probably do that. But I’m not sure for EA grantmakers. And of course sophisticated consequentialism seems better.
Maybe there’s a way we could pair this heuristic with some other heuristics or counter-examples such that the full package is quite useful. Or maybe adding more of this heuristic would already help “balance things out”, since grantmakers may already be focusing somewhat too much on downside risk. I really don’t know.
Hmm, I think this heuristic actually doesn’t make sense for ideal (Bayesian) reasoners, since ideal reasoners can just multiply the EVs out for all actions and don’t need weird approximations/heuristics.
I broadly think this heuristic makes sense in a loose way in situations where the downside risks are not disproportionately high. I’m not sure what you mean by “sophisticated consequentialism” here, but I guess I’d sort of expect sophisticated consequentialism (at least in situations where explicit EV calculations are less practical) to include a variant of this heuristic somewhere.
I now think sophisticated consequentialism may not be what I really had in mind. Here’s the text from the entry on naive consequentialism I linked to:
I think maybe what I have in mind is actually “consequentialism that accounts appropriately for biases, model uncertainty, optimizer’s curse, unilateralist’s curse, etc.” (This seems like a natural fit for the words sophisticated consequentialism, but it sounds like that’s not what the term is meant to mean.)
I’d be much more comfortable with someone having your heuristic if they were aware of those reasons why your EV estimates (whether implicit or explicit, qualitative or quantitative) should often be quite uncertain and may be systematically biased towards too much optimism for whatever choice you’re most excited about. (That’s not the same as saying EV estimates are useless, just that they should often be adjusted in light of such considerations.)