I strongly agree with this comment, except that I don’t think this issue is minor.
IMO, this issue is related to a very troubling phenomenon that EA is seemingly undergoing in the past few years: people in EA tend to sometimes do not think much about their EV, and instead strive to have as much impact as possible. “Impact” is a sign-neutral term (“COVID-19 had a large impact on international travel”). It’s very concerning that many people in EA now use it interchangeably with “EV”, as if EA interventions in anthropogenic x-risk domains cannot possibly be harmful. One can call this phenomenon “sign neglect”.
Having a major EA organization named “EV” (as an acronym for something that is not “expected value”) may exacerbate this problem by further decreasing the usage of the term “EV”, and making people use sign-neutral language instead.
I think when people talk about impact, it’s implicit that they mean positive impact. I haven’t seen anything that makes me think that someone in EA doesn’t care about the sign of their impact, although I’d certainly be interested in any evidence of that.
I haven’t seen anything that makes me think that someone in EA doesn’t care about the sign of their impact
It’s not about people not caring about the sign of their impact (~everyone in EA cares); it’s about a tendency to behave in a way that is aligned with maximizing impact (rather than EV).
I’d certainly be interested in any evidence of that
Consider this interview with one of the largest funders in EA (the following is based on the transcript from the linked page):
Rob: “What might be distinctive about your approach that will allow you to find things that all the other groups haven’t already found or are going to find?”
[...]
SBF: But having gotten that out of the way, I think that being really willing to give significant amounts is a real piece of this. Being willing to give 100 million and not needing anything like certainty for that. We’re not in a position where we’re like, “If you want this level of funding, you better effectively have proof that what you’re going to do is great.” We’re happy to give a lot with not that much evidence and not that much conviction — if we think it’s, in expectation, great. Maybe it’s worth doing more research, but maybe it’s just worth going for. I think that is something where it’s a different style, it’s a different brand. And we, I think in general, are pretty comfortable going out on a limb for what seems like the right thing to do.
.
Rob Wiblin: OK, so with that out of the way, what’s a mistake you think at least some nontrivial fraction of people involved in effective altruism are making?
[...]
SBF: Then the last thing is thinking about grantmaking. This is definitely a philosophical difference that we have as a grantmaking organization. And I don’t know that we’re right on it, but I think it’s at least interesting how we think about it. Let’s say we evaluate a grant for 48 seconds. After 48 seconds, we have some probability distribution of how good it’s going to be, and it’s quite good in expected value terms. But we don’t understand it that well; there’s a lot of fundamental questions that we don’t know the answer to that would shift our view on this.
Then we think about it for 33 more seconds, and we’re like, “What might this probability distribution look like after 12 more hours of thinking?” And in 98% of those cases, we would still decide to fund it, but it might look materially different. We might have material concerns if we thought about it more, but we think they probably won’t be big enough that we would decide not to fund it.
Rob Wiblin: Save your time.
SBF: Right. You can spend that time, do that, or you could just say, “Great, you get the grant, because we already know where this is going to end up.” But you say that knowing that there are things you don’t know and could know that might give you reservations, that might turn out to make it a mistake. But from an expected value of impact perspective —
Rob Wiblin: It’s best just to go ahead.
SBF: Yeah, exactly. I think that’s another example of this, where being completely comfortable doing something that in retrospect is a little embarrassing. They’ll go, “Oh geez, you guys funded that. That was obviously dumb.” I’m like, “Yeah, you know, I don’t know.” That’s OK.
[...]
Rob Wiblin: Yeah. It’s so easy to get stuck in that case, where you are just unwilling to do anything that might turn out to be negative.
SBF: Exactly. And a lot of my response in those cases is like, “Look, I hear your concerns. I want you to tell me — in writing, right now — whether you think it is positive or negative expected value to take this action. And if you write down positive, then let’s do it. If you write down negative, then let’s talk about where that calculation’s coming from.” And maybe it will be right, but let’s at least remove the scenario where everyone agrees it’s a positive EV move, but people are concerned about some…
Notably, the FTX Foundation’s regranting program “gave over 100 people access to discretionary budget” (and I’m not aware of them using a reasonable mechanism to resolve the obvious unilateralist’s curse problem). One of the resulting grants was a $215,000 grant for creating an impact market. They wrote:
This regrant will support the creation of an “impact market.” The hope is to improve charity fundraising by allowing profit-motivated investors to earn returns by investing in charitable projects that are eventually deemed impactful.
A naive impact market is a mechanism that incentivizes people to carry out risky projects—that might turn out to be beneficial—while regarding potential harmful outcomes as if they were neutral. (The certificates of a project that ended up being harmful are worth as much as the certificates of a project that ended up being neutral, namely nothing.)
If I understand correctly, you think that people in EA do care about the sign of their impact, but that in practice their actions don’t align with this and they might end up having a large impact of unknown sign?
That’s certainly a reasonable view to hold, but given that you seem to agree that people are trying to have a positive impact, I don’t see how using phrases like “expected value” or “positive impact” instead of just “impact” would help.
In your example, it seems that SBF is talking about quickly making grants that have positive expected value, and uses the phrase “expected value” three times.
Reasonably determining whether an anthropogenic x-risk related intervention is net-positive or net-negative is often much more difficult[1] than identifying the intervention as potentially high-impact. With less than 2 minutes to think, one can usually do the latter but not the former. People in EA can easily be unconsciously optimizing for impact (which tends to be much easier and aligned with maximizing status & power) while believing they’re optimizing for EV. Using the term “impact” to mean “EV” can exacerbate this problem.
I strongly agree with this comment, except that I don’t think this issue is minor.
IMO, this issue is related to a very troubling phenomenon that EA is seemingly undergoing in the past few years: people in EA
tend tosometimes do not think much about their EV, and instead strive to have as much impact as possible. “Impact” is a sign-neutral term (“COVID-19 had a large impact on international travel”). It’s very concerning that many people in EA now use it interchangeably with “EV”, as if EA interventions in anthropogenic x-risk domains cannot possibly be harmful. One can call this phenomenon “sign neglect”.Having a major EA organization named “EV” (as an acronym for something that is not “expected value”) may exacerbate this problem by further decreasing the usage of the term “EV”, and making people use sign-neutral language instead.
I think when people talk about impact, it’s implicit that they mean positive impact. I haven’t seen anything that makes me think that someone in EA doesn’t care about the sign of their impact, although I’d certainly be interested in any evidence of that.
It’s not about people not caring about the sign of their impact (~everyone in EA cares); it’s about a tendency to behave in a way that is aligned with maximizing impact (rather than EV).
Consider this interview with one of the largest funders in EA (the following is based on the transcript from the linked page):
.
Notably, the FTX Foundation’s regranting program “gave over 100 people access to discretionary budget” (and I’m not aware of them using a reasonable mechanism to resolve the obvious unilateralist’s curse problem). One of the resulting grants was a $215,000 grant for creating an impact market. They wrote:
A naive impact market is a mechanism that incentivizes people to carry out risky projects—that might turn out to be beneficial—while regarding potential harmful outcomes as if they were neutral. (The certificates of a project that ended up being harmful are worth as much as the certificates of a project that ended up being neutral, namely nothing.)
Thanks for the reply!
If I understand correctly, you think that people in EA do care about the sign of their impact, but that in practice their actions don’t align with this and they might end up having a large impact of unknown sign?
That’s certainly a reasonable view to hold, but given that you seem to agree that people are trying to have a positive impact, I don’t see how using phrases like “expected value” or “positive impact” instead of just “impact” would help.
In your example, it seems that SBF is talking about quickly making grants that have positive expected value, and uses the phrase “expected value” three times.
Reasonably determining whether an anthropogenic x-risk related intervention is net-positive or net-negative is often much more difficult[1] than identifying the intervention as potentially high-impact. With less than 2 minutes to think, one can usually do the latter but not the former. People in EA can easily be unconsciously optimizing for impact (which tends to be much easier and aligned with maximizing status & power) while believing they’re optimizing for EV. Using the term “impact” to mean “EV” can exacerbate this problem.
Due to an abundance of crucial considerations.