My estimate is that the EV of that $1M, conditional on person X winning, is several orders of magnitude larger for X=[Nick Bostrom] than for almost any other value of X.
I doubt this is literally true fwiw. If Bostrom, a very high-status figure within longtermist EA, has really good donation opportunities to the tune of 1 million, I doubt it’d be unfunded. I also feel like there are similar analogous experiments made in the past where relatively low oversight grantmaking power was given to certain high-prestige longtermist EA figures( eg here and here). You can judge for yourself whether impact “several orders of magnitude higher” sounds right, personally I very much doubt it.
Why? Do you believe in something analogous to the efficient-market hypothesis for EA grantmaking
I meant “should” as a normative claim, not an empirical claim. Sorry if I miscommunicated.
Some evidence in this direction: Eliezer Yudkowsky recently wrote on a Facebook post:
This is your regular reminder that, if I believe there is any hope whatsoever in your work for AGI alignment, I think I can make sure you get funded.
This implies that all the really good funding opportunities Eliezer is aware of have already been funded, and any that appear can get funded quickly. Eliezer is not Nick Bostrom, but they’re in similar positions.
(Note: Eliezer’s Facebook post is publicly viewable, so I think reposting this quote here is ok from a privacy standpoint.)
If Bostrom, a very high-status figure within longtermist EA, has really good donation opportunities to the tune of 1 million, I doubt it’d be unfunded.
Even ‘very high-status figures within longtermist EA’ can control a limited amount of funding, especially for requests that are speculative/weird/non-legible from the perspective of the relevant donors. I don’t know what’s the bar for “really good donation opportunities”, but the relevant thing here is to compare the EV of that $1M in the hands of Bostrom to the EV of that $1M in the hands of other longtermism aligned people.
Less importantly, you rely here on the assumption that being “a very high-status figure within longtermist EA” means you can influence a lot of funding, but the causal relationship may mostly be going in the other direction. Bostrom (for example) probably got his high-status in longtermist EA mostly from his influential work, and not from being able to influence a lot of funding.
I also feel like there are similar analogous experiments made in the past where relatively low oversight grantmaking power was given to certain high-prestige longtermist EA figures( eg here and here). You can judge for yourself whether impact “several orders of magnitude higher” sounds right, personally I very much doubt it.
To be clear, I don’t think my reasoning here applies generally to “high-prestige longtermist EA figures”. Though this conversation with you made me think about this some more and my above claim now seems to me too strong (I added an EDIT block).
Though this conversation with you made me think about this some more and my above claim now seems to me too strong (I added an EDIT block).
I’m glad to cause an update! Hopefully it’s in the right direction! :)
Even ‘very high-status figures within longtermist EA’ can control a limited amount of funding, especially for requests that are speculative/weird/non-legible from the perspective of the relevant donors. I don’t know what’s the bar for “really good donation opportunities”, but the relevant thing here is to compare the EV of that $1M in the hands of Bostrom to the EV of that $1M in the hands of other longtermism aligned people.
Again, there are proxies you can look at, like what Carl Shulman donates to vs what actual winners of the donor lottery donates to. But maybe you don’t consider this much evidence, if you posit that Nick Bostrom specifically has unusually high discernment, specifically enough to donate to things in the band of activities that are “speculative/weird/non-legible” from the perspective of the relevant donors, but not speculative/weird/non-legible enough that the donor lottery administration won’t permit this.
I guess my rejoinder here is just an intuitive sense of disbelief? Several (say >=3?) orders of magnitude above 1 million gets you >1B, and as can be deduced in the figures in the post above, this is already well over the annual long-termist spending every year. If we believe that Nick Bostrom can literally accomplish much more good with 1 million than money allocated by the rest of the longtermist EA movement combined (including all money sent to FHI, where he works), isn’t this really wild? Also why aren’t we sending more money to Nick Bostrom to regrant?
(Though perhaps you came to the same conclusion by now).
Less importantly, you rely here on the assumption that being “a very high-status figure within longtermist EA” means you can influence a lot of funding, but the causal relationship may mostly be going in the other direction
I’m confused about what you’re saying here. P(B| do A) is not evidence against P(A|B), except in very rare circumstances.
But maybe you don’t consider this much evidence, if you posit that Nick Bostrom specifically has unusually high discernment, specifically enough to donate to things in the band of activities that are “speculative/weird/non-legible” from the perspective of the relevant donors, but not speculative/weird/non-legible enough that the donor lottery administration won’t permit this.
My reasoning here is indeed based specifically on the track record of Nick Bostrom. (Also, I’m imagining here a theoretical donor lottery where the winner has 100% control over the money that they won.)
I guess my rejoinder here is just an intuitive sense of disbelief? Several (say >=3?) orders of magnitude above 1 million gets you >1B, and as can be deduced in the figures in the post above, this is already well over the annual long-termist spending every year. If we believe that Nick Bostrom can literally accomplish much more good with 1 million than money allocated by the rest of the longtermist EA movement combined (including all money sent to FHI, where he works), isn’t this really wild?
I was not comparing $1M in the hands of Bostrom to $1B in the hands of a random longtermism-aligned person. (The $1B would plausibly be split across many grants, and it’s plausible that Bostrom would end up controlling way more than $1M out of it.)
As an aside, without thinking about it much, it seems to me that the EV from the publication of the book Superintelligence is plausibly much higher than the total EV from everything else that was accomplished by the rest of the longtermist EA movement so far. (I can easily imagine myself updating away from that if I try to enumerate the things that were accomplished by the longtermist EA movement).
Also why aren’t we sending more money to Nick Bostrom to regrant?
To answer this I think that the word “we” should be replaced with something more specific. Why grantmakers at longtermism-aligned grantmaking orgs don’t send more money to Bostrom to regrant? One response is that there is probably nothing analogous to the efficient-market hypothesis for EA grantmaking (see the last paragraph here). Also, the grantmakers are in an implicitly completion with each other over influence on future grant funds. A grantmaker who makes grants that are speculative, weird, non-legible or have a high probability of failing may tend to lose influence over future grant funds, and perhaps reduce the amount of future longtermist funding that their org can give.
Imagine that Bostrom uses the additional $1M to hire another assistant, or some manager for FHI, that simply results in Bostrom being a bit more productive. Looking at this from the lens of the grantmakers’ incentives, how would that $1M grant compare to the average LTFF grant?
I’m confused about what you’re saying here. P(B| do A) is not evidence against P(A|B), except in very rare circumstances.
If we estimate P(A|B) based on a correlation that we observe between A and B then the existence of a causal relationship from A to B is indeed evidence that should update our estimate of P(A|B) towards a lower value.
I doubt this is literally true fwiw. If Bostrom, a very high-status figure within longtermist EA, has really good donation opportunities to the tune of 1 million, I doubt it’d be unfunded. I also feel like there are similar analogous experiments made in the past where relatively low oversight grantmaking power was given to certain high-prestige longtermist EA figures( eg here and here). You can judge for yourself whether impact “several orders of magnitude higher” sounds right, personally I very much doubt it.
I meant “should” as a normative claim, not an empirical claim. Sorry if I miscommunicated.
Some evidence in this direction: Eliezer Yudkowsky recently wrote on a Facebook post:
This implies that all the really good funding opportunities Eliezer is aware of have already been funded, and any that appear can get funded quickly. Eliezer is not Nick Bostrom, but they’re in similar positions.
(Note: Eliezer’s Facebook post is publicly viewable, so I think reposting this quote here is ok from a privacy standpoint.)
Even ‘very high-status figures within longtermist EA’ can control a limited amount of funding, especially for requests that are speculative/weird/non-legible from the perspective of the relevant donors. I don’t know what’s the bar for “really good donation opportunities”, but the relevant thing here is to compare the EV of that $1M in the hands of Bostrom to the EV of that $1M in the hands of other longtermism aligned people.
Less importantly, you rely here on the assumption that being “a very high-status figure within longtermist EA” means you can influence a lot of funding, but the causal relationship may mostly be going in the other direction. Bostrom (for example) probably got his high-status in longtermist EA mostly from his influential work, and not from being able to influence a lot of funding.
To be clear, I don’t think my reasoning here applies generally to “high-prestige longtermist EA figures”. Though this conversation with you made me think about this some more and my above claim now seems to me too strong (I added an EDIT block).
I’m glad to cause an update! Hopefully it’s in the right direction! :)
Again, there are proxies you can look at, like what Carl Shulman donates to vs what actual winners of the donor lottery donates to. But maybe you don’t consider this much evidence, if you posit that Nick Bostrom specifically has unusually high discernment, specifically enough to donate to things in the band of activities that are “speculative/weird/non-legible” from the perspective of the relevant donors, but not speculative/weird/non-legible enough that the donor lottery administration won’t permit this.
I guess my rejoinder here is just an intuitive sense of disbelief? Several (say >=3?) orders of magnitude above 1 million gets you >1B, and as can be deduced in the figures in the post above, this is already well over the annual long-termist spending every year. If we believe that Nick Bostrom can literally accomplish much more good with 1 million than money allocated by the rest of the longtermist EA movement combined (including all money sent to FHI, where he works), isn’t this really wild? Also why aren’t we sending more money to Nick Bostrom to regrant?
(Though perhaps you came to the same conclusion by now).
I’m confused about what you’re saying here. P(B| do A) is not evidence against P(A|B), except in very rare circumstances.
My reasoning here is indeed based specifically on the track record of Nick Bostrom. (Also, I’m imagining here a theoretical donor lottery where the winner has 100% control over the money that they won.)
I was not comparing $1M in the hands of Bostrom to $1B in the hands of a random longtermism-aligned person. (The $1B would plausibly be split across many grants, and it’s plausible that Bostrom would end up controlling way more than $1M out of it.)
As an aside, without thinking about it much, it seems to me that the EV from the publication of the book Superintelligence is plausibly much higher than the total EV from everything else that was accomplished by the rest of the longtermist EA movement so far. (I can easily imagine myself updating away from that if I try to enumerate the things that were accomplished by the longtermist EA movement).
To answer this I think that the word “we” should be replaced with something more specific. Why grantmakers at longtermism-aligned grantmaking orgs don’t send more money to Bostrom to regrant? One response is that there is probably nothing analogous to the efficient-market hypothesis for EA grantmaking (see the last paragraph here). Also, the grantmakers are in an implicitly completion with each other over influence on future grant funds. A grantmaker who makes grants that are speculative, weird, non-legible or have a high probability of failing may tend to lose influence over future grant funds, and perhaps reduce the amount of future longtermist funding that their org can give.
Imagine that Bostrom uses the additional $1M to hire another assistant, or some manager for FHI, that simply results in Bostrom being a bit more productive. Looking at this from the lens of the grantmakers’ incentives, how would that $1M grant compare to the average LTFF grant?
If we estimate P(A|B) based on a correlation that we observe between A and B then the existence of a causal relationship from A to B is indeed evidence that should update our estimate of P(A|B) towards a lower value.