Thanks for the post! I thought it was interesting and thought-provoking, and I really enjoy posts like this one that get serious about building models.
One thought I did have about the model is that (if I’m interpreting it right) it seems to assume a 100% probability of fast takeoff (from strong AGI to ASI/​the world totally changing), which isn’t necessarily consistent with what most forecasters are predicting. For example, the Metaculus forecast for years between GWP growth >25% and AGI assigns a ~25% probability that it will be at least 15 years between AGI and massive resulting economic growth. I would enjoy seeing an expanded model that accounted for this aspect of the forecast as well.
Thanks for the post! I thought it was interesting and thought-provoking, and I really enjoy posts like this one that get serious about building models.
Thanks :-)
One thought I did have about the model is that (if I’m interpreting it right) it seems to assume a 100% probability of fast takeoff (from strong AGI to ASI/​the world totally changing), which isn’t necessarily consistent with what most forecasters are predicting. For example, the Metaculus forecast for years between GWP growth >25% and AGI assigns a ~25% probability that it will be at least 15 years between AGI and massive resulting economic growth.
Good point! The model does assume that the funder’s spending strategy never changes. And if there was a slow takeoff the funder might try and spend quickly before their capital becomes useless etc etc.
I think I’m pretty sold on fast-takeoff that this consideration didn’t properly cross my mind :-D
I would enjoy seeing an expanded model that accounted for this aspect of the forecast as well.
Here’s one very simple way of modelling it
write ps for the probability of a slow takeoff
call C the interventions available during the slow take-off and write c as the (average) cost effectiveness of interventions as as fraction of a, the cost effectiveness of intervention A.[1]
Conditioning on AGI at t:
F spends fraction min(ft,1) of any saved capital on A
F spends fraction 1−min(ft,1) ofof any saved capital on C
Hence the cost effectiveness of a small donor’s donation to A this year is min(ft,1)+[1−min(ft,1)]⋅ps⋅c times the on-paper cost effectiveness of donating to A.
Taking
ps∼Normal(0.4,0.2) truncated to (0,1)
c∼Normal(0.1,0.2) truncated to (0,1)
the distribution for f in the post and Metaculus AGI timelines
gives the following result, around a 5pp increase compared to the results not factoring this in.
I think almost certainly that s<1 for GHD giving. Both because
(1) the higher spending rate requires a lower bar
(2) many of the best current interventions benefit people over many years (and so we have to truncate only consider the benefit accrued before full on AGI, something that I consider here).
Thanks for the post! I thought it was interesting and thought-provoking, and I really enjoy posts like this one that get serious about building models.
One thought I did have about the model is that (if I’m interpreting it right) it seems to assume a 100% probability of fast takeoff (from strong AGI to ASI/​the world totally changing), which isn’t necessarily consistent with what most forecasters are predicting. For example, the Metaculus forecast for years between GWP growth >25% and AGI assigns a ~25% probability that it will be at least 15 years between AGI and massive resulting economic growth. I would enjoy seeing an expanded model that accounted for this aspect of the forecast as well.
Thanks :-)
Good point! The model does assume that the funder’s spending strategy never changes. And if there was a slow takeoff the funder might try and spend quickly before their capital becomes useless etc etc.
I think I’m pretty sold on fast-takeoff that this consideration didn’t properly cross my mind :-D
Here’s one very simple way of modelling it
write ps for the probability of a slow takeoff
call C the interventions available during the slow take-off and write c as the (average) cost effectiveness of interventions as as fraction of a, the cost effectiveness of intervention A.[1]
Conditioning on AGI at t:
F spends fraction min(ft,1) of any saved capital on A
F spends fraction 1−min(ft,1) ofof any saved capital on C
Hence the cost effectiveness of a small donor’s donation to A this year is min(ft,1)+[1−min(ft,1)]⋅ps⋅c times the on-paper cost effectiveness of donating to A.
Taking
ps∼Normal(0.4,0.2) truncated to (0,1)
the distribution for f in the post and Metaculus AGI timelines
gives the following result, around a 5pp increase compared to the results not factoring this in.
I think this extension better fits faster slow-takeoffs (i.e. on the order of 1-5 years). In my work on AI risk spending I considered a similar model feature, where after an AGI ‘fire alarm’ funders are able to switch to a new regime of faster spending.
I think almost certainly that s<1 for GHD giving. Both because
(1) the higher spending rate requires a lower bar
(2) many of the best current interventions benefit people over many years (and so we have to truncate only consider the benefit accrued before full on AGI, something that I consider here).