I like that you can interact with this. It makes understanding models so much easier.
Playing with the calculator, I see that the result is driven to a surprising degree by the likelihood that “Compute needed by AGI, relative to a human brain (1e20-1e21 FLOPS)” is <1/1,000x (i.e. the bottom two options).[1]
I think this shows that your conclusion is driven substantially by your choice to hardcode “1e20-1e21 FLOPS” specifically, and then to treat this figure as a reasonable proxy for what computation an AGI would need. (That is, you suggest ~~1x as the midpoint for “Compute needed by AGI, relative to… 1e20-1e21 FLOPS”).
I think it’s also a bit of an issue to call the variable “relative to a human brain (1e20-1e21 FLOPS)”. Most users will read it as “relative to a human brain” while it’s really “relative to 1e20-1e21 FLOPS”, which is quite a specific take on what a human brain is achieving.
I value the fact that you argue for choosing this figure here. However, it seems like you’re hardcoding in confidence that isn’t warranted. Even from your own perspective, I’d guess that including your uncertainty over this figure would bump up the probability by a factor of 2-3, while it looks like other commenters have pointed out that programs seem to use much less computation than we’d predict with a similar methodology applied to tasks computers already do.
This is assuming a distribution on computation centred on ballpark ~100x as efficient in the future (just naively based on recent trends). If putting all weight on ~100x, nothing above 1⁄1,000x relative compute requirement matters. If putting some weight on ~1,000x, nothing above 1/100x relative compute requirement matters.
I like that you can interact with this. It makes understanding models so much easier.
Playing with the calculator, I see that the result is driven to a surprising degree by the likelihood that “Compute needed by AGI, relative to a human brain (1e20-1e21 FLOPS)” is <1/1,000x (i.e. the bottom two options).[1]
I think this shows that your conclusion is driven substantially by your choice to hardcode “1e20-1e21 FLOPS” specifically, and then to treat this figure as a reasonable proxy for what computation an AGI would need. (That is, you suggest ~~1x as the midpoint for “Compute needed by AGI, relative to… 1e20-1e21 FLOPS”).
I think it’s also a bit of an issue to call the variable “relative to a human brain (1e20-1e21 FLOPS)”. Most users will read it as “relative to a human brain” while it’s really “relative to 1e20-1e21 FLOPS”, which is quite a specific take on what a human brain is achieving.
I value the fact that you argue for choosing this figure here. However, it seems like you’re hardcoding in confidence that isn’t warranted. Even from your own perspective, I’d guess that including your uncertainty over this figure would bump up the probability by a factor of 2-3, while it looks like other commenters have pointed out that programs seem to use much less computation than we’d predict with a similar methodology applied to tasks computers already do.
This is assuming a distribution on computation centred on ballpark ~100x as efficient in the future (just naively based on recent trends). If putting all weight on ~100x, nothing above 1⁄1,000x relative compute requirement matters. If putting some weight on ~1,000x, nothing above 1/100x relative compute requirement matters.