I think this is a good way of thinking about it and I like your classification. Also agree that neartermist animal welfare interventions shouldn’t discounted to ~0. I disagree with the claim that scenarios 1–3 are not obviously more likely than 4–6. #6 seems somewhat plausible but I think #4 and #5 are highly unlikely.
The weakest possible version of AGI is something like “take all the technological and economic advances being made by the smartest people in the world, and now rapidly accelerate those advancements because you can run many copies of equally-smart AIs.” (That’s the minimum outcome; I think the more likely outcome is “AGI is radically smarter than the smartest human”.)
RE #5, I can’t imagine a world where an innovation like “greatly increase the number of world-class researchers” would be on par with the Internet in terms of impactfulness.
RE #4, if technological chance is happening that quickly, it seems implausible that McDonald’s will survive. They didn’t have anything comparable to McDonald’s 1000 years ago. They couldn’t have even imagined McDonald’s. I predict that a decade after TAI, if we’re still alive, then whatever stuff we have will look nothing like McDonald’s, in the same way that McDonald’s looks nothing like the stuff people had in medieval times.
I don’t think outcome #4 is crazy unlikely, but I do think it’s clearly less likely than #1–3.
RE #4, if technological chance is happening that quickly, it seems implausible that McDonald’s will survive. They didn’t have anything comparable to McDonald’s 1000 years ago. They couldn’t have even imagined McDonald’s. I predict that a decade after TAI, if we’re still alive, then whatever stuff we have will look nothing like McDonald’s, in the same way that McDonald’s looks nothing like the stuff people had in medieval times.
If we’re still alive, most of the same people will still be alive, and their tastes, habits and values will only have changed so much. Think conservatives, people against alt proteins and others who grew up with McDonald’s. 1000 years is enough time for dramatic shifts in culture and values, but 10 years doesn’t seem to be. I suspect shifts in culture and values are primarily driven by newer generations just growing up to have different values and older generations with older values dying, not people changing their minds.
And radical life extension might make some values and practices persist far longer than they would have otherwise, although I’m not sure how much people who’d still want to eat conventional meat would opt for radical life extension.
I would predict maybe 65% chance of outcome #1 (50% chance everyone dies, 15% chance we get mind uploads or something); 15% chance of #2; 15% chance of #6; low chance of #3 or #4; basically zero chance of #5.
I think this is a good way of thinking about it and I like your classification. Also agree that neartermist animal welfare interventions shouldn’t discounted to ~0. I disagree with the claim that scenarios 1–3 are not obviously more likely than 4–6. #6 seems somewhat plausible but I think #4 and #5 are highly unlikely.
The weakest possible version of AGI is something like “take all the technological and economic advances being made by the smartest people in the world, and now rapidly accelerate those advancements because you can run many copies of equally-smart AIs.” (That’s the minimum outcome; I think the more likely outcome is “AGI is radically smarter than the smartest human”.)
RE #5, I can’t imagine a world where an innovation like “greatly increase the number of world-class researchers” would be on par with the Internet in terms of impactfulness.
RE #4, if technological chance is happening that quickly, it seems implausible that McDonald’s will survive. They didn’t have anything comparable to McDonald’s 1000 years ago. They couldn’t have even imagined McDonald’s. I predict that a decade after TAI, if we’re still alive, then whatever stuff we have will look nothing like McDonald’s, in the same way that McDonald’s looks nothing like the stuff people had in medieval times.
I don’t think outcome #4 is crazy unlikely, but I do think it’s clearly less likely than #1–3.
If we’re still alive, most of the same people will still be alive, and their tastes, habits and values will only have changed so much. Think conservatives, people against alt proteins and others who grew up with McDonald’s. 1000 years is enough time for dramatic shifts in culture and values, but 10 years doesn’t seem to be. I suspect shifts in culture and values are primarily driven by newer generations just growing up to have different values and older generations with older values dying, not people changing their minds.
And radical life extension might make some values and practices persist far longer than they would have otherwise, although I’m not sure how much people who’d still want to eat conventional meat would opt for radical life extension.
I would predict maybe 65% chance of outcome #1 (50% chance everyone dies, 15% chance we get mind uploads or something); 15% chance of #2; 15% chance of #6; low chance of #3 or #4; basically zero chance of #5.