Just as our ancestors experienced before us, we face the prospect of losing the world we know in exchange for material progress and prosperity.
Your ancestors who adopted agriculture did so because they thought that they and their children would get to eat the bread, not that they were sowing the seeds of their own destruction. If they had known that planting crops would lead to invasion and replacement they likely would not have done it. This rather large dis-analogy makes me think your use of the word ‘just’ is a bit of a stretch here.
Our ancestors had less insight into the trade they were making than we do about our own situation. That’s true.
Yet they still made the trade, and in hindsight, was it a bad trade to make? I disagree with people like Jared Diamond who argue that the agricultural revolution was the “worst mistake in the history of the human race”. It certainly had some very negative consequences. But like most people, I think the agricultural revolution was still a good thing overall, despite the fact that it carried enormous negative side effects.
I suspect the transition to AI will be less calamitous and more peaceful than our transition to agriculture. In my view, this means our trade is even easier to make. Yet, I still recognize that we face similar tradeoffs. We risk losing our way of life. There is also a credible risk (even if I think it’s small), that the entire human species will go extinct. That would be very bad, but as I argued in the post, it would not be the same as losing all value in the universe.
Our ancestors did not make this trade at all for the most part. Mostly they stayed hunter gathers, until the people who adopted farming out populated them and then expanded and killed/outcompeted them. (technically, I guess “our” ancestors are the ones who adopted the agriculture)
Likewise, the vast majority of humanity is not directly developing AI. Therefore, in an important sense, “we” are not making the trade of whether to develop AI; only a small number of people are.
Your ancestors who adopted agriculture did so because they thought that they and their children would get to eat the bread, not that they were sowing the seeds of their own destruction. If they had known that planting crops would lead to invasion and replacement they likely would not have done it. This rather large dis-analogy makes me think your use of the word ‘just’ is a bit of a stretch here.
Our ancestors had less insight into the trade they were making than we do about our own situation. That’s true.
Yet they still made the trade, and in hindsight, was it a bad trade to make? I disagree with people like Jared Diamond who argue that the agricultural revolution was the “worst mistake in the history of the human race”. It certainly had some very negative consequences. But like most people, I think the agricultural revolution was still a good thing overall, despite the fact that it carried enormous negative side effects.
I suspect the transition to AI will be less calamitous and more peaceful than our transition to agriculture. In my view, this means our trade is even easier to make. Yet, I still recognize that we face similar tradeoffs. We risk losing our way of life. There is also a credible risk (even if I think it’s small), that the entire human species will go extinct. That would be very bad, but as I argued in the post, it would not be the same as losing all value in the universe.
Our ancestors did not make this trade at all for the most part. Mostly they stayed hunter gathers, until the people who adopted farming out populated them and then expanded and killed/outcompeted them. (technically, I guess “our” ancestors are the ones who adopted the agriculture)
Likewise, the vast majority of humanity is not directly developing AI. Therefore, in an important sense, “we” are not making the trade of whether to develop AI; only a small number of people are.