There’s a truism that technology is good—even if it creates winners and losers, it improves the world. Toby Ord argues that the conclusions about the benefits of technology is sensitive to the end of humanity—but this jumps over the transitions by starting from the assumption[1] that “long-term progress in science, technology, and values have tended to make people’s lives longer, freer, and more prosperous.” That is, looking back historically, the net impact misses the immense immediate harms of large scale technological changes that can last for generations.
As I’ll explain, the largest technological revolutions in human history are arguably the agricultural revolution and the industrial revolution. In both cases, the vast majority of those immediately affected were harmed, not helped. Of course, the longer term impact was positive; those benefits are not in question[2] - not that those alive during the transition should have cared.
The two obvious examples
The invention of agriculture led to increased food availability and around ten thousand years of greatly worsened health and lifespans[3]. The wealthiest and most powerful people benefited immensely from the population explosion, and from the wars that larger populations enabled and required; the population suffered from both malnutrition, and that same increase in the scale of violence[4].
The invention of industry was more beneficial to the consumer—but not to those directly involved. In 1840, over a third of the British population worked in a factory. This was bad, in part directly due to factory worker deaths, but also due to pollution and disease. Mortality shot up over the middle of the 1800s—the famed “urban penalty”, especially among children, albeit partially offset by reduced deaths because of sanitation later in the century[5].
3 more examples via ChatGPT/Manheim (which provided 5 including the 2 above; I omitted them—again emphasis mine):
Writing and external symbolic storage—Administration, law, history, mathematics, scripture, bureaucracy, long-distance coordination. Early writing mostly helped palaces, temples, tax systems, accounting, property claims, labor control, and bureaucracy before it helped ordinary people read novels or do science. So the near-term “users” benefited, but many affected subjects may have faced more legible extraction and administration. Evidence from early Mesopotamia links writing with larger government buildings and multi-level bureaucracies.
Metallurgy, especially iron—Tools, weapons, plows, empires, deforestation, intensified agriculture, military expansion. Bronze matters too, but iron’s scale and availability make it more transformative. Better tools helped agriculture and craft production, but weapons, fortifications, conquest, inequality, and elite control plausibly dominated early experience for many. The case is less clean because metal tools also had immediate productive benefits, but the war-and-hierarchy channel is very real.
Electricity + computation + telecommunications—I’d bundle these reluctantly as the “information-electrical stack”: telegraph, telephone, radio, electric grids, computers, internet, AI. This led to surveillance, labor displacement, attention capture, military command/control, financial acceleration, and dependence on fragile networks[8].
Manheim’s argument for what makes good technologies good:
There have been a couple of revolutionary changes in medicine and public health over the past couple centuries. The vaccine revolution, the advent of modern sanitation, and infection control each include a strong case that they were immediately beneficial, and stayed that way indefinitely[9]. Refrigeration, washing machines, and bicycles[10] are arguably more examples in this class. So some technologies really are just positive—but we need to ask which ones.
I think there’s a simple explanation; directly good things are good, but many other classes of transformative change end up disruptive in ways that hurt before they can help[11]. Technologies that have first order impacts on coordination and production, or that empower groups in other ways, tend to differentially benefit the powerful in ways that are harmful to others, either directly or indirectly[12].
I find myself instinctively resisting Manheim’s explanation, since I’m generally keen on improvements to empowerment and coordination, but have to admit it parsimoniously explains the small-n historical track record above. The issue, as always, seems to be the gap between the beautiful ideal (“more empowerment! more coordination!”) and the unavoidably-messy realities of implementation, people being people, etc.
Tangentially, this reminds me a bit of Holden Karnofsky’s maximally-conservative utopia, which is just “status quo minus clearly-bad things”.
This isn’t really a utopia in the traditional sense. It’s trying to lay out one end of a spectrum.
Start here:
In this world, everything is exactly like the status quo, with one exception: cancer does not exist.
It may not be very exciting, but it’s hard to argue with the claim that this would be better than the world as it is today.
This is basically the most conservative utopia I can come up with, because the only change it proposes is a change that I think we can all get on board with, without hesitation. Most proposed changes to the world would make at least some people uncomfortable (no inequality? No sadness?), but this one shouldn’t. If we got rid of cancer, we’d still have death, we’d still have suffering, we’d still have struggle, etc. - we just wouldn’t have cancer.
You can almost certainly improve this utopia further by taking more baby-steps along the same lines. Make a list of things that—like cancer—you think are just unambiguously bad, and would be happy to see no more of in the world. Then define utopia as “exactly like the status quo, except that all the things on my list don’t exist.” Examples could include:
Other diseases
Hunger
Non-consensual violence (not including e.g. martial arts, in which two people agree to a set of rules that allows specific forms of violence for a set period of time).
Racism, sexism, etc.
“Status quo, minus everything on my list” is a highly conservative utopia. Unlike literary utopias, it should be fairly clear that this world would be a major improvement on the world as it is.
I note that in my survey on fictional utopias, it was much easier to get widespread agreement (high average scores) for properties of utopia than for full utopian visions. For example, while no utopia description scored as high as 4 on a 5-point scale, the following properties all scored 4.5 or higher: “no one goes hungry”, “there is no violent conflict,” “there is no discrimination by race or gender.”
It’s pretty easy to argue with the claim that solving cancer or aging is good.
“Science advances one funeral at a time.” replace science with culture if you want.
I think the truth is the only way you can ensure technology is good is if you live in a non chaotic world, like a total dictatorship. We live in a chaotic world. Without a well defined, very high % enforceable social contract, you are simply guessing what might happen.
Broadly in agreement but I don’t think the examples are actually the pure version of what he is saying. It sounds like he is classifying goods based on the ratio of private benefit/social effect ish, so yes things with a very high private benefit and plausibly low externalities are definitely good but I feel less confident than him we could say that about refrigerators for instance.
David Manheim’s If AI is normal technology, history is not reassuring is a good read (emphasis mine):
3 more examples via ChatGPT/Manheim (which provided 5 including the 2 above; I omitted them—again emphasis mine):
Manheim’s argument for what makes good technologies good:
I find myself instinctively resisting Manheim’s explanation, since I’m generally keen on improvements to empowerment and coordination, but have to admit it parsimoniously explains the small-n historical track record above. The issue, as always, seems to be the gap between the beautiful ideal (“more empowerment! more coordination!”) and the unavoidably-messy realities of implementation, people being people, etc.
Tangentially, this reminds me a bit of Holden Karnofsky’s maximally-conservative utopia, which is just “status quo minus clearly-bad things”.
It’s pretty easy to argue with the claim that solving cancer or aging is good.
“Science advances one funeral at a time.” replace science with culture if you want.
I think the truth is the only way you can ensure technology is good is if you live in a non chaotic world, like a total dictatorship. We live in a chaotic world. Without a well defined, very high % enforceable social contract, you are simply guessing what might happen.
What do you think of Manheim’s simple explanation for what makes good technologies good?
Broadly in agreement but I don’t think the examples are actually the pure version of what he is saying. It sounds like he is classifying goods based on the ratio of private benefit/social effect ish, so yes things with a very high private benefit and plausibly low externalities are definitely good but I feel less confident than him we could say that about refrigerators for instance.