This post makes argues for three takeaways—“Make sure you’re actually in a race”, “Be careful about secrecy”, and that scientists shouldn’t give up power easily—primarily based on two case studies where people overestimated the extent to which they were in a technological race against their enemies.
However, in the middle of the post there is a caveat:
There are, of course, examples of mistaken intelligence in the other direction, that is overestimates of how long an adversary would take to achieve a capability—for example, the US estimate that the Soviets would take a decade to build a nuclear bomb. And there are examples of correct intelligence about other’s capabilities and intentions, for example perhaps the Dreadnought programme in the early 1900s.
This seems noteworthy, since it’s some evidence that people don’t have a general bias towards overestimating their enemies. However, these cases are, as far as I can see, not discussed in the discussion about takeaways.
I think it can be tricky to draw general lessons from case studies. In this case, it seems that both of the cases in focus supported a particular view, whereas cases that supported the opposite view were only briefly mentioned; and did not affect the conclusions. I think it could be better to give an overview of all relevant historical examples and try to establish what the overall pattern is (whether people generally overestimate their enemies, and the extent to which they’re in a race, or not).
You’re quite right that if this post were arguing that there is an overall pattern, it would quite clearly be inadequate. It doesn’t define the universe of cases or make clear how representative these cases are of that universe, the two main studies could be criticised for selecting on the dependent variable, and its based primarily on quotes from two books.
However, I didn’t set out to answer something like the research question “which is more common in 20th century history, mistakenly sprinting or mistakenly failing to sprint?”—though I think that’s a very interesting question, and would like someone to look into it!
My intention for this blog post was for it to be fairly clear and memorable, aimed at a general audience—especially perhaps a machine learning researcher who doesn’t know much about history. The main takeaway I wanted wasn’t for people to think “this is the most common/likely outcome” but rather to add a historic example to their repertoire that they can refer to—“this was an outcome”. It was supposed to be a cautionary tale, a prompt to people to think not “all sprints are wrong” but rather “wait am I in an Ellsberg situation?”—and if so to have some general, sensible recommendations and questions to ask.
My aim was to express a worry (“be careful about mistaken sprints”) and illustrate that with two clear, memorable stories. There’s a reasonable scenario in the next few decades that we’re in a situation where we feel we need to back a sprint, prompted by concern about another group/country’s sprint. If we do, and I’m not around to say “hey lets be careful about this and check we’re actually in a race” then I hope these two case studies may stick in someone’s mind and lead them to say “OK but lets just check, don’t want to make the same mistake as Szilard and Ellsberg...”
This post makes argues for three takeaways—“Make sure you’re actually in a race”, “Be careful about secrecy”, and that scientists shouldn’t give up power easily—primarily based on two case studies where people overestimated the extent to which they were in a technological race against their enemies.
However, in the middle of the post there is a caveat:
This seems noteworthy, since it’s some evidence that people don’t have a general bias towards overestimating their enemies. However, these cases are, as far as I can see, not discussed in the discussion about takeaways.
I think it can be tricky to draw general lessons from case studies. In this case, it seems that both of the cases in focus supported a particular view, whereas cases that supported the opposite view were only briefly mentioned; and did not affect the conclusions. I think it could be better to give an overview of all relevant historical examples and try to establish what the overall pattern is (whether people generally overestimate their enemies, and the extent to which they’re in a race, or not).
Hi Stefan,
Thanks for this response.
You’re quite right that if this post were arguing that there is an overall pattern, it would quite clearly be inadequate. It doesn’t define the universe of cases or make clear how representative these cases are of that universe, the two main studies could be criticised for selecting on the dependent variable, and its based primarily on quotes from two books.
However, I didn’t set out to answer something like the research question “which is more common in 20th century history, mistakenly sprinting or mistakenly failing to sprint?”—though I think that’s a very interesting question, and would like someone to look into it!
My intention for this blog post was for it to be fairly clear and memorable, aimed at a general audience—especially perhaps a machine learning researcher who doesn’t know much about history. The main takeaway I wanted wasn’t for people to think “this is the most common/likely outcome” but rather to add a historic example to their repertoire that they can refer to—“this was an outcome”. It was supposed to be a cautionary tale, a prompt to people to think not “all sprints are wrong” but rather “wait am I in an Ellsberg situation?”—and if so to have some general, sensible recommendations and questions to ask.
My aim was to express a worry (“be careful about mistaken sprints”) and illustrate that with two clear, memorable stories. There’s a reasonable scenario in the next few decades that we’re in a situation where we feel we need to back a sprint, prompted by concern about another group/country’s sprint. If we do, and I’m not around to say “hey lets be careful about this and check we’re actually in a race” then I hope these two case studies may stick in someone’s mind and lead them to say “OK but lets just check, don’t want to make the same mistake as Szilard and Ellsberg...”