I was also using the nuclear war example just to illustrate my argument. You could substitute in any other catastrophe/extinction event caused by violent actions of humans. Again, the same idea that “human nature” is variable and (most importantly) malleable would suggest that the potential for this extinction event provides relatively little evidence about the value of the long-term. And I think the same would go for anything else determined by other aspects of human psychology, such as short-sightedness rather than violence (e.g., ignoring consequences of AI advancement or carbon emissions), because again that wouldn’t show we’re irredeemably short-sighted.
Your mention of “one’s beliefs about technological development” does make me realise I’d focused only on what the potential for an extinction event might reveal about human psychology, not what it might reveal about other things. But most relevant other things that come to mind seem to me like they’d collapse back to human psychology, and thus my argument would still apply in just somewhat modified form. (I’m open to hearing suggestions of things that wouldn’t, though.)
For example, the laws of physics seem to me likely to determine the limits of technological development, but not whether it’s tendency to be “good” or “bad”. That seems much more up to us and our psychology, and thus it’s a tendency that could change if we change ourselves. Same goes for things like whether institutions are typically effective; that isn’t a fixed property of the world, but rather a result of our psychology (as well as our history, current circumstances, etc.), and thus changeable, especially over very long time scales.
The main way I can imagine I could be wrong is if we do turn out to be essentially unable to substantially shift human psychology. But it seems to me extremely unlikely that that’d be the case over a long time scale and if we’re willing to do things like changing our biology if necessary (and obviously with great caution).
Very interesting post.
But it seems to me that this argument assumes a relatively stable, universal, and fixed “human nature”, and that that’s a quite questionable assumption.
For example, the fact that a person was going to start a nuclear war that would’ve wiped out humanity may not give much evidence about how people tend to behave if in reality behaviours are quite influenced by situations. Nor would it give much evidence about how people in general tend to behave if behaviours vary substantially between different people. Even if behavioural patterns are quite stable and universal, if they’re at least quite manipulable then the fact that person would’ve started that war only gives strong evidence about current behavioural tendencies, not what we’re stuck with in the long term. (I believe this is somewhat similar to Cameron_Meyer_Shorb’s point.)
Under any of those conditions, the fact that person would’ve started that war provides little evidence about typical human behavioural patterns in the long term, and thus little evidence about the potential value of the long-term.
I suspect that there’s at least some substantial stability and universality to human behaviours. But on the other hand there’s certainly evidence that situational factors often important and that different people vary substantially (https://www.ncbi.nlm.nih.gov/pubmed/20550733).
Personally, I suspect the most important factor is how manipulable human behavioural patterns are. The article cited above seems to show a huge degree to which “cultural” factors influence many behavioural patterns, even things we might assume are extremely basic or biologically determined like susceptibility to optical illusions. And such cultural factors typically aren’t even purposeful interventions, let alone scientific ones.
It’s of course true that a lot of scientific efforts to change behaviours fail, and even when they succeed they typically don’t succeed for everyone. But some things have worked on average. And the social sciences working on behavioural change are very young in the scheme of things, and their methods and theories are continually improving (especially after the replication crisis).
Thus, it seems very plausible to me that even within decade we could develop very successful methods of tempering violent inclinations, and that in centuries far more could be done. And that’s all just focusing on our “software”—efforts focusing on our biology itself could conceivably accomplish far more radical changes. That, of course, if we don’t wipe ourselves out before this can be done.
I recently heard someone on the 80,000 Hours podcast (can’t remember who or which episode, sorry) discussing the idea that we may not yet be ready, in terms of our “maturity” or wisdom, for some of the technologies that seem to be around the corner. They gave the analogy that we might trust a child to have scissors but not an assault rifle. (That’s a rough paraphrasing.)
So I think there’s something to your argument, but I’d also worry that weighting it too heavily would be somewhat akin to letting the child keep the gun based on the logic that, if something goes wrong, that shows the child would’ve always been reckless anyway.