Error
Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?
Unrecognized LW server error:
Field "fmCrosspost" of type "CrosspostOutput" must have a selection of subfields. Did you mean "fmCrosspost { ... }"?
Since writing this post, Iāve learned of two additional approaches to risk analysis that could also likely be usefully applied to existential risks:
The bowtie method
Fault tree analysis
And hereās a generic example of the bowtie method (from the sourced linked to above):
I once created a causal map of all global risks starting from the beginning of evolution and accumulation of biases ā and up to the end. But it included too many high-knotted elements which make the reading of the map difficult. Smaller causal maps with less than 10 elements are better adapted for human understanding.
These diagrams look really useful for encouraging people to map out potential paths to existential risk and potential interventions more carefully!
Good to hear! We aim to provide useful tools for such work.
Also, for you and others who thought these diagrams looked useful, I do recommend Defence in Depth Against Human Extinction and Long-Term Trajectories of Human Civilization (if you havenāt read them already). This and those two papers each provide somewhat different visualisations, emphases, and ways of carving up this murky, complex territory, so I think all three sets of ideas are good tools to have in oneās toolkit.
Just a quick partial reaction: Some of your comment reminds me of distinctions in Bostromās original paper on information hazards, e.g. distinctions between idea hazards, data hazards, and attention hazards. You might find this summary post I wrote interesting (assuming you havenāt yet read it or the Bostrom paper).
Great post! I feel these diagrams will be really useful for clarifying the possible interventions and parts of the existential risks.
Do you think theyāll also serve for comparing different positions on a specific existential risk, like the trajectories in this post? Or do you envision the diagram for a specific risk as a summary of all causal pathways to this risk?
Thanks! I hope so.
By ācomparing different positions on a specific existential riskā, it seems to me that you could mean either:
1. Comparing what different āstagesā of a specific risk would be like
e.g., comparing what itād be like if weāre at the āimplementation of hazardous informationā vs āharmful eventsā stage of a engineered pathogen risk
2. Comparing different peopleās views on what stage a specific risk is currently at
e.g., identifying that one person believes the information required to develop an engineered pathogen just hasnāt been developed, while another believes that itās been developed but has yet to be shared or implemented
3. Comparing different peopleās views on a specific risk more generally
e.g., identifying that two people roughly agree on the chances an engineered pathogen could be developed, but disagree on how likely it is that itād be implemented or that a resulting outbreak would result in collapse/āextinction, and thatās why they overall disagree about the risk levels
e.g., identifying that two people roughly agree on the overall risks from an engineered pandemic, but this obscures the fact that they disagree, in ways that roughly cancel out, on the probabilities of progression from each stage to the next stage. This could be important because it could help them understand why they advocate for different interventions.
(Note that I just randomly chose to go with pathogen examples hereāas I say in the post, these diagrams can be used for a wide range of risks.)
I think that, if these diagrams can be useful at all (which I hope they can!), they can be useful for 1 and 3. And I think perhaps you had 3 in mind, as thatās perhaps most similar to what the state space model you linked to accomplishes. (Iād guess these models could also be useful for 2, but Iām not sure how often informed people would have meaningful disagreements about what stage a specific risk is currently at.)
Hopefully my examples already make it somewhat clear why I think that these diagrams could help with 1 and 3, and why thatās important. Basically, I think most things that help people make their more of their thinking more explicit, or that prompt/āforce them to do so, will help them identify precisely where they agree and disagree with each other. (I think this also applies to stating oneās probabilities/ācredences explicitly, as I sort of allude to in passing in a few places here.)
Another way to put that is these things will help or make people āfactor outā various inputs into their bottom line conclusions, so we can more easily point to those inputs that seem most uncertain or contestable, or conversely we can realise āOh, thatās actually a great pointāI should add that to my own internal model of the situationā. I think visualisation also generally makes that sort of thing easier and more effective.
And I think these diagrams can also work as āa summary of all causal pathways to this riskā, if Iām interpreting you correctly. For example, you could further flesh out my final diagram from this post (not the Defence in Depth diagram) to represent basically all the major causal pathways to existential catastrophes from bioengineering. And then you could also have people assign probabilities to moving from each stage to each other stage is connects to, or even contest which connections are shown (e.g., suggest how one step could ābypassā what I showed it as connecting to in order to connect to later steps). And then they could debate these things.
(But as I say in the post, I think if we wanted to get quite granular, weād probably want to ultimately use something like Guesstimate. And there are also various other models and approaches we could use to complement these diagrams.)
Thanks for that very in-depth answer!
I was indeed thinking about 3., even if 1. and 2. are also important. And I get that the main value of these diagrams is to force an explicit and as formal as possible statement to be made.
I guess my question was more about, given two different causal diagrams for the same risk (made by different researchers for example), have you an idea of how to compare them? Like finding the first difference along the causal path, or others means of comparison. This seems important because even with clean descriptions of our views, we can still talk past each other if we cannot see where the difference truly lies.
Hmm, Iām not sure I fully understand what you mean. But hopefully the following somewhat addresses it:
One possibility is that two different researchers might have different ideas of what the relevant causal pathways actually are. For a simple example, one researcher might not think of the possibility that a risk could progress right from the initial harms to the existential catastrophe, without a period of civilizational collapse first, or might think of that but dismiss it as not even worth considering because it seems so unlikely. A different researcher might think that that path is indeed worth considering.
If either of the researchers tried to make an explicit causal diagram of how they think the risk could lead to existential catastrophe, the other one would probably notice that their own thoughts on the matter differ. This would likely help them see where the differences in their views lie, and the researcher whoād neglected that path might immediately say āOh, good point, hadnāt thought of that!ā, or they might discuss why that seems worth considering to one of the researchers but not to the other.
(I would guess that in practice this would typically occur for less obvious paths than that, such as specific paths that can lead to or prevent the development/āspread of certain types of information.)
Another possibility is that two different researchers have essentially the same idea of what the relevant causal pathways are, but very different ideas of the probabilities of progression from certain steps to other steps. In that case, merely drawing these diagrams, in the way theyāre shown in this post, wouldnāt be sufficient for them to spot why their views differ.
But having the diagrams in front of them could help them talk through how likely they think each particular path or step is. Or they could each assign an actual probability to each path or step. Either way, they should then be able to see why and where their views differ.
In all of these cases, ideally, the researchers would go beyond just noticing where their views differ and instead discuss why each of them believes what they believe about the point on which they differ.
Does that answer your question?
If by āhow to compare themā you mean āhow to tell which one is betterā, then thatās something that this tool alone canāt do. But by facilitating clear, explicit thought and discussion, this tool could potentially help well-informed people form views about which diagrams/āmodels are more valid or useful.
That answers my question, yes. :)
Oh, just realised that I mentioned I hope to follow-up with those directions for future work at some point, and that itād also be great for others to do so, but I didnāt mention a third option: If anyoneās interested to collaborate with me on work along those linesāwhich could perhaps be as simple as making one, more fleshed out diagram for a specific riskāI might be keen for that.
In particular, if you have expertise relevant to a particular risk (e.g., AI safety, machine learning more generally, epidemiology, nanotech), collaborating on a fleshed out diagram for that risk could be really interesting. Likewise if you know a lot about one step of the causal path or one intervention type, e.g. civilizational collapse or recovery, or ways there could be a continuous progression directly from a harmful event to an existential catastrophe.