I understand some people don’t feel that certain posts are worth engaging with (which is fine) but at least don’t downvote then?
I disagree, I think it’s perfectly fine for people to downvote posts without commenting. The key function of the karma system is to control how many people see a given piece of content, so I think up-/downvotes should reflect “should this content be seen by more people here?” If I think a post clearly isn’t worth reading (and in particular not engaging with), then IMO it makes complete sense to downvote so that fewer other people spend time on it. In contrast, if I disagree with a post but think it’s well-argued and worth engaging with I would not downvote, and would engage in the comments instead.
Models such as the Carlsmith one, which treat AI x-risk as highly conjunctive (i.e. lots of things need to happen for an AI existential catastrophe), already seem like they’ll bias results towards lower probabilities (see e.g. this section of Nate’s review of the Carlsmith report). I won’t say more on this since I think it’s been discussed several times already.
What I do want to highlight is that the methodology of this post exacerbates that effect. In principle, you can get reasonable results with such a model if you’re aware of the dangers of highly conjunctive models, and sufficiently careful in assigning probabilities.[1] This might at least plausibly be the case for a single person giving probabilities, who has hopefully thought about how to avoid the multiple stage fallacy, and spent a lot of time thinking about their probability estimates. But if you just survey a lot of people, you’ll very likely get at least a sizable fraction of responses who e.g. just tend to assign probabilities close to 50% because anything else feels overconfident, or who don’t actually condition enough on previous steps having happened, even if the question tells them to. (This isn’t really meant to critique people who answered the survey—it’s genuinely hard to give good probabilities for these conjunctive models). The way the analysis in this post works, if some people give probabilities that are too low, the overall result will also be very low (see e.g. this comment).
I would strongly guess that if you ran exactly the same type of survey and analysis with a highly disjunctive model (e.g. more along the lines of this one by Nate Soares), you would get way higher probabilities of X-risk. To be clear, that would be just as bad, it would likely be an overestimate!
One related aspect I want to address:
There is a lot of disagreement about whether AI risk is conjunctive or disjunctive (or, more realistically, where it is on the spectrum between the two). If I understand you correctly (in section 3.1), you basically found only one model (Carlsmith) that matched your requirements, which happened to be conjunctive. I’m not sure if that’s just randomness, or if there’s a systematic effect where people with more disjunctive models don’t tend to write down arguments in the style “here’s my model, I’ll assign probabilities and then multiply them”.
If we do want to use a methodology like the one in this post, I think we’d need to take uncertainty over the model itself extremely seriously. E.g. we could come up with a bunch of different models, assign weights to them somehow (e.g. survey people about how good a model of AI x-risk this is), and then do the type of analysis you do here for each model separately. At the end, we average over the probabilities each model gives using our weights. I’m still not a big fan of that approach, but at least it would take into account the fact that there’s a lot of disagreement about the conjunctive vs disjunctive character of AI risk. It would also “average out” the biases that each type of model induces to some extent.
Though there’s still the issue of disjunctive pathways being completely ignored, and I also think it’s pretty hard to be sufficiently careful.