Thank you for taking the time to write this—I think it is a clear and concise entry point into the AGI ruin arguments.
I want to voice an objection / point out an omission to point 2: I agree that any plan towards a sufficiently complicated goal will include “acquire resources” as a sub-goal, and that “getting rid of all humans” might be a by-product of some ways to achieve this sub-goal. I’m also willing to grant that if all we now about the plan is that it achieves the end (sufficiently complicated) goal, it is likely that the plan might lead to the destruction of all humans.
However I don’t see why we can’t infer more about the plans. Specifically I think an ASI plan for a sufficiently complicated goal should be 1) feasible and 2) efficient (at least in some sense). If the ASI doesn’t believe that it can overpower humanity, then it’s plans will not include overpowering humanity. Even more, if the ASI ascribes a high enough cost to overpowering humanity, it would instead opt to acquire resources in another way.
It seems that for point 2 to hold you must think that an ASI can overpower humanity with 1) close to a 100% certainty and 2) at negligible cost to the ASI. However I don’t think this is (explicitly) argued for in the article. Or maybe I’m missing something?
I also think this was a not-so-good and somewhat misleading analogy—the association between Novik and Caroline in the example is strictly one-way (Caroline likes Novik, Novik has no idea who Caroline is), whereas the association between FTX and EA is clearly two-way (e.g. various EA orgs endorsing and promoting SBF, SBF choosing to earn-to-give after talking with 80k etc).