When tangling with entities much smarter than us, I’m sure we get screwed somewhere along the line.
This is probably the key point of disagreement. You seem to be “sure” that catastrophic outcomes happen when individual AIs are misaligned, whereas I’m saying “It could happen, but I don’t think the case for that is strong”. I don’t see how a high level of confidence can be justified given the evidence you’re appealing to. This seems like a highly speculative thesis.
Also, note that my argument here is meant as a final comment in my section about AI optimism. I think the more compelling argument is that AIs will probably care for humans to a large degree. Alignment might be imperfect, but it sounds like to get the outcomes you’re talking about, we need uniformity and extreme misalignment among AIs, and I don’t see why we should think that’s particularly likely given the default incentives of AI companies.
This is probably the key point of disagreement. You seem to be “sure” that catastrophic outcomes happen when individual AIs are misaligned, whereas I’m saying “It could happen, but I don’t think the case for that is strong”. I don’t see how a high level of confidence can be justified given the evidence you’re appealing to. This seems like a highly speculative thesis.
Also, note that my argument here is meant as a final comment in my section about AI optimism. I think the more compelling argument is that AIs will probably care for humans to a large degree. Alignment might be imperfect, but it sounds like to get the outcomes you’re talking about, we need uniformity and extreme misalignment among AIs, and I don’t see why we should think that’s particularly likely given the default incentives of AI companies.
“When tangling with entities much smarter than us, I’m sure we get screwed somewhere along the line.”
“This seems like a highly speculative thesis.”
I think it’s more of an anti-prediction tbh.