One thing you don’t really seem to be taking into account is inner alignment failure / goal misgeneralisation / mesaoptimisation. Why don’t you think this will happen?
I think we have doom by default for a number of independent disjunctive reasons. And by “default” I mean “if we keep developing AGI at the rate we currently are, without an indefinite global pause” (regardless of how many resources are poured into x-safety, there just isn’t enough time to solve it without a pause).
One thing you don’t really seem to be taking into account is inner alignment failure / goal misgeneralisation / mesaoptimisation. Why don’t you think this will happen?
I think we have doom by default for a number of independent disjunctive reasons. And by “default” I mean “if we keep developing AGI at the rate we currently are, without an indefinite global pause” (regardless of how many resources are poured into x-safety, there just isn’t enough time to solve it without a pause).