Going to push back a bit on the idea that preventing AI x-risk might be net negative as a reason for not trying to do this. Applying the reversal test: given the magnitude of expected values either way, positive or negative, from x-risk reduction, it would be an extreme coincidence if doing nothing was the best action. Why not work to increase x-risk? I appreciate there are other second order effects that may be overwhelming—the bad optics of being viewed as a “supervillain” for one! But this perhaps is a useful intuition pump for why in fact decreasing x-risk is likely to actually be +EV.
Going to push back a bit on the idea that preventing AI x-risk might be net negative as a reason for not trying to do this. Applying the reversal test: given the magnitude of expected values either way, positive or negative, from x-risk reduction, it would be an extreme coincidence if doing nothing was the best action. Why not work to increase x-risk? I appreciate there are other second order effects that may be overwhelming—the bad optics of being viewed as a “supervillain” for one! But this perhaps is a useful intuition pump for why in fact decreasing x-risk is likely to actually be +EV.