Error

Option value considerations dictate that we continue doing AI safety research even if we’re unsure of its value because it’s much easier to stop a research program than to start one.

I think the opposite is often true. Once there are people who get compensated for doing X it can be very hard to stop X. (Especially if it’s harder for impartial people, who are not experts-in-X, to evaluate X.)

I think this is a pretty important topic, and one I haven’t seen discussed as often as I’d like! Thanks for writing it up.

I think you could get more engagement with this topic if you spent some more time smoothing out the presentation of your writeup. For example, there are a few typos in the summary section that made me less excited to read the rest of the piece. Given that you now have a pretty interesting piece of thinking written, it might be pretty feasible to find a smart junior person who could give you copyedits and comments.