2. I agree I’m assuming there will be a slow takeoff (operationalized as let’s say a ~one year period where GPT-integer-increment-level-changes happen on a scale of months, before any such period where they happen on a scale of days).
3. AI companies being open to persuasion seems kind of trivial to me. They already have alignment teams. They already (I assume) have budget meetings where they discuss how many resources these teams should get. I’m just imagining inputs into this regular process. I agree that issues around politics could be a lesser vs. greater input.
1. I wouldn’t frame this as alignment is easy/hard, so much as “alignment is more refractory to 10,000 copies of GPT-6 working for a subjective century” vs. “alignment is more refractory to one genius, not working at a lab, coming up with a new paradigm using only current or slightly-above-current AIs as model organisms, in a sense where we get one roll at this per calendar year”.
Not to tell you what to do, but I’d love to see a longer ACX post making these arguments Scott :). Seems like it could be rich seed for discussion; almost all the writing I’ve seen in rationalist spaces around these issues has been anti-build-goodwill/influence-from-within, and I think your counterpoints here are strong
2. I agree I’m assuming there will be a slow takeoff (operationalized as let’s say a ~one year period where GPT-integer-increment-level-changes happen on a scale of months, before any such period where they happen on a scale of days).
3. AI companies being open to persuasion seems kind of trivial to me. They already have alignment teams. They already (I assume) have budget meetings where they discuss how many resources these teams should get. I’m just imagining inputs into this regular process. I agree that issues around politics could be a lesser vs. greater input.
1. I wouldn’t frame this as alignment is easy/hard, so much as “alignment is more refractory to 10,000 copies of GPT-6 working for a subjective century” vs. “alignment is more refractory to one genius, not working at a lab, coming up with a new paradigm using only current or slightly-above-current AIs as model organisms, in a sense where we get one roll at this per calendar year”.
Not to tell you what to do, but I’d love to see a longer ACX post making these arguments Scott :). Seems like it could be rich seed for discussion; almost all the writing I’ve seen in rationalist spaces around these issues has been anti-build-goodwill/influence-from-within, and I think your counterpoints here are strong