I think that the best work on AI alignment happens at the AGI labs
Based on your other discussion e.g. about public pressure on labs, it seems like this might be a (minor?) loadbearing belief?
I appreciate that you qualify this further in a footnote
This is a controversial view, but I’d guess it’s a majority opinion amongst AI alignment researchers.
I just wanted to call out that I weakly hold the opposite position, and also opposite best guess on majority opinion (based on safety researchers I know). Naturally there are sampling effects!
This is a marginal sentiment, and I certainly wouldn’t trade all lab researchers for non-lab researchers or vice versa. Diversification of research settings seems quite precious, and the dialogue is important to preserve.
I also question
Reasons include: access to the best alignment talent,
because a lot of us are very reluctant to join AGI labs, for obvious reasons! I know folks inside and outside of AGI labs, and it seems to me that the most talented are among the outsiders (but this also definitely could be an artefact of sample sizes).
it seems like this might be a (minor?) loadbearing belief?
Yes, if I changed my mind about this I’d have to rethink my position on public advocacy. I’m still pretty worried about the other disadvantages so I suspect it wouldn’t change my mind overall, but I would be more uncertain.
Based on your other discussion e.g. about public pressure on labs, it seems like this might be a (minor?) loadbearing belief?
I appreciate that you qualify this further in a footnote
I just wanted to call out that I weakly hold the opposite position, and also opposite best guess on majority opinion (based on safety researchers I know). Naturally there are sampling effects!
This is a marginal sentiment, and I certainly wouldn’t trade all lab researchers for non-lab researchers or vice versa. Diversification of research settings seems quite precious, and the dialogue is important to preserve.
I also question
because a lot of us are very reluctant to join AGI labs, for obvious reasons! I know folks inside and outside of AGI labs, and it seems to me that the most talented are among the outsiders (but this also definitely could be an artefact of sample sizes).
Yes, if I changed my mind about this I’d have to rethink my position on public advocacy. I’m still pretty worried about the other disadvantages so I suspect it wouldn’t change my mind overall, but I would be more uncertain.