[weirdness-filter: ur weird if you read m commnt n agree w me lol]
Doing private capabilities research seems not obviously net-bad, for some subcategories of capabilities research. It constrains your expectations about how AGI will unfold, meaning you have a narrower target for your alignment ideas (incl. strategies, politics, etc.) to hit. The basic case: If an alignment researcher doesn’t understand how gradient descent works, I think they’re going to be less effective at alignment. I expect this to generalise for most advances they could make in their theoretical understanding of how to build intelligences. And there’s no fundamental difference between learning the basics and doing novel research, as it all amounts to increased understanding in the end.
That said, it would in most cases be very silly to publish about that increased understanding, and people should be disincentivised from doing so.
(I’ll delete this comment if you’ve read it and you want it gone. I think the above can be very bad advice to give some poorly aligned selfish researchers, but I want reasonable people to hear it.)
[weirdness-filter: ur weird if you read m commnt n agree w me lol]
Doing private capabilities research seems not obviously net-bad, for some subcategories of capabilities research. It constrains your expectations about how AGI will unfold, meaning you have a narrower target for your alignment ideas (incl. strategies, politics, etc.) to hit. The basic case: If an alignment researcher doesn’t understand how gradient descent works, I think they’re going to be less effective at alignment. I expect this to generalise for most advances they could make in their theoretical understanding of how to build intelligences. And there’s no fundamental difference between learning the basics and doing novel research, as it all amounts to increased understanding in the end.
That said, it would in most cases be very silly to publish about that increased understanding, and people should be disincentivised from doing so.
(I’ll delete this comment if you’ve read it and you want it gone. I think the above can be very bad advice to give some poorly aligned selfish researchers, but I want reasonable people to hear it.)