I appreciated that for the claim I was most skeptical of: “There’s also the basic intuition that more people with new expertise working on a hard problem just seems better”, my skepticism was anticipated and discussed.
For me one of the most important things is:
Patch the gaps that others won’t cover
E.g., if more academics start doing prosaic alignment work, then ‘big-if-true’ theoretical work may become more valuable, or high-quality work on digital sentience.
There’s probably predictable ‘market failures’ in any discipline – work that isn’t sexy but still very useful (e.g., organizing events, fixing coordination problems, distilling the same argument into new language, etc.).
Generally track whether top priority work is getting covered (e.g., information security, standards and monitoring)
This, plus avoiding and calling out safety washing, keeping an eye out for overall wrongheaded activities and motions (for instance, probably a lot of regulation is bad by default and some could actively make things worse), seem like the strongest arguments against making big naive shifts because the field is in a broad sense less neglected.
More generally, I think a lot of the details of what kinds of engagements we have with the broader world will matter (and I think in many cases “guide” will be a less accurate description of what’s on the table than “be one of the players in the room”, and some might be a lot more impactful than others, but I don’t have super fleshed out views on which yet!
I liked this!
I appreciated that for the claim I was most skeptical of: “There’s also the basic intuition that more people with new expertise working on a hard problem just seems better”, my skepticism was anticipated and discussed.
For me one of the most important things is:
This, plus avoiding and calling out safety washing, keeping an eye out for overall wrongheaded activities and motions (for instance, probably a lot of regulation is bad by default and some could actively make things worse), seem like the strongest arguments against making big naive shifts because the field is in a broad sense less neglected.
More generally, I think a lot of the details of what kinds of engagements we have with the broader world will matter (and I think in many cases “guide” will be a less accurate description of what’s on the table than “be one of the players in the room”, and some might be a lot more impactful than others, but I don’t have super fleshed out views on which yet!