Thank you for the thoughtful reply. I think I largely agree with your point that different roles do not require the same depth of technical knowledge, and that a strong generalist can often develop a sufficiently deep understanding of a field while contributing to it.
I would go even further: someone may have only ordinary overall abilities, but if they possess some degree of cross-disciplinary knowledge, the capacity to keep learning and revising their understanding, or even a genuine willingness to develop these qualities, I believe there may still be a meaningful place for them in AI safety. That place is not merely something conferred by the existing field or its institutions; it may arise from the fact that such a person is genuinely well suited to contributing to a complex and still-emerging domain.
I may not have expressed my original point clearly enough. I was not arguing that every person working in AI safety must first become an expert in AI research, nor that operations, management, communications, or policy roles require the same depth as technical research roles.
My concern is at a different level. It is not mainly about whether each individual contributor understands AI deeply enough. It is about whether the field as a whole has a sufficiently reliable understanding of what AI is before it defines the safety problem and commits heavily to particular solutions.
Generalists can rely on specialists, and specialists can divide the work among themselves. But this works only if the specialist community has not collectively overlooked a foundational issue. If the field has an incomplete model of the object it is trying to make safe, then adding more capable people may accelerate useful work, but it may also accelerate work within an incomplete framing.
For example, much of current AI safety work focuses on controlling outputs, behavior, incentives, or capabilities. These may all be necessary. My question is whether they are sufficient if the underlying cognitive architecture itself contains risks that have not been isolated at the design level.
So I do not disagree that talented generalists can make important contributions—quite the opposite. Good generalists may be especially valuable because they can connect different areas, notice gaps between specialties, and help identify assumptions that individual experts may take for granted.
My original comment was therefore less about setting a high entry barrier for people, and more about maintaining epistemic humility at the level of the field: strong commitment and rapid progress are valuable, but they should not be mistaken for evidence that we already understand the object of safety well enough.
> But this works only if the specialist community has not collectively overlooked a foundational issue. If the field has an incomplete model of the object it is trying to make safe, then adding more capable people may accelerate useful work, but it may also accelerate work within an incomplete framing.
Thanks! I largely agree, and would add 2 points onto what you you said here: 1. In my opinion, the goal of adding generalists are to take work off specialists plates that may not be delivering the most value for their capabilities, may not be their largest strengths, etc. That in turn should free up more of their time focus on the toughest questions, like: Are we on the right path? 2. The power of generalists are the diverse careers and passions (outside of work too) they carry. As we look at new problems, we stitch all of that history and context in. Sometimes that generates novel ideas or questions that haven’t been asked. In short, bringing in generalists brings diversity and expansion of potential ideas, as well as how to execute work.
Thank you for the thoughtful reply. I think I largely agree with your point that different roles do not require the same depth of technical knowledge, and that a strong generalist can often develop a sufficiently deep understanding of a field while contributing to it.
I would go even further: someone may have only ordinary overall abilities, but if they possess some degree of cross-disciplinary knowledge, the capacity to keep learning and revising their understanding, or even a genuine willingness to develop these qualities, I believe there may still be a meaningful place for them in AI safety. That place is not merely something conferred by the existing field or its institutions; it may arise from the fact that such a person is genuinely well suited to contributing to a complex and still-emerging domain.
I may not have expressed my original point clearly enough. I was not arguing that every person working in AI safety must first become an expert in AI research, nor that operations, management, communications, or policy roles require the same depth as technical research roles.
My concern is at a different level. It is not mainly about whether each individual contributor understands AI deeply enough. It is about whether the field as a whole has a sufficiently reliable understanding of what AI is before it defines the safety problem and commits heavily to particular solutions.
Generalists can rely on specialists, and specialists can divide the work among themselves. But this works only if the specialist community has not collectively overlooked a foundational issue. If the field has an incomplete model of the object it is trying to make safe, then adding more capable people may accelerate useful work, but it may also accelerate work within an incomplete framing.
For example, much of current AI safety work focuses on controlling outputs, behavior, incentives, or capabilities. These may all be necessary. My question is whether they are sufficient if the underlying cognitive architecture itself contains risks that have not been isolated at the design level.
So I do not disagree that talented generalists can make important contributions—quite the opposite. Good generalists may be especially valuable because they can connect different areas, notice gaps between specialties, and help identify assumptions that individual experts may take for granted.
My original comment was therefore less about setting a high entry barrier for people, and more about maintaining epistemic humility at the level of the field: strong commitment and rapid progress are valuable, but they should not be mistaken for evidence that we already understand the object of safety well enough.
> But this works only if the specialist community has not collectively overlooked a foundational issue. If the field has an incomplete model of the object it is trying to make safe, then adding more capable people may accelerate useful work, but it may also accelerate work within an incomplete framing.
Thanks! I largely agree, and would add 2 points onto what you you said here:
1. In my opinion, the goal of adding generalists are to take work off specialists plates that may not be delivering the most value for their capabilities, may not be their largest strengths, etc. That in turn should free up more of their time focus on the toughest questions, like: Are we on the right path?
2. The power of generalists are the diverse careers and passions (outside of work too) they carry. As we look at new problems, we stitch all of that history and context in. Sometimes that generates novel ideas or questions that haven’t been asked. In short, bringing in generalists brings diversity and expansion of potential ideas, as well as how to execute work.