I was quietly thinking that, in the AI safety space, advocacy is more important than research at this point, precisely for the reasons you outline in the ‘research vs lobbying section’. Thank you for giving a voice on the EA forum to these concerns of mine!
I would tentatively add that we already know that policy solutions (e.g. AI pause, global cooperation, mandating safety-first approaches to AI products) would be most effective in mitigating most X- and S- risks from AI (as opposed to finding technical solutions to the risks), but because of the general belief that policy is less tractable than technical improvement, we observe a lot more energy and funding being directed towards technical research into making AI products safer (which, as you say, isn’t a sinecure because you can find technical solutions all you want, they won’t necessarily be implemented at scale unless you have robust legislation making this compulsory!) rather than policy.
However, this is a self-fulfilling prophecy. The less people believe policy is tractable, the less they invest in it, and the less tractable it actually is.
I’m interested in the reasons this happens and I can think of two possible ones:
Negative bias towards humanities: I believe with moderate certainty that the EA movement is unfortunately not immune to some of the broader negative bias towards humanities that we’ve noticed in recent years. This would explain the funding asymmetry you describe.
Talent constraint: A perhaps more palatable way of explaining the funding asymmetry would be that the EA movement really lacks excellent people in fields like policy, and is unwilling to invest in entreprises that rely on outstanding talent it doesn’t have.[1]
I have actually made some recommendations on how this could be mitigated which you can read about here. Separately, because of all the recently made redundant policy specialist at e.g. USAID, there’s probably an opportunity to reverse that gap quite quickly if the movement becomes more receptive to policy/lobbying as a priority.
Thank you for this post!
I was quietly thinking that, in the AI safety space, advocacy is more important than research at this point, precisely for the reasons you outline in the ‘research vs lobbying section’. Thank you for giving a voice on the EA forum to these concerns of mine!
I would tentatively add that we already know that policy solutions (e.g. AI pause, global cooperation, mandating safety-first approaches to AI products) would be most effective in mitigating most X- and S- risks from AI (as opposed to finding technical solutions to the risks), but because of the general belief that policy is less tractable than technical improvement, we observe a lot more energy and funding being directed towards technical research into making AI products safer (which, as you say, isn’t a sinecure because you can find technical solutions all you want, they won’t necessarily be implemented at scale unless you have robust legislation making this compulsory!) rather than policy.
However, this is a self-fulfilling prophecy. The less people believe policy is tractable, the less they invest in it, and the less tractable it actually is.
I’m interested in the reasons this happens and I can think of two possible ones:
Negative bias towards humanities: I believe with moderate certainty that the EA movement is unfortunately not immune to some of the broader negative bias towards humanities that we’ve noticed in recent years. This would explain the funding asymmetry you describe.
Talent constraint: A perhaps more palatable way of explaining the funding asymmetry would be that the EA movement really lacks excellent people in fields like policy, and is unwilling to invest in entreprises that rely on outstanding talent it doesn’t have.[1]
I have actually made some recommendations on how this could be mitigated which you can read about here. Separately, because of all the recently made redundant policy specialist at e.g. USAID, there’s probably an opportunity to reverse that gap quite quickly if the movement becomes more receptive to policy/lobbying as a priority.