What makes you say this? I agree that it is likely that Aschenbrennerâs report was influential here, but did we make Aschenbrenner write chapter IIId of Situational Awareness the way he did?
But the background work predates Leopoldâs involvement.
Is there some background EA/âaligned work that argues for an arms race? Because the consensus seems to be against starting a great power war.
I think a non-trivial fraction of Aschenbrennerâs influence as well as intellectual growth is due to us and the core EA/âAI-Safety ideas, yeah. I doubt he would have written it if the extended community didnât exist, and if he wasnât mentored by Holden, etc.
I donât disagree with this at all. But does this mean that blame can be attributed to the entire EA community? I think not.
Re mentorship/âfunding: I doubt that his mentors were hoping that he would accelerate the chances of an arms race conflict. As a corollary, I am sure nukes wouldnât have been developed if the physics community in the 1930s didnât exist or mentored different people or adopted better ethical norms. Even if they did the latter, it is unclear if that would have prevented the creation of the bomb.
(I found your comments under Ben Westâs posts insightful; if true, it highlights a divergence between the beliefs of the broader EA community and certain influential EAs in DC and AI policy circles.)
Currently, it is just a report, and I hope it stays that way.
For as long as itâs existed the âAI safetyâ movement has been trying to convince people that superintelligent AGI is imminent and immensely powerful. You canât act all shocked pikachu that some people would ignore the danger warnings and take that as a cue to build it before someone else does. This was all quite a predictable result of your actions.
I have no idea what you are advocating for here. I have no inherent interest in trying to convince people that AGI is likely powerful, but it does seem likely true. Should I lie to people?
Many have chosen the path of keeping their beliefs to themselves. My guess is that wasnât very helpful as the âimminent and powerfulâ part is kind of obvious as it starts happening.
What is the predictable result here? What is the counterfactual? How does anything better happen if you donât say anything, and why are you falsely claiming that itâs been consensus that itâs a good idea to publicly talk about the power and capabilities of AI systems? A substantial fraction of the AI safety movement did not do this, and indeed strongly advocated against (again, I think mistakenly), so even if you assign blame, you obviously canât assign blame uniformly.
Do you see advocating for export controls as fundamentally different from an arms race? Because it seems like export controls are pretty popular among AI policy people.
I do think they are a sufficient condition to kick off an arms race. Export controls are a declaration of hostility, and they force the two countries to decouple from each other. China and the US being decoupled makes the downside of an arms race much lower and the upside much higher.
Itâs hard to justify export controls unless you believe that we actually are in an arms race, sooner or later. If you wanted to prevent an arms race you couldnât pick a worse policy to put your weight behind. That leads me to conclude that AI policy people who back export controls find an arms race to be acceptable.
I donât think something as strong as this, but I did think at the time that the work on export controls was bad and likely to exacerbate arms race dynamics, and continue to believe this (and the celebration of export controls as a great success of the EA policy efforts was one of the things that caused me to update on future EA-driven AI policy efforts probably being net harmful, though FTX played a bigger role).
What makes you say this? I agree that it is likely that Aschenbrennerâs report was influential here, but did we make Aschenbrenner write chapter IIId of Situational Awareness the way he did?
Is there some background EA/âaligned work that argues for an arms race? Because the consensus seems to be against starting a great power war.
I think a non-trivial fraction of Aschenbrennerâs influence as well as intellectual growth is due to us and the core EA/âAI-Safety ideas, yeah. I doubt he would have written it if the extended community didnât exist, and if he wasnât mentored by Holden, etc.
I donât disagree with this at all. But does this mean that blame can be attributed to the entire EA community? I think not.
Re mentorship/âfunding: I doubt that his mentors were hoping that he would accelerate the chances of an arms race conflict. As a corollary, I am sure nukes wouldnât have been developed if the physics community in the 1930s didnât exist or mentored different people or adopted better ethical norms. Even if they did the latter, it is unclear if that would have prevented the creation of the bomb.
(I found your comments under Ben Westâs posts insightful; if true, it highlights a divergence between the beliefs of the broader EA community and certain influential EAs in DC and AI policy circles.)
Currently, it is just a report, and I hope it stays that way.
For as long as itâs existed the âAI safetyâ movement has been trying to convince people that superintelligent AGI is imminent and immensely powerful. You canât act all shocked pikachu that some people would ignore the danger warnings and take that as a cue to build it before someone else does. This was all quite a predictable result of your actions.
I have no idea what you are advocating for here. I have no inherent interest in trying to convince people that AGI is likely powerful, but it does seem likely true. Should I lie to people?
Many have chosen the path of keeping their beliefs to themselves. My guess is that wasnât very helpful as the âimminent and powerfulâ part is kind of obvious as it starts happening.
What is the predictable result here? What is the counterfactual? How does anything better happen if you donât say anything, and why are you falsely claiming that itâs been consensus that itâs a good idea to publicly talk about the power and capabilities of AI systems? A substantial fraction of the AI safety movement did not do this, and indeed strongly advocated against (again, I think mistakenly), so even if you assign blame, you obviously canât assign blame uniformly.
Do you see advocating for export controls as fundamentally different from an arms race? Because it seems like export controls are pretty popular among AI policy people.
I honestly donât know. When I think of an arms race, I typically think of rapid manufacturing and accumulation of âweapons.â
Do you think export controls between two countries are a sufficient condition for an arms race?
I do think they are a sufficient condition to kick off an arms race. Export controls are a declaration of hostility, and they force the two countries to decouple from each other. China and the US being decoupled makes the downside of an arms race much lower and the upside much higher.
Itâs hard to justify export controls unless you believe that we actually are in an arms race, sooner or later. If you wanted to prevent an arms race you couldnât pick a worse policy to put your weight behind. That leads me to conclude that AI policy people who back export controls find an arms race to be acceptable.
I donât think something as strong as this, but I did think at the time that the work on export controls was bad and likely to exacerbate arms race dynamics, and continue to believe this (and the celebration of export controls as a great success of the EA policy efforts was one of the things that caused me to update on future EA-driven AI policy efforts probably being net harmful, though FTX played a bigger role).