There’s been quite a bit written on the “pro” side:
Also ARCHES, Concrete Problems in AI safety, etc
But not so much on the “con” side—people have generally just thought about opportunity cost. Your point that it might speed up harmful (due to safety, misuse or structural risks) applications is a really useful and important one! Would be hard to weigh things up—getting into tricky differential technological development territory. Would love for there to be more thinking on this topic.