I think the critical crux here is the assumption about human competence, individually and working in groups. And Iām afraid I agree; humans have an optimism bias by many measures. Our track record on doing even easy projects right on the first try (or even the first few tries) is not good.
I also think optimists are often asking the question could we solve alignment, while pessimists are asking will we solve alignment, which includes a lot more practical difficulties so more opportunities for failure.
Of course there are many other relevant cruxes, but I think those two are pretty common and the first is the biggest contribution of this particular contribution.
I think the critical crux here is the assumption about human competence, individually and working in groups. And Iām afraid I agree; humans have an optimism bias by many measures. Our track record on doing even easy projects right on the first try (or even the first few tries) is not good.
I also think optimists are often asking the question could we solve alignment, while pessimists are asking will we solve alignment, which includes a lot more practical difficulties so more opportunities for failure.
Of course there are many other relevant cruxes, but I think those two are pretty common and the first is the biggest contribution of this particular contribution.