Bandwagoning onto this sensible post, another problem with this argument is that differential technological development is very fuzzy to reason about, since most of the mechanisms by which it could advance alignment are things that haven’t happened yet. This means it’s possible to reach any conclusion (“this work is good on net”, “this work is bad on net”) and motivated reasoning will make people want to reach the conclusion that the work they are doing is good on net. It’s a classic case of suspicious and surprising convergence.
Bandwagoning onto this sensible post, another problem with this argument is that differential technological development is very fuzzy to reason about, since most of the mechanisms by which it could advance alignment are things that haven’t happened yet. This means it’s possible to reach any conclusion (“this work is good on net”, “this work is bad on net”) and motivated reasoning will make people want to reach the conclusion that the work they are doing is good on net. It’s a classic case of suspicious and surprising convergence.