If timelines are long... ...we can prioritize work that takes a long time to complete
In short timelines, we might have a small amount of calendar time with which to work on more difficult tasks, but a large amount of thinking/research timecourtesy of automated AI researchers / research assistants. This might be important to the value of things like foundational research under short timelines.
If we’re on track to solving AI alignment... ...the shape of the future will be determined by an aligned ASI
Solving the alignment problem does not necessarily mean “the shape of the future will be determined by an aligned ASI”. For example, an aligned earth-originating ASI might run into a more powerful alien AI; and the shape of the future would be determined by the alien AI. In that case, we care about the long-term effects of earth-originating AI only insofar as they influence the decisions of the alien one. More prosaically, if we solve intent-alignment but someone uses this to launch acoup the resulting ASI might not be aligned to the interests of all humanity / good values.
If we’re not on track to solving AI alignment... ...none of those other types of work listed above will pay off
This isn’t always true. For example, decision theory research could informinterventions on unaligned ASIsthat make the future go better for sentient beings in expectation, without us having solved the full alignment problem.
Some objections:
In short timelines, we might have a small amount of calendar time with which to work on more difficult tasks, but a large amount of thinking/research time courtesy of automated AI researchers / research assistants. This might be important to the value of things like foundational research under short timelines.
Solving the alignment problem does not necessarily mean “the shape of the future will be determined by an aligned ASI”. For example, an aligned earth-originating ASI might run into a more powerful alien AI; and the shape of the future would be determined by the alien AI. In that case, we care about the long-term effects of earth-originating AI only insofar as they influence the decisions of the alien one. More prosaically, if we solve intent-alignment but someone uses this to launch a coup the resulting ASI might not be aligned to the interests of all humanity / good values.
This isn’t always true. For example, decision theory research could inform interventions on unaligned ASIs that make the future go better for sentient beings in expectation, without us having solved the full alignment problem.