I think I am unsure how long it is possible for an indefinite moratorium to last, but I think I probably fall, and increasingly fall, much closer to supporting it than I guess you do.
In answer to these specific points, I basically seem maintaining a moratorium as an example of Differential Technology Development. As long as the technologies that we can use to maintain a moratorium (both physical and social technologies) outpace the rate of progress towards ASI, we can maintain the moratorium. I do think this would require drastically slowing down a specific subset of scientific progress in the long term, but am not convinced it would be so general as you suggest. I guess this is some mixture of both 1 and 2, although with both I do think this means that neither position ends up being so extreme.
In answer to your normative judgement, if 1 allows a flourishing future, which I think a drastically slowed sense of progress could, then it seems desirable from a longtermist perspective. I’m also really unsure that, with sufficient time, we can’t access significant parts of technology space without an agentic ASI, particularly if we increase our defences against an agentic ASI using technologies like narrow AIs sufficiently. It also strikes me that assigning significant normative value to accessing all areas (or even extremely large areas) of science and technology space seems like a value set that is related to ‘progress’/transhumanism as an end of itself, rather than a means to an end (like totalist utilitarians with transhumanist bents do).
For me, its really hard to tell how long we could hold a moratroium for, and how long its desireable. But certainly, if feasible, it seems timescales well beyond decades would be very desirable
I think I am unsure how long it is possible for an indefinite moratorium to last, but I think I probably fall, and increasingly fall, much closer to supporting it than I guess you do.
In answer to these specific points, I basically seem maintaining a moratorium as an example of Differential Technology Development. As long as the technologies that we can use to maintain a moratorium (both physical and social technologies) outpace the rate of progress towards ASI, we can maintain the moratorium. I do think this would require drastically slowing down a specific subset of scientific progress in the long term, but am not convinced it would be so general as you suggest. I guess this is some mixture of both 1 and 2, although with both I do think this means that neither position ends up being so extreme.
In answer to your normative judgement, if 1 allows a flourishing future, which I think a drastically slowed sense of progress could, then it seems desirable from a longtermist perspective. I’m also really unsure that, with sufficient time, we can’t access significant parts of technology space without an agentic ASI, particularly if we increase our defences against an agentic ASI using technologies like narrow AIs sufficiently. It also strikes me that assigning significant normative value to accessing all areas (or even extremely large areas) of science and technology space seems like a value set that is related to ‘progress’/transhumanism as an end of itself, rather than a means to an end (like totalist utilitarians with transhumanist bents do).
For me, its really hard to tell how long we could hold a moratroium for, and how long its desireable. But certainly, if feasible, it seems timescales well beyond decades would be very desirable