I thought this post was particularly cool because it seems to be applicable to lots of things, at least in theory (I have some concerns in practice). I’m curious about further reviews of this post.
I find myself using the reasoning described in the post in a bunch of places related to the prioritization of longtermist interventions. At the same time, I’m not sure I ever get any useful conclusions out of it. This might be because the area of application (predicting the impact of new technologies in the medium-term future) is particularly challenging. (One could argue that there’s a “Which world gets saved” argument against “Which world gets saved” arguments: In worlds where the consideration is useful, we start out with wide uncertainty. Having wide uncertainty is bad for having lots of impact.)
I wonder if there’s something a bit fishy about imagining only/exactly the worlds where one’s contribution makes a pivotal difference (the post hints at this with the discussion of evidential decision procedures).
I wonder if the reasoning in this post can be used to strengthen particular combinations of interventions. E.g., “AI alignment research is particularly impactful in worlds where alignment is neither too hard nor too easy. In those worlds, there will be people who would build misaligned AIs if it weren’t for our inputs. Accordingly, AI alignment research becomes particularly important if we combine it with raising awareness of safety mindset and building leading coalitions of safety-minded research teams.”
I thought this post was particularly cool because it seems to be applicable to lots of things, at least in theory (I have some concerns in practice). I’m curious about further reviews of this post.
I find myself using the reasoning described in the post in a bunch of places related to the prioritization of longtermist interventions. At the same time, I’m not sure I ever get any useful conclusions out of it. This might be because the area of application (predicting the impact of new technologies in the medium-term future) is particularly challenging. (One could argue that there’s a “Which world gets saved” argument against “Which world gets saved” arguments: In worlds where the consideration is useful, we start out with wide uncertainty. Having wide uncertainty is bad for having lots of impact.)
I wonder if there’s something a bit fishy about imagining only/exactly the worlds where one’s contribution makes a pivotal difference (the post hints at this with the discussion of evidential decision procedures).
I wonder if the reasoning in this post can be used to strengthen particular combinations of interventions. E.g., “AI alignment research is particularly impactful in worlds where alignment is neither too hard nor too easy. In those worlds, there will be people who would build misaligned AIs if it weren’t for our inputs. Accordingly, AI alignment research becomes particularly important if we combine it with raising awareness of safety mindset and building leading coalitions of safety-minded research teams.”