Post GPT-4, I think the most urgent thing is to get a global AGI moratorium (or Pause) into effect as alignment is too far behind to be likely to save us in time. Advocating for a Pause is accessible to anyone, although learning the arguments as to why it is needed in some depth can help with being convincing (for a decent level of depth without too much time commitment, you can go through the AGI Safety Fundamentals syllabus in about 20 hours). Or just try and grok the 3 essential components of AGI x-risk: the Orthogonality Thesis (Copernican Revolution applied to mind-space; the “shoggoth”; the necessity of outer alignment), Basic AI Drives (convergent instrumental goals leading to power seeking) and Mesaoptimization (even if aimed at the right target, keeping it aimed there without value drift/corruption is hard; the necessity of inner alignment). Along with the facts that AGI seems near (could GPT-5 code as well as the top AI engineers and be able to make GPT-6, leading to an intelligence explosion?), and alignment progress has been slow.
Post GPT-4, I think the most urgent thing is to get a global AGI moratorium (or Pause) into effect as alignment is too far behind to be likely to save us in time. Advocating for a Pause is accessible to anyone, although learning the arguments as to why it is needed in some depth can help with being convincing (for a decent level of depth without too much time commitment, you can go through the AGI Safety Fundamentals syllabus in about 20 hours). Or just try and grok the 3 essential components of AGI x-risk: the Orthogonality Thesis (Copernican Revolution applied to mind-space; the “shoggoth”; the necessity of outer alignment), Basic AI Drives (convergent instrumental goals leading to power seeking) and Mesaoptimization (even if aimed at the right target, keeping it aimed there without value drift/corruption is hard; the necessity of inner alignment). Along with the facts that AGI seems near (could GPT-5 code as well as the top AI engineers and be able to make GPT-6, leading to an intelligence explosion?), and alignment progress has been slow.