This looks like an interesting platform for sharing ideas about AI safety.
You mention that AI safety is a ‘pre-paradigmatic field’—however, to a newcomer like me, the safety ideas and projects on the AI Safety Ideas site so far look pretty ‘paradigmatic’, in the sense of closely following the standard EA AGI X-risk paradigm that’s centered around the ideas of Yudkowsky, Bostrom, MIRI, utility maximization, instrumental convergence, deep learning, fast takeoff, etc.
I worry that this reflects & encourages a premature convergence onto a governing paradigm that may deter newcomers from contributing new ideas that fall outside the current ‘Overton window’ of AI alignment. For example: (1) a lot of AI alignment work seems based on a tacit assumption that AI won’t become an global catastrophic risk until it reaches AGI level, but I can see reasonable arguments that even quite narrow AI could be severely risky long before AGI is reached. Also, (2) a lot of AI alignment seems focused much more on how to align specific AI systems with specific human users, rather than on how to align human groups that are using AI, with other potentially conflicting groups that are using AI.
So, I guess the question arises: to what extent do you want AI Safety Ideas to elicit new ideas within the current paradigm, versus new ideas that stray outside the current paradigm?
You raise a very good point that I agree with. Right now, the platform is definitely biased towards the existing paradigm. This will probably be the case during the first few months, but we hope that it will help make the exploration of new directions and paradigms easier at the same time.
This also raises the point of the ideas currently playing into the canon of AI safety instead of looking at the vast literature outside of AI safety that concerns itself with the same topics but with another framing.
So to answer your questions; we want AISI to make it easier to elicit new ideas in all paradigms and directions with our personal bias moving that more towards new perspectives as we implement better functionality.
Esben—thanks very much for your reply. That all makes sense—to develop a gradual broadening-out from the current paradigm to welcoming new perspectives from other existing research traditions.
This looks like an interesting platform for sharing ideas about AI safety.
You mention that AI safety is a ‘pre-paradigmatic field’—however, to a newcomer like me, the safety ideas and projects on the AI Safety Ideas site so far look pretty ‘paradigmatic’, in the sense of closely following the standard EA AGI X-risk paradigm that’s centered around the ideas of Yudkowsky, Bostrom, MIRI, utility maximization, instrumental convergence, deep learning, fast takeoff, etc.
I worry that this reflects & encourages a premature convergence onto a governing paradigm that may deter newcomers from contributing new ideas that fall outside the current ‘Overton window’ of AI alignment. For example: (1) a lot of AI alignment work seems based on a tacit assumption that AI won’t become an global catastrophic risk until it reaches AGI level, but I can see reasonable arguments that even quite narrow AI could be severely risky long before AGI is reached. Also, (2) a lot of AI alignment seems focused much more on how to align specific AI systems with specific human users, rather than on how to align human groups that are using AI, with other potentially conflicting groups that are using AI.
So, I guess the question arises: to what extent do you want AI Safety Ideas to elicit new ideas within the current paradigm, versus new ideas that stray outside the current paradigm?
You raise a very good point that I agree with. Right now, the platform is definitely biased towards the existing paradigm. This will probably be the case during the first few months, but we hope that it will help make the exploration of new directions and paradigms easier at the same time.
This also raises the point of the ideas currently playing into the canon of AI safety instead of looking at the vast literature outside of AI safety that concerns itself with the same topics but with another framing.
So to answer your questions; we want AISI to make it easier to elicit new ideas in all paradigms and directions with our personal bias moving that more towards new perspectives as we implement better functionality.
Esben—thanks very much for your reply. That all makes sense—to develop a gradual broadening-out from the current paradigm to welcoming new perspectives from other existing research traditions.