There is a distinction between “control” and “alignment. ”
The control problem addresses our fundamental capacity to constrain AI systems, preventing undesired behaviors or capabilities from manifesting, regardless of the system’s goals. Control mechanisms encompass technical safeguards that maintain human authority over increasingly autonomous systems, such as containment protocols, capability limitations, and intervention mechanisms.
The alignment problem, conversely, focuses on ensuring AI systems pursue goals compatible with human values and intentions. This involves developing methods to specify, encode, and preserve human objectives within AI decision-making processes. Alignment asks whether an AI system “wants” the right things, while control asks whether we can prevent it from acting on its wants.
I believe AI is soon to have wants, and it’s critical to align those wants with increasingly capable AIs.
As far as I’m concerned I don’t see humanity not eventuallycreating superintelligence and thus it should be the main focus of EA and other groups concerned with AI. As I mentioned in another comment I don’t have many ideas for how the average EA person can do this aside from making a career change into AI policy or something similar.
There is a distinction between “control” and “alignment. ”
The control problem addresses our fundamental capacity to constrain AI systems, preventing undesired behaviors or capabilities from manifesting, regardless of the system’s goals. Control mechanisms encompass technical safeguards that maintain human authority over increasingly autonomous systems, such as containment protocols, capability limitations, and intervention mechanisms.
The alignment problem, conversely, focuses on ensuring AI systems pursue goals compatible with human values and intentions. This involves developing methods to specify, encode, and preserve human objectives within AI decision-making processes. Alignment asks whether an AI system “wants” the right things, while control asks whether we can prevent it from acting on its wants.
I believe AI is soon to have wants, and it’s critical to align those wants with increasingly capable AIs.
As far as I’m concerned I don’t see humanity not eventually creating superintelligence and thus it should be the main focus of EA and other groups concerned with AI. As I mentioned in another comment I don’t have many ideas for how the average EA person can do this aside from making a career change into AI policy or something similar.