If AI is an uncontrollable God, then alignment is impossible. Alignment to me implies some sort of control. Uncontrollable superintelligent AI sounds like a horrific idea. There’s no guarantees or incentives for God to solve any of our problems. God will work in mysterious ways. God might be cruel and merciless.
So even if we do succeed in preventing the creation of God, then that means we still need to do everything else EA is concerned about.
There is a distinction between “control” and “alignment. ”
The control problem addresses our fundamental capacity to constrain AI systems, preventing undesired behaviors or capabilities from manifesting, regardless of the system’s goals. Control mechanisms encompass technical safeguards that maintain human authority over increasingly autonomous systems, such as containment protocols, capability limitations, and intervention mechanisms.
The alignment problem, conversely, focuses on ensuring AI systems pursue goals compatible with human values and intentions. This involves developing methods to specify, encode, and preserve human objectives within AI decision-making processes. Alignment asks whether an AI system “wants” the right things, while control asks whether we can prevent it from acting on its wants.
I believe AI is soon to have wants, and it’s critical to align those wants with increasingly capable AIs.
As far as I’m concerned I don’t see humanity not eventuallycreating superintelligence and thus it should be the main focus of EA and other groups concerned with AI. As I mentioned in another comment I don’t have many ideas for how the average EA person can do this aside from making a career change into AI policy or something similar.
>inherently uncontrollable, and thus not a tool.
If AI is an uncontrollable God, then alignment is impossible. Alignment to me implies some sort of control. Uncontrollable superintelligent AI sounds like a horrific idea. There’s no guarantees or incentives for God to solve any of our problems. God will work in mysterious ways. God might be cruel and merciless.
So even if we do succeed in preventing the creation of God, then that means we still need to do everything else EA is concerned about.
There is a distinction between “control” and “alignment. ”
The control problem addresses our fundamental capacity to constrain AI systems, preventing undesired behaviors or capabilities from manifesting, regardless of the system’s goals. Control mechanisms encompass technical safeguards that maintain human authority over increasingly autonomous systems, such as containment protocols, capability limitations, and intervention mechanisms.
The alignment problem, conversely, focuses on ensuring AI systems pursue goals compatible with human values and intentions. This involves developing methods to specify, encode, and preserve human objectives within AI decision-making processes. Alignment asks whether an AI system “wants” the right things, while control asks whether we can prevent it from acting on its wants.
I believe AI is soon to have wants, and it’s critical to align those wants with increasingly capable AIs.
As far as I’m concerned I don’t see humanity not eventually creating superintelligence and thus it should be the main focus of EA and other groups concerned with AI. As I mentioned in another comment I don’t have many ideas for how the average EA person can do this aside from making a career change into AI policy or something similar.