Agreement karma indicates agreement, separate from overall quality.
Is there a plausible (but perhaps impractical?) steelman of “merging with the AI” as building an AI with direct access to your values/preferences, and optimizing for those by design? It could simulate outcomes of actions, your judgements are queried to filter actions, and then you choose. Maybe you need separate simulations of your brain to keep up. The AI is trained with simulation accuracy as its objective, and is not an RL agent.
Maybe it could show you something to fool you or otherwise take over?
Just pick a human to upload and let them recursively improve themselves into an SAI. If they’re smart enough to start out with, they might be able to keep their goals intact throughout the process.
(This isn’t a strategy I’d choose given any decent alternative, but it’s better than nothing. Likely to be irrelevant though, since it looks like we’re going to get GAI before we’re even close to being able to upload a human.)
2 votes
Overall karma indicates overall quality.
Total points: 0
Agreement karma indicates agreement, separate from overall quality.
Is there a plausible (but perhaps impractical?) steelman of “merging with the AI” as building an AI with direct access to your values/preferences, and optimizing for those by design? It could simulate outcomes of actions, your judgements are queried to filter actions, and then you choose. Maybe you need separate simulations of your brain to keep up. The AI is trained with simulation accuracy as its objective, and is not an RL agent.
Maybe it could show you something to fool you or otherwise take over?
3 votes
Overall karma indicates overall quality.
Total points: 0
Agreement karma indicates agreement, separate from overall quality.
Just pick a human to upload and let them recursively improve themselves into an SAI. If they’re smart enough to start out with, they might be able to keep their goals intact throughout the process.
(This isn’t a strategy I’d choose given any decent alternative, but it’s better than nothing. Likely to be irrelevant though, since it looks like we’re going to get GAI before we’re even close to being able to upload a human.)