Very interesting!
I’d be interested to hear a bit more about what a restrained system would be able to do.
For example, could I make two restrained AGIs, one which has the goal:
A) “create a detailed plan plan.txt for maximising profit”
And another which has the goal:
B) “execute the plan written in plan.txt”?
If not, I’m not clear on why “make a cure for cancer” is scope-insensitive but “write a detailed plan for [maximising goal]” is scope-sensitive
Some more test case goals to probe the definition:
C) “make a maximal success rate cure for cancer”
D) “write a detailed plan for generating exactly $10^100 USD profit for my company”
Very interesting!
I’d be interested to hear a bit more about what a restrained system would be able to do.
For example, could I make two restrained AGIs, one which has the goal:
A) “create a detailed plan plan.txt for maximising profit”
And another which has the goal:
B) “execute the plan written in plan.txt”?
If not, I’m not clear on why “make a cure for cancer” is scope-insensitive but “write a detailed plan for [maximising goal]” is scope-sensitive
Some more test case goals to probe the definition:
C) “make a maximal success rate cure for cancer”
D) “write a detailed plan for generating exactly $10^100 USD profit for my company”