I find this submission very low on detail in the places that matter, namely the anthropomorphic AGI itself. It is not clear how this could be build, or why it is more realistic that such an AGI gets build than other AGIs.
and educated and reared much like a human child, in a caring and supportive environment.
How would this look like? Why would the AGI respond to this like a well-behaved human child?
Its value system would be, like that of humans, dynamic, high dimensional, and to some degree ineffable.
Would it have inconsistent values? How do you know there won’t be any mesaoptimization?
I find this submission very low on detail in the places that matter, namely the anthropomorphic AGI itself. It is not clear how this could be build, or why it is more realistic that such an AGI gets build than other AGIs.
How would this look like? Why would the AGI respond to this like a well-behaved human child?
Would it have inconsistent values? How do you know there won’t be any mesaoptimization?