The question is what is the mechanism of value spreading.
If the mechanism is having rational discussions then it is not necessarily urgent to have these discussions right now. Once we create a future in which there is no death and no economic pressures to self-modify in ways that are value destructive, we’ll have plenty of time for rational discussions. Things like “experience machine” also fit into this framework, as long as the experiences are in some sense non-destructive (this rules out experiences that create addiction, for example).
If the mechanism is anything but rational discussion then
It’s not clear in what sense the values you’re spreading are “correct” if it’s impossible to convince other people through rational discussion.
I would definitely consider this sort of intervention as evil and would fight rather than cooperate with it (at least assuming the effect cannot be reversed by rational discussion; I also consider hedonistic utilitarianism abhorrent except as an approximate model in very restricted contexts).
Regarding MIRI in particular, I don’t think the result of their work depends on the personal opinions of its director in the way you suggest. I think that any reasonable solution to the FAI problem will be on the meta-level (defining what does it mean for values to be “correct”) rather than the object level (hard-coding specific values like animal suffering).
I mostly agree with you. I am less confident than you are that a solution to the FAI problem will be on the meta-level. I think you’re probably right, but I have enough uncertainty about it that I much prefer someone who’s doing AI safety research to share my values so I can be more confident that they will do research that’s useful for all sentient beings and not just humans.
The question is what is the mechanism of value spreading.
If the mechanism is having rational discussions then it is not necessarily urgent to have these discussions right now. Once we create a future in which there is no death and no economic pressures to self-modify in ways that are value destructive, we’ll have plenty of time for rational discussions. Things like “experience machine” also fit into this framework, as long as the experiences are in some sense non-destructive (this rules out experiences that create addiction, for example).
If the mechanism is anything but rational discussion then
It’s not clear in what sense the values you’re spreading are “correct” if it’s impossible to convince other people through rational discussion.
I would definitely consider this sort of intervention as evil and would fight rather than cooperate with it (at least assuming the effect cannot be reversed by rational discussion; I also consider hedonistic utilitarianism abhorrent except as an approximate model in very restricted contexts).
Regarding MIRI in particular, I don’t think the result of their work depends on the personal opinions of its director in the way you suggest. I think that any reasonable solution to the FAI problem will be on the meta-level (defining what does it mean for values to be “correct”) rather than the object level (hard-coding specific values like animal suffering).
I mostly agree with you. I am less confident than you are that a solution to the FAI problem will be on the meta-level. I think you’re probably right, but I have enough uncertainty about it that I much prefer someone who’s doing AI safety research to share my values so I can be more confident that they will do research that’s useful for all sentient beings and not just humans.