Even a 10 year delay is worth a huge amount (in expectation). We may well have a very different view of alignment by then (including perhaps being pretty solid on it’s impossibility?
Agreed on all counts! Though as someone who’s been working in this area for 10 years, I have a newfound appreciation for how little intellectual progress can easily end up happening in a 10-year period...
(Or even the seemingly very unlikely ”..there’s nothing to worry about”)
I have a lot of hopes that seem possible enough to me to be worth thinking about, but this specific hope isn’t one of them. Alignment may turn out to be easier than expected, but I think we can mostly rule out “AGI is just friendly by default”.
But it could have a huge effect in terms of shifting the Overton Window on this.
In which direction?
:P
I’m joking, though I do take seriously that there are proposals that might be better signal-boosted by groups other than MIRI. But if you come up with a fuller proposal you want lots of sane people to signal-boost, do send it to MIRI so we can decide if we like it; and if we like it as a sufficiently-realistic way to lengthen timelines, I predict that we’ll be happy to signal-boost it and say as much.
As I’ve said, it doesn’t make sense for this not to be part of any “Death with Dignity” strategy. The sensible thing when faced with ~0% survival odds is to say “FOR FUCK’S SAKE CAN WE AT LEAST TRY AND PULL THE PLUG ON HUMANS DOING AGI RESEARCH!?!”, or even “STOP BUILDING AGI YOU FUCKS!” [Sorry for the language, but I think it’s appropriate given the gravity of the situation, as assumed by talk of 100% chance of death etc.]
I strongly agree and think it’s right that people… like, put some human feeling into their words, if they agree about how fucked up this situation is? (At least if they find it natural to do so.)
Agreed on all counts! Though as someone who’s been working in this area for 10 years, I have a newfound appreciation for how little intellectual progress can easily end up happening in a 10-year period...
I have a lot of hopes that seem possible enough to me to be worth thinking about, but this specific hope isn’t one of them. Alignment may turn out to be easier than expected, but I think we can mostly rule out “AGI is just friendly by default”.
In which direction?
:P
I’m joking, though I do take seriously that there are proposals that might be better signal-boosted by groups other than MIRI. But if you come up with a fuller proposal you want lots of sane people to signal-boost, do send it to MIRI so we can decide if we like it; and if we like it as a sufficiently-realistic way to lengthen timelines, I predict that we’ll be happy to signal-boost it and say as much.
I strongly agree and think it’s right that people… like, put some human feeling into their words, if they agree about how fucked up this situation is? (At least if they find it natural to do so.)