I can accept that it seemed to make sense at the start, but can you explain how it would still make sense now given what’s happened (or, rather, didn’t happen) in the meantime?
Basically, I don’t know . I think it’s good to start off by emphatically stating I don’t have any real knowledge of MIRI.
A consideration is that the beliefs in MIRI are still on very short timelines. A guess is that because of the nature of some work relevant to short timelines, maybe some projects could have bad consequences if made public (or just don’t make sense to ever make public).
Again, this is presumptuous, but my instinct is not to have attitudes of instructing org policy in a situation like this, because of dependencies we don’t see. (Just so this doesn’t read like a statement that nothing can ever change: I guess the change here would be a new org or new leaders, obviously this is hard).
Also, to be clear, this is accepting the premise of MIRI. IMO one should take seriously the premise of shorter timelines, like, it’s a valid belief. Under this premise, the issue here is really bad execution, like actively bad.
If your comment was alluding to shifting of beliefs away from short timelines, that seems like a really different discussion.
No, I’m saying the nearer and more probable you thing doom-causing AGI is, and the longer you stagnate on solving the problem, the less it makes sense to not let the rest of the world in on the work. If you don’t, you’re very probably doomed. If you do, you’re still very probably doomed, but at least you have orders of magnitude more people collaborating with you to prevent it, this increasing the chance of success.
(As a presumptuous comment) I don’t have a positive view about the work from strong circumstantial evidence. However, as sort of devils advocate:
There are very few good theories of change for very short timelines and one of them is build it yourself. So, I don’t see how that’s good to share.
Alignment might be entangled in this to the degree that sharing even alignment might be capabilities research.
The above might be awful beliefs but I don’t see how it’s wrong.
By the way, just to calibrate so people can read if I’m crazy:
It reads like MIRI or closely related people have tried to build AGI or find the requisite knowledge, many times over the years. The negative results seems to be an update about their beliefs.
Thanks. That kinda sorta makes sense. I still think if they’re trying to build an aligned AGI, it’s arrogant and unrealistic to think you can achieve it with a small group that’s not collaborating with others, faster than the entire AI capabilities community who are basically collaborating together can.
I can accept that it seemed to make sense at the start, but can you explain how it would still make sense now given what’s happened (or, rather, didn’t happen) in the meantime?
Basically, I don’t know . I think it’s good to start off by emphatically stating I don’t have any real knowledge of MIRI.
A consideration is that the beliefs in MIRI are still on very short timelines. A guess is that because of the nature of some work relevant to short timelines, maybe some projects could have bad consequences if made public (or just don’t make sense to ever make public).
Again, this is presumptuous, but my instinct is not to have attitudes of instructing org policy in a situation like this, because of dependencies we don’t see. (Just so this doesn’t read like a statement that nothing can ever change: I guess the change here would be a new org or new leaders, obviously this is hard).
Also, to be clear, this is accepting the premise of MIRI. IMO one should take seriously the premise of shorter timelines, like, it’s a valid belief. Under this premise, the issue here is really bad execution, like actively bad.
If your comment was alluding to shifting of beliefs away from short timelines, that seems like a really different discussion.
No, I’m saying the nearer and more probable you thing doom-causing AGI is, and the longer you stagnate on solving the problem, the less it makes sense to not let the rest of the world in on the work. If you don’t, you’re very probably doomed. If you do, you’re still very probably doomed, but at least you have orders of magnitude more people collaborating with you to prevent it, this increasing the chance of success.
I think what you said makes sense.
(As a presumptuous comment) I don’t have a positive view about the work from strong circumstantial evidence. However, as sort of devils advocate:
There are very few good theories of change for very short timelines and one of them is build it yourself. So, I don’t see how that’s good to share.
Alignment might be entangled in this to the degree that sharing even alignment might be capabilities research.
The above might be awful beliefs but I don’t see how it’s wrong.
By the way, just to calibrate so people can read if I’m crazy:
It reads like MIRI or closely related people have tried to build AGI or find the requisite knowledge, many times over the years. The negative results seems to be an update about their beliefs.
Thanks. That kinda sorta makes sense. I still think if they’re trying to build an aligned AGI, it’s arrogant and unrealistic to think you can achieve it with a small group that’s not collaborating with others, faster than the entire AI capabilities community who are basically collaborating together can.