Here are some examples reasons why I think my high probabilities are plausible:
The training proposal I gave is pretty close to how models like GPT4 are trained. These models are pretty general and are quite strategic etc. Adding more FLOP makes a pretty big qualitative difference.
It doesn’t seem to me like you have to generalize very far for this to succeed. I think existing data trains you to do basically everything humans can do. (See GPT4 and prompting)
Even if this proposal is massively inefficient, we’re throwing an absurd amount of FLOP at it.
It seems like the story for why humans are intelligent looks reasonably similar to this story: have big, highly functional brains, learn to predict what you see, train to achieve various goals, generalize far. Perhaps you think humans intelligence is very unlikely ex-ante (<0.04% likely).
Here are some examples reasons why I think my high probabilities are plausible:
The training proposal I gave is pretty close to how models like GPT4 are trained. These models are pretty general and are quite strategic etc. Adding more FLOP makes a pretty big qualitative difference.
It doesn’t seem to me like you have to generalize very far for this to succeed. I think existing data trains you to do basically everything humans can do. (See GPT4 and prompting)
Even if this proposal is massively inefficient, we’re throwing an absurd amount of FLOP at it.
It seems like the story for why humans are intelligent looks reasonably similar to this story: have big, highly functional brains, learn to predict what you see, train to achieve various goals, generalize far. Perhaps you think humans intelligence is very unlikely ex-ante (<0.04% likely).