Thanks, this and particularly the Medium post was helpful.
So to restate what I think your model around this is, it’s “the efficiency gap determines how tractable social solutions will be (if < 10% they seem much more tractable), and technical safety work can change the efficiency gap.”
Thanks for the link. So I guess I should amend what Paul and OpenAI’s goal seems like to me, to “create AGI, make sure it’s aligned, and make sure it’s competitive enough to become widespread.”
OK, this is what I modeled AI alignment folks as believing. But doesn’t the idea of first-past-the-post-is-the-winner rely on a “hard takeoff” scenario? This is a view I associate with Eliezer. But Paul in the podcast says that he thinks a gradual takeoff is more likely, and envisions a smooth gradient of AI capability such that human-level AI comes into existence in a world where slightly stupider AIs already exist.
The relevant passage:
and in particular, when someone develops human level AI, it’s not going to emerge in a world like the world of today where we can say that indeed, having human level AI today would give you a decisive strategic advantage. Instead, it will emerge in a world which is already much, much crazier than the world of today, where having a human AI gives you some more modest advantage.
So I get why you would drop everything and race to be the first to build an aligned AGI if you’re Eliezer. But if you’re Paul, I’m not sure why you would do this, since you think it will only give you a modest advantage.
(Also, if the idea is to build your AGI first and then use it to stop everyone else from building their AGIs—I feel like that second part of the plan should be fronted a bit more! “I’m doing research to ensure AI does what we tell it to” is quite a different proposition from “I’m doing research to ensure AI does what we tell it to, so that I can build an AI and tell it to conquer the world for me.“)
Thanks Ozzie, this is helpful!
The former. To your other comment—yes, I’ve gotten a number of emails! :)
Thanks very much for the comment Ozzie.
I share the idea that U.S. educational issues are not the most efficient ones to be working on, all else equal. My question arises because it’s not obvious to me that all else is equal in my case. (Though I think the burden of proof should be on me here.) For example, I have a pretty senior role in the organization, and therefore presumably have higher leverage. How should I factor considerations like that in? (Or is it misguided to do so?)
I’m curious also about your statement that it’s hard to have much counterfactual impact in the for-profit world. I’ve been struggling with similar questions. Why do you think so?