but I’m arguing more for making the real AI Safety problems concrete enough that they will tackle it.
I agree that this would be immensely valuable if it works. Therefore, I think it’s important to try it. I suspect it likely won’t succeed because it’s hard to usefully simplify problems in a pre-paradigmatic field. I feel like if you can do that, maybe you’ve already solved the hardest part of the problem.
(I think most of my intuitions about the difficulty of usefully simplifying AI alignment relate to it being a pre-paradigmatic field. However, maybe the necessity of “security mindset” for alignment also plays into it.)
In my view, progress in pre-paradigmatic fields often comes from a single individual or a tight-knit group with high-bandwidth internal communication. It doesn’t come from lots of people working on a list of simplified problems.
(But maybe the picture I’m painting is too black-and-white. I agree that there’s some use to getting inputs from a broader set of people, and occasionally people who isn’t usually very creative can have a great insight, etc.)
Don’t discount the originality of academics, they can also be quite cool :)
That’s true. What I said sounded like a blanket dismissal of original thinking in academia, but that’s not how I meant it. Basically, my picture of the situation is as follows:
Few people are capable of making major breakthroughs in pre-paradigmatic fields because that requires a rare kind of creativity and originality (and probably also being a genius). There are people like that in academia, but they have their quirks and they’d mostly already be working on AI alignment if they had the relevant background. For the sort of people I’m thinking about, they are drawn to problems like AI risk or AI alignment. They likely wouldn’t need things to be simplified. If they look at a simplified problem, their mind immediately jumps to all the implications of the general principle and they think through the more advanced version of the problem because that’s way more interesting and way more relevant.
In any case, there are a bunch of people like that in long-termist EA because EA heavily selects for this sort of thinking. People from academia who excel at this sort of thinking often end up at EA aligned organizations.
So, who is left in academia and isn’t usefully contributing to alignment but could maybe contribute to it if we knew what we wanted from them? Those are the people who don’t invent entire fields on their own.
It sounds like our views are close!
I agree that this would be immensely valuable if it works. Therefore, I think it’s important to try it. I suspect it likely won’t succeed because it’s hard to usefully simplify problems in a pre-paradigmatic field. I feel like if you can do that, maybe you’ve already solved the hardest part of the problem.
(I think most of my intuitions about the difficulty of usefully simplifying AI alignment relate to it being a pre-paradigmatic field. However, maybe the necessity of “security mindset” for alignment also plays into it.)
In my view, progress in pre-paradigmatic fields often comes from a single individual or a tight-knit group with high-bandwidth internal communication. It doesn’t come from lots of people working on a list of simplified problems.
(But maybe the picture I’m painting is too black-and-white. I agree that there’s some use to getting inputs from a broader set of people, and occasionally people who isn’t usually very creative can have a great insight, etc.)
That’s true. What I said sounded like a blanket dismissal of original thinking in academia, but that’s not how I meant it. Basically, my picture of the situation is as follows:
Few people are capable of making major breakthroughs in pre-paradigmatic fields because that requires a rare kind of creativity and originality (and probably also being a genius). There are people like that in academia, but they have their quirks and they’d mostly already be working on AI alignment if they had the relevant background. For the sort of people I’m thinking about, they are drawn to problems like AI risk or AI alignment. They likely wouldn’t need things to be simplified. If they look at a simplified problem, their mind immediately jumps to all the implications of the general principle and they think through the more advanced version of the problem because that’s way more interesting and way more relevant.
In any case, there are a bunch of people like that in long-termist EA because EA heavily selects for this sort of thinking. People from academia who excel at this sort of thinking often end up at EA aligned organizations.
So, who is left in academia and isn’t usefully contributing to alignment but could maybe contribute to it if we knew what we wanted from them? Those are the people who don’t invent entire fields on their own.