“You’ll need to get hands-on. The best ML and alignment research engages heavily with neural networks (with only a few exceptions). Even if you’re more theoretically-minded, you should plan to be interacting with models regularly, and gain the relevant coding skills. In particular, I see a lot of junior researchers who want to do “conceptual research”. But you should assume that such research is useless until it cashes out in writing code or proving theorems, and that you’ll need to do the cashing out yourself (with threat modeling being the main exception, since it forces a different type of concreteness). …”
Yeah, I agree on priors & some arguments about feedback loops, although note that I don’t really have relevant experience. But I remember hearing someone try to defend something like the opposite claim to me in some group setting where I wasn’t able to ask the follow-up questions I wanted to ask — so now I don’t remember what their main arguments were and don’t know if I should change my opinion.
I expect a bunch of more rationalist-type people disagree with this claim, FWIW. But I also think that they heavily overestimate the value of the types of conceptual research I’m talking about here.
This seems strongly true to me
Yeah, I agree on priors & some arguments about feedback loops, although note that I don’t really have relevant experience. But I remember hearing someone try to defend something like the opposite claim to me in some group setting where I wasn’t able to ask the follow-up questions I wanted to ask — so now I don’t remember what their main arguments were and don’t know if I should change my opinion.
I expect a bunch of more rationalist-type people disagree with this claim, FWIW. But I also think that they heavily overestimate the value of the types of conceptual research I’m talking about here.