Right now it seems like AI safety is more of a scene (centered around the Bay Area) than a research community. If you want to attract great scientists and mathematicians (or even mediocre scientists and mathematicians), something even more basic than coming up with good “nerd-sniping” problems is changing this. There are many people who could do good work in technical AI safety, but would not fit in socially with EAs and rationalists.
I’m sympathetic to arguments that formal prepublication peer review is a waste of time. However, I think formalizing and writing up ideas to academic standards, such that they could be submitted for peer review, is definitely a very good use of time, it’s not even that hard, and there should be more of it. This would be one step towards making a more bland, boring, professionalized research community where a wider variety of people might want to be involved.
Let’s hope that AGI Alignment goes mainstream in the coming months. I think it has a good chance with the Pause letter, and the wide reporting it has had in the mainstream media.
Right now it seems like AI safety is more of a scene (centered around the Bay Area) than a research community. If you want to attract great scientists and mathematicians (or even mediocre scientists and mathematicians), something even more basic than coming up with good “nerd-sniping” problems is changing this. There are many people who could do good work in technical AI safety, but would not fit in socially with EAs and rationalists.
I’m sympathetic to arguments that formal prepublication peer review is a waste of time. However, I think formalizing and writing up ideas to academic standards, such that they could be submitted for peer review, is definitely a very good use of time, it’s not even that hard, and there should be more of it. This would be one step towards making a more bland, boring, professionalized research community where a wider variety of people might want to be involved.
Let’s hope that AGI Alignment goes mainstream in the coming months. I think it has a good chance with the Pause letter, and the wide reporting it has had in the mainstream media.