Finally, I think your points about the ‘agreed upon methods’ sort of thing is really good,
Since you liked that though let me think out loud a bit more.
I think it’s practically impossible to be rigorous without a paradigm.
Old sciences have paradigms and mostly work well but the culture is not nice to people trying to form ideas outside the paradigm, because that is necessarily less rigours. I remember some academic complaining on this on a podcast. They where doing some different approach within cognitive science and had problem with pear review because they where not enough focused on measuring the standard things.
On the other had there is EA/LW style AI Safety research, where everyone talks abut how preparadigmatic we are. Vague speculative ideas, with out inferential depth, get more appreciation and attention. By now there are a few paradigms, the clearest case being Vanessas research, which almost no one understand. I think part of the reason her work is hard to undertand is exactly because it is rigours within a paradigm research. It’s specific proof with in a specific framework. It has both more details and more prerequisites. While reading pre paradigmatic blogposts is like reading the first intro chapter in a text book (which is always less technical), the with in paradigmatic stuff is more like reading chapter 11, and you really have to have read the previous chapters, which makes it less accessible. Especially since no one collected the previous chapters for you, and the person writing it is not selected for their pedagogical skills.
Research has to start as pre paradigmatic. But I think that the dynamic described above makes it hard to move on, to pick some paradigm to explore and start working out the details. Maybe a field at some point needs to develop a culture of looking down at less rigours work, for any rigours work to really take hold? I’m really not sure. And I don’t want to loose the explorative part of EA/LW style AI Safety research either. Possibly rigour will just develop naturally over time?
I think this is pretty interesting and thanks for sharing your thoughys! There’s things here I agree with, things I disagree with, and I might say more when I’m on my computer not phone!. However, I’d love to call about this to talk more, and see
Since you liked that though let me think out loud a bit more.
I think it’s practically impossible to be rigorous without a paradigm.
Old sciences have paradigms and mostly work well but the culture is not nice to people trying to form ideas outside the paradigm, because that is necessarily less rigours. I remember some academic complaining on this on a podcast. They where doing some different approach within cognitive science and had problem with pear review because they where not enough focused on measuring the standard things.
On the other had there is EA/LW style AI Safety research, where everyone talks abut how preparadigmatic we are. Vague speculative ideas, with out inferential depth, get more appreciation and attention. By now there are a few paradigms, the clearest case being Vanessas research, which almost no one understand. I think part of the reason her work is hard to undertand is exactly because it is rigours within a paradigm research. It’s specific proof with in a specific framework. It has both more details and more prerequisites. While reading pre paradigmatic blogposts is like reading the first intro chapter in a text book (which is always less technical), the with in paradigmatic stuff is more like reading chapter 11, and you really have to have read the previous chapters, which makes it less accessible. Especially since no one collected the previous chapters for you, and the person writing it is not selected for their pedagogical skills.
Research has to start as pre paradigmatic. But I think that the dynamic described above makes it hard to move on, to pick some paradigm to explore and start working out the details. Maybe a field at some point needs to develop a culture of looking down at less rigours work, for any rigours work to really take hold? I’m really not sure. And I don’t want to loose the explorative part of EA/LW style AI Safety research either. Possibly rigour will just develop naturally over time?
End of speculation
I think this is pretty interesting and thanks for sharing your thoughys! There’s things here I agree with, things I disagree with, and I might say more when I’m on my computer not phone!. However, I’d love to call about this to talk more, and see