I do think there is often a tension in what you write on this front. On the one hand, you seem to support radical democratic control of (every?) decision made by anyone anywhere. And on the other hand, you think we should all defer to experts.
On the one hand, I agree there is a tension, and one that I definitely haven’t figure out (although I do think this is actually somewhat different to my views on EA structure and organising, particularly because I am not here actually talking about EA).
I am deeply concerned with the problem that I’m really unsure it make sense to use the term ‘expert’ with reference to anyone in XRisk, given the events haven’t occurred and so its really unclear what would confer the experience to make one an expert. I think my greater reticence to see expertise as something that can be had in XRisk seperates me from others somewhat similar to me like Luke Kemp, and is definitely somewhere I differ substantially from people like David Thorstad. On the other hand, I definitely think peer review can play a really important role, including in promoting the establishment of a strong methodological basis, in the creation of trusted ‘techniques of futuring’ which help us to construct and performed shared futures (although the relative success of AI Safety in doing this without a strong reliance on peer review definitely reduces this argument). I also think peer review guards against a strong ‘Matthew Effect’ explained well here (https://www.liamkofibright.com/uploads/4/8/9/8/48985425/is_peer_review_a_good_idea_.pdf) which I actually think is realtively prevelant in xrisk. However, this is about supporting greater conversation about what the role is, rather than even saying such role needs to be prominent; rather, I think we ought to be much more deliberate about whether or not and how we want peer review to be present in our research culture as a community, rather than simply relying on the current status quo either in ERS or in mainstream academia.
I do find the emphasis on peer review and expertise hard to square with the radical democratic view, and I don’t think that is a needle that can be threaded. If the majority were climate sceptics and were in favour of repealing all climate policy, it seems like you would have to be in favour of that given your radical democratic views but opposed to it because it is violently at odds with peer reviewed science.
My understanding (appreciating I may be somewhat biased on this), is that the demand for greater expertise comes from what you and others perceive to be the lack of deference to peer reviewed science by EAs working on climate change (which I think isn’t true fwiw because the ‘standard EA view’ is in line with the expert consensus on climate change) and the fact that there is not much peer reviewed work in AI and to a lesser extent bio (I’m sympathetic on AI).
That aside, Yeah I have somewhat conflicted and not worked out thoughts on peer review in EA.
As a statement of how I view things, I would generally be more inclined to trust a report with paid expert reviewers by Open Phil, or a blogpost or report by someone like Scott Alexander, Carl Shulman or Toby Ord, than the median peer reviewed study on a topic area. I think who write something matters a lot and explains a lot of the variation in quality, independent of the fora in which something is published.
I generally think peer review is a bit of a mess compared to what we might hope for the epistemic foundation of modern society. Published definitely doesn’t mean true. Most published research is false. Reviewers don’t usually check the maths going into a paper. Political bias and seniority influences publishing decisions. The bias is worse in the most prestigious journals. Some fields are far worse than others and some should be completely ignored (eg continental philosophy, nutritional epidemiology). ‘Experts’ who know a lot of factual information on a topic area can systematically err because they have bad epistemics (witness the controversy about the causes of the Holocene megafauna extinction)
That being said, I think the median peer reviewed study is usually better than the typical EA forum or lesswrong blogpost. Given how thin the literature on AI is, the marginal value of yet another blogpost that isn’t related to an established published literature seems low. In AI, the marginal value of more peer reviewed work seems high. But I also think the marginal value of more open phil reports with paid expert reviewers with published reviewer reports would probably be higher than peer review given how flawed peer review is and how much better the incentives are for the open phil-type approach
Ye, in response to the first point I basically think you’ve misinterpreted my position on this John, so thanks for laying this out so clearly.
I have never (as far as I know, and if I have, I must apologise because I am wrong) accused you or other EA’s of lack of deference to peer reviewed science in the climate space. I essentially agree that you are more in line with the climate expert consensus on climate and xrisk than I am, and I would (and have) defended you against accusations that you don’t know and utilise this literature very well (althouigh I think you do fail to engage with the literature on climate and xrisk, but we’ve already had this discussion and its very tangential to my main point). I just don’t think the present peer reviewed literature actually does a very good job at all at addressing climate and xrisk questions (although again, I think tis is tangential).
I full agree with your second bullet point. But note, again, the statement says “more care put into publication practices for those engaged in research, including greater consideration and conversation of the appropriate role of different fora in the epistemic process, including peer reviewed publications”. I interpreted this as suggesting not ‘peer review is perfect and amazing’ but rather if we are to make this community as epistemically healthy as possible, we need to have consideration for what the appropriate role for peer reviewed publications are. For me anyway, one motivating factor behind this is a question of if we ought to or not have an xrisk journal, or whether we ought to set up an xrisk specific forum (like the alignment forum is). Each of these can play different epistemic roles, and I think there is often a polarised response of peer review is perfect or peer review is the worst. I do think there is a strong argument against peer review in xrisk work, although I also think there are compelling arguments in favour of peer review in certain forms. Indeed, many signatories have made public defences of peer review (eg David Thorstad) and others have been critical (eg Erica Thompson’s Escape from Modelland contains critiques of peer review), but I think we broadly felt that it was a discussion much more worth having in the xrisk space
I do think there is often a tension in what you write on this front. On the one hand, you seem to support radical democratic control of (every?) decision made by anyone anywhere. And on the other hand, you think we should all defer to experts.
On the one hand, I agree there is a tension, and one that I definitely haven’t figure out (although I do think this is actually somewhat different to my views on EA structure and organising, particularly because I am not here actually talking about EA).
I am deeply concerned with the problem that I’m really unsure it make sense to use the term ‘expert’ with reference to anyone in XRisk, given the events haven’t occurred and so its really unclear what would confer the experience to make one an expert. I think my greater reticence to see expertise as something that can be had in XRisk seperates me from others somewhat similar to me like Luke Kemp, and is definitely somewhere I differ substantially from people like David Thorstad. On the other hand, I definitely think peer review can play a really important role, including in promoting the establishment of a strong methodological basis, in the creation of trusted ‘techniques of futuring’ which help us to construct and performed shared futures (although the relative success of AI Safety in doing this without a strong reliance on peer review definitely reduces this argument). I also think peer review guards against a strong ‘Matthew Effect’ explained well here (https://www.liamkofibright.com/uploads/4/8/9/8/48985425/is_peer_review_a_good_idea_.pdf) which I actually think is realtively prevelant in xrisk. However, this is about supporting greater conversation about what the role is, rather than even saying such role needs to be prominent; rather, I think we ought to be much more deliberate about whether or not and how we want peer review to be present in our research culture as a community, rather than simply relying on the current status quo either in ERS or in mainstream academia.
I do find the emphasis on peer review and expertise hard to square with the radical democratic view, and I don’t think that is a needle that can be threaded. If the majority were climate sceptics and were in favour of repealing all climate policy, it seems like you would have to be in favour of that given your radical democratic views but opposed to it because it is violently at odds with peer reviewed science.
My understanding (appreciating I may be somewhat biased on this), is that the demand for greater expertise comes from what you and others perceive to be the lack of deference to peer reviewed science by EAs working on climate change (which I think isn’t true fwiw because the ‘standard EA view’ is in line with the expert consensus on climate change) and the fact that there is not much peer reviewed work in AI and to a lesser extent bio (I’m sympathetic on AI).
That aside, Yeah I have somewhat conflicted and not worked out thoughts on peer review in EA.
As a statement of how I view things, I would generally be more inclined to trust a report with paid expert reviewers by Open Phil, or a blogpost or report by someone like Scott Alexander, Carl Shulman or Toby Ord, than the median peer reviewed study on a topic area. I think who write something matters a lot and explains a lot of the variation in quality, independent of the fora in which something is published.
I generally think peer review is a bit of a mess compared to what we might hope for the epistemic foundation of modern society. Published definitely doesn’t mean true. Most published research is false. Reviewers don’t usually check the maths going into a paper. Political bias and seniority influences publishing decisions. The bias is worse in the most prestigious journals. Some fields are far worse than others and some should be completely ignored (eg continental philosophy, nutritional epidemiology). ‘Experts’ who know a lot of factual information on a topic area can systematically err because they have bad epistemics (witness the controversy about the causes of the Holocene megafauna extinction)
That being said, I think the median peer reviewed study is usually better than the typical EA forum or lesswrong blogpost. Given how thin the literature on AI is, the marginal value of yet another blogpost that isn’t related to an established published literature seems low. In AI, the marginal value of more peer reviewed work seems high. But I also think the marginal value of more open phil reports with paid expert reviewers with published reviewer reports would probably be higher than peer review given how flawed peer review is and how much better the incentives are for the open phil-type approach
Ye, in response to the first point I basically think you’ve misinterpreted my position on this John, so thanks for laying this out so clearly.
I have never (as far as I know, and if I have, I must apologise because I am wrong) accused you or other EA’s of lack of deference to peer reviewed science in the climate space. I essentially agree that you are more in line with the climate expert consensus on climate and xrisk than I am, and I would (and have) defended you against accusations that you don’t know and utilise this literature very well (althouigh I think you do fail to engage with the literature on climate and xrisk, but we’ve already had this discussion and its very tangential to my main point). I just don’t think the present peer reviewed literature actually does a very good job at all at addressing climate and xrisk questions (although again, I think tis is tangential).
I full agree with your second bullet point. But note, again, the statement says “more care put into publication practices for those engaged in research, including greater consideration and conversation of the appropriate role of different fora in the epistemic process, including peer reviewed publications”. I interpreted this as suggesting not ‘peer review is perfect and amazing’ but rather if we are to make this community as epistemically healthy as possible, we need to have consideration for what the appropriate role for peer reviewed publications are. For me anyway, one motivating factor behind this is a question of if we ought to or not have an xrisk journal, or whether we ought to set up an xrisk specific forum (like the alignment forum is). Each of these can play different epistemic roles, and I think there is often a polarised response of peer review is perfect or peer review is the worst. I do think there is a strong argument against peer review in xrisk work, although I also think there are compelling arguments in favour of peer review in certain forms. Indeed, many signatories have made public defences of peer review (eg David Thorstad) and others have been critical (eg Erica Thompson’s Escape from Modelland contains critiques of peer review), but I think we broadly felt that it was a discussion much more worth having in the xrisk space