Luke Kemp on:
Climate Change and Existential Risk
The role of Horizon Scans in Existential Risk Studies
His views on what EA gets wrong about XRisk
Deep Systems Thinking and XRisk.
Alternatively for another Climate Change and XRisk that would be narrower and less controversial/critical of EA than Luke is, Constantin Arnsschedit would be good
I think another discussion presenting SRM in the context of GCR might be good; there has now been a decent amount of research on this which probably proposes actions rather different from what SilverLining presents.
SilverLining is also decently controversial I the SRM community, so some alternative perspectives would probably be better than Kelly
Send me a DM if you’re interested, I’d be happy to provide a bunch of resources and to put you in contact with some people who could help send a bunch of resources
Hi John,
Sorry to revisit this, and I understand if you don’t. I must apologies if my previous comments felt a bit defensive from my side, as I do feel your statements towards me were untrue, but I think I have more clarity on the perspective you’ve come from and some of the possible baggage brought to this conversation, and I’m truly sorry if I’ve be ignorant of relevant context.
I think this comment is more going to address the overall conversation between us two on here, and where I perceive it to have gone, although I may be wrong, and I am open to corrections.
Firstly, I think you have assumed this statement is essentially a product of CSER, perhaps because it has come from me, who was a visit at CSER, and has been similarly critical of your work in a way that I know some at CSER have. [I should say, for the record on this, I do think your work is of high quality, and I hope you’ve never got the impression that I don’t. Perhaps some of my criticisms last year towards the review process your report went through felt poor quality (and I can’t remember what they were and may not stand by them today), but if so, I am sorry.] Nonetheless, I think its really important to keep in mind that this statement is absolutely not a ‘CSER’ statement; I’d like to remind you of the signatories, and whilst every signatory doesn’t agree with everything, I hope you can see why I got so defensive when you claimed that the signatories weren’t being transparent and actually attempting to just make EA another left-wing movement. I tried really hard to get a plurality of voices in this document, which is why such an accusation offended me, but ultimately I shouldn’t have got defensive over this, and I must apologise.
Secondly, on that point, I think we may have been talking about different things when you said ‘heterodox CSER approaches to EA.’ Certainly, I think Ehrlich and much of what he has called for is deeply morally reprehensible, and the capacity for ideas like his to gain ground is a genuine danger of pluralistic xrisk, because it is harder to police which ideas are acceptable or not (similarly, I have recieved criticism because this letter fails to call out eugenics explicitly, another danger). Nonetheless, I think we can trust as a more pluralistic community develops it would better navigate where the bounds of acceptable or unacceptable views and behaviours are, and that this would be better than us simply suggesting this now. Maybe this is a crux we/the signatories and much of the commens section disagree on. I think we can push for more pluralism and diversity in response to our situation whilst trusting that the more pluralistic ERS community will police how far this can go. You disagree and think we need to lay this out now otherwise it will either a) end up with anything goes, including views we find moral reprehensible or b) will mean EA is hijaked by the left. I think the second argument is weaker, particularly because this statement is not about EA, but about building a broader field of Existential Risk Studies, although perhaps you see this as a bit of a trojan horse. I understand I am missing some of the historical context that makes you think it is, but I hope that the signatories list may be enough to show you that I really do mean what I say when I call for pluralism.
I also must apologise if the call for retraction of certain parts of your comment seemed uncollegiate or disrespectful to you; this was certainly not my intention. I, however, felt that your painting of my views was incorrect, and thought you may, in light of this, be happy to change; although given you are not happy to retract, I assume you are either trying to make the argument that these are in fact my underlying beliefs (or that I am being dishonest, although I have no reason to suspect you would say this!).
I think there are a few more substantive points we disagree on, but to me this seems like the crux of the more heated discussion, and I must apologise it got so heated
Hi John Since I’ve corrected you that neither me nor Luke would agree with your characterisation of our positions, would you mind correcting this?
in response to your first point, I think one of the hopes of creating a pluralistic xrisk community is so that different parts of the community actually understand what work and persepctives each are doing, rather than either characturing them/misrepresenting them (for example, I’ve heard people outside EA assuming all EA XRisk work is basically just what Bostrom says) or just not knowing what other have to say. Ultimately, I think the workshop that this statement came out of did this really well, and so I hope if there is desire to move towards a more pluralistic community (which, perhaps from this forum, there isn’t) then we would better understand each others persepctives and why we disagree, and gain value from this disagreement. One example here is I think I personally have gained huge value from my discussions with John Halstead on climate, and really trying to understand his position.
I agree on the last paragraph, and is definitely a tension we will have to try anda resolve over time. This is one of the reasons we spoke about “we suggest that the power to confer support for different approaches should be distributed among the community rather than allocated by a few actors and funders, as no single individual can adequately manifest the epistemic and ethical diversity we deem necessary.” which would hopefully go someway to make sure that more forms of pluralism can assert themselves. Obviously, though, this won’t be perfect, and we will have to create spaces where voices that may previously not have been heard, because they don’t have all the money or aren’t loud and assertive, would get heard; this will be hard, and will definitely be difficult for someone like me who is clearly quite loud and likes to get my opinion out there.
NB: (I would also like to comment, and I really don’t want to be antagonistic to John as I do deeply respect him, but I do think his representation of ‘CSER-type heterodoxy’ or at least how he’s framed it with his two chief examples being me and Luke seems to me to be a misrepresentation. I know this may be arguing back too much, but given he’s said I believe something I don’t, I think its important to put the record straight (I’d hope its unintentional, although we have actually spoken a lot about my views))
I’m honestly rather confused with how people can disagree vote with this. I’d I get these stats wrong?
Apologies, my bad! DRR= Disaster Risk Reduction RCG = Riesgos Catastroficos Globales STS = Science and Technology Studies/ Science, Technology and Society ANT= Actor Network Theory
Some of those definitions definitely do, and some probablg do, so yes! However, I think there is a valid question is studying S-Risks gains much from being part of the ERS community or whether it would be more beneficial to be its own thing. I’m unsure (genuinely) how much either side gains from its involvement (maybe a lot!) In general, to me the two areas that may be commonly associated with xrisk that I don’t know how useful it is for it to be fully in ERS (although in conversation with definitely helps) is pure technical alignment (when divorced from AI Strategy) and maybe srisks, but I’m pretty unsure of this take, and many signatories would probably disagree.
Once again, I think the accusation that we are not being transparent is deeply disingenuous.
If you agree that saying ‘diversity is a strength’ is equivalent to ‘diversity is always a strength and there are no problems increasing diversity in anyway then I can see your concern; I’m pretty confused how this is your assumption of what we mean, and to me is far from the common usage of the phrase. But yes, I agree even if our epistemic situation demands diversity, there are ways this could go wrong, and its not an easy problem ‘where the tent stops’, and whilst it is a very important conversation to have and to negotiate, I too often think that having this conversation in response to any calls to diversify ends up doing much more harm than good.
Once again, these post is not talking about EA, and I’m not sure it’s particularly advocating for ‘non-merit based’ practices (some signatories may agree, some may not). One example of initiatives that could be done to increase demograohic diversity are efforts like magnify mentoring, or doing more outreach in developing countries, or funding initiatives in a broader geographic distribution, or even improving the advertising of projects and job positions. But sure, if we think increasing demographic diversity is important, we might want to have a conversation about other things that can be done.
Also, much of the diversity we speak about is about pluralism of method, core assumptions etc, which only have something to do with ’merit’if you are judging from already a very particular perspective, and it is having this singular perspective is one of the things we are arguing against.
On your final point, you have definitely entirely misrepresentation my position and I am shocked from the conversations we have had that you would come to this conclusion about my work. I’m also pretty surprised this would be your conclusion of Luke’s work as well, which has included everything from biosecurity work for the WHO, work on AI governance and work on climate change, but I don’t know how much of his stuff your reading. I can safely say Luke disagrees that ERS should basically just be XR. I know far less about Dasgupta’s work. Also, i really don’t understand how we can be seen as fully representative of CSER-style xrisk work either. I don’t quite understand how you can claim people hold beliefs, be counteracted, then fail to give evidence for your point whilst maintaining that you are right.
Firstly I should say the vagueness, whilst frustrating, ie there to both reflect and open up a discussion; it can appear ‘applause-lighty’ if one doesn’t recognise that this is the point of the statement, but I’m not quite sure how it does if you see tye statement as providing a statement of intent and justification for positions that ought to be debated, contested and refined.
On the topic of excluding racists, I think it is basically possible to do given how negatively it impacts a huge range of people from engaging with the community and doing good work. Whilst in my mind it’s clearly motivated by dedp concerns that the racism is unethical and the future that may emerge from a community that has racism so prevelant is deeply problematic, although I’m pretty sure it could still be justified from purely utilitarian perspectives due to the negative impact racism has on the community to function.
I think your demand for specifics here is both admirable, and I can give some that I agree with, and a little besides the point. One of the points we are making is the community at present is such that it doesn’t allow practitioners to explore the sorts of methods we could be using, and many important concepts or assumptions that could aid just aren’t present in this space at present (one example here is I bet if there had been a community containing active discussions around both geoengineering and AI then the idea of simply pausing or stopping AGI development would have been explored much early than it did in ERS at present). Now onto different conceptual bases. A few examples: we could use concepts from DRR like vulnerability or exposure much more, and I think a vulnerability focused account of xrisk would look very different. We could look at all the different xrisk cascades using causal mapping and find the best points of leverage in this. We could expand out an agents of doom like agenda to target those organisations that produce xrisk most effectively and study specifically how to reduce their power to cause risk the most. Black Swan approaches to xrisk may look further different. Or we may work from assumptions, either ethical (like RCG does) or epistemic (kind of like I do) that GCR is not very seperable from xrisk, so looking at cascades through critical systems may be deeply important. One may take as STS inspired approaches, either directly using methods from there (eg ANT or ethnograohy) or better utilising concepts. As said earlier, Id be interested in a Burkeian political philosophy of xrisk. These are just the research agendas i may be excited about, which is a very narrow slice of what is possible or optimal. The problem is none of these research agendas have the scope and airtime to develop, because current communal structures are ill designed to allow this to happen, and we think this move to pluralism which we lay out (and this forum clearly disagrees with) could allow for many very useful and exciting research agendas to form
I do basically agree that we should have people who we would say are on the right (hence my suggestion) and I can see why my previous comment may come across dismissive (apologies), I just don’t agree that breaking it down to party politics(eg Republican) or left-right given how broad it is is necessarily that useful. (Again, please remember I’m not just talking about EA but ERS, but I think your point still applies)
So I think you are pointing to something real here, although even then I don’t think it actually constitutes a great defence of the status quo of ERS. If we are to get people with preexisting experience in the disciplines we want, as most disciplines are far from equal, and if you are straying into an interdisciplinary soace like this job security and lack of mentorship may mean being more senior is very benefiical, we are likely to not get the sorts of demographic diversity we may want. However, ERS at present also rarely included and reaches out to people with deep expertise in these fields as well, so in some ways it feels we get the worst of both worlds; we bring in relatively young and inexperienced people and yet churn out people who broadly think very similarly and have the ability to influence very similar spaces . Sure, I’m not saying juggling all this is easy, nor that it will be perfect, but this status quo seems really suboptimal
I’m not quite sure how to address most of your comment, as I think in many ways we are critiquing much of the underlying logic of how we evaluate what we ought to be funded. Its essentially suggesting, not really from a place of justice at all, that the current structures fail to optimise or assess the things that are useful for the community. And I think it may also be implicitly suggesting that centralising this power as is done currently is counter to the aims, and so power in the community ought to be better distributed. One response would be to change these criteria to better optimise for things that we may think are more valuable eg novelty of ideas/approach. Another would be to randomise funding ie every proposal that crosses a certain bar gets entered into a lottery. Ultimately, I’m (not necessarily the other signatories)pretty comfortable making the argument for this on purely utilitarian grounds, although some people may feel a pull towards talking in terms of justice; a community that have metrics and evaluations that better encourage scientific creativity and a pluralism of ideas and approaches will be better off, and evaluation criteria that optimise for these sorts of pluralism are considerable better than the status quo.
I also worry that your focus is essentially on individual epistemics over a small range of positions, rather than the overall questions of funding structures and where it goes. For example, Open Phil could have decided to fund fellowships in developing countries rather than ERA in Cambridge, where I can’t imagine the applicants are any less good but didn’t. I think this expanding of geographic diversity may be a realtively easy win, although definitely isn’t the only criteria we could have followed. (Also, as a side point, I would suggest there is good cost-effectiveness reasons to do this as well; for example, the DEGREES initiative (that OpenPhil funds), has now got the largest geoengineering research team in the world, all based in Developing Countries, with significant impact now on informing policy, for a cost that was less than the budget of ERA I believe. Here, increasing geographic diversity must have played into their decision, again purely from a utilitarian and not justice perspectuve. )
In response to the trade off stuff, to some extent this is true, although I’m unconvinced this is that different to the trade offs we discuss. In response to your examples, I would say that the proposals we make are definitely promoting ‘novelty of ideas’ much heavier than the status quo, and indeed we are making a clear trade off between more ‘novel ideas’ and more depth along the current ‘orthodox’ research paths
Hi Quinn,
I think you may have strawmanned the case quite a lot here, likely unintentionally (so sorry if the strawman accusation comes off harsh). So let me clarify:
This statement is about ERS not about EA. We want to make a broad, thriving field which EA could and I hope would be a part of creating
I basically think this is wrong. Sure, I, for example, don’t want racists in my community (and I spoek about this to John). But this is a genuine attempt to make a community with a plurality of methods, visions of the futures, ways of doing things. We explicitly don’t want agreement, and we say as much. If you look at the signatories, these are people who hold a whole host of different views (although its probably more homogenous than I would like actually!)
This statement was based on a workshop, and the signatories are only drawn from there. There was large amounts of disagreement at the workshop about a bunch of things, and we definitely would want a space that could sustain this. Indeed, much of the point of setting up an ERS space is to facilitate active disagreement that we can learn from, rather than shut it down and replace one political view with another. So I essentially think your comment here is wrong
I think its important to not simplify political pluralism down to simply party politics, particularly in one country, so I’m simply not sure this simple left-right axis can at all accurately conceptualise the politics of the space and I think if we are to fully capitalise on the politics of this space and how to diversify them, we really need to look more expansively.
Addressing your question directly, I firstly don’t claim to know or have all the answers, although I can think of places it would definitely be useful. One potential example would be to develop a political philosophy of xrisk reduction based on Burkeian ideas, which I think may be a useful addition to an ‘agents of doom’ agenda. I’m sure other intellectual currents in conseravtive thought could also be useful, particularly given how inherently conservative the idea of XRisk reduction is (eg see Mitchell and Chaudrey’s paper, and I actually agree this inherent conseravtivism is problematic) and so conservatives ought to have a lot to say about xrisk reduction.
Ye, in response to the first point I basically think you’ve misinterpreted my position on this John, so thanks for laying this out so clearly.
I have never (as far as I know, and if I have, I must apologise because I am wrong) accused you or other EA’s of lack of deference to peer reviewed science in the climate space. I essentially agree that you are more in line with the climate expert consensus on climate and xrisk than I am, and I would (and have) defended you against accusations that you don’t know and utilise this literature very well (althouigh I think you do fail to engage with the literature on climate and xrisk, but we’ve already had this discussion and its very tangential to my main point). I just don’t think the present peer reviewed literature actually does a very good job at all at addressing climate and xrisk questions (although again, I think tis is tangential).
I full agree with your second bullet point. But note, again, the statement says “more care put into publication practices for those engaged in research, including greater consideration and conversation of the appropriate role of different fora in the epistemic process, including peer reviewed publications”. I interpreted this as suggesting not ‘peer review is perfect and amazing’ but rather if we are to make this community as epistemically healthy as possible, we need to have consideration for what the appropriate role for peer reviewed publications are. For me anyway, one motivating factor behind this is a question of if we ought to or not have an xrisk journal, or whether we ought to set up an xrisk specific forum (like the alignment forum is). Each of these can play different epistemic roles, and I think there is often a polarised response of peer review is perfect or peer review is the worst. I do think there is a strong argument against peer review in xrisk work, although I also think there are compelling arguments in favour of peer review in certain forms. Indeed, many signatories have made public defences of peer review (eg David Thorstad) and others have been critical (eg Erica Thompson’s Escape from Modelland contains critiques of peer review), but I think we broadly felt that it was a discussion much more worth having in the xrisk space
Firstly John, before I address the ( interesting and useful) substantive point you make here, I think the first paragraph is clearly blatantly false. Firstly, you accuse all these signatories and authors of not being transparent. This is a deeply disrespectful accusation of bad faith, which is clearly untrue. Please recind this statement or I will be reporting you to the forum team for breaking forum norms.
I don’t speak for every signatory, although I would urge you to look at the list of signatories; does this look to you like a group of people who want to fold EA into XR. The letter is explicitly talking about ERS as a field and not just EA; it wants to create a more pluralistic field with EA engaging as a part of this more pluralistic and diverse field, not replace EA with this field. It is also not about making ERS just CSER-type approaches by any margin. Also, please point me where in the letter you see any calls for things that look at all like ‘Folding EA into extinction rebellion’; we are talking about a plurality of methods to study XRisk, and a plurality of individuals and visions of the future to do this, and a community that best supports this, and I can’t think this misreading of our letter comes from anywhere else but pattern matching to other pieces rather than actually engaging with what is said. Please correct this.
Your also are simply wrong about ‘Folding EA into extinction rebellion’ is the ‘main aim of heterdox CSER-type approaches to EA’; the point of this letter isn’t to advance CSER, but I think this blatant misinformation ought to be corrected. How is ‘Governing Boring Apocalypses (https://www.sciencedirect.com/science/article/pii/S0016328717301623) or ’Classifying Global Catastrophic Risks https://www.sciencedirect.com/science/article/pii/S0016328717301957‘, two classic works in the CSER-type approaches folding EA into extinction rebellion? Even the work on climate and xrisk, such as ‘Climate Endgame’ of ‘Climate Change’s contribution to GCR’ has very little to do with XR. Again, unless you can provide me evidence for your point that is not a gross misrepresentation of a large body of work by a large amount of people, please recind and correct your statement.
Onto your more substantive point; I basically agree this is an issue with calls for diversity; ultimately the tent has to stop somewhere, and that somewhere will clearly be ‘politically (not necessarilly party political) charged’. I agree, I don’t want racists, homophobes or ableists in the ERS community. I think negotitating what the boundaries of this community is and should be is a really difficult task, and indeed, in many ways I think you undersell it. Why shouldn’t we include people who think that God will bring about the apocalypse? Indeed, focusing on a variety of methods was also a key part, so how do we constrain what methods we should use, and how we can decide if two methods disagree what to do? I think this problem is tricky, and is definitely something that will need to be iterated on, negotiated and researchered and understood conceptually more. But you here throw the baby out with the bathwater; its a deeply unsastifying solution when we have a good reason to have pluralism of method, vision of the future, epistemology and also greater diversities of many different factors to suggest that just because it may be possible to justify the inclusion of those you don’t like on this logic, then we have to throw out the entire argument full stop.
I definitely think a weakeness of the statement is the vagueness; it had to be to be so ecumenical.
Nonetheless, I do think it is untrue that its not worth saying sans specifics. As I say above, ‘greater conversation as to the role of peer review’ does not mean ‘peer review and credentialism everywhere’. Rather, it means better understanding the epistemic role it plays, which can be pretty useful, and deliberately deciding whether and how this means we should and could deploy peer review in the field.
Secondly, the vagueness overall does mean it says less of substance than many people I have spoken about this would like. The hope is that this lays a stake in the ground suggesting that actually the epsitemic status of XRisk does mean pluralism OF SOME KIND AND TO SOME EXTENT is required to make this field better. I think the argument we make in that section, and the sketches we give of pluralism ought to be informative enough to at least start a discussion and debate about this. I’ve felt its been a bit unfortunate that people seem to have solely focused on the diversity side (which is really important, and definitely some signatories think is most important) rather than the pluralism of approaches, methodologies, core assumptions etc. I think even in this piece it sets out a decently substantive vision that we ought to use new and more approaches, and construct a community that is better suited to this
Some of the scholars who’ve worked on Insects or Decapod and Pain/Sentience (Jonathan Birch, Meghan Barrett, Lars Chittika etc)
Bob Fischer on comparing interspecies welfare